Optimization Tips

The model needs to be int8/uint8 quantized. Operators that take in other types are not supported on the DLPU.

Tensor sizes must be constant at compile time. Dynamic sizes are not supported.

Quantize the model using per-tensor quantization. This performs much better than per-channel quantization.

Prefer regular Conv2d over DepthwiseConv2d.

The optimal kernel size for convolutions is 3x3.

Use a stride of 2 for convolutions whenever possible as this is natively supported by the convolution engine. For other cases, consider using pooling.

The number of filters per convolution block is optimal when it is a multiple of 6.

The DLPU has zero-skipping, meaning outputs of exactly zero is preferred over outputs close to zero, for all operations. Example: ReLU is much more efficient than Leaky ReLU.