Supported layer types - DigitalMediaProfessionals/dv-sdk GitHub Wiki
Currently, not all the layer types are supported by AI processor.
In the following table, a quick review of the layer types and their mapping to FPGA-block type in AI processor is given.
Full details for Caffe and Keras layer types will be given after.
In the case that a layer is not supported and also if there is no user provided custom layer definition Python script, it will be detected by the network convertor (refered to as the convertor), and an error message will be generated by the convertor.
| Layer Type |
Mapping |
| Input |
Ignore |
| Data |
Ignore |
| Convolution |
Convolution |
| BatchNorm |
Convolution |
| LRN |
Convolution. Available for zia-firmware-c3-sd-100mhz/zia-firmware-c3-emmc-100mhz version 7.0.20180215 or the older. |
| Scale |
Convolution |
| Concat |
Convolution or SW implementation |
| Eltwise |
Convolution |
| Pooling |
Convolution* |
| InnerProduct |
Fully Connected for zia-firmware-c3-sd-100mhz/zia-firmware-c3-emmc-100mhz version 7.0.20180215 or the older. |
| ReLU |
Convolution or Fully Connected |
| TanH |
Convolution or Fully Connected |
| Flatten |
SW implementation or ignore |
| Reshape |
Ignore |
| Softmax |
SW implementation |
| Dropout |
Ignore |
*Note: Pooling layer by itself is not a considered as a layer. In case a pooling layer can not be merged with an existing convolution layer, then an extra convolution layer is be created, with convolution disabled and pooling parameters set accordingly.
NOTE: The convertor cannot convert a network whose input size is not defined.
Caffe layer support
In this section, lists of supported layers for Caffe framework are given.
In the following tables, the Support column means currently supported by the convertor, and the Future column means that layer maybe currently supported by AI processor, but not yet implemented in the convertor; or means that it is planned to be supported in later versions.
Caffe vision layer support
| Group |
Support |
Limitation |
| Convolution |
:heavy_check_mark: |
Maximum kernel size is 7x7 |
| Pooling |
:heavy_check_mark: |
|
| Spatial Pyramid Pooling (SSP) |
:x: |
|
| Crop |
:x: |
|
| Upsample |
:heavy_check_mark: |
Only support size 2, stride 2 and nearest neighbor setting |
| Deconvolution |
:heavy_check_mark: |
Supported by zia-firmware-c3-emmc/zia-firmware-c3-sd of version 7.0.20190410 or newer. Dilated Deconvolution is not supported. |
| Im2Col |
:x: |
|
Caffe recurrent layer support
| Group |
Support |
Limitation |
| Recurrent |
:x: |
|
| RNN |
:x: |
|
| Long Short Term Memory (LSTM) |
:x: |
|
Caffe common layer support
| Group |
Support |
Limitation |
| Inner product |
:heavy_check_mark: |
Not all parameter settings are supported. The maximum input size is 16384. |
| Dropout |
:x: |
|
| Embed - hot encode |
:x: |
|
Caffe normalization layer support
| Group |
Support |
Limitation |
| Local Response Normalization (LRN) |
:heavy_check_mark: |
Only those settings used by AlexNet and GoogLeNet are supported. Available for zia-firmware-c3-sd-100mhz/zia-firmware-c3-emmc-100mhz version 7.0.20180215 or the older. |
| Mean Variance Normalization (MVN) |
:x: |
|
| Batch Normalization (BR) |
:heavy_check_mark: |
Currently merge with previous Convolution layer |
Caffe activation layer support
| Group |
Support |
Limitation |
| ReLU |
:heavy_check_mark: |
|
| Rectified ReLU |
:heavy_check_mark: |
|
| Leaky ReLU |
:heavy_check_mark: |
|
| Parametric ReLU |
:heavy_check_mark: |
|
| Exponential ELU |
:x: |
|
| Sigmoid |
:heavy_check_mark: |
|
| TanH |
:heavy_check_mark: |
|
| Absolute Value |
:heavy_check_mark: |
|
| Power func |
:x: |
|
| Exp func |
:x: |
|
| Log func |
:x: |
|
| BNLL func |
:x: |
|
| Threshold |
:x: |
|
| Bias |
:heavy_check_mark: |
Adds bias during conversion (i.e. not a HW feature) |
| Scale |
:x: |
|
Caffe utility layer support
| Group |
Support |
Limitation |
| Flatten |
:heavy_check_mark: |
Implemented by software (i.e. not a HW feature) |
| Reshape |
:heavy_check_mark: |
Ignored |
| Batch Reindex |
:x: |
|
| Split |
:x: |
|
| Concat |
:heavy_check_mark: |
|
| Slicing |
:x: |
|
| Eltwise |
:heavy_check_mark: |
|
| Filter / Mask |
:x: |
|
| Parameter |
:x: |
|
| Reduction |
:x: |
|
| Silence |
:x: |
|
| ArgMax |
:x: |
|
| Softmax |
:heavy_check_mark: |
Implemented by software (i.e. not a HW feature) |
| Python |
:x: |
|
Caffe loss layer support
| Group |
Support |
Limitation |
| Multinomial Logistic Loss |
:x: |
|
| Infogain Loss |
:x: |
|
| Softmax with Loss |
:x: |
|
| Sum-ofSquares / Euclidean |
:x: |
|
| Hinge / Margin |
:x: |
|
| Accuracy / Top-k layer |
:x: |
|
| Contrastive Loss |
:x: |
|
Keras layer support
In this section, lists of supported layers for Keras framework are given. In the following tables, the Support column means currently supported by the convertor, and the Future column means that layer maybe currently supported by AI processor, but not yet implemented in the convertor; or means that it is planned to be supported in later versions.
None of the function in the groups Noise, Own Layers, Loss and Metrics are supported.
Keras convolution layer support
| Group |
Support |
Limitation |
| Conv1D |
:heavy_check_mark: |
|
| Conv2D |
:heavy_check_mark: |
Maximum kernel size is 7x7 |
| SeparableConv2D (Depthwise) |
:heavy_check_mark: |
Maximum kernel size is 7x7 |
| Conv2DTranspose |
:heavy_check_mark: |
Supported by zia-firmware-c3-emmc/zia-firmware-c3-sd of version 7.0.20190410 or newer. Dilated Deconvolution is not supported. |
| Conv3D |
:x: |
|
| Cropping1D |
:x: |
|
| Cropping2D |
:x: |
|
| Cropping3D |
:x: |
|
| UpSampling1D |
:heavy_check_mark: |
Only support nearest neighbor setting |
| UpSampling2D |
:heavy_check_mark: |
Only support size 2, stride 2 and nearest neighbor setting |
| UpSampling3D |
:x: |
|
| ZeroPadding1D |
:heavy_check_mark: |
|
| ZeroPadding2D |
:heavy_check_mark: |
|
| ZeroPadding3D |
:x: |
|
Keras pooling layer support
| Group |
Support |
Limitation |
| MaxPooling1D |
:heavy_check_mark: |
|
| MaxPooling2D |
:heavy_check_mark: |
|
| MaxPooling3D |
:x: |
|
| AveragePooling1D |
:heavy_check_mark: |
|
| AveragePooling2D |
:heavy_check_mark: |
|
| AveragePooling3D |
:x: |
|
| GlobalMaxPooling1D |
:heavy_check_mark: |
|
| GlobalAveragePooling1D |
:heavy_check_mark: |
|
| GlobalMaxPooling2D |
:heavy_check_mark: |
|
| GlobalAveragePooling2D |
:heavy_check_mark: |
|
Keras locally connected layer support
| Group |
Support |
Limitation |
| LocallyConnected1D |
:x: |
|
| LocallyConnected2D |
:x: |
|
Keras recurrent layer support
| Group |
Support |
Limitation |
| Simple RNN |
:x: |
|
| GRU |
:x: |
|
| LSTM |
:x: |
|
Keras merge layer support
| Group |
Support |
Limitation |
| Add |
:heavy_check_mark: |
|
| Multiply |
:x: |
|
| Average |
:x: |
|
| Maximum |
:x: |
|
| Concatenate |
:heavy_check_mark: |
Implemented by software (i.e. not a HW feature) |
| Dot |
:heavy_check_mark: |
|
Keras advanced activation layer support
| Group |
Support |
Limitation |
| LeakyReLU |
:heavy_check_mark: |
|
| PReLU |
:heavy_check_mark: |
|
| ELU |
:x: |
|
| ThresholdedReLU |
:x: |
|
Keras normalization layer support
| Group |
Support |
Limitation |
| Normalization |
:heavy_check_mark: |
|
Keras activation function support
| Group |
Support |
Limitation |
| Softplus |
:x: |
|
| Softsign |
:x: |
|
| ReLU |
:heavy_check_mark: |
|
| TanH |
:heavy_check_mark: |
|
| Sigmoid |
:heavy_check_mark: |
|
| Hard_sigmoid |
:x: |
|
| Softmax |
:heavy_check_mark: |
|
| Linear |
:heavy_check_mark: |
|