Custom layer - DigitalMediaProfessionals/dv-sdk GitHub Wiki

Since deep Learning is currently a rapid research area, not all types of layers can be supported by AI processor. The network convertor (referred to as the convertor) and driver can support some standard layer like SoftMax in software, but there are still lots of layers not supported by the convertor. In this chapter, the steps of how to support the PriorBox layer in SSD networks as an example to how to add custom layer support will be given.

Overview

In order to support custom layers, one needs to follow these steps:

Make sure the original network data file contains all required parameters. Implementation of non-standard layers in CNN frameworks may not store all necessary parameters to the network. Section export custom layer parameters will describe how to export all parameters to the network data file for the Keras framework.
Implement custom layer definition script and pass it to the convertor. In order for the convertor to recognize these custom layers, one needs to write simple Python script to tell the convertor how to determine the output dimensions and what parameters needed to be exported by the convertor. Section custom layer definition script will provide detailed information.
Implement custom layer callback function for the FPGA network class to call while running the network. The convertor will generate custom layer definition and store all parameters in the generated source code. The FPGA network class framework calls the callback functions to run each corresponding custom layer. One needs to implement the callback functions and section runtime custom layer implementation will describe the interfaces provided by the network class to help one implement these functions.

Currently only the network originally using Keras framework can have custom layer supports. It is because the Caffe framework uses the Protobuf format, which is not that easily extended by the convertor.

The following sections in this chapter will also describe