Network API - DigitalMediaProfessionals/dv-sdk GitHub Wiki

• Network Class Interface
• Layer Data Structure

The CDMP_Network class wraps upon user space driver, and provides these interfaces for the converted network application to run:

class CDMP_Network {
  // Internal member definitions.
  // ...
 public:
  std::vector<fpga_layer*>& get_output_layers();
  fpga_layer& get_layer(int i);
  int get_total_layer_count() const;
  int get_output_layer_count() const;
  virtual bool Initialize() = 0;
  bool LoadWeights(const std::string& filename);
  bool Commit();
  bool RunNetwork();
  void *get_network_input_addr_cpu();
  void get_final_output(std::vector<float>& output, int i_output = 0);
  int get_conv_usec() const;
  int get_fc_usec() const;
  int get_cpu_usec() const;
  void Verbose(int level);
  uint8_t *get_io_ptr() const;
  const dmp_dv_info_v0& get_dv_info() const;
};

Here are detailed descriptions of each function.

  std::vector<fpga_layer*>& get_output_layers()

Returns reference to output layer definition.

  fpga_layer& get_layer(int i)

Returns reference to the specific layer definition by given index i.

  int get_total_layer_count()

Returns total number of layers.

  int get_output_layer_count()

Returns number of output layers. In most cases the final layer of the network will be the only output layer, so this function will return 1. But for some complicated network, it is possible that the conversion tools will generate more than one output layers; which will result in this function returns number more than 1.

  virtual bool Initialize()

Initializes the network.

  bool LoadWeights(const std::string& filename)

Loads packed weights from file.
Note: Use the same file generated by the conversion tool. Currently there is no meta information stored in the packaged weight file, so one needs to make sure to load the correct weight file for the network or the execution results will be undefined.

  bool Commit()

Commits the network configuration. Must be called after LoadWeights if network contains fully connected layer.

  bool RunNetwork()

Runs the network. All layers in the network will be executed at once. This includes software implemented layers like SoftMax layers or Flatten layers.

  void *get_network_input_addr_cpu()

Returns address of the input buffer of the first layer.

  void get_final_output(std::vector<float>& output, int i_output = 0)

• output vector to store output data. Previous contents in the vector will be erased and be resized to the size of the output layer.
• i_output Index of the output layer. If there is only one output layer, can be omitted.

Copies output data to the provided buffer in single-precision float format.

  int get_conv_usec()

Returns last time spent for executing convolutional layers in microseconds.

  int get_fc_usec()

Returns last time spent for fully connected layers in microseconds.

  int get_cpu_usec()

Returns last time spent for CPU layers in microseconds.

  void Verbose(int level);

Sets verbosity level.

  uint8_t *get_io_ptr()

Returns pointer to the base of input/output buffer.

  const dmp_dv_info_v0& get_dv_info()

Returns reference to information about the SDK context.

The fpga_layer struct is defined in application/common/include/dmp_network.h as following data structure:

enum layer_type {
  LT_INPUT,
  LT_CONV,
  LT_FC,
  LT_FLATTEN,
  LT_CONCAT,
  LT_COPY_CONCAT,
  LT_SOFTMAX,
  LT_CUSTOM,
};

typedef void (*run_custom_callback_proc)(fpga_layer& layer, void *custom_param, uint8_t *io_ptr);

struct fpga_layer {
  layer_type type;
  std::string name;
  union {
    dmp_dv_cmdraw_conv_v0 conv_conf;
    dmp_dv_cmdraw_fc_v0 fc_conf;
    int softmax_axis;
    struct {
      int input_layer_num;
      fpga_layer **input_layers;
    };
    struct {
      run_custom_callback_proc custom_proc_ptr;
      void *custom_param;
    };
  };
  size_t input_offs;
  size_t output_offs;
  size_t output_size;
  int input_dim[3];
  int input_dim_size;
  int output_dim[3];
  int output_dim_size;

  bool is_output;
  bool is_f32_output;
  bool is_input_hw_layout;

  dmp_dv_cmdlist *cmdlist;
  int cmdlist_pos;
  int cmdlist_size;
};

Enum of the layer type. Currently the tool generate these layer types:

• LT_INPUT Input Layer
• LT_CONV Convolution layer. Note that pooling and batched normalization are merged into convolution layer in FPGA
• LT_FC Fully connected layer
• LT_FLATEN Flatten layer. This layer will only be generated if necessary.
• LT_CONCAT Concatenate layer. Generated only for preserving layer connection information
• LT_COPY_CONCAT Copy concatenate layer that actually need to be copied to work around FPGA buffer layout constraints.
• LT_SOFTMAX Softmax layer.
• LT_CUSTOM Custom layer. Will be generated only when one provides additional custom layer definition to the tool. See Custom layer for detail

Contains converted layer information for FPGA. The documentation of each member is as follows:

• layer_type type Type of the layer
• std::string name Name of the layer
• dmp_dv_cmdraw_conv_v0 conv_conf For convolution layer, contains config data for FPGA
• dmp_dv_cmdraw_fc_v0 fc_conf For fully connected layer, contains config data for FPGA
• int softmax_axis For softmax layer, the axis where the softmax should be applied
• int input_layer_num For copy concatenate layer, number of input layers
• fpga_layer **input_layers For copy concatenate layer, pointer array of input layers
• run_custom_callback_proc custom_proc_ptr For custom layer, function pointer to the software implementation function
• void *custom_param For custom layer, pointer to the parameter that will be passed to the above function when this layer is executed
• size_t input_offs Offset for an input from the base of the input/output buffer
• size_t output_offs Offset for an output from the base of the input/output buffer
• size_t output_size Size of the output in bytes
• int input_dim[3] Input dimensions
• int input_dim_size Length of input dimension vector
• int output_dim[3] Output dimensions
• int output_dim_size Length of output dimension vector
• bool is_output Indicates if the layer is output layer
• bool is_f32_output Indicates if the output is in 32-bit float format
• bool is_input_hw_layout Indicates if the input is using FPGA layer layout
• dmp_dv_cmdlist *cmdlist Command list containing this layer, can be NULL. See driver interface for more detail
• int cmdlist_pos Position of this layer in the command list
• int cmdlist_size Size of the command list