Sample Application API - DigitalMediaProfessionals/dv-sdk GitHub Wiki

Here describes common API among the sample applications. This API enables us to implement an DNN application for an AI FPGA module. The concrete features of the API is as below.

• Load images for DNN input from a file or a camera
• Preprocessing images for DNN
• Drawing and outputing a result from DNN to a monitor.

The API can be used for not only DNN but also other purposes.

• Monitor Output / Drawing
  • Utility API
  • COverlayRGB
• INPUT
  • Camera API
  • Image API
  • Preprocessing API
  • Others

These APIs are declared in common/include/util_draw.h.

The below is an example of monitor output.

```c++
#include <iostream>
#include <cstdlib>
#include "util_draw.h"

using namespace std;
using namespace dmp;
using namespace util;


int main()
{
    // 1st: Initialize framebuffer
    if (!init_fb()) {
        cerr << "Failed to init FrameBuffer" << endl;
        return -1;
    ]

    // 2nd: Create a COverlayRGB instance and allocate memory.
    uint32_t width = get_screen_width();
    uint32_t height = get_screen_height();
    uint32_t image_w = 256;
    uint32_t image_h = 256
    COverlayRGB overlay(width, height);
    overlay.alloc_mem_overlay(image_w, image_h);

    // 3rd: Drawing to the memory
    // Here the memory is filled with random pixels
    for (uint32_t x = 0; x < image_w; x++) {
        for (uint32_t y = 0; y < image_h; y++) {
            bool s = overlay.set_pixel(x, y, rand(), rand(), rand());
            if (!s) goto fin;
        }    
    }
fin:

    // 4th: draw the drawn to the framebuffer
    uint32_t left_top_x = 10;
    uint32_t left_top_y = 10;
    overlay.print_to_display(left_top_x, left_top_y);

    // 5th: Swap buffers (since framebuffer is double bufferred) and output the drawn to a monitor
    swap_buffer();

    // 6th: Finalization
    shutdown();
}

dmp::util::COverlayRGB class is responsible for drawing and monitor output. To show something in a monitor, allocate a buffer for drawing by alloc_mem_overlay() method, draw it by each drawing method and output the drawn to a monitor by print_to_display() method. (After this, you have to call swap_buffer() function to switch framebuffer.)

The followings are a list of methods of COverlayRGB. For more detail, see commnets in the source code.

API for INPUT is declared in common/include/util_input.h.

To use a camera, open a camera by open_cam() and load an image by capture_cam(). Please close the camera by close_cam() when it is no longer in use.

int dmp::util::open_cam(int capture_w, int capture_h, int fps);

Opens a camera.

• Arguments
  • capture_w Width of an image from a camera
  • capture_h Height of an image from a camera
  • fps FPS of a camera
• Returned value
  • 0 on success, non-zero value otherwise

int dmp::util::close_cam();

Closes a camera.

int dmp::util::capture_cam(uint32_t* vImg, int capture_w, int capture_h,
                int crop_xoffs, int crop_yoffs, int crop_w, int crop_h);

Loads an image from a camera.

• Arguments
  • vImg Buffer for an image
  • capture_w Height of an image from a camera
  • capture_h Width of an image from a camera
  • crop_xoffs x offset for cropping
  • crop_yoffs y offset for cropping
  • crop_w width of cropping
  • crop_h height of cropping
• Returned value
  • 0 on success, non-zero value otherwise

void get_jpeg_image_names(const std::string &input_image_path, std::vector<std::string> &image_names);

Searchs JPEG files recursively and get those paths. This function is declared in demo_common.h.

• Arguments
  • input_image_path Directory to be searched in.
  • image_names Buffer to store the JPEG file paths.

int dmp::util::decode_jpg_file(const std::string &imgFileName, uint32_t *imgView,
                    int img_width, int img_height);

Decodes a JPEG file into RGB data.

• Arguments
  • imgFileName Path to a JPEG file.
  • imgView Destination buffer.
  • img_width Width of the image.
  • img_height Height of the image.
• Returned value
  • 0 on success, non-zero value otherwise

void dmp::util::preproc_image(uint32_t *__restrict imgView, __fp16 *__restrict imgProc,
                   int img_width, int img_height, float r_offs, float g_offs,
                   float b_offs, float sf, bool transpose, bool is_bgr = true);

void dmp::util::preproc_image(uint32_t *__restrict imgView, __fp16 *__restrict imgProc,
                   int inimg_width, int inimg_height, int outimg_width,
                   int outimg_height, float r_offs, float g_offs, float b_offs,
                   float sf, bool transpose, bool is_bgr = true);

Converts a RGB image to an input image for a DNN model.

• Arguments
  • imgView a RGB image.
  • imgProc Destination buffer.
  • inimg_width Width of the RGB image.
  • inimg_height Height of the RGB image.
  • outimg_width Width of the input image for a DNN.
  • outimg_height Height of the input image for a DNN.
  • r_offs Offset of R channel. A value of R channel in the input image for a DNN is calculated by adding r_offs and a value or R channel in the RGB image.
  • g_offs Offset of G channel. A value of G channel in the input image for a DNN is calculated by adding g_offs and a value or G channel in the RGB image.
  • b_offs Offset of B channel. A value of B channel in the input image for a DNN is calculated by adding b_offs and a value or B channel in the RGB image.
  • sf Scale coefficient. The final value of each channel is determined by multiplying sf after applying *_offs.
  • transpose If true, this function transposes the image. Please use true since SDK supposes that the input is transposed.
  • is_bgr If true, this function converts the input image to a BGR image.

void handle_keyboard_input(int &exit_code, bool &pause);

Configure exit_code or pause according to keyboard inputs. If ESC key is pressed, exit_code will be 0. If SPACE key is pressed, pause will be !pause.

This function is declared in demo_common.h.