TI DL Inferer

tidlinferer is a plugin that wraps around Open Source RunTimes (TFLite,ONNX,TVM). It calls the API to run inference on input frame and produce resultant tensor.

Hierarchy

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseTransform
                         +----GstTIDLInferer

Pad templates

sink

      application/x-tensor-tiovx
               num-dims: 3
              data-type: [ 2, 10 ]
          channel-order: { (string)NCHW, (string)NHWC }
          tensor-format: { (string)RGB, (string)BGR }
           tensor-width: [ 1, 8192 ]
          tensor-height: [ 1, 8192 ]

Availability – always

Direction – sink

Object type – GstPad

src

      application/x-tensor-tiovx
               num-dims: 3
              data-type: [ 2, 10 ]
          channel-order: { (string)NCHW, (string)NHWC }
          tensor-format: { (string)RGB, (string)BGR }
           tensor-width: [ 1, 8192 ]
          tensor-height: [ 1, 8192 ]

Availability – always

Direction – src

Object type – GstPad

Properties

in-pool-size

Number of buffers to allocate in input pool

flags: readable, writable

Unsigned Integer Range: 2-16

Default: 2

model

TIDL Model directory with params, model and artifacts

flags: readable, writable

String

Default: NULL

name

The name of the object

flags: readable, writable

String

Default : "tidlinferer0"

out-pool-size

Number of buffers to allocate in output pool

flags: readable, writable

Unsigned Integer Range: 2 - 16

Default : 2

parent

The parent of the object

flags: readable, writable

GstObject

Default : N/A

qos

Handle Quality-of-Service events

flags: readable, writable

Boolean

Default : false

target

C7x target to offload the inference

flags: readable, writable, controllable

Enum "GstTIDLInfererTarget"

Default: 1, "C7x1"

(1): C7x1 - C7x instance 1

(2): C7x2 - C7x instance 2 (only for AM69A)

(3): C7x3 - C7x instance 3 (only for AM69A)

(4): C7x4 - C7x instance 4 (only for AM69A)

C7x Offload

This plugin uses conditional compilation based on flag ENABLE_TIDL. If the flag is true, the plugin is compiled with offloading to C7x turned on, if it is false, the plugin is compiled such that the inference runs on arm only mode i.e no C7x offload. ENABLE_TIDL flag is set to true by default for devices like TDA4VM, AM68A, AM69A, AM62A.

Pipeline examples

Input with explicitly defined preprocessing properties

gst-launch-1.0                                                                                             \
multifilesrc location=/opt/edgeai-test-data/videos/video_0000_h264.h264 !                                  \
h264parse ! avdec_h264 ! ticolorconvert !                                                                  \
video/x-raw, format=NV12 !                                                                                 \
                                                                                                           \
tee name=split                                                                                             \
                                                                                                           \
split. ! queue ! tiscaler ! video/x-raw, width=416, height=416 !                                           \
                 tidlpreproc data-type=3 channel-order=0 tensor-format=bgr  !                              \
                 tidlinferer model=/opt/model_zoo/ONR-OD-8200-yolox-nano-lite-mmdet-coco-416x416 !         \
                 queue ! post.tensor                                                                       \
                                                                                                           \
split. ! queue ! tiscaler ! video/x-raw,width=1280,height=720 ! post.sink                                  \
                                                                                                           \
tidlpostproc name=post model=/opt/model_zoo/ONR-OD-8200-yolox-nano-lite-mmdet-coco-416x416 !               \
kmssink sync=false driver-name=tidss