tiovxdlcolorblend - TexasInstruments/edgeai-gst-plugins GitHub Wiki

TIOVX DL Color Blend

tiovxdlcolorblend Applies a mask defined by an input tensor over an input image using the TIOVX Modules API.

Hierarchy

GObject
 ╰──GInitiallyUnowned
     ╰──GstObject
         ╰──GstElement
             ╰──GstAggregator
                 ╰──GstTIOVXMiso
                     ╰──GstTIOVXDLColorBlend

Pad templates

`sink`

      video/x-raw
                 format: { (string)RGB, (string)NV12 }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:batched)
                 format: { (string)RGB, (string)NV12 }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
           num-channels: [ 1, 16 ]

Availability – on request

Direction – sink

Object type – GstTIOVXMisoPad

Pad properties:

emit-signals: Send signals to signal data consumption

flags: readable, writable

Boolean.

Default: false

pool-size: Pool size of the internal buffer pool

flags: readable, writable, controllable

Integer.

Range: 2 - 16

Default: 2

repeat-after-eos: Flag to indicate if the pad will repeat the last buffer after an EOS is received. Only valid for sink pads

flags: readable, writable, controllable

Boolean.

Default: true

`tensor`

      application/x-tensor-tiovx
               num-dims: 3
           tensor-width: [ 1, 8192 ]
          tensor-height: [ 1, 8192 ]
              data-type: [ 2, 10 ]
      application/x-tensor-tiovx(memory:batched)
               num-dims: 3
           tensor-width: [ 1, 8192 ]
          tensor-height: [ 1, 8192 ]
              data-type: [ 2, 10 ]
           num-channels: [ 1, 16 ]

Availability – on request

Direction – sink

Object type – GstTIOVXMisoPad

Pad properties:

emit-signals: Send signals to signal data consumption

flags: readable, writable

Boolean.

Default: false

pool-size: Pool size of the internal buffer pool

flags: readable, writable, controllable

Integer.

Range: 2 - 16

Default: 2

repeat-after-eos: Flag to indicate if the pad will repeat the last buffer after an EOS is received. Only valid for sink pads

flags: readable, writable, controllable

Boolean.

Default: true

`src`

      video/x-raw
                 format: { (string)RGB, (string)NV12 }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:batched)
                 format: { (string)RGB, (string)NV12 }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
              framerate: [ 0/1, 2147483647/1 ]
           num-channels: [ 1, 16 ]

Availability – always

Direction – src

Object type – GstTIOVXMisoPad

Pad properties:

emit-signals: Send signals to signal data consumption

flags: readable, writable

Boolean.

Default: false

pool-size: Pool size of the internal buffer pool

flags: readable, writable, controllable

Integer.

Range: 2 - 16

Default: 2

repeat-after-eos: Flag to indicate if the pad will repeat the last buffer after an EOS is received. Only valid for sink pads

flags: readable, writable, controllable

Boolean.

Default: true

Properties

data-type

Data Type of tensor at the output

Flags : Read / Write / Controllable

Enum Default : 10, ("float32")

Enum "GstTIOVXDLColorBlendDataType"

(2): int8 - VX_TYPE_INT8

(3): uint8 - VX_TYPE_UINT8

(4): int16 - VX_TYPE_INT16

(5): uint16 - VX_TYPE_UINT16

(6): int32 - VX_TYPE_INT32

(7): uint32 - VX_TYPE_UINT32

(10): float32 - VX_TYPE_FLOAT32

latency

Additional latency in live mode to allow upstream to take longer to produce buffers for the current position (in nanoseconds)

Flags : Read / Write

Unsigned Integer64. Range: 2 - 18446744073709551615 Default value : 0

min-upstream-latency

When sources with a higher latency are expected to be plugged in dynamically after the aggregator has started playing, this allows overriding the minimum latency reported by the initial source(s). This is only taken into account when larger than the actually reported minimum latency. (nanoseconds).

Flags : Read / Write

Unsigned Integer64. Range: 2 - 18446744073709551615 Default value : 0

name

The name of the object

Flags : Read / Write

String Default value : "tiovxdlcolorblend0"

num-classes

Number of classes in mask

Flags : Read / Write

Unsigned Integer. Range: 0 - 4294967295 Default value : "8"

parent

The parent of the object

Flags : Read / Write

GstObject Default value : N/A

start-time

Start time to use if start-time-selection=set

Flags : Read / Write

Unsigned Integer64. Range: 2 - 18446744073709551615 Default value : 18446744073709551615

start-time-selection

Decides which start time is output

Flags : Read / Write

**Enum ** Default value : 0 ("zero")

Enum "GstAggregatorStartTimeSelection

(0): zero - Start at 0 running time (default)

(1): first - Start at first observed input running time

(2): set - Set start time with start-time property

target

TIOVX target to use by this element

Flags : Read / Write / Controllable

Enum Default : 0, ("DSP-1")

Enum "GstTIOVXDLColorBlendTarget"

(0): DSP-1 - DSP instance 1, assigned to C66_0 core

(1): DSP-2 - DSP instance 2, assigned to C66_1 core

Pipeline examples

Single DLColorblend

TENSOR=<input-mask-file>
OUTPUT=<output-file>

GST_DEBUG=tiovxmiso:4,tiovxdlcolorblend:4 gst-launch-1.0                                                                                                     \
tiovxdlcolorblend data-type=3 name=blend videotestsrc is-live=true ! "video/x-raw,format=NV12,width=640,height=480" ! blend.sink                             \
filesrc location=$TENSOR blocksize=76800 ! "application/x-tensor-tiovx,data-type=3,num-dims=3,tensor-width=320,tensor-height=240" ! blend.tensor blend.src ! \
filesink location=$OUTPUT -e

Batched processing


TENSOR_A=<input-mask-file>
TENSOR_B=<input-mask-file>
OUTPUT_LOCATION=<output-file>

gst-launch-1.0                                                                                                                               \
videotestsrc num-buffers=10 is-live=true pattern=0    ! "video/x-raw, width=640, height=480, format=NV12, num-channels=1" ! mux_a.           \
videotestsrc num-buffers=10 is-live=true pattern=ball ! "video/x-raw, width=640, height=480, format=NV12, num-channels=1" ! mux_a.           \
tiovxmux name=mux_a ! "video/x-raw(memory:batched), width=640, height=480, format=NV12, num-channels=2" ! blend.sink                         \
multifilesrc num-buffers=10 loop=true location=$TENSOR_A blocksize=76800 !                                                                   \
"application/x-tensor-tiovx, data-type=2, num-dims=3, tensor-width=320, tensor-height=240, num-channels=1" ! mux_b.                          \
multifilesrc num-buffers=10 loop=true location=$TENSOR_B blocksize=76800 !                                                                   \
"application/x-tensor-tiovx, data-type=2, num-dims=3, tensor-width=320, tensor-height=240, num-channels=1" ! mux_b.                          \
tiovxmux name=mux_b !                                                                                                                        \
"application/x-tensor-tiovx(memory:batched), data-type=2, num-dims=3, tensor-width=320, tensor-height=240, num-channels=2" !                 \
blend.tensor                                                                                                                                 \
tiovxdlcolorblend num-classes=8 data-type=2 name=blend ! "video/x-raw(memory:batched), width=640, height=480, format=NV12, num-channels=2" ! \
tiovxdemux name=demux                                                                                                                        \
demux. ! queue ! multifilesink location=$OUTPUT_LOCATION/image_a_%02d.nv12                                                                   \
demux. ! queue ! multifilesink location=$OUTPUT_LOCATION/image_b_%02d.nv12