Before going into this tutorial we would suggest to at least take a look at the official documentation and try using the basic tools described in the GStreamer introduction. Also, before developing make sure that there is nothing already out there that can do what you need! there are many elements and it may be that you overlooked an element that does what you need. You can always ask questions on the mailing list or on irc.

In this tutorial we will develop a simple element for printing out the timestamps of buffers using python for fast development. You will require:

• A setup that can run GStreamer in python
• Some basic knowledge of GStreamer, concepts and basic application/element concepts.
• Make sure the GStreamer GObject Introspection overrides are installed:

apt-get update
apt-get install python3-gst-1.0

• Access to the Python GI API Reference

Before starting with developing an element we will need an application pipeline to run it in:

import sys
import gi

gi.require_version('Gst', '1.0')
gi.require_version('GstBase', '1.0')

from gi.repository import GObject, Gst

Gst.init(sys.argv)
GObject.threads_init()
Gst.segtrap_set_enabled(False)

def connect(bus, name):
    def _connect(f):
        bus.connect(name, f)
        return f
    return _connect

def main():
  pipeline = Gst.parse_launch('videotestsrc ! ximagesink')

  bus = pipeline.get_bus()
  bus.add_signal_watch()

  @connect(bus, "message::error")
  def on_error(bus, message):
      pipeline.set_state(Gst.State.NULL)
      exit(message.parse_error())

  @connect(bus, "message::eos")
  def on_eos(bus, message):
      pipeline.set_state(Gst.State.NULL)
      exit(0)

  pipeline.set_state(Gst.State.PLAYING)
  loop = GObject.MainLoop()
  try:
      loop.run()
  except(KeyboardInterrupt):
      pass

  pipeline.set_state(Gst.State.NULL)


if __name__ == '__main__':
  main()

There is nothing out of the ordinary here. A simple videotestsource/ximagesink pipeline with some bindings to buss messages and handling keyboard interrupts. add the above code in a .py file and give it a try. You may ask 'why not use gst-launch?' gst-launch requires that you perform correct registration of elements. We don't want to have to go into handling OS specific details on how GStreamer detects and loads elements. We want to focus on developing an actual element before publishing it. So for now we will manually register the element and worry about distribution later.

Now time for the element code:

import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstBase', '1.0')

from gi.repository import Gst, GObject, GstBase
Gst.init(None)

class GstTimestamp(GstBase.BaseTransform):
  __gstmetadata__ = ('<longname>', '<classification>',
                     '<description>', '<author>')

  __gsttemplates__ = (Gst.PadTemplate.new("src",
                                           Gst.PadDirection.SRC,
                                           Gst.PadPresence.ALWAYS,
                                           Gst.Caps.new_any()),
                      Gst.PadTemplate.new("sink",
                                           Gst.PadDirection.SINK,
                                           Gst.PadPresence.ALWAYS,
                                           Gst.Caps.new_any()))

  def do_transform_ip(self, buffer):
    print("timestamp(buffer):%s" % (Gst.TIME_ARGS(buffer.pts)))
    return Gst.FlowReturn.OK

def plugin_init(plugin):
    Gst.Element.register(plugin, 'timestamper', 0,
                         GObject.type_register(GstTimestamp))
    return True

def init():
  version = Gst.version()
  Gst.Plugin.register_static(
      version[0], version[1], 'timestamper', 'Timestamper',
      plugin_init, '1.0', 'GPL', 'timestamper',
      'plugins-demo', 'demo')

init()

To get the element to load you need to import the module in the application. Then changing Gst.parse_launch to 'videotestsrc ! timestamper ! ximagesink' should launch the same x window. You should see the buffer timestamp messages on the console.

This is more obfuscated. 'do_transform_ip' is part of the PyGI documentation, so there is no surprise here. The messages being print are also simple to comprehend and using documented functions. But what about __gstmetadata__ and __gsttemplates__? The Py GI overrides uses these variables to load the GstElement metadata and pad templates. Unfortunately there is no real documentation on these, only the source code. __gstmetadata__ expects 4 strings and is equivalent to the gst_element_class_set_metadata function in c. __gsttemplates__ is a tuple containing all the GstPadTemplate definitions.

And what about auto detecting the element for applications like gst-launch? lets change the plugin code a bit:

import gi
gi.require_version('Gst', '1.0')
gi.require_version('GstBase', '1.0')

from gi.repository import Gst, GObject, GstBase
Gst.init(None)

class GstTimestamp(GstBase.BaseTransform):
  __gstmetadata__ = ('longname', 'classification',
                     'description', 'author')

  __gsttemplates__ = (Gst.PadTemplate.new("src",
                                           Gst.PadDirection.SRC,
                                           Gst.PadPresence.ALWAYS,
                                           Gst.Caps.new_any()),
                      Gst.PadTemplate.new("sink",
                                           Gst.PadDirection.SINK,
                                           Gst.PadPresence.ALWAYS,
                                           Gst.Caps.new_any()))

  def do_transform_ip(self, buffer):
    print("timestamp(buffer):%s" % (Gst.TIME_ARGS(buffer.pts)))
    return Gst.FlowReturn.OK

GObject.type_register(GstTimestamp)
__gstelementfactory__ = ("timestamper", Gst.Rank.NONE, GstTimestamp)

We removed the last couple of functions and replaced them with a type registration and an variable declaration. The type registration was happening in the plugin_init function before, so that shouldn't come as a surprise. The second line is another standard variable that the element registration code looks for. In this case it corresponds to a call of gst_element_register. However, if you do gst-inspect, you will see that the plugin isn't detected. We need to add it to a folder that the gstremer registry will look into for detecting new elements. In Linux there is an enviromental variable that handles this: GST_PLUGIN_PATH. So assuming that the module is under a folder called 'plugins' in your current directory you can do:

GST_PLUGIN_PATH=$PWD/plugins gst-inspect-1.0

And now the plugin should be listed.

Developing elements in python3 using GStreamer is a simple task as long as we are aware of certain caveats. This also shows one of the downsides of making python based elements: There are some features that do not get interpreted well via the GObject introspection. However, especially for rapid development, this can prove a powerful solution.