By Roland Elek

The aim of this project was to create a set of Java bindings for the GStreamer 1.x series, automatically generating everything where applicable. The primary target platform was Linux throughout the project. The end result is not meant to be Linux only, however.

The first approach taken was to use JNAerator to automatically parse the low-level native code (GLib, GObject, etc.), generate the corresponding java files, then build GStreamer bindings on top of this layer from GObject Introspection data. These efforts were forked from a temporary playground to test some ideas to rewrite the gstreamer-java bindings so they can support GStreamer 1.x. See the original repository for details, and a starting point for building on OSX. We selected BridJ as the native interoperability library, mainly because it is the one JNAerator supports the best. Less important factors were that I had no in-depth knowledge about any interoperability library, and BridJ offered ease of use, and high performance, compared to the other library considered, which was JNA. Original GLib tests were ported to Java to test the new bindings. JUnit was chosen as a test framework.

There is no such thing as a free lunch, and using JNAerator was no exception. It is a young project, generating bindings for GLib is a complex and hard job, and we don’t know about any previous attempts at doing it with JNAerator. Therefore, we came across uncommon cases that required changes to JNAerator. Luckily, its author, Olivier Chafik, was willing to help us by fixing and adding things, and some problems could be worked around. See the following issues for further details: https://github.com/ochafik/nativelibs4java/issues/132 https://github.com/ochafik/nativelibs4java/issues/518 https://github.com/ochafik/nativelibs4java/issues/511 https://github.com/ochafik/nativelibs4java/issues/516 https://github.com/ochafik/nativelibs4java/issues/512 https://github.com/ochafik/nativelibs4java/issues/509

By around the mid-term evaluations, we had a set of bindings generated using JNAerator, and some tests to prove that we can use a simple GList, sort an array of random integers using qsort (which requires a comparator callback), and subclass GObject (which requires multiple callbacks, and accessing struct fields) to create a singleton class, all through the new bindings.

The source code, along with instructions for building, and trying things out can be found in a GitHub repository at https://github.com/octachoron/gstreamer-1.x-java.

After having created the low-level bindings, we started to address the other end of the problem: building a higher level layer on top of the low-level bindings using GObject Introspection data, which comes in two different flavors: .gir XML files, and a binary format compiled from said XML files, called typelibs. After extensive research on possible approaches and previous similar efforts, we settled on using the XML files, and writing a parser that generates Java code from them. XOM is used for parsing the XMLs. The reason for this is again ease of use, and that it has already been used to accomplish a very similar goal as part of java-gnome GObject Introspection support. We also needed an intermediate data structure to parse the XML files into, and to use for final Java code generation. Both of these requirements are provided by a library called CodeModel. Another candidate was FreeMarker. We chose CodeModel, mainly because we found it first, and it already provided everything we needed. The native interoperability library used is BridJ, in anticipation of using the low-level bindings which were also based on BridJ. We are currently not using a test framework in connection with this approach.

The main challenge associated with this step was scarce documentation - we ended up figuring out most of the data structures in .gir files ourselves. Another challenge was that after seeing that GLib/GObject also had associated .gir files, we were very unsure until late in the project whether the JNAerator-based low-level bindings were necessary. While it currently seems that a modern system provides enough information in .gir files to generate a GStreamer bindings set from them down to GLib primitive types, GObject and below were previously expected to be special and not able to be generated from .gir files, especially following a discussion on gir-devel-list. Dependency resolution was also problematic, as the .gir files depend on each other, and types can be freely used before their definitions within a single .gir file.

By the final evaluations, we were able to parse a significant subset of the .gir format. The main features are currently:

• generating enums for and elements found in .girs
• using elements to generate package names
• creating empty interfaces for opaque structs
• creating classes for and elements with fields
• tracking types defined in the .gir files, and resolving references to them
• handling array types to some degree
• reading a static mappings list
• keeping a diagnostic list of referenced, and defined types, and their difference to help identify any external dependencies
• parsing the given files in topologically sorted order based on include dependencies, which means the parser should see
• type definitions before references to them in other files
• generating class-specific and namespace-wide functions/methods

This set of features is enough to generate GStreamer bindings to the point where a video file can be played from Java using a playbin. A test was written to prove this. Another test proves that a simple GList can be used through code generated using this approach as well, apparently supporting the idea that GObject and below do not need special hand-written bindings anymore.

The source code, along with instructions for building, and trying things out can be found in a GitHub repository at https://github.com/gstreamer-java/gir2java.

Currently, the main direction is completing the parser, and using it for also generating the low-level bindings, not using the JNAerator-based bindings. The next steps towards this goal are:

• Fixing type reference resolution within a single file. There is already a two-pass approach implemented, but it fails non-deterministically.
• Implementing callback parsing and generation.
• Designing and implementing and parsing and generation.
• Adding another layer on top of the generated bindings that provides a view of the actual element types on the system, based on the information that can be queried using gst-inspect. (The current approach does allow the creation of any element, but does not provide a truly object-oriented view of them at the Java level.)
• Adding automatic reference counting.

Some progress has already been made after the GSoC time frame. The non-deterministic build failure has been fixed. There is also another test that assembles a pipeline from individual elements, and runs it. Named callback types are parsed and generated, and the playbin test now uses a bus callback to handle the EOS message instead of halting playback after a hardwired delay. Typedefs can be given manually to the parser in a file similar to the static mappings file, these are applied recursively to the c:type attributes of type references. This is currently used for defining gpointer, so that it is not treated as a struct by value. This fixes the generation of functions taking or returning gpointer, for example gst_object_unref(). The unref calls have been added to test programs where necessary. Field setters and getters are now prefixed with the class name to prevent unwanted inheritance. The way class/record fields are parsed is order dependent, so second pass snapshots are now collected on a deque. Private field accessors are no longer used, as BridJ does not count these as actual field accessors, which leads to it believing the native structure is smaller than it really is. Enum classes may have methods, and these are now usable. A less notable change is that the test video file is now shorter. For more details and updates, see the commits after 18 August (or after the one tagged GSoC-2014)