Overview

This page describes some hints for getting started with GStreamer on Buildroot Raspberry Pi 3B+.

Introduction

GStreamer is a powerful framework for building multimedia applications. It enables developers to create seamless pipelines for processing audio and video, making it ideal for tasks like media playback, streaming, and recording. For more information, refer to the Gstreamer website.

The main purpose of this exercise is to use the gstreamer in the client-server environment with two Raspberry Pi's and a Logitech C270 camera. The GStreamer setup for Yocto- Project is used as a reference to start with the setup.

Setup Steps

Server

The buildroot image is built by referring to the following pages:

https://github.com/cu-ecen-aeld/buildroot-assignments-base/wiki/Setting-up-Buildroot-for-Hardware-Builds

https://github.com/cu-ecen-aeld/buildroot-assignments-base/wiki/Raspberry-Pi-Hardware-Support

After verifying the basic image built with the above commands, various configurations related to gstreamer and the dependent configurations are enabled from the menuconfig. The buildroot configuration corresponding to the yocto packages referenced from gstreamer-yocto are:

• gstreamer1.0-plugins-ugly:

  • BR2_PACKAGE_GST1_PLUGINS_UGLY

• gstreamer1.0-plugins-bad:

  • BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_X264
  • BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_VOAACENC
  • BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_RTMP2
  • BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_RTMP
  • BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_V4L2CODECS
  • BR2_PACKAGE_GST1_PLUGINS_BAD_PLUGIN_AUTOCONVERT

• gstreamer1.0-plugins-good:

  • BR2_PACKAGE_GST1_PLUGINS_GOOD_PLUGIN_V4L2
  • BR2_PACKAGE_GST1_PLUGINS_GOOD_PLUGIN_V4L2_PROBE
  • BR2_PACKAGE_GST1_PLUGINS_GOOD_JPEG
  • BR2_PACKAGE_GST1_PLUGINS_GOOD_PLUGIN_ALPHA
  • BR2_PACKAGE_GST1_PLUGINS_GOOD_PLUGIN_AUTODETECT

Refer to the following commits for the list of configurations including the graphics, display, utilities and dependent configurations:

• https://github.com/cu-ecen-aeld/final-project-krishnasuhagiya29/commit/06377bc79c204a1a02e8add810617abc6db69fe7
• https://github.com/cu-ecen-aeld/final-project-krishnasuhagiya29/commit/b62ec1ff254d5d5cb761954153e7f79551ad96bf
• https://github.com/cu-ecen-aeld/final-project-krishnasuhagiya29/commit/e6eea2306f36b894ba8d6a37da579b54a82ffce9
• https://github.com/cu-ecen-aeld/final-project-krishnasuhagiya29/commit/942b8b26024366447392374c1eed1faddabb7c3c
• https://github.com/cu-ecen-aeld/final-project-krishnasuhagiya29/commit/c544c1bc0b8890f445331b1911d246c183fec63d
• https://github.com/cu-ecen-aeld/final-project-krishnasuhagiya29/commit/67803ac0166ae7db9719fdf76216f70ae5dbbe6f

Client

The Buildroot image for the client can be built with the above mentioned configurations. The 'autovideosink' option sinks to the screen. The reference output with autovideosink with the above configurations on client looks like following:

Note: Additional work is required for getting autovideosink to work. Refer to the Raspberry Pi X11 Support for the list of display configurations.

Testing

Setup

Server: The buildroot image built with the Setup Steps

Client setup for RPi OS:

OS Version Details: Release date: March 15th 2024 System: 64-bit Kernel version: 6.6 Debian version: 12 (bookworm) Size: 1,105MB

sudo apt-get install libx264-dev libjpeg-dev

sudo apt-get install libgstreamer1.0-dev

sudo apt-get install libgstreamer-plugins-base1.0-dev

sudo apt-get install libgstreamer-plugins-bad1.0-dev

sudo apt-get install gstreamer1.0-plugins-ugly

sudo apt-get install gstreamer1.0-tools

sudo apt-get install gstreamer1.0-gl

sudo apt-get install gstreamer1.0-gtk3

The server has the logitech C270 camera located at /dev/video0 as source and sinks to the IP address of the other Raspberry Pi. The receiving Raspberry Pi has UDP port 5200 as source and sinks to the screen (autovideosink).

• The following command can be used to test the gstreamer installation on the client:

gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink

Output:

• The following command can be used on the server to get the information about the resolution, FPS and formats supported by the camera:

v4l2-ctl --list-formats-ext -d /dev/video0

Note: Replace /dev/video0 with the number of the video device detected.

Output:

There are many types of streaming possible with GStreamer. UDP and TCP are most used to connect two devices. The name of the streaming refers to the Ethernet protocol used. The steps are provided below to use UDP based streaming. Both the Raspberry Pis are connected to the same network.

• The following command can be used on the server Raspberry Pi by choosing any of the supported width, height and framerate configuration to start the streaming of camera feed:

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=-1 ! video/x-raw, width=1280, height=720, framerate=10/1 ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=10.0.0.15 port=5200

Note: Replace 10.0.0.15 with the client Raspberry Pi's IP address

Output:

• The following command can be used on the client Raspberry Pi to stream the camera feed received from the server

gst-launch-1.0 -v udpsrc port=5200 ! application/x-rtp, media=video, clock-rate=90000, payload=96 ! rtpjpegdepay ! jpegdec ! videoconvert ! autovideosink

Output:

References

GStreamer setup for Yocto- starter page

GStreamer setup for Yocto- Project

GStreamer-setup-for Buildroot- Raspberry Pi 3B+

Raspberry Pi X11 Support