Introduction

Video4Linux, v4l in short, is an API and a set of drivers implementing it, for real time capture of video in Linux. There are multiple USB devices, webcam and TV tuners supporting this API, thus making it easier for software developers to add video support to their applications. Currently, the supported version of v4l is version 2, or v4l2. For additional information you can check the official site as well as the Linux Media Subsystem Documentation

Section Overview

For the Qtechnology cameras, we support the v4l2 API. Thus, the features of the camera can be controlled via your applications. Not only do we support the C API, but we also provide a Python3 binding of the API, as well as the v4l2wrapper, a library that provides a simplified interface for handling all the features of the camera. In this tutorial we will go over the following:

• [Installation] (#Installation)
• [Adding the wrapper to a Python3 script] (#addingchap)
• [Capture an image] (#capture)
• [Get and set controls] (#getsetctrl)
• [Streaming video] (#streaming)
• [Conclusion] (#conclusion)

For now we will only talk about using v4l2 in Python. If you wish to see how to add v4l2 to a C application, please check out the tutorial on GCC

Installation

On the command line run:

apt-get update
apt-get install python3-v4l2wrapper

This will install the v4l2 bindings and the v4l2 wrapper. If you just want the bindings just run:

apt-get update
apt-get install python3-v4l2

Adding the bindings and wrapper to a Python3 script

Adding the bindings and wrapper is as simple as:

import v4l2
from v4l2wrapper import create_device_wrapper

create_device_wrapper is the function that will generate the wrapper for us. All we have to do is:

w = create_device_wrapper()

Capture an image

Capturing an image is as simple as doing the following:

img = wrapper.capture()

The returned image is an numpy.ndarray, which can be used in ScyPy and OpenCV-Python, so if we want to capture an image and present it we can do this using e.g. the PIL library:

from PIL import Image
from v4l2wrapper import create_device_wrapper

w = create_device_wrapper()
npimg = w.capture()
#this is format specific, you may need to change 'RGB' to e.g. 'BGR'
pilimg = Image.fromarray(npimg, 'RGB')
pilimg.show()

Get and set controls

Controls can be set and changed in the application by using the 'get_ctrl' and 'set_ctrl' functions. The wrapper knows the type of the control and will try to convert the input to the correct type if possible. if you check the name space of the wrapper (this can be done by either the variable followed by a period '.', so 'wrapper.' and pushing 'tab' twice, or you can type 'dir(wrapper)'), you will see that there are a series of functions with certain standard prefixes. these are:

• get_ctrl_{control name}, set_ctrl_{control name}: These are the most standard controls, used for getting and setting variables like integers and strings. If you input a value that does not fit the type of the control, the wrapper will try and convert it into the correct format.
• turn_on_{control name}, turn_off_{control name}, switch_{control name}: These represent switches, that can either be on or off. There is also a utility function for inverting the control value.
• select_{control name}_{selection name}: These represent a menu of possible options. running one of these functions will change the specific menu to the selected value.

Alternatively, if you wish to get an entire list of all available controls, you can do:

w.pprint_ctrls() 

This will give you an entire list of the controls, their type, minimum, maximum and default values.

As a note, there is a UI in GWT for manipulating v4l2 controls, which is more useful if the user wishes to change values on the fly.

Streaming video

As we saw before, we can capture images from the device via the capture() function. Video is a similar task as it is nothing more than capturing a series of images in succession at a certain rate. However, the capture function is not as efficient at performing such a task, since it is slow, due to a) having to calculate formatting information per capture and b) requiring to perform additional copies in order to retrieve the image from the device. The v4l2 API takes care of this by providing a streaming capability for video capture. Below is a simple python3 script that performs capture and presentation using opencv to output the image to the screen:

import v4l2, cv2
from v4l2wrapper import create_device_wrapper

w = create_device_wrapper()

w.apply_format(1024,720,v4l2.V4L2_PIX_FMT_BGR24)
w.set_fps(30)
w.request_buffers(3)
w.stream_on()

streaming = True
while streaming:
  buf = w.get_formatted_frame()
  cv2.imshow('image',buf)
  if cv2.waitKey(1) != -1:
    break

cv2.destroyAllWindows()
w.stream_off()

Lets go over the code. First, we do the required imports and create a wrapper to the v4l2 device. next we do:

w.apply_format(1024,720,v4l2.V4L2_PIX_FMT_BGR24)
w.set_fps(30)
w.request_buffers(3)

We are setting certain characteristics for the device. Usually these should be the minimum settings an application should do before running. First, we set the width height and format. The format is a v4l2 identifier for the bgr format, with no additional padding. We need this format since OpenCV expects data to be passed to it in bgr. for a complete list of v4l2 formats you can check the official API documentationhere. To see what formats are available to the device you can using the print_device_info` function. This will print out device info, amongst which is also the format info. Next, we set the fps and then we request 3 buffers from the driver. The buffers we request are memory mapped regions that the driver and application will commonly use for pushing images to the application. The implementation details are not important here, just that we need to request the buffers from the device before streaming.

With everything set up and redy to go we can now start streaming. We start the stream by setting w.stream_on(). At this point, the camera will start capturing. Now we need to read the captured images and push them to OpenCV. This is done in the while loop. get_formatted_frame() recieves a frame from the driver and formats it into a numpy buffer. In this format, we can now pass the buffer to OpenCV using the 'imshow' function. we check the return value for any keystrokes and if we detect a keystroke we terminate the application.

Conclusion

In this tutorial we looked at how to perform some basic operations using the python interface for v4l2. specifically we looked at getting images, video and manipulating the device controls. For using the v4l2 interface in C check out the GCC tutorial. If you wish to build a more advanced application using video or image capture we suggest looking at the GStreamer tutorial.

Useful tools

When developing on a v4l2 device you may need to debug or test the capabilities of the v4l2 device you are working on. To help you can install the v4l-utils package. this contains a series of packages to aid development. A couple of useful tools are:

qv4l2: A QT v4l2 control panel application that is targeted to be compliant to the v4l2 specification. useful for testing device functionality and output.

v4l2-ctl: A tool for controlling v4l2 controls from the cmdline. Can help when you want to see what controls are available and to test if controls are working across the raw interface.

For more information on the package and tools you can go here.

As a final note, remember that the QTec camera device driver is exposed under /dev/qt5023_video{number}. So you should always target this for your video applications.