Tutorial: Making a Module using Depth Imaging - ColoradoSchoolOfMines/interface_sdk GitHub Wiki

Making a module that uses depth imaging is a straightforward process. Follow these simple steps to add depth image handling to your module.

Initial Setup

If you haven't yet made your project, see [Creating Your First Module] (https://github.com/ColoradoSchoolOfMines/interface_sdk/wiki/Tutorial:-The-Undisputed-Step-by-Step-Guide-to-Developing-with-the-Interface-SDK#3-creating-your-first-module) to get started. Make sure that you've defined a proper package name and made a class that extends a module.

Writing the First Lines

Your module should follow the format specified by the module type you're extending. Consult Extending Modules for more information on how to start your specific module type.

Getting Depth Images

In order to use depth imaging, you will need to add an input tag to your module manifest that looks like this:

<input input-type="depth" />

This specifies that your module will need access to depth imaging data, and this signals the [HardwareManager] (https://github.com/ColoradoSchoolOfMines/interface_sdk/wiki/API:-Hardware-Management) to give you permission to those devices.

You will also need to add a HardwareManager to your variables, and instantiate it like this:

private HardwareManager hardwareManager;
...
try {
	hardwareManager = HardwareManager.getInstance();
} catch (HardwareManagerManifestException e) {
	e.printStackTrace();
}

To get the driver itself, you will need to create a DepthImageInterface driver. The simple way to get the driver is to do this:

private DepthImageInterface depthDriver;
...
try {
	depthDriver = (DepthImageInterface) hardwareManager.getInitialDriver("depth");	
} catch (BadFunctionalityRequestException e) {
	e.printStackTrace();
} catch (InvalidConfigurationFileException e) {
	e.printStackTrace();
} catch (UnknownDriverRequest e) {
	e.printStackTrace();
}

getInitialDriver("depth") will return the first driver that supports depth imaging. If you would like to specify which driver you get data from, call getDevices("depth") and store it as a list, then call inflateDriver(devicePath, "depth"), where devicePath is one of the elements in the aforementioned list. Note that you must cast this driver as a DepthImageInterface as shown above.

To get a depth image, use the following function:

ShortBuffer depthBuffer = depthDriver.getDepthImageData();

This will produce a ShortBuffer containing the depth image data, which can be converted to a BufferedImage using the provided DepthImageUtilities:

NOTE: There is a bug in this conversion process that produces a low-quality image. If possible, use a different function to convert the buffer into an image.

BufferedImage img = DepthImageUtilities.shortBuffToImage(depthBuffer, 
	depthDriver.getDepthImageWidth(), depthDriver.getDepthImageHeight());

Using the Depth Image

Once you've retrieved the image, you can draw it as you would any BufferedImage. For Processing, use this function to convert a BufferedImage to a PImage:

public static PImage buffImagetoPImage(BufferedImage bimg) {
	PImage img = new PImage(bimg.getWidth(), bimg.getHeight(), PConstants.ARGB);
	bimg.getRGB(0, 0, img.width, img.height, img.pixels, 0, img.width);
	return img;
}

From there, draw it as you would any other image.