Overview

This page details a starter project which combines the Jetson Nano 2GB Devkit, the Yocto BSP for NVIDIA jetson at OE4T, and NVIDIA’s NGC docker container registry

While the Jetson Nano 2GB Devkit is EOL, it represents a very inexpensive compute platform for starting out with AI and computer vision projects. At the time of this writing it can be found for less than $150 on ebay.

The purpose is to demonstrate a basic AI and computer vision deployment using docker on an embedded device. In this example, we’ll use the deepstream-nvdsanalytics-docker repository to demonstrate AI and computer vision capabilities.

Necessary Components

• One build machine, running Ubuntu 20.04. Using 22.04 will likely also work but is not specifically tested. You will need at least 100GB of disk space available. Ensure you have met all base yocto System Requirements for dunfell and required packages installed.
• One Jetson Nano 2GB devkit. Other Jetson hardware is also supported, you just need to modify the MACHINE references below to reference the appropriate MACHINE type.
• One USB 3.0 power supply or USB 3.0 power port, and USB 3.0 cable with type C connector.
• One USB cable with micro connector.
• One SDCard (16GB minimum).

Building The OS Image

NVIDIA refers to “Jetpack” as their software deployment for the Jetson series. NVIDIA’s support for the Jetson Nano 2GB ends with Jetpack 4. The latest release currently available at the time of this writing, Jetpack 4.6.4, can be found at this link. Using the instructions provided by NVIDIA, it would be possible to build a root filesystem Ubuntu deployment image which you could then use for deepstream development. These instructions will skip over this step and instead build the image using Yocto (and the content covered in the ECEN 5713 course). If you experience problems with the yocto image build, or deployment of software components on the yocto image, you may want to build a standard Jetpack image for comparison/troubleshooting purposes.

Setup Image Build

We’ll use the tegra-demo-distro repository to build our Yocto image, and specifically the dunfell branch as this is the most recent available version for the Nano 2GB which also works properly with docker and deepstream. Note that I’ve got a temporary fork setup at https://github.com/cu-ecen-aeld/tegra-demo-distro/tree/dunfell%2Bdeepstream-docker to work around an issue with docker support.

Refer to the instructions in the README at tegra-demo-distro for getting started, but the basic steps are:

git clone [email protected]:cu-ecen-aeld/tegra-demo-distro.git
git checkout --track origin/dunfell+deepstream-docker
source setup-env --machine jetson-nano-2gb-devkit

Customizing the image

To support compiling from within the docker container on target, you’ll need to add one package to the image. Use this command to add cuda-cudart-dev to your image via your local.conf file, as well as the git application.

echo  'IMAGE_INSTALL_append = " cuda-cudart-dev git" ' >> conf/local.conf

Note the underscore '_' here after 'IMAGE_INSTALL' instead of ':'. When using the dunfell branch, you need to use an underscore '_' separator for overrides where you would use a ':' in kirkstone or later branches as used in assignments.

TODO: These files should be added into a custom image specific to your project.

Building the image

Build the image using

bitbake demo-image-full

Deploying the image

When your image build completes, you can use the Flashing the Devkit instructions to deploy to the target. One way is to put the device in recovery mode and then use the deploy script to deploy to target.

1. The Nano 2GB device Force Recovery pin can be grounded by jumping the FRC REC pin to GND on the header at the back of the PCB. Place this jumper before applying power.

2. Apply power using a USB 3.0 cable and power supply.

3. Connect the micro USB connector to your host PC. Using lsusb on your host machine should list a NVIDIA APX target.

4. Run the deploy scripts using a scheme like this from the build directory:

mkdir deploy
cd deploy
ln -s ../tmp/deploy/images/jetson-nano-2gb-devkit/demo-image-full-jetson-nano-2gb-devkit.tegraflash.tar.gz
tar -xvf demo-image-full-jetson-nano-2gb-devkit.tegraflash.tar.gz
sudo ./doflash.sh

Building and running the deepstream container

After deploying, connect to your target over SSH or via a keyboard and monitor.

Modifying default docker runtime

Modify your docker runtime file at /etc/docker/daemon.json to look like this:

{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime" : "nvidia"
}

Then systemctl restart docker to pick up the change.

TODO: This step should be moved into your yocto customizations to avoid the need to modify at runtime.

Building and Running

Next, clone this repository on the target

git clone https://github.com/cu-ecen-aeld/deepstream-nvdsanalytics-docker.git

Follow the instructions in the README to build and run:

cd deepstream-nvdsanalytics-docker
./docker/build.sh
./docker/run.sh

Examining the Output

The first run will take several minutes to build a model used for conversion with messages like this:

[NvMultiObjectTracker] Initialized
ERROR: Deserialize engine failed because file path: /data/cfg/deepstream/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine open error
0:00:04.500013872    16   0x559ea4d430 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/data/cfg/deepstream/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine failed
0:00:04.501164464    16   0x559ea4d430 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/data/cfg/deepstream/resnet34_peoplenet_pruned.etlt_b1_gpu0_fp16.engine failed, try rebuild

After a few minutes, the logs will contain messages like this, indicating that the input video is being processed:

**PERF:  4.50 (6.53)
Frame Number = 8 of Stream = 0,   LineCrossing Cumulative peds = 0 LineCrossing Cumulative lane1 = 0 LineCrossing Current Frame peds = 0 LineCrossing Current Frame lane1 = 0
Frame Number = 9 of Stream = 0,   LineCrossing Cumulative peds = 0 LineCrossing Cumulative lane1 = 0 LineCrossing Current Frame peds = 0 LineCrossing Current Frame lane1 = 0
Frame Number = 10 of Stream = 0,   LineCrossing Cumulative peds = 0 LineCrossing Cumulative lane1 = 0 LineCrossing Current Frame peds = 0 LineCrossing Current Frame lane1 = 0
Frame Number = 11 of Stream = 0,   LineCrossing Cumulative peds = 0 LineCrossing Cumulative lane1 = 0 LineCrossing Current Frame peds = 0 LineCrossing Current Frame lane1 = 0
Frame Number = 12 of Stream = 0,   LineCrossing Cumulative peds = 0 LineCrossing Cumulative lane1 = 0 LineCrossing Current Frame peds = 0 LineCrossing Current Frame lane1 = 0
**PERF:  4.83 (5.79)

If you copy the resulting data-default/output/video.mp4 you should see a video like this:

Containing annotations for detected people and line crossing counts.

Next Steps

• You can customize the input video used by overwriting the video at data-default/input/video.mp4
• You can customize the location of lines for annotation by modifying [add link]
• You can customize the actions firmware takes on detection by modifying the associated source files in the src directory and rebuilding/running the container.
• You can modify the example to use a camera input instead of an input video
• You can modify the example to use an RTSP output stream instead of writing an output video.