• Embedded plateform: PX2 AutoChauffeur(P2379)
• Ubuntu: 16.04
• ROS: Kinetic
• OpenCV: 2.4.9.1
• CUDA: 9.2

For NVIDIA is continuously updating the software support for Drive PX2, methods on this page are only guaranteed to be valid before November 27, 2019. Please follow up on any updates from NVIDIA after this date; get official support in the NVIDIA developer forum for Drive PX2.

NVIDIA SDK Manager is provided to flash the board. The needed GPU driver, CUDA and cuDNN are included. The highest version is SDK Manager 0.9.14.4964 with CUDA 9.2 for now. To know GPU information, run sample /usr/local/cuda/samples/1_Utilities/deviceQuery.

Install DRIVE with SDK Manager provides the step-by-step installation guide for using the SDK Manager. Note that during the installation you will be asked in SDK-Manager about the HARDWARE CONFIGURATION at STEP 01, if you want to use Nsight toolkit for monitoring GPU usage, please select both "Host Machine" and "Target Hardware(PX2 AutoChauffeur)". The reason is in the later section Measuring GPU utilization and performance. During the flashing process: connecte the USB 2.0 for Debug(PX2 interface) and the host PC with USB A-A cable.(Source: Hardware QuickStart Guides-->DRIVE PX 2 AutoChauffeur)

Source: https://devtalk.nvidia.com/default/topic/1066116/gpu-driver-of-px2-autochauffeur-p2379-/

If the lower version of CUDA is needed, choose past versions of SDK to flash the board.

Source: https://devtalk.nvidia.com/default/topic/1066757/zed-camera-working-on-drive-px-2/

Follow the instruction below

Install ROS Kinetic

Errors you might meet:

1. If broken ROS dependencies show up when installing ROS or when running the simulation they can be fixed by executing:

sudo apt install ros-kinetic-'name of package'

In our case we had a rviz package missing which got fixed by executing:

sudo apt install ros-kinetic-jsk-rviz-plugins

2. When you run sudo apt-get install ros-kinetic-desktop-full, you might get this error:

Reading package lists... Done

Building dependency tree

Reading state information... Done

Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation:

The following packages have unmet dependencies:  ros-kinetic-desktop-full :

Depends: ros-kinetic-desktop but it is not going to be installed

Depends: ros-kinetic-perception but it is not going to be installed

Depends: ros-kinetic-simulators but it is not going to be installed E: Unable to correct problems, you have held broken packages.

You should run sudo apt-get install aptitude sudo aptitude install libssl-dev to downgrade the version of libssl-dev

(Tip: It's very important to notice "0 to remove" of the provided solution, if it's not, rerun this command again)

adeye@tegra-ubuntu:/etc/apt/sources.list.d$ sudo aptitude install libssl-dev
The following NEW packages will be installed:
  libssl-dev{b} libssl-doc{a} 
0 packages upgraded, 2 newly installed, 0 to remove and 30 not upgraded.
Need to get 1,077 kB/2,123 kB of archives. After unpacking 9,388 kB will be used.
The following packages have unmet dependencies:
 libssl-dev : Depends: libssl1.0.0 (= 1.0.2g-1ubuntu4.2) but 1.0.2g-1ubuntu4.5 is installed.
The following actions will resolve these dependencies:

     Keep the following packages at their current version:
1)     libssl-dev [Not Installed]                         

Accept this solution? [Y/n/q/?] n
The following actions will resolve these dependencies:

     Install the following packages:                                            
1)     libssl-dev [1.0.2g-1ubuntu4 (xenial)]                                    

     Downgrade the following packages:                                          
2)     libssl1.0.0 [1.0.2g-1ubuntu4.5 (<NULL>, now) -> 1.0.2g-1ubuntu4 (xenial)]

Accept this solution? [Y/n/q/?] y
The following packages will be DOWNGRADED:
  libssl1.0.0 
The following NEW packages will be installed:
  libssl-dev libssl-doc{a} 
0 packages upgraded, 2 newly installed, 1 downgraded, 0 to remove and 30 not upgraded.
Need to get 2,849 kB of archives. After unpacking 9,457 kB will be used.
Do you want to continue? [Y/n/?] y
Get: 1 http://ports.ubuntu.com/ubuntu-ports xenial/main arm64 libssl1.0.0 arm64 1.0.2g-1ubuntu4 [726 kB]
Get: 2 http://ports.ubuntu.com/ubuntu-ports xenial/main arm64 libssl-dev arm64 1.0.2g-1ubuntu4 [1,046 kB]
Get: 3 http://ports.ubuntu.com/ubuntu-ports xenial-security/main arm64 libssl-doc all 1.0.2g-1ubuntu4.15 [1,077 kB]
Fetched 2,849 kB in 0s (5,572 kB/s)   
Preconfiguring packages ...
dpkg: warning: downgrading libssl1.0.0:arm64 from 1.0.2g-1ubuntu4.5 to 1.0.2g-1ubuntu4
(Reading database ... 166815 files and directories currently installed.)
Preparing to unpack .../libssl1.0.0_1.0.2g-1ubuntu4_arm64.deb ...
Unpacking libssl1.0.0:arm64 (1.0.2g-1ubuntu4) over (1.0.2g-1ubuntu4.5) ...
Selecting previously unselected package libssl-dev:arm64.
Preparing to unpack .../libssl-dev_1.0.2g-1ubuntu4_arm64.deb ...
Unpacking libssl-dev:arm64 (1.0.2g-1ubuntu4) ...
Selecting previously unselected package libssl-doc.
Preparing to unpack .../libssl-doc_1.0.2g-1ubuntu4.15_all.deb ...
Unpacking libssl-doc (1.0.2g-1ubuntu4.15) ...
Processing triggers for libc-bin (2.23-0ubuntu11) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libssl1.0.0:arm64 (1.0.2g-1ubuntu4) ...
Setting up libssl-dev:arm64 (1.0.2g-1ubuntu4) ...
Setting up libssl-doc (1.0.2g-1ubuntu4.15) ...
Processing triggers for libc-bin (2.23-0ubuntu11) ...

Follow the steps here: Install SSDCAFE and most of the errors met and their solutions can be found in the same website in Note section.

Solution: the path to vision-ssd-detect is incorrect, it should be changed to the correct path. The path is set by the file deploy.prototxt which should be found in a path similar to what we found: /home/nvidia/AD-EYE_Core/AD-EYE/Data/ssdcaffe_models/AD-EYE_SSD_Model/SSD_512x512. The folder path code in deploy.prototxt can be found at lines 1821, 1824 and 1825. If you cannot find the file or line simply use the linux grep command, e.g. grep -Hrn 'search term' path/to/files where path/to/files can be omitted if you're already in the correct folder.

If the methods given here doesn't work, another method can be tried: create a file caffe.conf in the folder /etc/ld.so.conf.d; add the path of libcaffe.so.1.0.0-rc3 into the file caffe.conf; run sudo ldconfig.

1. As a complement to this Modifications; for PX2, sm=61 and sm=62.
2. During the compilation process, make runtest will report several broken tests, but this won't cause a real error on PX2.

Source: https://devtalk.nvidia.com/default/topic/1066619/errors-when-build-the-single-shot-detector-ssd-on-px2/

Install Autoware and AD EYE

Error message: cmake error at /opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:83 Could not find a package configuration file provided by "autoware_build_flags"

Solution: sudo apt-get update, sudo apt-get install -y + your missing package name

Sometimes when running adeye on PX2 some out of memory errors can happen if proper swap space is not allocated. Check available swap space using free -h . If swap space is shown as 0, you have to create a swap partion.Used a swap file of 7GB on TegraB setup. Follow the below link for reference.

https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-16-04

If the connection/communication between the prescan computer (host) and the PX2 is not working but no error messages are displayed on the host computer, it is most likely due to the argument of the command used to setup the connection. The command used is rosinit('IP_OF_COMPUTER') where IP_OF_COMPUTER can either be the network address or the set name associated with the IP. Due to a Prescan bug, the command should always use the name which is, unless changed, "tegra-ubuntu". To associate the IP with a name add the IP address and name to the file C:\Windows\System32\drivers\etc\hosts.

NVIDIA Nsight systems tools (including nvprof and nsight visual profiler) are performance tools provided by NVIDIA. They are part of the CUDA toolkit which should already be installed when the PX2 is flashed (the steps under Driver and CUDA, also given here)

Note however, that the tools can only be used remotely to profile the PX2 via an SSH connection between the host and target hardware (PX2). If the PX2 is flashed using an SDK manager, on the host, the SDK manager will install the CUDA toolkit that matches the one installed on the target, it's important that they match. The CUDA toolkits and nsight systems performance tools that can be downloaded directly from the website are not supported on the PX2. Please refer to the link below: https://devtalk.nvidia.com/default/topic/1052052/profiling-drive-targets/connection-error/

Nvidia Nsight systems via SSH connection was not used here because the profiling process needs to be able to terminate and restart the application multiple times. This is problematic since we would need to terminate and restart the prescan simulation as well but this is difficult because we have no control or knowledge of when the tool is doing this.

You can generate a timeline using nvprof in the terminal, locally on the PX2. However, to visualise and get some statistics and recommendations of optimisations, use the host computer, import the timeline into visual profiler to get some statistics on gpu utilisation of cuda applications(nodes).

on target run: nvprof --export-profile <path/to/file/timeline%p.prof> --profile-child-processes roslaunch adeye manager.launch

move files to any directory of your choice on host: go to directory where the files are saved and run the following /usr/local/cuda-9.2/libnvvp/nvvp <timeline#.prof> where <timeline#.prof> with correct filename and /usr/local/cuda-9.2/libnvvp/nvvp is the path to the visual profiler in the CUDA toolkit installed by the SDK manager.

For more information on nvprof and visual profiler refer to the NVIDIA documentation website: https://docs.nvidia.com/cuda/profiler-users-guide/index.html Please also note that tegrastats does not provide correct dGPU statistics on the Drive PX2.

By compiling the code in the file gpustats.cu and running the executable file, information about all present GPUs will be printed in the terminal followed by the memory usage in percentage for the currently used GPU. To compile the code, execute the following command in the terminal. Note: CUDA has to be installed before doing this step.

nvcc /path to file/gpustats.cu -o gpustats

To execute the runnable file created, execute the following command.

./path to file/gpustats

As stated above, the program starts by retrieving and printing the info for the present GPUs. It does so by using a function from the CUDA Runtime API which returns a cudaDeviceProp structure containing 69 data fields corresponding to the GPU. The function is executed on the host (CPU) and is stated as follows.

cudaGetDeviceProperties(cudaDeviceProp* prop, int  device)

where *prop is a pointer to a cudaDeviceProp structure and device is the id of the wanted device. More information on what data fields are available in the structure and more information about the function could be found here.

After retrieving and printing the GPU info the program continues into a loop that retrieves the free and total device memory which is used to later calculate the used memory. Before calculating and printing the memory usage the program retrieves the device currently being used. All functions used can be found in the CUDA Runtime API.

The limited disk space of PX2 may cause errors during installation steps, so always keep an eye on the remaining space and clean up useless files. Maybe useful tips:

1. Download stuff on a mobile hard drive; but care about the dependencies if installed software on a mobile hard drive.
2. Use rosclean purge to clean up the ros log files. Source: http://wiki.ros.org/rosclean

sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) universe"
sudo add-apt-repository universe
sudo apt-get update
sudo apt-get install exfat-fuse exfat-utils 

Source: https://unix.stackexchange.com/questions/321494/ubuntu-16-04-package-exfat-utils-is-not-available-but-is-referred-to-by-anothe

Ubuntu 16.04 on PX2 has less than 6 Gb RAM space, while Autoware installation may need more space. This process will always get stuck and finally terminated with errors, besides keeping applications with large RAM space occupied closing (e.g. browser)，swap space may be needed. There is a trade-off between disk space and ram space, in our cases, we allocate 2G space for swapfile. Guide to Creating Swap Space