There were two new major hardware additions the SPOT spacecraft, increased processing power and new camera system, which allowed for on board vision-based tracking systems to be implemented. The two new hardware additions were the NVIDIA JetsonTX2 and the ZED Stereo camera by StereoLabs.

NVIDIA Jetson TX2

The Jetson TX2 and development board were selected to add more processing power and graphics processing unit, GPU, to be enable the SPOT spacecraft to handle more demanding tasks such as machine learning and vision processing. The development board allows for easy interfacing to the Jetson while fitting within the physical constraints of the SPOT spacecraft. The Jetson specifications are as follows, 256-core NVIDIA Pascal GPU, Dual-Core NVIDIA Denver 64-bit CPU and QuadCore ARM Cortex A57 MPCore processor, 8GB of LPDDR4 memory and 32GB of internal storage. The Jetson power draw is 15W which also allows the on board battery to provide power to it.

ZED Camera

In addition to the new processing board, a stereo camera was added to the spacecraft as well. The camera selected was the ZED Stereo Camera by Stereolabs. The ZED camera is capable of capturing up to 2K video at 15fps, however for this work the camera was used at the 720p and 30-60fps mode. The camera comes with proprietary software provided by Stereloabs to resolved depth information relative to the camera, however this software was not used for this work.

Integration into SPOT Platforms

In order to enable the Jetson TX2 to communicate to the GN&C Raspberry Pi an ethernet cable was directly connected between both computers. This was chosen over wireless mainly to reduce latency as well as to avoid potential interference between other devices on the wireless network which could inadvertently impact performance. Setting both computers with static IP addresses allowed for consistent connections between devices and an easy way to debug communication problems if they were to occur. This allowed for consistent and trouble free communications between both computers.

On the software side, the Jetson acted as the communications server with the Raspberry Pi being a client to the server. The two would communicate using TCP/IP to ensure a connection between the Raspberry Pi and the Jetson was made prior to any experiment starting. If a connection was not established the Raspberry Pi would wait until the connection was made, effectively causing the experiment to pause. This would prevent experiments from being run without communications between the two computers and effectively notify the researcher there is something wrong with the current experiment. A network diagram of how the communications is relayed across the computers is shown in Figure 2.5 below.

The Jetson has a second network layer that handles how external information from the Raspberry Pi is internally broadcasted to programs that might need this information. This is done by custom software, written in Python, called the Jetson Repeater which acts like a wifi repeater taking in data from the external network and broadcasting it on the local 127.0.0.1 network of the Jetson. The same is true for data going to the Raspberry Pi with anything published onto the 127.0.0.1 network being relayed to the Raspberry Pi. The Jetson Repeater allows for multiple programs to listen to the local network for the information that is relevant to them without having to establish an external connection to the Raspberry Pi. This makes it easier for new programs to be written and debugged since they only have to listen to the local host network. If the Raspberry Pi network information changes or additional external connections are added only the Jetson Repeater needs to be updated and all preexisting programs running on the Jetson will continue to work without being updated.

The Raspberry Pi TCP/IP client software was created using MATLAB and a custom simulink block to allow for data to be transferred to and from the Jetson. This custom simulink block is customizable to be flexible for future experiments and can be customizable depending on the data being passed to and from the Jetson.