Sim partial concepts - terrytaylorbonn/auxdrone GitHub Wiki

("Partial simulation concepts")

Previously we ran simulation of everything. A sim camera on a sim copter recognizes a sim person and tells the copter to land near the person (search and rescue). All this in a sim world.

image

Such sim all is vital because it can verify the control SW on the sim companion computer (CC) that runs the mission. Note that although the sim copter is not an accurate representation of the real copter I plan to use, that (strangely enough) is not a problem. Quadcopters are impossible for a human to fly (the only control surfaces are the blades of the 4 motors), so the FC (flight controller) does all the work. To fly the copter just requires simple Mavlink protocol commands (“go forward 5m”). The copter maintains position using GPS/INS. You just send the commands, and the FC does the rest. So, any real-world copter is going to behave almost exactly as the sim copter.

But sim all has a big shortcoming. The sim camera and the sim CC that does the sim object recognition are probably not accurate representations of the actual CC (below left shows my Nano) and camera (PI camera below right).

image

The solution is “partial” simulation shown in the diagram below: 1 A real PI camera sends video to the real Nano. 2 The real Nano (or PI4) performs object recognition. 3 The sim drone (SITL running on Ubuntu PC) sends sim sensor data (UDP) to the mission software program (on the Nano). 4 The mission program running on the Nano sends flight control commands (TCP) to the sim drone.

image

This setup verifies that the mission program (Pymavlink) correctly (and safely) operates the drone before doing an actual flight. I have never read anything about doing this kind of “hybrid” sim, so I’m not sure it will work.

Below is my planned implementation of the above sim setup. The sim drone flies in an empty sim world (upper left) with no sim person (crossed out). A real toy person (or just a picture) is placed in front of the real PI camera (right). The real CC Nano (middle red circle) recognizes the real person and sends a command to the sim FC (lower left) to land the sim copter (in the sim world). This is a very simple test of the basic functionality.

image

The following shows a test on 24.0111 where the real Nano and the sim world connected via UDP/TCP.

  • Green circle: Jetson nano dev kit (companion computer).
  • Orange square: Nano screen, upper left code, right run, lower UDP sensor info previously from sim (SITL).
  • Pink square: Shows gazebo sim running on SITL ubuntu pc (right).
  • Pink circle: Sim drone. this did not work because requires ROS, and I did not set it up (intentionally to see if required).
  • Big blue square: Sim computer.
  • Small blue square: Sim FC with Mav proxy.
  • Blue circle: Mav proxy console. This shows enough sim info to verify without needing gazebo (Gazebo and ROS are quite big and complex to install).

image