z Progress (old) - terrytaylorbonn/auxdrone GitHub Wiki

PART A Sim all. Completed in Dec 2023.

PART B Sim partial. Current focus. Run various AI algorithms on a real Nvidia Jetson Nano or PI4 (2 Real AI CC + camera). This is my primary goal because I believe that

  • The focus in the future will be on (semi-) autonomous drones that are primarily a carrier for intelligence (like the big-time military drones).
  • The level of autonomy will increase rapidly (the Elbit Lanius is a lethal example https://www.youtube.com/watch?v=G7yIzY1BxuI), but the same trend will apply to non-lethal drones).
  • Small AI packages (that can fit on a small drone) will rapidly gain capability while requiring less power.

Below: Cameras (PICam, Logi920s) and CC's (PI4, Nano)

image

Note: I am using the Nvidia Jetson Nano (devkit $160) and the PI4 ($30) as the CC. Both are quite old models (4 years or more). I knew there would be problems when I got them, but I wanted to keep costs down on my first try. I will probably switch to Nvidia Orin Nano (devkit $500) and PI5 ($100) in the near future. There are lot of driver issues with the older models (the Jetson Nano camera drivers have been a major issue).

Based on the AI output, send flight commands (3 Real CC flight commands -> Sim FC) to a sim drone following a flight plan in a sim world (4 Flight plan -> Sim FC).

PART C Real drone. Build (5 Drone build) and "fly-by-AI" (6 Real CC flight commands -> Real FC (autonomous flight)) real drone (Holybro X500).

PART D Further study. ROS, more AI algorithms, etc.

Latest updates

Note: The list below describes the latest core updates to the docs. These docs are very draft test docs. After I have put all the puzzle pieces together, I will create documentation that contains only the valuable "distilled" info from these docs.

  • 24.0127 3b_pymavlink_sim-fc_v21_24.0127 chapter "*24.0127-1 mavwp (WORKS!!)" shows pymavlink program that imports mission then runs a mission, making stops, land, continue, etc. Took me several days to figure out the details (Ardupilot incomplete docs, skittish controller behavior)
  • 24.0121b facemesh / mediapipe / pi4 / Logi920s (see 2c_ai-pi4-raspbian_sim-fc_.docx).
  • 24.0121 tflite with PI4 / Logi920s (see 2c_ai-pi4-raspbian_sim-fc_.docx).
  • 24.0120 Implementation of Part A that implements basic simulated autonomous flight using Sim FC (SITL), PI4, openCV Haarcascades, Logitech 920s camera (see 2c_ai-pi4-raspbian_sim-fc_.docx).
  • 24.0119 Completed 2 Real AI CC + camera with PI4 running openCV, Logi 920s camera.
  • 24.0111 Flight control with QGC and Mission Planner (ongoing tasks) 4-Flight-plan-Sim-FC.
  • 23.1217 Completed 1 Sim all