Wiki organization - terrytaylorbonn/auxdrone GitHub Wiki

24.0705 (epics 4,5 reorg'd into 4,5,6) (after 6 months of working on this project, I might have finally gotten the core wiki org right)

This wiki (a work in progress) starts out with

  • Home
  • Getting into the air ASAP. Focuses on the absolute essentials to get a basic drone into the air within 1-3 weeks (and to get an AI drone into the air within another month). If you want to build a drone ASAP, then click the link (no need to read further on this page).
  • Reference.
    • Drone/AI Concepts 24.0620
    • Shopping list. shows in detail how I ordered the parts I used. Most of the parts are from mainland China, so getting the right parts (with decent quality) and feeling confident about the safety of payment menthods was important.
    • Tech overviews. Tech overviews + config files for the FPV/AI drone builds.
    • Config files

The rest of the wiki is organized into 6 "epics":

  • Epic 1: Build/fly FPV drone. Build/fly a basic FPV drone (the best platform to learn the basics). These are cheaper, simpler drones that are not used as AI platforms. You could compare an FPV to a Linux computer.

    image

  • Epic 2: Build/fly Pixhawk drone. Build/fly a Pixhawk drone (that will be used for AI). You could compare a Pixhawk to a (open source) Windows computer.

    image

  • Epic 3: Add AI to Pixhawk drone. Add AI to the Pixhawk drone.

    image

  • Epic 4: Advanced AI. This epic is about taking the simple AI example from epic 3 to a new level. Writing (more) advanced Python programs and using advanced AI algorithms to control copter movement (the copter becomes "intelligent" and/or "autonomous"). A very sophisticated example would be a copter that could flight through the gates of a drone racing track (screenshot below).

    image

  • Epic 5: Advanced platforms. Build/fly drones for real-world mission platforms and run total simulations.

    image

  • Epic 6: Pie in the sky. HITL, SITL, Matlab, and ROS. Great (?) tools that (based on my limited understanding) are more trouble than they are worth for the purposes of ZiptieAI. I spent a lot of time on them because a lot of videos convinced me they were needed.

    image

Epic 1: Build/fly FPV drone

What makes building/flying your first drone so difficult are confusing tools, a massive hodgepodge of technologies, components from China, and a surprising lack of basic documentation. Its best to get acquainted with the drone ecosystem by first building a simple FPV drone.

  • (1) FPV simulators (Gdrive) (24.0529). These are useful simulators for beginners (unlike the amazingly complicated simulators I used in early 2024 (Gazebo, etc) for Pixhawk drones (1 Total simulation)). I added this to the wiki after crashing my first FPV drone endless times because I had no training on the RC sticks (I actually thought I would not need it).

    image

  • (2) SBeeF405/INAV (1a) (Gdrive). The entire build process from charging batteries to outdoor flight testing. SpeedyBee is the FC (flight controller) manufacturer. INAV is the firmware (the code the FC runs to keep the copter stable). SBee/INAV is not really suitable for AI, but it (or SBee/Betaflight) is the best FC/firmware to start out with.

    image image

  • (3) SBeeF405/BF (1b) (Gdrive). Simply switch the previous SBee/INAV copter firmware to BF. How to backup platform 1 INAV then replace with Betaflight with minimal disruption/effort.

  • (4) SBeeF405/AP (Ardupilot) (1c) (Gdrive). Not recommended, but I got to work partially. AP is normally used on Pixhawk drones, but it might make it possible to add AI to a SBee AP-based drone.

Epic 2: Build/fly Pixhawk drone

Pixhawk is a type of FC that comes in a plastic case (not just a bare metal board). Pixhawk FCs have more advanced functionality that makes them the best choice for drones with a companion computer (CC).

  • (5) Pix6c/PX4 (2a) (Gdrive). This is the best platform for an AI drone. The Pixhawk 6c FC and the PX4 firmware have the required functionality to support AI (and the copter frame is much bigger, making it possible to carry AI companion computers).

    image image

  • (6) Pix6c/AP (2b) (Gdrive). My experiences with Ardupilot have been less than positive. I spent a lot of time on it, and could not get the motors to work. In in any case, I documented my efforts (and I will figure out the motor problem eventually).

Epic 3: Add AI to Pixhawk drone

The CC (companion computer) makes the drone "intelligent" by (1) gathering input from external sensors, (2) performing AI (such as object detection/recognition), and (3) sending (text-based) flight commands to the FC (basically the CC does what the pilot normally does with the sticks and switches on a radio controller (RC)).

How it works:

  • The (1) camera output is (2) is analyzed (object recognition) by the CC which then (3) sends mission flight control messages to the flight controller. Note: The AI commands to the FC (3) are very basic and can be tested without HITL or SITL simulation (which are complicated to setup). But future AI programs will be more complex, and should be tested with HITL/SITL (the next topics).

    image image

Epic 4: Advanced AI

  • (9a) SIH (Software in hardware simulation) (Pix6c/PX4) (3) (Gdrive). Simulate the world that the AI drone will fly in (using SIH; use the real drone peripherals such as the RC, Rx, camera, etc). This allows testing of the AI flight algorithms safely before the first real flight.

    image

  • (11) FC firmware API (Pix6c/PX4) (5) (Gdrive). The Mavlink "server" runs on the FC. Other devices (CC, GCS's, etc) can use the FC Mavlink API (server) to interact with the FC by exchanging Mavlink messages with the FC using the following "local" (not on the FC; on the PC, CC, etc) "APIs":

  • (13) CC AI (+ Mavlink) (Pix6c/PX4) (2) (Gdrive).

    • Create the
      • AI program (that performs, for example, object recognition) and the
      • C++/Python program (that, based on AI program outputs, sends flight commands (mavlink) to the FC).
    • Flight test in the
      • (1) simulated world (HITL, part (9)), ensuring that the Mavlink commands are safe, and then the
      • (2) real world (field test).
        • image

Epic 5: Advanced platforms

Firmware customization, Matlab , Specific mission platforms (such as minefield detection), special projects (such as the "stealth" drone).

Epic 6: Pie in the sky

image

HITL, SITL, Matlab, and ROS for ZiptieAI drone purposes are basically PITS "Pie in the sky". PITS means

  • Too complex. You require an entire dev team to integrate. Especially with the AI routines (AI is a core focus of ZiptieAI). Since I am learning this stuff on my own, a big focus is avoid rabbit holes and PITS.
  • Whats the point? Sure if you are building customized proprietary drones with a big dev team, then fine. But that's not the ZiptieAI focus (ziptie'ing components is a core focus on ZiptieAI).

In particular for each PITS item:

  • HITL. Takes too much PC power, slow, and dont need the fancy 3d graphics. The objective is to "sanity-check" AI routines. Just use SIH. It works, gets the job done.

  • SITL. I got my start with SITL with an Intelligent Quads tutorial. But he stayed totally on simulation, and I think I am starting to understand why (more below).

  • ROS. With all its crazy versioning complexity (and the silly version names with the silly turtles, rather than just using version numbers). And

  • SITL/ROS. How to get these running on JNano and functioning realistically while running AI routines... what a rabbit hole. Seems that all the Youtube demos/tutorials are either

    • (1) custom hardware you can buy (ROS),
    • (2) running on a Linux machine (a clean machine with a specific version), or
    • (3) using simulation (convenient; but going from simulation to real HW?).

  • Matlab. Good chance I am wrong about this (exploring Matlab more is near term focus). I need to study more. But again for ZiptieAI purposes, seems to me that:

    • Proprietary. Very. Probably expensive. You might need quite a few modules to do anything.
    • Outdated, chaotic docs.
    • Whats the value for ZiptieAI?

  • (9) HITL (Partial simulation) (Pix6c/PX4) (3) (Gdrive). Simulate the world that the AI drone will fly in (HITL; use the real drone peripherals such as the RC, Rx, camera, etc). This allows testing of the AI flight algorithms safely before the first real flight.

    image image image image

  • (10) SITL (Total simulation) (SITL) (Pix6c/PX4) (1) (Gdrive). I dont see the value of SITL for the ZiptieAI goals. I left it in this wiki for now, however, because its

    • good overview of SITL and copter function in general, and its
    • Total simulation is based on a Youtube series from Intelligent Quads(that I started out with) that is a great complete intro to drones (other than ZiptieAI, its the only complete drone intro that I know of (the IQ tutorial only covers simulation))
    • SITL is in some ways easier and probably accurate enough for simple AI flight algorithms.
    • image

  • (14b) Matlab (added 24.0625) (Gdrive). Matlab has a very large and advanced set of capabilites for developing drones. But it has a big learning curve and convoluted, confusing docs. I have only done a few simple examples. Matlab is good about giving out demo/student trial licenses. Below: A lidar simulation on my PC that was surprisingly easy to run).

    image

  • (12) ROS ecosystem (Pix6c/PX4) (Gdrive). ROS is a messaging software whose node master runs on the CC and slave nodes run on ROS-enabled devices (cameras, lidar, the CC itself, etc). Rather than dealing with complicated drivers or custom interfaces, ROS nodes used the ROS master to publish messages that other devices can subscribe to.

    • image