Using TensorFlow in Freight Frenzy - WestsideRobotics/FTC-training GitHub Wiki
FTC teams can use TensorFlow Lite, a lightweight version of Google's TensorFlow machine learning technology that is designed to run on mobile devices such as an Android smartphone. A trained TensorFlow model was developed to recognize game elements for the 2021-2022 Freight Frenzy challenge.
This season's TFOD model can recognize Freight elements.
TensorFlow Object Detection (TFOD) has been integrated into the FTC control system software, to identify and track these game pieces during a match. The FTC software (SDK version 7.0) contains TFOD Sample Op Modes that can recognize the Freight elements Duck, Box (or Cube), and Cargo (or Ball).
Also, FTC teams can soon use a new tool to train their own TFOD models, to recognize their custom Team Shipping Element (TSE) and/or to improve recognition of Freight elements. This training could take into account certain conditions of distance, angle, lighting and background.
This new tool, FTC Machine Learning Toolchain, was announced 10/7/2021 for upcoming beta testing by interested FTC teams.
For this season's challenge, during the pre-Match stage a single die is rolled and the field is randomized.
Randomization
At the beginning of the match's Autonomous period, a robot can use TensorFlow to "look" at the Barcode area and determine whether the Duck or optional Team Shipping Element (TSE) is in position 1, 2 or 3. This indicates the preferred scoring level on the Alliance Shipping Hub. A bonus is available for using the TSE instead of a Duck.
Alliance Shipping Hub
TensorFlow Lite runs on Android 6.0 (Marshmallow) or higher, a requirement met by all currently allowed FTC devices. If you are a Blocks programmer using an older/disallowed Android device that is not running Marshmallow or higher, TFOD Blocks will automatically be missing from the Blocks toolbox or design palette.
The FTC software (SDK version 7.0 and higher) contains sample Blocks and Java op modes that demonstrate TensorFlow recognition of Freight elements Duck, Box (cube) and Cargo (ball). The sample op modes also show where in the camera's field of view a detected object is located.
Click on the following links to learn more about these sample Op Modes.
Teams have the option of using a custom inference model with the FIRST Tech Challenge software. As in the past, some teams may want to use the TensorFlow Object Detection API to create an enhanced model of the Freight elements or TSE, or to create a custom model to detect other entirely different objects. Other teams might also want to use an available pre-trained model to build a robot that can detect common everyday objects (for demo or outreach purposes, for example).
The FTC software includes sample op modes (Blocks and Java versions) that demonstrate how to use a custom inference model:
These tutorials use examples from a previous FTC season (Skystone), but the process remains generally valid for Freight Frenzy.
As noted above, soon FTC teams will have a streamlined tool for training their own TFOD models. Watch for announcements regarding the FTC Machine Learning Toolchain, already scheduled for beta testing.
You can use a pretrained TensorFlow Lite model to detect everyday objects, such as a clock, person, computer mouse, or cell phone. The following advanced tutorial shows how you can use a free, pretrained model to recognize numerous everyday objects.
TensorFlow can recognize everyday objects.
updated 10/21/21