Using a Custom TensorFlow Model with Blocks - ftctechnh/FtcRobotController GitHub Wiki
The Blocks Development Tool includes a sample op mode that demonstrates how a user can load a custom inference model. In this example, we will load a TensorFlow inference model from a prior FIRST Tech Challenge season.
The Robot Controller allows you to load an inference model that is in the form of a TensorFlow Lite (.tflite) file. If you are a very advanced user, you can use Google's TensorFlow Object Detection API to create your own custom inference model. In this example, however, we will use a .tflite file from a prior season (Skystone).
The inference model is available on GitHub at the following link:
Press the "Download" button to download the file from GitHub to your local device.
Press the Download button to download the .tflite file.
Once you have the file downloaded to your laptop, you need to upload it to the Robot Controller. Connect your laptop to your Robot Controller's wireless network and navigate to the "Manage" page of the system:
Connect to the Robot Controller's wireless network and navigate to the Manage page.
Scroll towards the bottom of the screen, and click on the "Select TensorflowLite Model File" button to open a dialog box that you can use to select your .tflite file.
Press the "Select TensorflowLite Model File" button.
Use the dialog box to browse for and to select the .tflite file.
Use the dialog box to find and select your .tflite file.
Press the "Upload" button to upload the file to the Robot Controller. The words "Tensorflow Lite model upload complete" will appear if the upload was successful.
Press the "Upload" button to upload the file to the Robot Controller.
Click on the "Blocks" tab at the top of the screen to navigate to the Blocks Programming page. Click on the Create New Op Mode Button to display the Create New Op Mode dialog box. Specify a name for your new op mode. Select "ConceptTensorFlowObjectDetectionCustomModel" as the sample op mode that will be used as a template for your new op mode. Note that if you do not have a webcam configured for your Robot Controller, the dialog box might display a warning message. You can ignore this warning message. Press "OK" to create your new op mode.
Use "ConceptTensorFlowObjectDetectionCustomModel" as the template for your new op mode.
The new op mode should appear in edit mode in your browser. By default, the sample op mode assumes that you are using the smartphone camera with your robot controller. If you are using a webcam, disable the phone-specific initialization block and enable the webcam-specific block.
Use the appropriate initialization block for the type of camera used on your robot.
Look in the op mode for the programming block "TensorFlowObjectDetectionCustomModel.setModelFromFile" and change the "tfliteModelFilename" from its default value of "WiffleBalls.tflite" to the name of the .tflite file that you uploaded to your robot controller.
Change the tflitemodelFilename to match the name of your uploaded file.
When you load an inference model, you must specify a list of labels that describe the known objects that are included in the model. For the "skystone.tflite" model, there are two known objects. The first object has a label of "Stone" and the second object has a label of "Skystone". Click on the little gear icon for the "create list with" block, and then drag a second item onto the list block, to change the number of items in the list from one to two.
Click on gear icon to configure the number of items in the list.
After you have added the additional item to the list, click on the gear icon a second time to hide the list configuration window. Change the names of the list items to "Stone" and "Skystone".
Specify the labels to be used for the elements in the model.
If the object that you are trying to detect will be at a distance of 24" (61cm) or greater, you might want to set the digital zoom factor to a value greater than 1. This will tell the TensorFlow to use an artificially magnified portion of the image, which can result in more accurate detections at greater distances.
You can specify a zoom factor if the object is located at a distance greater than 24" (61cm) from the camera.
Press the "Save Op Mode" button to save your changes. Then run the op mode. The robot controller should use the new Skystone inference model to identify and track the Stone and Skystone elements from the Skystone challenge.
The op mode should detect the game elements from the Skystone challenge.
You can use the following images (and point the camera at your computer's screen) to test your op mode.
