TensorFlow-Micro EthosU face detection on EK-RA8P1

Branch information: https://github.com/renesas/zephyr/tree/wip/face_detection_sample

Application: https://github.com/renesas/zephyr/tree/wip/face_detection_sample/samples/modules/tflite-micro/face_detection_ethos

README: https://github.com/renesas/zephyr/blob/wip/face_detection_sample/samples/modules/tflite-micro/face_detection_ethos/README.rst

Hardware requirement:

• Connect USB cable → J10 for flashing and USB UART
• Connect Camera Expansion Board Arducam CMOS OV5640 Camera Module → Camera Expansion Port (JP35)
• Connect MIPI Graphics Shield RTKMIPILCDB00000BE MIPI Display → MIPI Graphics Expansion Port (JP32)
• SW4: SW4-1 to SW4-8 → OFF, except SW4-6 → ONs

Sample setup:

1. Setting up Zephyr repository:

   a. If you haven’t already setup Zephyr development environment please follow the guideline at Getting Started Guide — Zephyr Project Documentation:

   b. Add Renesas github repository to your remote and checkout to wip/face_detection_sample branch.

   $ cd <path to your zephyrproject>/zephyr/
   $ git remote add renesas_github https://github.com/renesas/zephyr.git 
   $ git fetch renesas_github wip/face_detection_sample
   $ git checkout wip/face_detection_sample
   

   c. Update lvgl, tflite-micro and hal_renesas revision:

   $ west config manifest.project-filter -- +tflite-micro
   $ west update lvgl tflite-micro hal_renesas
   

   d. Source Zephyr build environment, at “/zephyrproject/zephyr”

   $ cd <path to your zephyrproject>/zephyr/
   $ source zephyr-env.sh
   

2. Setting sample application: Build and Flash the Application: Use west build to compile the Face Detection application:

$ west build -b ek_ra8p1/r7ka8p1kflcac/cm85 -p always samples/modules/tflite-micro/face_detection_ethos/ --shield rtkmipilcdb00000be --shield arducam_cu450_ov5640_dvp

Flashing the application is simple as building, after do the build command just do in console:

$ west flash

Console log will print corrdinate information of detect face from camera:

Application run on board will The application captures image frames from an Arducam OV5640 camera, runs a TensorFlow Lite Micro face detection model, and displays the results on a MIPI LCD display connected to the Renesas EK-RA8P1development kit.

Appendix

How to generate Vela-compiled model for Ethos U55 256: Downloading the model from here or following command:

$  wget https://github.com/emza-vs/ModelZoo/blob/master/object_detection/yolo-fastest_192_face_v4.tflite

1. Open the Bash and create a virtual environment:

$ cd path/to/your/model/folder
$ python3 -m venv .venv

2. For Windows with Git bash:

$ source .venv/Scripts/activate 
$ pip install ethos-u-vela

For Linux:

$ source .venv/bin/activate 
$ pip install ethos-u-vela

3. Optimizing the model for Ethos-U using Vela (RA8P1 is using ETHOS U55 256 variant):

$ vela yolo-fastest_192_face_v4.tflite --output-dir . --accelerator-config ethos-u55-256 --system-config Ethos_U55_High_End_Embedded --memory-mode Shared_Sram

4. Converting to C array:

$ xxd -c 16 -i yolo-fastest_192_face_v4_vela.tflite yolo-fastest_192_face_v4_vela.tflite.h

Note: This model (yolo-fastest_192_face_v4_vela.tflite.h) has already been added to the sample.