Real-time object identification in Python

The object detection method uses the frames captured by the smartphone's camera to detect the objects presents in the scene. To use this method, it is necessary to first create an instance of the Robobo class, send the IP of the smartphone and connect it. Then, the readDetectedObject() method can be called and it returns an object containing all the information.

readDetectedObject( )

Returns the last object detected by the robot.

Returns
	A DetectedObject object

Return type
	DetectedObject

The object detection method is based on the "MobileNet V3" ANN, retrained to adapt it to our educational robotics scope. This ANN can recognize the following tags:

"Robobo, bowl, laptop, clock, scissors, fork, knife, spoon, cat, dog, remote control, keyboard, book, person, car, traffic light, stop sign, bottle, glass of wine, cup, banana, apple, orange, mouse, cell phone, vase."

You can download the model in the next link: MobileNet V3

To use it, you must put the “detect.tflite” and the “labelmap.txt” archives in a smartphone’s folder called “properties”, at the root path.

The steps that must be followed to use the method are shown below:

IP = "10.113.36.145"
rob = Robobo(IP)
rob.connect( )
# Start Object Recognition method
rob.startObjectRecognition()
obj = rob.readDetectedObject( )

# Stop Object Recognition method
rob.stopObjectRecognition()

The object returned by the method contains 6 values:

• The label of the detected object.
• The confidence level of the detected object (probability of success).
• The X coordinate of the center of the object in the image.
• The Y coordinate of the center of the object in the image.
• The width of the detection rectangle that contains the object.
• The height of the detection rectangle that contains the object

The width, the height and the (X,Y) values are used to locate the object in the image and draw the bounding box, as displayed in the following image:

Each value can be accessed individually by typing the following:

label = obj.label
confidence = obj.confidence
x = obj.x
y = obj.y
height = obj.height
width = obj.width

Usage example

The following script shows an example where each time an object is detected, the Robobo will say by voice (speech production) the detected object and the detection confidence percentage, and it will display the same information on the screen.

def SayDetectObject( ):
    obj = rob.readDetectedObject( )
    text = f'{obj.label} detected at {round(obj.confidence*100)} percent'
    print(text)
    rob.sayText(text)

for i in range(10):
    SayDetectObject( )

The following image shows the script output (bottom) when the robot detects a bottle that is placed in front of it as shown in the top part. The detection box has been drawn with the values returned by the method (X, Y, height and width):

Acknowledgement

Developing an artificial intelligence curriculum adapted to european high schools 2019-1-ES01-KA201-065742 More information: aiplus.udc.es

This project has been funded with support from the European Commission. This web reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.