The main idea of this project is to design a conceptual robot tour guide for an art gallery using Dagu Thumper. The kit's task is to scan the art space, detect the exibits by detecting their color, recall background information and give the visitors a brief description about the history of the art piece, e.x. artist, period, location ..etc. Please, check the GitHub repo here

Social robots has been recently used to serve multiple purposes, like personal assistance and performing tasks within the domain of domestic services and healthcare. Currently, there is a growing interest in the use of tour guide mobile robots in various environments such as museums, exhibitions and fairs, to provide help for the visitors by offering them a set of services, such as giving them general tours, helping them find points of interest, and providing information about different exhibits.

• MP3 Audio Playing: To give a brief description about an art piece.
• Color Detection: To recognize the art piece through dtecting different colors.
• Line Following: To use a line tracker array to follow a certain path.
• User Input Taking: By using an HC-05 bluetooth bridge.

The Kit will take the user input through a bleutooth module. The Dagu will then start the track searching for the spicific color associated with this exhibit. When the Robot finds the exhibit it will stop and retrieve the information related to it. A black track will represent the exhibition hall and two IR arrays will be used to follow this track.

The following is a grahical representation of the full system design showing all the sensors and controllers.

The following is a real representation of the whole system on top of the Dago Thumper kit.

IR array sensor fixation to read the black line:

• 2 STM32 Nucleo Boards (L432KC): to controle the communication between all the actuators and sensors.
• Dagu Thumper
• Pololu TReX Motor Controller : To control the Dagu Thumper.
• Color Sensor (TCS34725): to detect and recognize a certain exhibit.
• MP3 Mini Player Module (MP3－TF－16P): to play an audio file describing a given exhibit.
• MicroSD Memory Card: for audio files data storage.
• Speakers: To retrieve information.
• Line Follower Sensor: to help the Dagu Thumper follow a given path.
• HC-05 Bluetooth Bridge: to take input from the user about the exhibit they want to visit.
• Android Smart Phone: to send commands to the system about the exhibit they want to visit.

Initially, we wanted to design our system using the TCS3200 color sensor, which works as follows. It first senses color light with the help of an 8 x 8 array of photodiodes. Then, using a Current-to-Frequency Converter the readings from the photodiodes are converted into a square wave with a frequency directly proportional to the light intensity. Finally, using the Microcontroller Board we can read the square wave output and get the results for the color.

How it Works:

• First set the input pins as input and output pins as output. No need to use analog pins.
• Set S0 and S1 to high or low to set desired frequency scaling.
• In loop, activate each filters by setting S2 and S3 to HIGH or LOW and measure frequency ‘fo’ from 6th pin to get corresponding colour intensity. Compare frequencies of each colour to determine the colour of the object.

Each 16 photodiodes are connected in parallel, so using the two control pins S2 and S3 we can select which of them will be read. So for example, if we want to detect red color, we can just use the 16 red filtered photodiodes by setting the two pins to low logic level according to the following table:

The sensor has two more control pins, S0 and S1 which are used for scaling the output frequency. The frequency can be scaled to three different preset values of 100 %, 20 % or 2%. This frequency-scaling function allows the output of the sensor to be optimized for various frequency counters or microcontrollers.

After running our initial experiments using the TCS3200 sensor, we later decided to replace it with the TCS34725 color sensor due to the fact that it is provided with an IR blocking filter, integrated on-chip and localized to the color sensing photodiodes, that minimizes the IR spectral component of the incoming light and allows color measurements to be made accurately. Additionally, it can be easily integrated with our Nucleo Board since it doesn't depend on timers to measure color frequencies, unlike TCS3200. To integrate it in our system, we made use of the library provided by Adafruit, and made the necessary adjustments to make it compatible with our C project since the library is implemented in C++. I2C synchronous serial communication was used to interface it with the MCU.

The tracing module sensor has 3 way signal collection and output, respectively the left, the middle, and the right of the 3 pair’s infrared TCRT5000 photoelectric switches. The communication with STM32 is directly through the GPIO port communication. The MCU handles the signals by collecting the signals from the 3 way sensor digital signal port. When the signal from the TCRT5000 was reflected by a white object it outputs high level. When meeting the black line, the infrared signal will be absorbed which won’t be reflected back. Hence, signal port will output low level. We used two modules to accomodate for the high speed of the Dagu sinse using one module only wasn't effecient in reading the line correctly.

The mp3 module is connected to the microcontroller using an asynchronous serial link, where it can read audio mp3 files from the microSD memory card and display them using the speaker, by taking command from the microcontroller on which audio file to play, volume level, ..etc. We connected it to the STM32 board using asynchronous serial communication, namely UART2, with the default baud rate for the module (9600), as well as the default clock configurations. Additionally, we made use of aready made library , and added to it some functionality to play a certain audio file by taking the track number as a parameter. For more information, check out the tutorial available in the references section below.

Additionally, we're using a bluetooth bridge to communicate with a secondary MCU via UART interface, to take input from an Android user via the bluetooth, and send it to the secondary MCU via the asynchronous link. Then, the secondary MCU transmit the user input to the primary MCU via a synchronous serial link using SPI.

• Keil uVision5
• STM32CubeMX
• Serial Bluetooth Terminal (or any equivalent application).

• The map of the gallery is predefined and positions of exhibits .
• The human interaction partners follow the robot’s suggestions without the need for the robot to check for their compliance.

1. We decided not to proceed working with Loomo Segway for number of reasons. These include the difficulty to deploy native C++ robotics applications on Loomo robot. The only Repository found that works as a bridge between JAVA SDK and C++ interfaces was in beta phase with no sufficient documentation and number of deprecated modules/libraries.
2. There were no suffecient resources that explain interfacing between the old color sensor (TCS3200) and the STM32 Nucleo Board (L432KC). Thus, we decided to replace it with another color sensor, namely TCS34725, which is performing noticeably better than the first one and has better documentations.

1. First Milestone (Following a Given Map and MP3 Module Implementation) DONE: Using a previously known localization map, the Dugo Thumper can guide the users and give them information about different exhibits, following a predefined path from start to end.
2. Second Milestone (Art pieces detection & Information Retrieval) DONE: We will utilize the color sensor to detect the exhibits and then retrieve the related information.
3. Third Milestone (Point-to-Point Mapping & User input taking) DONE: In this milestone, we shall add the feature of the robot being able to direct the users to their desired art piece from a any point in the exhibition. The User shall send the name of the art piece to the kit trough Bluetooth Module.

To build this project clone both the Slave and Master projects. Upload the slave projcet on our primary Nucleo board and the Master project on the secodary Nucleo board.

Follow this Cube diagram here for the primary Nucleo board connections:

Follow this Cube diagram for the secondary Nucleo board connections:

Using the serial Bluetooth Terminal application on your Android Mobile Phone, send the following Letters for each mood of operation:
"R" : to take the user to the red exhibit.
"B" : to take the user to the blue exhibit.
"G" : to take the user to the green exhibit.
"T" : to take the user in a general tour passing through all the exhibits.

• TCS3200 Color Sensor Tutorial
• TCS3200 Color Sensor Datasheet
• MP3 Mini Player Datasheet
• MP3 Mini Player Tutorial
• TCS34725 Color Sensor Datasheet
• Adafruit TCS34725 Library

Kindly, find the proposal video at this link.
For the slides, check this link.

Kindly, find the demo for the first milestone at this link. Find the MS1 presentation video at this link.
For the slides, check this link.

Kindly, find the demo for the final milestone of the general tour at this link.
Find the demo for the customized tour at this link . For the slides, check this link .

• Hadeel Mabrouk - 900163213
• Israa Fahmy - 900171831
• Samah Ayman - 900171848