ROS PRD - rosielab/pepper-documentation GitHub Wiki
Change history of PRD
| Author | Date | Describe Changes |
|---|---|---|
| Clare Provenzano | Nov. 8 2023 | Created, added section headings. |
| Clare Provenzano | Nov. 10 2023 | Added Context section, user personas for Veronica and Hamid, begun executive summary, use cases and scope. |
| Clare Provenzano | Nov. 29-30 2023 | Hamid is split into 2 personas, Hamid and Sara. Requirements are also added. |
This project will focus on creating custom ROS (Robot Operation System) applications for the Pepper robots in RosieLab. Our goal is to enhance Pepper’s capabilities to facilitate research by leveraging ROS’s robustness and customizability to create applications that researchers can use to test algorithms and unlock new experiments and possible research.
RosieLab at SFU is interested in researching robots with empathy. They want to build robots that interact naturally with humans and develop AI tools to help robots understand human behaviour.
They have multiple Pepper robots in their lab, but are currently limited with what they can use Pepper for. Developing new Android apps on Pepper has proven to be nearly impossible, so certain research activities aren’t possible on Pepper at this time.
Using ROS, we can unlock new capabilities on Pepper which will unlock new possible research and experiments.
Current research areas that are high prioirity to have Pepper uses include AI text to speech, embodied AI, teleoperation, and motion capture.
Angelica
Background:
Age: 30
Occupation: RosieLab Director, previously worked at Softbank as a team lead on Pepper’s Expressivity Team
Role: Supports students in their research, also does her own research
Fun facts: Did her undergrad at SFU, is also a mom!
Personality:
Passionate: Angelica is passionate about studying and improving human-robot interactions, and is excited about the work her students are doing in her lab.
Supportive: She wants her students to succeed and do their best possible work.
Approachable: Angelica is friendly, and students feel comfortable talking to her and approaching her.
Goals and Needs:
She wants her students to succeed, and have all the possible tools to do so.
She would like the Pepper robots to help students in their research more, and to be used more in general.
Veronica
Background:
Age: 20
Occupation: Undergraduate Computer Science student at SFU
Role: Currently doing research about human-robot interactions in addition to her undergrad homework.
Fun facts: Loves to knit, and is obsessed with butterflies. She knows all about their biology and behaviour, and is always wearing a necklace with a butterfly pendant.
Personality:
Curious: She is always eager to learn more, and is genuinely very curious about how humans perceive and interact with human-like robots.
Enthusiastic: She attends all the conferences and lectures she can, and enjoys talking to the grad students about their research.
Insecure: Veronica is younger and has less experience than the graduate students that comprise the majority of RosieLab, and is a little insecure about fitting in socially, and being seen as smart.
Goals and Needs:
Veronica wants to know if she should pursue a Master’s degree, and specifically if she should pursue it at SFU RosieLab.
She also wants to build up a network or researchers and peers in the robotics field.
She wants to do some research on her own or with a grad student alongside her undergrad courses for her resume and to decide if she wants to do a Master’s.
Hamid
Background:
Age: 27
Occupation: Masters student at SFU in RosieLab.
Role: Currently researching computer vision and motion capture
Fun facts: Hamid moved from Iran to Canada recently, and is trying to start up a side business as a wedding photographer. His undergrad was in computer engineering.
Personality:
Meticulous: Hamid pays close attention to his research, and has very accurate and reliable work.
Patient: Research in computer vision can be challenging and may involve extensive experimentation. Sara's patience allows her to persist in solving problems and refining her algorithms.
Romantic: As an up and coming wedding photographer, Hamid is a romantic at heart. He enjoys having meaningful conversations with those around him and often tries to be a matchmaker among his friends.
Goals and Needs:
Short term: To complete his master’s thesis with a working prototype or algorithm of his computer vision system.
Long term: He is still deciding between pursuing a PhD, or getting industry experience in robotics company to see if he enjoys working at a corporate company more than academia.
Challenges:
Time management: Balancing research, looking out for potential jobs, and taking care of himself is tricky,
Measuring outcomes: Hamid wants to test his computer vision algorithm on a robot people may actually interact with, not just using an emulator or online software.
Sara
Background:
Age: 25
Occupation: Masters student at SFU in RosieLab.
Role: Currently researching computer-robot interactions and is trying to make them more natural
Fun facts: Sara loves hiking and exploring. She even hiked up Mount Kilimanjaro, where she unfortunately injured herself leading to a permanent disability. She also has a pet bird named Cupcake who has begun to imitate Sara’s speaking and singing.
Personality:
Extroverted: He’s very social, striking up a conversation with anyone around him and connecting quickly. This is also why he’s so interested in making robots more social!
Empathetic: As an emotionally intelligent person, Hamid is able to pick up on small cues and gestures from others. He thinks if robots become able to do this, they will fit into society better and have more uses, such as in hospitals or nursing homes.
Eco-Conscious: She is environmentally conscious and promotes sustainable practices in her work and personal life.
Goals and Needs:
Short term: Sarah wants to publish her thesis and contribute new knowledge to her field. There are certain experiments in mind she wants to run, but the Pepper robots currently can’t support exactly what she wants to do.
Long term: Sara is really passionate about ethical AI and use of robotics, and she would like to promote these values in the future. She sees herself going to conferences, and being part of committees to help write policies regarding ethical AI and robotics.
Challenges:
Resource limitations: Sara finds it difficult to find a robot that is available with enough capabilities to have interesting and novel interactions with humans. The current capabilities of Pepper are quite limited, and the Pepper-human interactions that Sara is able to study right now are not complex enough to write a thesis about.
Finding participants: Sara is putting out calls for research’s participants on various email lists, but not many students are interested in participating in any experiments.
As a lab director, I want a tutorial on how to use the ROS Naoqi bridge so that my grad students are able to connect to Pepper.
As a graduate researcher, I want an embodied GPT in Pepper so I can research how people have conversations with a robot.
As a graduate researcher, I want to be able to teleoperate Pepper so that I can begin my new research study involving teleoperation.
As an undergraduate interested in research, I want a tutorial on how to use ROS so that I can begin new experiments and using Pepper.
We will begin by taking on 3 requests for custom ROS applications based on what is highest priority at RosieLab.
Teleoperations
Text-to-speech
Playing motion capture files on Pepper
Robust documentation must be provided.
Teleoperations:
User should be able to control Pepper using a remote device, like a game controller.
There should be clear documentation on how Pepper’s motions are mapped to the controller.
Pepper should be able to move around and turn.
A user should be able to use the controller to move Pepper so that someone interacting with Pepper does not realize a human is controlling Pepper.
TTS:
The TTS application will allow users to communicate with Pepper in a natural way, the user will be able to prompt Pepper in a conversational tone rather than a clear and direct order.
Pepper will respond out loud using TTS in a human-like way
A large language model will be used to formulate Pepper’s responses.
Program should be created and run using ROS.
The GPT should be running locally to avoid network issues.
Motion Capture:
Pepper will move the way the file specifies.
Pepper will run a bvh and/or smpl file.
If Pepper’s range of motion cannot perform the motion specified in the file, Pepper will run a similar motion.