Bin Picking using Baxter + Dexnet

• Dexnet is a recent state-of-the-art bin-picking algorithm.
• It employs CNNs and depth images to learn to perform dexterous bin picking of objects.
• Dexnet uses ROS to implement the system on a real robot.

While this code uses deep learning which is not part of a course, one can replicate their work using pre-trained models on the yumi simulations to deliver a simulated solutions that involves setting up the system and running it.

Or use their code on a real-robot to redo training. While this requires some knowledge of deep networks, you do not have to design the system, but rather adapt to Baxter and re-run it.

• The more interesting part of the project would be to use the already defined code/datasets/structures and re-train on the baxter robot.

Dexnet has had 5 periods of evolution with each period described in the documentation (link below). For this project, any level of implementation starting from 2.0 or above is welcome. Notice that the GQ-CNNS that were originally programed in Dexnet 2.0 used python 2.7. But since, the package has been updated to support:

• Facilitate training/deployment using Dexnet 2.0, 2.1, 3.0
• Integrate recent research
• Update python API. and are more efficient.

Quicklinks

Dexnet homepage

• https://github.com/BerkeleyAutomation/dex-net
• https://berkeleyautomation.github.io/gqcnn/install/install.html
• https://bair.berkeley.edu/blog/2017/06/27/dexnet-2.0/

Task Deliverable:

• The task deliverable here would be to perform a one arm bin-picking (if you develop the two arm solution that would be welcome as well) task where the robot tries to pick up as many objects as possible from a bin.
• I do not expect similar performance output as in the original paper but if Baxter is able to grasp objects better than chance using this specific approach, you can receive full credit.

Equipment:

• We are currently setting up the environment and expect to have a working Baxter, with dual grippers, bins, and objects ready to go for the project.
• You can access the Baxter robot and associated equipment in Academic Building 1st floor, 102 after some initial training

Lab Usage:

If more than one team is interested in using the real-robot, we will create a schedule to share access to the real robot platform.

Demonstration

For teams that manage to succeed in demonstrating the task, I would like to invite you to present your work during a real-live demonstration where attendees/friends/family can come to see the demonstration (as much as physical space allows).

Report

• Everyone needs to submit a .zip file named 2021Robotics-FinalProject.zip to Gradescope by Monday, May 24, 2021. Note for Team Submissions:
• If you are part of a team, only one person should submit a full zip file, but other team members please submit a zip file that only includes a text file indicating the

1. students in the team
2. the name of the student who submitted the complete zip file on behalf of the team.

Your zip file should include:

1. Video of your simulation (independently of placing your video on youtube)
2. PDF report of your project

Project Day

• Tuesday, May 25th (can be adjusted If necessary)
• Academic Building 1st floor, 104.
• 4:00pm (can be adjusted according to participation).