Back to Main Project Page

Related Challenges

The following challenges related to this project are listed below. In each directory, a README addresses the goal of each challenge, and the logs from these challenges have been merged into this wiki:

1. Challenge 9
2. Challenge 12
3. Challenge 15
4. Challenge 18
5. Challenge 21
6. Challenge 25

Project Documentation

Challenge 9

In this challenge, I start searching for an algorithm/workload that I would like to accelerate using a hardware implementation. Through this challenge, I went through 2 additional iterations of project searching before I landed on my finalized project idea, which is to make an accelerator chip for the SHA-256 algorithm for SPHINCS+.

Iteration #1: RL-based PCB Layout Optimizer

In my first iteration, I found a repository for RL-based component layout optimization called RL PCB. When I first attempted to follow the README from the RL_PCB repo. There were several issues with the installation and setup of the repo.

• Since the repo is nearly 2 years old, a lot of version dependencies have changed. This caused a lot of issues with using the Git repository, and it needed to be cleaned up to get it working properly.
• The repo uses some binaries that are stored on the creator’s GitLab, which is locked. I can pull from it, but some of the binaries have some issues compiling, which was rooted in a missing library import for a uint_16t as a type.
• Also, due to the changes in the binaries, their names also changed. Thus, I had to scrape around the scripts to find any inconsistencies in file names to get the experiments to work.

After solving these issues, I tried to run the model for several hours to see if I could get a trained model that I could visualize, but I realized that training the agents for this particular application takes a lot of time. Looking at the paper provided by the repository, it can take 3-4 hours of training for a single run, and my kernel was killing the process due to resource starvation.

After talking with Prof. Teuscher about the problems I was facing, he recommended that I look for another workload that would not be too intensive to run, and pretrained agents may be used to reduce training time.

Iteration #2: RL Car Racing

After my first failed attempt in my first iteration, I still wanted to do a project related to some kind of ML workload using reinforcement learning. I knew that I had to scale down my project when I saw that RL-based car driving could be a good candidate. OpenAI has a powerful library of environments packaged in their library called Gymnasium.

To start, I asked my LLM about how I could set this up to see if this could be a good project to do:

Can you provide me with a Python program that initializes a gymnasium environment for car racing, trains the model with a parameterized timestep, and outputs the agent binary in the same directory?

The major issue with this workload is that training takes a long time for testing, and also trying to work with a library like gymnasium is difficult once I get to profiling. Thus, I moved to the next idea, which was my SHA-256 accelerator.