Project: Challenge Documentation Overview - tnl3pdx/ece510-HwAIML GitHub Wiki
Related Challenges
The following challenges related to this project are listed below. In each directory, a README addresses the goal of each challenge, and the logs from these challenges have been merged into this wiki:
- Challenge 9
- Challenge 12
- Challenge 15
- Challenge 18
- Challenge 21
- Challenge 25
Project Documentation
Challenge 9
In this challenge, I start searching for an algorithm/workload that I would like to accelerate using a hardware implementation. Through this challenge, I went through 2 additional iterations of project searching before I landed on my finalized project idea, which is to make an accelerator chip for the SHA-256 algorithm for SPHINCS+.
Iteration #1: RL-based PCB Layout Optimizer
In my first iteration, I found a repository for RL-based component layout optimization called RL PCB. When I first attempted to follow the README from the RL_PCB repo. There were several issues with the installation and setup of the repo.
- Since the repo is nearly 2 years old, a lot of version dependencies have changed. This caused a lot of issues with using the Git repository, and it needed to be cleaned up to get it working properly.
- The repo uses some binaries that are stored on the creator’s GitLab, which is locked. I can pull from it, but some of the binaries have some issues compiling, which was rooted in a missing library import for a uint_16t as a type.
- Also, due to the changes in the binaries, their names also changed. Thus, I had to scrape around the scripts to find any inconsistencies in file names to get the experiments to work.
After solving these issues, I tried to run the model for several hours to see if I could get a trained model that I could visualize, but I realized that training the agents for this particular application takes a lot of time. Looking at the paper provided by the repository, it can take 3-4 hours of training for a single run, and my kernel was killing the process due to resource starvation.
After talking with Prof. Teuscher about the problems I was facing, he recommended that I look for another workload that would not be too intensive to run, and pretrained agents may be used to reduce training time.
Iteration #2: RL Car Racing
After my first failed attempt in my first iteration, I still wanted to do a project related to some kind of ML workload using reinforcement learning. I knew that I had to scale down my project when I saw that RL-based car driving could be a good candidate. OpenAI has a powerful library of environments packaged in their library called Gymnasium.
To start, I asked my LLM about how I could set this up to see if this could be a good project to do:
Can you provide me with a Python program that initializes a gymnasium environment for car racing, trains the model with a parameterized timestep, and outputs the agent binary in the same directory?
The major issue with this workload is that training takes a long time for testing, and also trying to work with a library like gymnasium is difficult once I get to profiling. Thus, I moved to the next idea, which was my SHA-256 accelerator.