Week 10, Wednesday Morning

• Demo format on Thursday
• Try out class server with GPU for self-hosting AI (45 minutes)
  • Finetuning OpenAI GPT-2 weights with your dataset
  • Running inference on your entire dataset, pre-pended as a "system prompt" to your query
• Break
• Upload your trained models to HuggingFace
• Lab, make progress on your GPTs for demo on Thursday

• Send any questions about summer, next fall, and next year courses, computer science, or learning in general, for faculty to Winston winston dot shine at evergreen dot edu
  • say whether you'd like to be anonymous or not
  • faculty will not know in advance, feel free to ask hard questions that you think will benefit you and your classmates' learning
  • new faculty
    • Catherine Kehl
    • Jessica Carey
    • Omar Dawish
  • you can also ask questions in-person at the townhall
• Combined CSF and SOS, mixed between 2610 and 2619
• Thursday morning, 11am - 12noon setup and practice
• Two shifts
  • first shift stays at computers and presents from 1pm - 1:40pm, second shift walks around and visits
  • second shift 1:50pm - 2:40pm
• For each visitor, plan 4 minutes presentation, 1 minute Q & A

• List any public keys you already have that you don't mind re-using
  • cat ~/.ssh/id_*.pub
• Create a new (temporary) SSH keypair for this class if you wish
  • ssh-keygen -t ecdsa
  • Then try listing the new keypair using the cat command above
• Copy and paste your chosen public key to respond to Paul via Canvas message
• You'll receive back a message with your login name (and your default password, but you shouldn't need it)
• Test logging in
  • ssh <username>@indira.arcology.builders -p 9999
  • If you have more than one keypair, you can specify which one with the path to your private key, not the *.pub part of the keypair ssh <username>@indira.arcology.builders -p 9999 -i ~/.ssh/id_ecdsa
  • You'll see a login message for the host anti-villain

• Check your home directory

  • pwd

• Change into upper-division-cs repo that has been cloned for you

  • Run the command cd src/upper-division-cs
  • git pull to make sure you are up-to-date on main

• Check that you are a member of the group ai-24sp

  • Run the command id

• Try activating the Python virtual environment

  • Run the command . /opt/virtualenvs/ai-24sp/bin/activate (you can tab to autocomplete)
  • Run which pip3 and which python3

Run the command

nvidia-smi -a

Scroll up in your terminal to examine the output.

In particular, pay attention to the power usage.

Temperature
        GPU Current Temp                  : 35 C
        GPU T.Limit Temp                  : N/A
        GPU Shutdown Temp                 : 98 C
        GPU Slowdown Temp                 : 95 C
        GPU Max Operating Temp            : 93 C
        GPU Target Temperature            : 83 C
        Memory Current Temp               : N/A
        Memory Max Operating Temp         : N/A
    GPU Power Readings
        Power Draw                        : 82.11 W
        Current Power Limit               : 320.00 W
        Requested Power Limit             : 320.00 W
        Default Power Limit               : 320.00 W
        Min Power Limit                   : 100.00 W
        Max Power Limit                   : 320.00 W

• Run tmux and open up two panes.
  • C-b %
  • C-b <left-arrow> to move focus to the left pane
  • C-b <right-arrow> to move focus to the right pane

In one pane

• Go to your assignments directory and run data.sh to download your dataset.

• Go to your week9 directory and run your gpt_train.py code

python3 gpt_train.py train

• Run nvidia-smi -a in another pane. How much does power consumption go up? Memory?

Join our HuggingFace organization

Upload your model.pth to share it between computers, or exchange with your classmates.

Create a directory for week10 work in your assignments directory and change into it

cd <repo_dir>/ai-24sp/assignments/<username>
mkdir week10

Follow the code from the Raschka book, Chapter 5, Section 5.5

"Loading pretrained weights from OpenAI"

https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/01_main-chapter-code/ch05.ipynb

• Run tmux and open up two panes.
  • C-b %
  • C-b <left-arrow> to move focus to the left pane
  • C-b <right-arrow> to move focus to the right pane
• Try finetuning GPT-2 with your chosen dataset, in one pane.
• Run nvidia-smi -a in another pane. How much does power consumption go up? Memory?
• Stretch goal: pass in the entire text of your chosen dataset as a system prompt, and make a statement to autocomplete about it.
  • Example: "The main point of this book is "

Try running these with your account on the shared class server, or your own laptop if you have a GPU / AI hardware acceleration.