AI Self-Hosting, Spring 2024

Syllabus

Schedule

• Wednesday morning, 10am-12noon
  • SEM II C 1105
• Thursday afternoon, 1-3pm
  • LIB 2619

Program Description

AI Self-Hosting is a 4 credit thread of the Student-Originated Software program in upper division computer science. It presents the building blocks of artificial intelligence focusing on the mathematical and software engineering techniques of neural networks as they are used in deep learning and generative pre-trained transformers. AI is also studied as an engineering discipline which can be hosted on machines owned and operated by its community of users, with attendant practical and governance concerns.

The main class text is "Building Large Language Models from Scratch" by Sebastian Raschka. It is supplemented by the 3blue1brown online lessons by Grant Sanderson; "Neural Networks and Deep Learning" by Mike Nielsen; and "Deep Learning" by Ian Goodfellow and Yoshua Bengio and Aaron Courville.

Topics

• Axes and indexing into tensors of first, second, third, and higher ranks.
• Linear algebra concepts needed to represent and operate on machine learning features:
  • vectors, vector spaces, dot product, dimensionality, and matrix multiplication.
• Supervised learning with training, validation, and test datasets and input, target pairs
  • using the MNIST handwritten digit classification lab
• Models as probabilistic compiled files from input dataset sources
  • model weights and files, saving and loading, distributing models, pre-training and finetuning
• The operation of AI models from (pre-)training, finetuning, multiple-shot examples, and inference.
• The history of artificial intelligence technology and research.
• Introduction to multi-modal learning with text-to-speech (TTS) synthesis

Students are evaluated based on 4 afternoon labs, 2 morning exercises, and 5 weekly homeworks asking them to write an original essay prompted by article readings on AI ethics and social impact, submitting written answers based on readings from the class texts in dev diaries, coding two projects (MNIST classifier and GPT autocomplete) in Python submitted with the git version control system, and presenting at a final project fair.