AI Self-hosting, Spring 2024

2024-04-10 - Week 02 - Morning

Back to AI Self-Hosting, Spring '24

Neural Network Review

We'll be discussing these notes from 3Blue1Brown and answering questions from there

• You can think of "neural network" as "graph of nodes in layers"

• Inspired by biological neurons

• Input edges are from "parents", or nodes in earlier layers, that influence us

• Output edges are to "children", or nodes in succeeding layers, that we influence

• Parameters:

  • weights are positive or negative multipliers, or correlation
  • whether a particular parent has positive or negative influence
  • bias is a threshold

Each node in our network is like

Google Colab

If you have a Google account, you can join this shared notebook to try evaluation math expressions for this class.

https://colab.research.google.com/drive/1E4MfdKbZ8r_fhRewM3Zv5cXLP3iAR0sW?usp=sharing

Questions to Think About in Small Groups

1. Let's look at these weights and inputs for a single neuron (a neural network of one)

• $w_1 = 2.07$
• $w_2 = -1.51$
• $w_3 = 0.43$
• $b = 10$
• $input1 = a_1 = 23$
• $input2 = a_2 = 105$
• $input3 = a_3 = 211$

Try guessing whether this neural node will "fire", that is, output a positive or negative value by evaluating this formula, and checking whether it is greater than 0, or less than 0.

$$ w_1 a_1 + w_2 a_2 + w_3 a_3 + b $$

What about the output after we pass it through a sigmoid function to normalize or "squish" the output for children of this node?

3. If you can access Google Colab, start a new Notebook and try typing the following

Preparation for Lab Tomorrow

In lab tomorrow, we'll continue to type and complete the network.py Python class and script for running MNIST training.

In particular, we'll need to finish writing our own data loader https://github.com/3b1b/videos/blob/master/_2017/nn/mnist_loader.py#L72-L80