Lecture 8 - AsyDynamics/CS231n GitHub Wiki
Review of lecture 7
- optimization: SGD+momentum, Nesterov, RMSProp, Adam
- Regularization: Dropout
- other recall: weight, loss
Compare CPU and GPU
balabala
?
- Numpy, problem: can't run on GPU; have to compute own gradients
- Tensorflow, could call function to calculate gradient; could determine run on gpu or cpu
- pytorch
Tensorflow
- define computational graph
- create placeholder
- forward pass, compute prediction and loss, i.e., L2 distance between y and y_predicted
- calculate gradients
- after building graph, enter a session to run the graph
- create numpy arrays to fill the place holders above
- run the graph, get loss
- train the network
- change weights from place holder to variables
- assign and update weighs using learning rate*gradients
- use optimizer, predefined loss, initializer
Tensorboard
Keras
- define model object as a sequence of layers
- define optimizer
- build the model, specify the loss function
- train the model