Notes - UdayKiranJ/CS231n-Convolution-Neural-Networks-for-Visual-Recognition GitHub Wiki

Score Function

Maps raw input to class scores, for e.g. y = f(x) = Wx + b, where W is Weight Vector and b is Bias

Loss Function

Quantifies difference between predicted class scores and ground truth labels, for e.g. 0.5*(t-y)^2

Types of Score Functions

• Linear Classifier
  Computes the score of a class as weighted sum of raw inputs: y = W*x + b

• Logistic Unit
  Output is logistic of weighted sum of read valued inputs. Output is real valued and bounded between 0 and 1. Logistic output is interpreted as probability that an example belongs to a class.

• Softmax Classifier Output is multinomial logistic of weighted sum of real valued inputs. Output is real valued and bounded between 0 and 1. Sum of all outputs is 1. So, each node output is the probability of an example belonging to that class

Data Preprocessing

In Machine Learning, it is common practice to always normalize input data before model build/validation/test such that it'll have zero mean and standard deviation 1 for all input features.

Types of Cost/Loss Functions

The loss function quantifies our unhappiness with predictions on the training set

• Sum of Squared Errors
• Hinge Loss
• Cross Entropy
• MultiClass SVM Loss:
  SVM loss is set up so that SVM wants correct class for each image to have score higher than the incorrect classes by at least a fixed margin.
  Hyper Parameters:
  • Delta - Margin Parameter
  • Lambda - Regularization Parameter Hyper parameters (excluding Delta) are set during cross validation