Appendix Linear Classification - AsyDynamics/CS231n GitHub Wiki

This is the notes of study material of linear classification for CS231n.
Source: http://cs231n.github.io/linear-classify/

Previous K-Nearest Neighbor, KPN, has disadvantages that the classifier should store all the training data and the classifying process is expensive.

A new approach is developed, composed of

  • score function - maps the raw data to class scores
  • loss function - quantifies the agreement between the predicted scores and the ground truth labels
  1. Mapping from image to label
  • f(x,w,b)=w*x+b
  • N example, D pixel, K class; w[K*D], b[k*1]
  1. Interpreting a linear classifier
  • Analogy of images as high-dimensional points
  • Interpretation of linear classification as template matching - each row of w corresponds to a template for one class; use negative inner product instead of L1 and L2 distance
  • Bias trick - combine the weights and bias into a single matrix
  • Image data processing - normalize the input or center the data by subtracting the mean from every feature

  1. Loss function

  2. Multiclass support vector machine loss (SVM) SVM wants to correct class for each image to have a score higher than the incorrect class be fixed margin， notation /delta. SVM wants the outcome to be lower.