Machine Learning Tutorial

Support Vector Machines (SVM)

SVM is one of the most popular algorithm in linear classification.

Here, there is a parameter C which controls the regularization. A higher C means less regularization. In other words, when you use a high value for the parameter C, SVM tries to fit the training set as best as possible, while with low values of the parameter C, the model puts more emphasis on finding a coefficient vector (w) that is close to zero. Using low values of C also will cause the algorithms to try to adjust to the “majority” of data points, while using a higher value of C stresses the importance that each individual data point be classified correctly.

Similarly to the case of regression, linear models for classification might seem very restrictive in low-dimensional spaces, only allowing for decision boundaries that are straight lines or planes. Again, in high dimensions, linear models for classification become very powerful, and guarding against overfitting becomes increasingly important when considering more features.

from sklearn.svm import LinearSVC

svm = LinearSVC(C=100).fit(X_train, y_train)