Machine Learning Tutorial

Multiclass Classification

Many linear classification models are for binary classification only, and don’t extend naturally to the multiclass case (with the exception of logistic regression). A common technique to extend a binary classification algorithm to a multiclass classification algorithm is the one-vs.-rest approach. In the one-vs.-rest approach, a binary model is learned for each class that tries to separate that class from all of the other classes, resulting in as many binary models as there are classes. To make a prediction, all binary classifiers are run on a test point. The classifier that has the highest score on its single class “wins,” and this class label is returned as the prediction.

Example

from sklearn.datasets import make_blobs

X, y = make_blobs(random_state=42) # The default is to create 3 classes in y

linear_svm = LinearSVC().fit(X, y)

# X.shape: (100, 2)
# y.shape: (100,)

# Each row contains the coefficient vector for one of the classes
# Coefficient shape:  (3, 2)
# Intercept shape:  (3,)