Machine Learning Tutorial

Create nonlinear models from linear models

We are going to use a example to illustrate the process. Suppose the following dataset:

A linear model for classification would separate the classes like this:

Which clearly not perform well.

The trick is, as this dataset is in 2 dimensions, we can create a new dimension, completely dependent of the ones before so a linear model in 3D would be able to separate better both classes. In this example, we are going to change the dataset from (feature0, feature1) to (feature0, feature1, feature1 ** 2). (OBS: A question that arises and is better addressed by the Kernelized Support Vector Machines is how many new features we should add and how should their functions be like in order to increase performance?).

Now, a linear model is able to do a better job:

But the point is that when looking back to the original 2 dimensional dataset and its decision function, it gets like this: