Previous: PCL Feature Representation

---

Once a feature set has been defined that can differentiate between the classes, the last step of the process is to create and train a classifier that can classify clusters based on those features. Much of the body of machine learning work done on lidar data has been done using pattern analysis algorithms, of which a well-known example is a Support Vector Machine. Support Vector Machines attempt to differentiate between classes of data by generating hyperplanes that separate clusters of the data in the n-dimensional feature-space. The form these hyperplanes take is dependent on the kernel, and can be linear, radial, polynomial, or more exotic.

The team's dataset exhibited significant non-linearities around certain pairs of parameters. Linear regression performed quite poorly in this case, averaging a correct guess on 30-40% of the test dataset, as compared to the 25% correct guess rate of random chance.

In order to account for the non-linearity, the team applied a SVM with a radial-based function kernel. This kernel allows for non-linear fitting to the data, at the cost of a risk of overfitting if the model parameters are tuned incorrectly.

In order to better understand the relationship between features, the team did some explorations of subsets of the data, sub-selecting features and classes to compare.

Experiment 1

The first experiment created was a basic linear regression model trained on density and volume, with a subselected dataset of cars and pedestrians. The resulting model and confusion matrix is shown in Figure 1. This experiment was run to show that cars and people are very different in terms of density and volume - cars are big and sparse, while people are small and dense.
Classes: Cars, Pedestrians
Features: Density, Volume
Accuracy: 96.5%

[191212_ml_linear-density-vol_1-2_graph.png]] ](/AmyPhung/ml_person_detection/wiki/[[191212_ml_linear-density-vol_1-2_confusion.png) :-------------------------------:|:--------------------------------: Figure 1a: Linear SVM predicting car vs person | Figure 1b: Covariance matrix of Fig. 1a SVM

As shown by the figure above, the proposed relationship across density and volume between cars and pedestrians holds out for the most part, although there are some pedestrians with surprisingly high volume. This may be due to the pedestrians walking on the sidewalk, which in some frames was tall enough not to get selected by the ground plane cutoff. As such, the sidewalk beneath the pedestrian would also be selected, increasing the overall volume.

A similar pattern is true between signs and vehicles - signs are much smaller and denser than vehicles, and using only these features to predict the difference provides a reasonable accuracy.
Classes: Cars, Signs
Features: Density, Volume
Accuracy: 84.5%

[191212_ml_linear-density-vol_1-3_graph.png]] ](/AmyPhung/ml_person_detection/wiki/[[191212_ml_linear-density-vol_1-3_confusion.png) :----------------------------------------------:|:--------------------------------------------------: Figure 2a: Linear SVM predicting car vs sign | Figure 2b: Covariance matrix of Fig. 2a SVM

The team also compared signs to pedestrians with the same features. As might be expected from the above two figures, these two classes do not differ along these features by much at all, and the model performed exceedingly poorly in classifying this distinction. In terms of volume and density, people and signs look fairly similar, as the volume is calculated as ground area multiplied by height. This metric is ambiguous, so the model mostly guesses pedestrian.

Classes: Pedestrians, Signs
Features: Density, Volume
Accuracy: 64%

[191212_ml_linear-density-vol_2-3_graph.png]] ](/AmyPhung/ml_person_detection/wiki/[[191212_ml_linear-density-vol_2-3_confusion.png) :----------------------------------------------:|:--------------------------------------------------: Figure 3a: Linear SVM predicting pedestrian vs sign | Figure 3b: Covariance matrix of Fig. 3a SVM

To wrap up the experiment, the team looked at using the basic SVM with all four classes, to determine an overall accuracy. As expected, the model performed poorly. Classes 1 and 4 are more clustered together, so they are classified correctly. Since classes 2 and 3 are more spread out and overlap with points from 1 and 4, they are more ambiguous and their accuracy is lower. Classes: Pedestrians, Signs, Cars, Bicyclists
Features: Density, Volume
Accuracy: 59%

[191212_ml_linear-density-vol_1-2-3-4_graph.png]] ](/AmyPhung/ml_person_detection/wiki/[[191212_ml_linear-density-vol_1-2-3-4_confusion.png) :----------------------------------------------:|:--------------------------------------------------: Figure 4a: Linear SVM predicting all classes | Figure 4b: Covariance matrix of Fig. 4a SVM

---