ICP 6 - AsharGit/Python-ICP GitHub Wiki

Welcome to the Python-ICP-6 wiki!

This ICP we use KMeans and PCA for clustering data

Video for this ICP: https://youtu.be/Ca7iwYcC08U

The first task for this ICP is to use KMeans for clustering on our dataset

We start by reading the csv file and removing null values from our data by replacing them with mean values

Code

Left output showing two variables with null values

Right output showing no more null values in our data

Output

Next, we use the elbow method to find optimal number of clusters to use for our algorithm

Code below runs a loop to find the fit using WCSS and then plots the result

Code

Looking at the graph, we choose 3 clusters for our algorithm

Graph

Finally we are asked to provide a silhouette score for this algorithm

We build the KMean model by assigning 3 clusters we found from the elbow method and apply a fit

We can now use prediction to place data points into cluster

Code

We print out our silhouette score

Output

Second task asks us to use the same data and use scaling with KMeans and to compare our result

Scaling algorithm is applied which transforms our data

KMeans is applied the same as before to our scaled data and we print the silhouette score for the scaled data

Code

The resultant silhouette score is worse with the scaled data

Output

Final task of the ICP asks us to apply PCA and KMeans to our data

We accomplish this by first scaling our data same as before and applying the PCA fit to transform our scaled data

KMeans is once again applied and the resulting silhouette score is printed

Code

The silhouette score for the PCA and KMeans data is higher than the scaled data but still slightly worse than normal data with KMeans

Output

An additional bonus question asks us to visualize the previous cluster and it can be seen below

Graph