Lyrics Clustering Experiment - AumitLeon/million-songs GitHub Wiki

In order to cluster songs by their lyrics, we used the musiXmatch dataset —official lyrics collection of the Million Song Dataset (https://labrosa.ee.columbia.edu/millionsong/musixmatch).

Description of MXM Dataset

  • Provides lyrics for many tracks of the Million Songs Dataset
  • The lyrics come in bag-of-words format
  • Each track is described as the word-counts for a dictionary of the top 5,000 words across the set
  • There are 210,519 training bag-of-words and 27,143 testing ones. However, given that we are not predicting anything for clustering we combined these two sets

Converting the data to a usable format

They MXM Dataset provides 2 txt files formatted as follow (per line):

  • '# - comment, ignore
  • %word1,word2,... - list of top words, in popularity order
  • TID,MXMID,idx:cnt,idx:cnt,... - track ID from MSD, track ID from musiXmatch,
  • word index : word count (word index starts at 1!)

To deal with this data, we created a function read_txt (found in lyrics-experiment.py) that takes a filename as parameter and returns a tuple with:

  1. list of 5,000 words,
  2. list with the word index: count,
  3. list with track ID from MSD, and
  4. list with track ID from musiXmatch

Afterwards, the function create_vectors takes two parameters -- a list with the 'word index: count' and the number of words (in this case, 5,000). It returns a list of numpy arrays where:

  • each numpy array contains 5,000 entries (every index represents index of a word)
  • the entries were: 0 if the word did not appear in the track, and its frequency otherwise. However, this resulted in a bigger cost J. Therefore, we used 1 for present words and 0 for absent words.

The returned list was later used to compute the clusters.

Experiments and Results

Using HW5 KMeans program

1. K = 1 to 30. Using 1,000 trackss

Using scikit-learn KMeans

  1. K = 1 to 30. Using all 237,662 trackss

List of Costs

  • K = 1. J = 65.71692241
  • K = 2. J = 62.5335295123
  • K = 3. J = 61.3349518861
  • K = 4. J = 60.6993668968
  • K = 5. J = 60.2466308327
  • K = 6. J = 59.8453981631
  • K = 7. J = 59.6566719753
  • K = 8. J = 59.4478679349
  • K = 9. J = 59.4097613168
  • K = 10. J = 58.9807209343
  • K = 11. J = 58.9704330173
  • K = 12. J = 58.8117860097
  • K = 13. J = 58.6006389571
  • K = 14. J = 58.6592381745
  • K = 15. J = 58.285167777
  • K = 16. J = 58.6456692443
  • K = 17. J = 58.531918413
  • K = 18. J = 58.2487263237
  • K = 19. J = 58.0855498083
  • K = 20. J = 58.1151177936
  • K = 21. J = 57.998003996
  • K = 22. J = 57.8869643512
  • K = 23. J = 57.9908727148
  • K = 24. J = 57.9498609159
  • K = 25. J = 57.9868073971
  • K = 26. J = 57.8420365321
  • K = 27. J = 57.7488046343
  • K = 28. J = 57.7168243756
  • K = 29. J = 57.7724285232
  • K = 30. J = 57.7025893269

Using scikit-learn MiniBatchKMeans

1. K = 1 to 30. Using all 237,662 trackss

2. K = 10

J = 60.4381434773

Cluster 0: 1

Cluster 1: 155786

Cluster 2: 36060

Cluster 3: 1

Cluster 4: 1

Cluster 5: 9

Cluster 6: 45801

Cluster 7: 1

Cluster 8: 1

Cluster 9: 1

As shown above, most of the tracks were assigned to just 3 clusters out of the 10. This would be a interesting fact to investigate for future work.

Choosing value of K

By applying the “elbow method” to the above graph for J vs. K=1-30, we can see how from K=1 the cost J goes down rapidly, and after it reaches K=3, the cost starts going down slowly.

Furthermore, the results obtained when clustering with K=10 also suggest that K=3 would be a reasonable value for the number of clusters.

Year distribution in each cluster with K=3

Cluster 0

Cluster 1

Cluster 2