ICP 3 - a190884810/Big-Data-Programming GitHub Wiki

Lesson 3 Plan, Map Reduce 2nd

Fulfilling matrix multiplication by using map reduce technology.

  • Let's skimming the input data first, here is the input and output matrix M,N which size is 999 rows and 100 cols. The first col of the data is the name of the matrix which is M in this case. The second col of the data represents the current row in the matrix, and the third col of the data represents the current col in the matrix, and the last col data represent the val of the <row,col> in the matrix.

The Mapping stage for the program

  • This code snippet contains a mapping program. This program will scan the input data, and generate a pair with such a format <<i,k><M/N,j,val>>, which can match the oncoming computation pair together.

The reducing stage for the program

  • Mainly reduce those pairs which generated by the mapping reduce. sort them by the row, push them into two separate hash_map and then finally compute the product of each pair to sum them up get the value of a single cell.

Result for the multiplication

  • Output all value of the multiplication by using nearly same format as the input data.
⚠️ **GitHub.com Fallback** ⚠️