Module 1: ICP #3 - VidyullathaKaza/BigData_Programming_Spring2020 GitHub Wiki

Problem Statement:

Create a Map-Reduce Program to perform the task of matrix multiplication.

Features:

Local Input and Output Files

The name of the matrix, the row number, the column number, its value is indicated. We are giving 2 different matrices.

The result matrix we are going to get is in the format shown above the row number, the column number, it's value.

Configurations / Steps followed:

  1. Opened IntelliJ in cloud era using command "./idea.sh" by moving into bin folder of the IntelliJ.
  2. Opened code given by instructor and edited with the my matrix code and performed configuration.

  1. Updated Pom.xml file with the existing java version in CloudEra.
  2. Configure .JAR/Artifacts by following below steps: a.) Go to to your Project Settings by doing: Ctrl+Shift+Alt+S and click on the Artifacts tab on the side. b.) Then click the Green + at the top, and be able to configure the project to build an artifact. c.) Then Build > Build Artifacts

Procedure

The input we have produced is done mapping with the logic shown below

The end result is gonna have the format as ordered pairs such as ((i,k),[(M,j,mij),(M,j,mij),......,(N,j,njk),(N,j,njk),.....]) The reduce phase is gonna take the output from the mapped phase and sorts the matrices and sums up the multiplication value from two pairs provided. And the product Matrix is generated.

Reduce with the logic shown below

  1. Created M and N matrix file, pushed data to file and executed the code.

My Learning Outcomes

From this exercise I got hands on in implementing mathematical functions in map reduce. BFS and DFS concepts were revisited and effort was put in to implement it is map reduce.