ICP8 - bhargavi1411/BigDataProgramming GitHub Wiki

Name : Bhargavi Saipoojitha Chennupati

Class id : 4

Objective :

  1. To Write a spark program with an interesting use case using text data as the input and program should have at least Two Spark Transformations and Two Spark Actions.

2.Secondary Sorting in Map Reduce.

Prerequisites:

  • Install Intellij.

  • Install scala tools while installing Intellij.

  • Spark shell needs to be installed

Task 1 :

Spark Programming

In this program we have to take an input text file apply word count on that file and place the output in an output directory.

Inverted Index :

  1. Secondary Sorting:

Code Screenshot :