Lab 2 - Apoorvag2597/BDP_Revised GitHub Wiki

Name - Apoorva Geetanjali Avadhanula

Class ID - 34

Part 1

To Implement MapReduce Algorithm for finding Facebook common friends problem and implement the MapReduce task on Apache Spark.

Code-

Input-

Output

Part 2

To create a spark dataset using one of dataset and to use all different StructType Code

To perform any 10 questions on dataset, implement any 5 queries in Spark RDD's and Spark Dataframes

Dataset

Queries

RDD

Output -

8

Part 3 Spark Streaming - Perform Word-Count on Twitter Streaming Data using Spark

Output