ICP_13 - PallaviArikatla/Big-Data-Programming GitHub Wiki

INTRODUCTION: analysis on Graph Frames and GraphXAlgorithms in Spark.

IMPLEMENTATION:

Question 1:

Import the dataset as a csv file and create data frames directly on import than create graph out of the data frame created.

• Make initial setup.
• Install all the necessary libraries.

• Import CSV files.

• Stations data frame created.

• Then create vertices and edges from the imported data frames to create the graph.

• Create edges from Trips data frame.

• Create graph frame using vertices and edges created.

Question 2: Triangle Count.

• Triangle count on the created graph.

Question 3:

Find Shortest Paths w.r.t. Landmarks.

• Shortest path will be calculated and gets created on the graph.

Question 4: Apply Page Rank algorithm on the dataset.

• Apply Page rank algorithm, it identifies important vertices on the graph based on connections.

Question 5: Save graphs to a file.

Bonus Question:

Question 1: Apply Label Propagation Algorithm.

Question 2: BFS Algorithm.