ICP8 Apache Spark - gabriellawillis/BigData GitHub Wiki

Source code can be found here: SourceCode

Spark Programming - Word Count

• Write a spark program using text data as the input to count number of words

IntelliJ Word Count Code Snippet:

Word Count Output:

Secondary Sorting in Map Reduce

• Sort the values in the reducer phase

IntelliJ Sorting Code Snippet:

Sorting Output:

References:

• Lecture: https://umkc.app.box.com/s/i7u5y88fk02460r8rnsf4cuiox0o6q9o

• Lesson Plan: https://umkc.app.box.com/s/h7o4epqq1vd76mbfyswo4qcjzkg2d9rb

• http://allaboutscala.com/tutorials/chapter-1-getting-familiar-intellij-ide/scala-tutorial-first-hello-world-application/

• https://www.whizlabs.com/blog/apache-spark-limitations/

• http://spark.apache.org/downloads.html