SparkScalaInterpreterTopology: A Scala prototyping engine for Spark - IKANOW/Aleph2-examples GitHub Wiki

TODO

Example working code for MR scenario

 val cnt = inputs.rdd
                      .map({ case (o, (l, record)) => (l, record)})
                      .mapPartitions(mapper_enricher.inMapPartitionsPreGroup("key"))
                      .groupByKey() 
                      .map((r) => (r._1, r._2.asJava)) 
             .mapPartitions(reducer_enricher.inMapPartitionsPostGroup())
                      .count