Spark Internals - pykello/pykello.github.com GitHub Wiki

Attaching debugger

./spark-shell --conf spark.driver.extraJavaOptions=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005

Physical Plan

• ColumnarToRowExec
• BroadcastExchangeExec

Optimization Techniques

• Dynamic Partition Pruning
  • Talk