Spark SQL - animeshtrivedi/notes GitHub Wiki

About

This page contains atr's notes about Spark SQL performance investigation. Compiling spark

set -e 
MAVEN_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1 -Xmx8g -XX:ReservedCodeCacheSize=2g"
branch=$(git branch | sed -n 's/^\* //p')
echo "Building the branch of : $branch" 
time mvn -T 2C -Phive -Pyarn -Phadoop-2.7 -Dhadoop.version=2.7.3 -DskipTests package
#./build/mvn -pl :spark-streaming_2.11 clean install - specific subsystem