2. 서버 인프라 구축 - map-search/map-search-etl GitHub Wiki

설치 목록

  1. jvm
  2. scala
  3. hadoop
  4. spark

환경변수 설정 내용

export JAVA_HOME=/usr/local/lib/java
export PATH=$PATH:$JAVA_HOME/bin/
export CLASSPATH="."

export SCALA_HOME=/usr/local/lib/src/scala
export PATH=$SCALA_HOME/bin:$PATH

export SPARK_HOME=/usr/local/lib/spark
export PATH=$SPARK_HOME/bin:%PATH

export HADOOP_HOME=/usr/local/lib/hadoop
export HADOOP_CONFIG_HOME=$HADOOP_HOME/etc/hadoop
export HADOOP_OPTS="$HADOOP_OPTS -
Djava.library.path=$HADOOP_HOME/lib/native"
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export PATH=$PATH:/usr/bin:$JAVA_HOME/bin:$HADOOP_HOME/bin

export HADOOP_PID_DIR=$HADOOP_HOME/pids
export HADOOP_CLASSPATH=$JAVA_HOME/lib/tools.jar
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_LIBRARY_PATH

export YARN_HOME=$HADOOP_HOME
export PATH=$PATH:$YARN_HOME

참고 링크

  1. 하둡설치 : https://blog.naver.com/gladiator67/197624564
  2. spark 설치 : http://pubdata.tistory.com/17
  3. 하둡 standalone : https://m.blog.naver.com/PostView.nhn?blogId=alice_k106&logNo=220436293186&proxyReferer=https%3A%2F%2Fwww.google.co.kr%2F
  4. ssh 설정 : http://avilos.codes/system/linux/centos-ssh-installation/