进入spark shell提示不能加载native hadoop - lg1011/SparkLearn GitHub Wiki
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
解决方法如下:
方法一:在Linux环境变量里设置Linux共享库
vim /etc/profile
增加如下内容:
export LD_LIBRARY_PATH=$HADOOP_HOME/lib/native/:$LD_LIBRARY_PATH
source /etc/profile后重新进入就没有这样的提示了
方法二:设置环境变量和spark-env.sh
环境变量
vi /etc/profile
export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native/
conf/spark-env.sh
vi conf/spark-env.sh
export LD_LIBRARY_PATH=$JAVA_LIBRARY_PATH