spark-2.3.2-bin-hadoop2.6執行在yarn client上
阿新 • • 發佈：2018-12-15
- 修改配置檔案/hadoop_opt/spark-2.3.2-bin-hadoop2.6/conf/spark-env.sh
export HADOOP_CONF_DIR=/hadoop_opt/hadoop-2.6.1/etc/hadoop
export JAVA_HOME=/hadoop_opt/jdk1.8.0_181
export HADOOP_HOME=/hadoop_opt/hadoop-2.6.1
export SCALA_HOME=/hadoop_opt/scala-2.12.7
export SPARK_MASTER_IP=192.168.197.128
修改/hadoop_opt/spark-2.3.2-bin-hadoop2.6/conf/slaves，在最後加入如下程式碼
master
slave1
slave2
2.slave執行和master相同的操作,登陸master執行向slave分發操作
scp -r /hadoop_opt/spark [email protected] /hadoop_opt
scp -r /hadoop_opt/spark [email protected] /hadoop_opt
3.master 終端輸入
spark-shell --master yarn-client即可，最後出現如下截圖
