1。編輯spark-env.sh

export HADOOP_HOME=/usr/hdp/current/hadoop-client
export HADOOP_CONF_DIR=/usr/hdp/current/hadoop-client/conf
這兩個其實就是為了獲得hadoop中的hdfs和yarn的配置檔案。

2。編輯spark-defaults.conf

spark.history.fs.logDirectory hdfs:///spark2-history/
spark.history.provider org.apache.spark.deploy.history.FsHistoryProvider
spark.history.ui.port 18081
spark.eventLog.enabled true
spark.eventLog.dir hdfs:///spark2-history/

3。在hdfs上建立檔案

hadoop fs -mkdir /spark2-history

4。啟動服務

//啟動job歷史埠
./start-history-server.sh