安裝spark-1.5.0-cdh5.5.2所踩過的坑

阿新 • • 發佈：2019-01-06

我一開始想安裝spark-1.5.0-cdh5.5.2的standalone模式，於是乎（已安裝有hadoop叢集）：

[[email protected] ~]$ tar -zxvf spark-1.5.0-cdh5.5.2.tar.gz

[[email protected] ~]$ vi spark-1.5.0-cdh5.5.2/conf/spark-env.sh

export JAVA_HOME=/usr/jdk1.7.0_25
export SPARK_MASTER_IP=h71
export SPARK_MASTER_PORT=7077  
export SPARK_WORKER_CORES=1  
export SPARK_EXECUTOR_INSTANCES=1  
export SPARK_WORKER_MEMORY=1g

[[email protected] ~]$ vi spark-1.5.0-cdh5.5.2/conf/slaves

h72
h73

[[email protected] ~]$ scp -r spark-1.5.0-cdh5.5.2/ h72:/home/hadoop/

[[email protected] ~]$ scp -r spark-1.5.0-cdh5.5.2/ h73:/home/hadoop/

然後啟動spark叢集卻報錯：

[[email protected] spark-1.5.0-cdh5.5.2]$ sbin/start-all.sh

starting org.apache.spark.deploy.master.Master, logging to /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-h71.out
failed to launch org.apache.spark.deploy.master.Master:
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 6 more
full log in /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-h71.out
h72: starting org.apache.spark.deploy.worker.Worker, logging to /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-h72.out
h73: starting org.apache.spark.deploy.worker.Worker, logging to /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-h73.out
h72: failed to launch org.apache.spark.deploy.worker.Worker:
h72:    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
h72:    ... 6 more
h72: full log in /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-h72.out
h73: failed to launch org.apache.spark.deploy.worker.Worker:
h73:    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
h73:    ... 6 more
h73: full log in /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-h73.out

檢視日誌：

[[email protected] spark-1.5.0-cdh5.5.2]$ cat logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-h71.out

Spark Command: /usr/jdk1.7.0_25/bin/java -cp /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../conf/:/home/hadoop/spark-1.5.0-cdh5.5.2/lib/spark-assembly-1.5.0-cdh5.5.2-hadoop2.6.0-cdh5.5.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/etc/hadoop/ -Xms1g -Xmx1g -XX:MaxPermSize=256m org.apache.spark.deploy.master.Master --ip h71 --port 7077 --webui-port 8080
========================================
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/Logger
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2521)
        at java.lang.Class.getMethod0(Class.java:2764)
        at java.lang.Class.getMethod(Class.java:1653)
        at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 6 more

原因：無法載入slf4j相關jar包

解決：在spark-env.sh中新增

export HADOOP_HOME=/home/hadoop/hadoop-2.6.0-cdh5.5.2

for f in $HADOOP_HOME/share/hadoop/common/lib/*.jar; do
  if [ "$SPARK_CLASSPATH" ]; then
   export SPARK_CLASSPATH=$SPARK_CLASSPATH:$f
  else
   export SPARK_CLASSPATH=$f
  fi
done
 
for f in $HADOOP_HOME/share/hadoop/common/*.jar; do
  if [ "$SPARK_CLASSPATH" ]; then
   export SPARK_CLASSPATH=$SPARK_CLASSPATH:$f
  else
   export SPARK_CLASSPATH=$f
  fi
done

並複製到其他兩個從節點：

[[email protected] spark-1.5.0-cdh5.5.2]$ scp conf/spark-env.sh h72:/home/hadoop/spark-1.5.0-cdh5.5.2/conf/

[[email protected] spark-1.5.0-cdh5.5.2]$ scp conf/spark-env.sh h73:/home/hadoop/spark-1.5.0-cdh5.5.2/conf/

再啟動spark叢集：

[[email protected] spark-1.5.0-cdh5.5.2]$ sbin/start-all.sh

starting org.apache.spark.deploy.master.Master, logging to /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-h71.out
h73: starting org.apache.spark.deploy.worker.Worker, logging to /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-h73.out
h72: starting org.apache.spark.deploy.worker.Worker, logging to /home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-h72.out

用jps命令檢視你會在主節點上看到Master程序在從節點上看到Worker程序（可能你都看不到，因為它很快就又shut down了），檢視日誌：

[[email protected] spark-1.5.0-cdh5.5.2]$ cat logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-h71.out
Spark Command: /usr/jdk1.7.0_25/bin/java -cp /home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/activation-1.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/asm-3.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/avro-1.7.6-cdh5.5.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-codec-1.4.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-digester-1.8.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-el-1.0.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-lang-2.6.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/commons-net-3.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/curator-client-2.7.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/curator-framework-2.7.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/gson-2.2.4.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/guava-11.0.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/hadoop-annotations-2.6.0-cdh5.5.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/hadoop-auth-2.6.0-cdh5.5.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/hamcrest-core-1.3.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/htrace-core4-4.0.1-incubating.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/httpclient-4.2.5.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/httpcore-4.2.5.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jackson-jaxrs-1.8.8.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jackson-xc-1.8.8.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jersey-json-1.9.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jets3t-0.9.0.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jettison-1.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jetty-6.1.26.cloudera.4.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jetty-util-6.1.26.cloudera.4.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jsch-0.1.42.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jsp-api-2.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/junit-4.11.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/logredactor-1.0.3.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/paranamer-2.3.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/servlet-api-2.5.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/stax-api-1.0-2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/xmlenc-0.52.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/xz-1.0.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/lib/zookeeper-3.4.5-cdh5.5.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/hadoop-common-2.6.0-cdh5.5.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/hadoop-common-2.6.0-cdh5.5.2-tests.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/share/hadoop/common/hadoop-nfs-2.6.0-cdh5.5.2.jar:/home/hadoop/spark-1.5.0-cdh5.5.2/sbin/../conf/:/home/hadoop/spark-1.5.0-cdh5.5.2/lib/spark-assembly-1.5.0-cdh5.5.2-hadoop2.6.0-cdh5.5.2.jar:/home/hadoop/hadoop-2.6.0-cdh5.5.2/etc/hadoop/ -Xms1g -Xmx1g -XX:MaxPermSize=256m org.apache.spark.deploy.master.Master --ip h71 --port 7077 --webui-port 8080
========================================
17/03/17 17:26:27 INFO master.Master: Registered signal handlers for [TERM, HUP, INT]
17/03/17 17:26:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/03/17 17:26:28 INFO spark.SecurityManager: Changing view acls to: hadoop
17/03/17 17:26:28 INFO spark.SecurityManager: Changing modify acls to: hadoop
17/03/17 17:26:28 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(hadoop); users with modify permissions: Set(hadoop)
17/03/17 17:26:29 INFO slf4j.Slf4jLogger: Slf4jLogger started
17/03/17 17:26:29 INFO Remoting: Starting remoting
17/03/17 17:26:31 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:7077]
17/03/17 17:26:31 INFO Remoting: Remoting now listens on addresses: [akka.tcp://[email protected]:7077]
17/03/17 17:26:31 INFO util.Utils: Successfully started service 'sparkMaster' on port 7077.
17/03/17 17:26:32 INFO master.Master: Starting Spark master at spark://h71:7077
17/03/17 17:26:32 INFO master.Master: Running Spark version 1.5.0-cdh5.5.2
17/03/17 17:26:34 INFO server.Server: jetty-8.y.z-SNAPSHOT
17/03/17 17:26:39 INFO server.AbstractConnector: Started [email protected]:8080
17/03/17 17:26:39 INFO util.Utils: Successfully started service 'MasterUI' on port 8080.
17/03/17 17:26:39 INFO ui.MasterWebUI: Started MasterWebUI at http://192.168.8.71:8080
17/03/17 17:26:39 INFO server.Server: jetty-8.y.z-SNAPSHOT
17/03/17 17:26:40 INFO server.AbstractConnector: Started [email protected]:6066
17/03/17 17:26:40 INFO util.Utils: Successfully started service on port 6066.
17/03/17 17:26:40 INFO rest.StandaloneRestServer: Started REST server for submitting applications on port 6066
17/03/17 17:26:40 ERROR actor.ActorSystemImpl: Uncaught fatal error from thread [sparkMaster-akka.actor.default-dispatcher-15] shutting down ActorSystem [sparkMaster]
java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/Module
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:270)
        at org.apache.spark.util.Utils$.classForName(Utils.scala:172)
        at org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:190)
        at org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:186)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
        at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
        at org.apache.spark.metrics.MetricsSystem.registerSinks(MetricsSystem.scala:186)
        at org.apache.spark.metrics.MetricsSystem.start(MetricsSystem.scala:100)
        at org.apache.spark.deploy.master.Master.onStart(Master.scala:152)
        at org.apache.spark.rpc.akka.AkkaRpcEnv$$anonfun$actorRef$lzycompute$1$1$$anon$1$$anonfun$preStart$1.apply$mcV$sp(AkkaRpcEnv.scala:100)
        at org.apache.spark.rpc.akka.AkkaRpcEnv.org$apache$spark$rpc$akka$AkkaRpcEnv$$safelyCall(AkkaRpcEnv.scala:197)
        at org.apache.spark.rpc.akka.AkkaRpcEnv$$anonfun$actorRef$lzycompute$1$1$$anon$1.preStart(AkkaRpcEnv.scala:99)
        at akka.actor.ActorCell.create(ActorCell.scala:562)
        at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:425)
        at akka.actor.ActorCell.systemInvoke(ActorCell.scala:447)
        at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:262)
        at akka.dispatch.Mailbox.run(Mailbox.scala:218)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.Module
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 26 more
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/driver/kill,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/app/kill,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/api,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/static,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/json,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/history/not-found/json,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/history/not-found,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/app/json,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/app,null}
17/03/17 17:26:40 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/,null}
17/03/17 17:26:40 WARN metrics.MetricsSystem: Stopping a MetricsSystem that is not running
17/03/17 17:26:40 ERROR akka.AkkaRpcEnv: Ignore error: null
java.lang.NullPointerException
        at org.apache.spark.deploy.master.Master.onStop(Master.scala:198)
        at org.apache.spark.rpc.akka.AkkaRpcEnv$$anonfun$actorRef$lzycompute$1$1$$anon$1$$anonfun$postStop$1.apply$mcV$sp(AkkaRpcEnv.scala:143)
        at org.apache.spark.rpc.akka.AkkaRpcEnv.org$apache$spark$rpc$akka$AkkaRpcEnv$$safelyCall(AkkaRpcEnv.scala:197)
        at org.apache.spark.rpc.akka.AkkaRpcEnv$$anonfun$actorRef$lzycompute$1$1$$anon$1.postStop(AkkaRpcEnv.scala:142)
        at akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:201)
        at akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:163)
        at akka.actor.ActorCell.terminate(ActorCell.scala:338)
        at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:431)
        at akka.actor.ActorCell.systemInvoke(ActorCell.scala:447)
        at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:262)
        at akka.dispatch.Mailbox.run(Mailbox.scala:218)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
17/03/17 17:26:40 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
17/03/17 17:26:40 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
17/03/17 17:26:41 INFO Remoting: Remoting shut down
17/03/17 17:26:41 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.

原因：CLASSPATH中缺少jackson包

解決：在spark-env.sh中再新增

for f in $HADOOP_HOME/share/hadoop/mapreduce*/lib/*.jar; do
  if [ "$SPARK_CLASSPATH" ]; then
   export SPARK_CLASSPATH=$SPARK_CLASSPATH:$f
  else
   export SPARK_CLASSPATH=$f
  fi
done

並複製到其他兩個從節點後再啟動spark叢集則一切正常。

測試（執行這個spark自帶的程式可成功執行）：

[[email protected] spark-1.5.0-cdh5.5.2]$ ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master spark://h71:7077 --executor-memory 1G --total-executor-cores 1 ./lib/spark-examples-1.5.0-cdh5.5.2-hadoop2.6.0-cdh5.5.2.jar 100

注意：

在網上https://zhidao.baidu.com/question/2269421342018900708.html有看到解決報錯一的另一種方法，即在spark-env.sh中新增如下內容

export HADOOP_HOME=/home/hadoop/hadoop-2.6.0-cdh5.5.2
export SPARK_DIST_CLASSPATH=$(/home/hadoop/hadoop-2.6.0-cdh5.5.2/bin/hadoop classpath)

按這種方法解決報錯一後再新增相應內容解決報錯二後也是雖然能正常啟動但過一段時間相應的程序還是會掛掉，檢視日誌有：

17/03/17 18:43:02 ERROR actor.ActorSystemImpl: Uncaught fatal error from thread [sparkMaster-akka.actor.default-dispatcher-7] shutting down ActorSystem [sparkMaster]
java.lang.VerifyError: (class: org/jboss/netty/channel/socket/nio/NioWorkerPool, method: createWorker signature: (Ljava/util/concurrent/Executor;)Lorg/jboss/netty/channel/socket/nio/AbstractNioWorker;) Wrong return type in function
        at akka.remote.transport.netty.NettyTransport.<init>(NettyTransport.scala:282)
        at akka.remote.transport.netty.NettyTransport.<init>(NettyTransport.scala:239)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:78)
        at scala.util.Try$.apply(Try.scala:161)
        at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:73)
        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84)
        at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84)
        at scala.util.Success.flatMap(Try.scala:200)
        at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84)
        at akka.remote.EndpointManager$$anonfun$8.apply(Remoting.scala:618)
        at akka.remote.EndpointManager$$anonfun$8.apply(Remoting.scala:610)
        at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:722)
        at scala.collection.Iterator$class.foreach(Iterator.scala:727)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:721)
        at akka.remote.EndpointManager.akka$remote$EndpointManager$$listens(Remoting.scala:610)
        at akka.remote.EndpointManager$$anonfun$receive$2.applyOrElse(Remoting.scala:450)
        at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
        at akka.actor.ActorCell.invoke(ActorCell.scala:456)
        at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
        at akka.dispatch.Mailbox.run(Mailbox.scala:219)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
17/03/17 18:43:02 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
17/03/17 18:43:02 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
17/03/17 18:43:02 INFO Remoting: Remoting shut down
17/03/17 18:43:03 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
[ERROR] [03/17/2017 18:43:12.812] [main] [Remoting] Remoting error: [Startup timed out] [
akka.remote.RemoteTransportException: Startup timed out
        at akka.remote.Remoting.akka$remote$Remoting$$notifyError(Remoting.scala:129)
        at akka.remote.Remoting.start(Remoting.scala:191)
        at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
        at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:579)
        at akka.actor.ActorSystemImpl._start(ActorSystem.scala:577)
        at akka.actor.ActorSystemImpl.start(ActorSystem.scala:588)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:111)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:104)
        at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
        at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)
        at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52)
        at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1912)
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
        at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1903)
        at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55)
        at org.apache.spark.rpc.akka.AkkaRpcEnvFactory.create(AkkaRpcEnv.scala:253)
        at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:53)
        at org.apache.spark.deploy.master.Master$.startRpcEnvAndEndpoint(Master.scala:1074)
        at org.apache.spark.deploy.master.Master$.main(Master.scala:1058)
        at org.apache.spark.deploy.master.Master.main(Master.scala)
Caused by: java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds]
        at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
        at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
        at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
        at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
        at scala.concurrent.Await$.result(package.scala:107)
        at akka.remote.Remoting.start(Remoting.scala:173)
        ... 18 more
]
Exception in thread "main" java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds]
        at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
        at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
        at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
        at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
        at scala.concurrent.Await$.result(package.scala:107)
        at akka.remote.Remoting.start(Remoting.scala:173)
        at akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
        at akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:579)
        at akka.actor.ActorSystemImpl._start(ActorSystem.scala:577)
        at akka.actor.ActorSystemImpl.start(ActorSystem.scala:588)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:111)
        at akka.actor.ActorSystem$.apply(ActorSystem.scala:104)
        at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
        at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)
        at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52)
        at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1912)
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
        at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1903)
        at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55)
        at org.apache.spark.rpc.akka.AkkaRpcEnvFactory.create(AkkaRpcEnv.scala:253)
        at org.apache.spark.rpc.RpcEnv$.create(RpcEnv.scala:53)
        at org.apache.spark.deploy.master.Master$.startRpcEnvAndEndpoint(Master.scala:1074)
        at org.apache.spark.deploy.master.Master$.main(Master.scala:1058)
        at org.apache.spark.deploy.master.Master.main(Master.scala)

後來經過自己反覆嘗試，應該將SPARK_DIST_CLASSPATH改為SPARK_CLASSPATH方可好使，真尼瑪坑！

說明：

我安裝從spark官網下載的spark-1.6.3-bin-hadoop2.6或者spark-1.3.1-bin-hadoop2.6也沒這麼多的事啊，按我的這篇文章http://blog.csdn.net/m0_37739193/article/details/74908865中的安裝步驟就可以，這安裝個cdh版的咋尼瑪這麼費事啊。。。

補充：

將spark-1.6.3-bin-hadoop2.6.tgz解壓到相應的目錄後什麼都不用做就可以執行local單機模式：

[[email protected] spark-1.6.3-bin-hadoop2.6]$ ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master local[1] ./lib/spark-examples-1.6.3-hadoop2.6.0.jar 100

Yarn叢集模式：

client模式（可在SecureCRT終端中看見結果）：

[[email protected] spark-1.6.3-bin-hadoop2.6]$ ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn-client --executor-memory 1G --num-executors 1 ./lib/spark-examples-1.6.3-hadoop2.6.0.jar 100

cluster模式（結果只能在h71:8088裡面可見！）：

[[email protected] spark-1.6.3-bin-hadoop2.6]$ ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master yarn-cluster --executor-memory 1G --num-executors 1 ./lib/spark-examples-1.6.3-hadoop2.6.0.jar 100

注意：

我見網上Yarn叢集模式需要在spark-env.sh新增如下配置：

export HADOOP_CONF_DIR=$HADOOP_INSTALL/etc/hadoop
export YARN_CONF_DIR=$HADOOP_INSTALL/etc/hadoop
export SPARK_HOME=/usr/hadoopsoft/spark-1.6.3-bin-hadoop2.6
export SPARK_JAR=/usr/hadoopsoft/spark-1.6.3-bin-hadoop2.6/lib/spark-assembly-1.6.3-hadoop2.6.0.jar
export PATH=$SPARK_HOME/bin:$PATH

可是我在h71主節點中將spark-1.6.3-bin-hadoop2.6.tgz解壓後什麼都沒有做，Yarn叢集模式卻可以正常使用。。。

standalone叢集模式：

client模式（可在SecureCRT終端中看見結果）：

[[email protected] spark-1.6.3-bin-hadoop2.6]$ ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master spark://h71:7077 --executor-memory 1G --total-executor-cores 1 ./lib/spark-examples-1.6.3-hadoop2.6.0.jar 100

cluster模式（結果只能在h71:8080裡面可見！）：

[[email protected] spark-1.6.3-bin-hadoop2.6]$ ./bin/spark-submit --class org.apache.spark.examples.SparkPi --master spark://h71:7077 --deploy-mode cluster --supervise --executor-memory 1G --total-executor-cores 1 ./lib/spark-examples-1.6.3-hadoop2.6.0.jar 100

安裝spark-1.5.0-cdh5.5.2所踩過的坑

安裝spark-1.5.0-cdh5.5.2所踩過的坑

重新編譯、安裝spark assembly，使CDH5.5.1支援sparkSQL

Hadoop-2.5.0-cdh5.3.2 搭建單機偽分佈

編譯打包spark-1.6.0-cdh5.11.0-src詳細過程及問題記錄

Ubuntu16.04下載編譯Android6.0原始碼以及我所踩的坑

安裝Oozie4.1.0-cdh5.5.2

Hadoop 2.3.0-cdh5.0.2升級Hadoop 2.5.0-cdh5.3.1

centos7.5版本安裝nginx.1.14.0時需要安裝的依賴

在docker上安裝 Spark 1.2.0

Windows10下Anaconda3-5.0.1安裝TensorFlow 1.10.0-py36

presto-0.147+postgresql-9.5.3+msql-5.0.7+hadoop-2.5.2+hive-1.2.1環境構築以及測試

利用hadoop-2.5.0-cdh5.3.6版本，搭建完全分散式HA詳細記錄（怕忘）

hadoop-2.5.0-cdh5.3.0 HA線上升級

HDP2.5.0 + Spark1.6.2 通過IDEA(Win64)遠端提交spark jobs On YARN

CENTOS 7.5離線安裝Cloudera Manager 5和CDH5(5.15.0) 完全教程

【伊利丹】Hadoop-2.5.0-CDH5.2.0/Hive與Hbase整合實驗

最新版大資料平臺安裝部署指南，HDP-2.6.5.0，ambari-2.6.2.0

[Hadoop] CentOS7 安裝flume-ng-1.6.0-cdh5.7.0

ThinkPHP(5.1.x ~ 5.1.31 5.0.x ~ 5.0.23)GetShell漏洞

thinkphp 最新版本5.0到5.1高危漏洞爆發可直接提權getshell

安裝spark-1.5.0-cdh5.5.2所踩過的坑

相關推薦