問題描述:

在用MapReduce實現資料去重時遇到報錯

Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses

除此之外還有:java.lang.NoClassDefFoundError；java.io.IOException: Cannot initialize Cluster.

問題解決:

這類問題都是由於MapReduce和hadoop的依賴沒有導全,匯入:

<dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-common</artifactId>
      <version>3.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-hdfs</artifactId>
      <version>3.2.0</version>
    </dependency>
    <dependency>
      <groupId>log4j</groupId>
      <artifactId>log4j</artifactId>
      <version>1.2.17</version>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-client</artifactId>
      <version>3.2.0</version>
    </dependency>

    <!--mapreduce-->
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-mapreduce-client-core</artifactId>
      <version>3.2.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-mapreduce-client-common</artifactId>
      <version>3.2.0</version>
    </dependency>
 

至此,問題解決.