i本人初學Spark，記錄點滴收穫，自己mark一下，也希望對各位有幫助。

平臺及版本：

ubuntu12.04 LTS /  python-2.7.3 / hadoop-2.7.1 / spark-1.6.0-bin-without-hadoop

基本安裝配置不做介紹，推薦參考廈門大學 林子雨 老師的網上教程（非常詳細）。

執行《Spark 機器學習》教材中的1.6節程式碼出錯：

"""檔名為 pythonapp.py"""
from pyspark import SparkContext

sc = SparkContext("local[2]", "First Spark App")
# we take the raw data in CSV format and convert it into a set of records of the form (user, product, price)
data = sc.textFile("data/UserPurchaseHistory.csv").map(lambda line: line.split(",")).map(lambda record: (record[0], record[1], record[2]))
# let's count the number of purchases
numPurchases = data.count()
# let's count how many unique users made purchases
uniqueUsers = data.map(lambda record: record[0]).distinct().count()
# let's sum up our total revenue
totalRevenue = data.map(lambda record: float(record[2])).sum()
# let's find our most popular product
products = data.map(lambda record: (record[1], 1.0)).reduceByKey(lambda a, b: a + b).collect()
mostPopular = sorted(products, key=lambda x: x[1], reverse=True)[0]

# Finally, print everything out
print "Total purchases: %d" % numPurchases
print "Unique users: %d" % uniqueUsers
print "Total revenue: %2.2f" % totalRevenue
print "Most popular product: %s with %d purchases" % (mostPopular[0], mostPopular[1])

# stop the SparkContext
sc.stop()

-----------------------------------------------------報錯顯示-------------------------------------

Traceback (most recent call last):
  File "/usr/local/spark/bin/pythonapp.py", line 8, in <module>
    numPurchases = data.count()
  File "/usr/local/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 1004, in count
  File "/usr/local/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 995, in sum
  File "/usr/local/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 869, in fold
  File "/usr/local/spark/python/lib/pyspark.zip/pyspark/rdd.py", line 771, in collect
  File "/usr/local/spark/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 813, in __call__
  File "/usr/local/spark/python/lib/py4j-0.9-src.zip/py4j/protocol.py", line 308, in get_return_value
py4j.protocol.Py4JJavaError

啟動 hadoop，在hdfs中建立一個路徑：/usr/hadoop/input/

並將"pythonapp.py"中，data 的路徑改為："/usr/hadoop/input/UserPurchaseHistory.csv"

在spark/bin目錄下，執行pythonapp.py

解決問題，得到預期結果。