graphframes本地windows環境搭建
阿新 • • 發佈:2018-12-29
1.從github上下載graphframes包,或者從連結下載:https://download.csdn.net/download/weixin_42247685/10591772
2.解壓後將graphframes-master\python下的graphframes資料夾拷貝到python/Lib/site-packages檔案下。
3.在python中輸入import graphframes ,檢視是否正常。
示例程式碼:
from graphframes import * from pyspark.sql import SparkSession import os import graphframes #下面這段必須有,否則報錯, os.environ["PYSPARK_SUBMIT_ARGS"] = ( "--packages graphframes:graphframes:0.5.0-spark2.1-s_2.11 pyspark-shell" ) sqlContext = SparkSession.builder.appName("Python Spark SQL basic example").config("spark.some.config.option", "some-value").getOrCreate() v = sqlContext.createDataFrame([ ("a", "Alice", 34), ("b", "Bob", 36), ("c", "Charlie", 30), ], ["id", "name", "age"]) # Create an Edge DataFrame with "src" and "dst" columns e = sqlContext.createDataFrame([ ("a", "b", "friend"), ("b", "c", "follow"), ("c", "b", "follow"), ], ["src", "dst", "relationship"]) # Create a GraphFrame g = GraphFrame(v, e) # Query: Get in-degree of each vertex. g.inDegrees.show() # Query: Count the number of "follow" connections in the graph. g.edges.filter("relationship = 'follow'").count() # Run PageRank algorithm, and show results. results = g.pageRank(resetProbability=0.01, maxIter=20) results.vertices.select("id", "pagerank").show()