轉自https://www.cnblogs.com/wupeiqi/articles/6283017.html

一.requests

Python標準庫中提供了：urllib、urllib2、httplib等模塊以供Http請求，但是，它的 API 太渣了。它是為另一個時代、另一個互聯網所創建的。它需要巨量的工作，甚至包括各種方法覆蓋，來完成最簡單的任務。

Requests 是使用 Apache2 Licensed 許可證的 基於Python開發的HTTP 庫，其在Python內置模塊的基礎上進行了高度的封裝，從而使得Pythoner進行網絡請求時，變得美好了許多，使用Requests可以輕而易舉的完成瀏覽器可有的任何操作。

1、GET請求

#<1>、GET請求
#1、無參數實例
ret=requests.get("https://github.com/timeline.json")

print (ret.url)
#輸出：https://github.com/timeline.json
print (ret.text)
#輸出：{"message":"Hello there, wayfaring stranger. If you’re reading this then you probably didn’t see our blog post a couple of years back announcing that this API would go away: http://git.io/17AROg Fear not, you should be able to get what you need from the shiny new Events API instead.","documentation_url":"https://developer.github.com/v3/activity/events/#list-public-events"}
 


# 2、有參數實例
payload = {‘key1‘: ‘value1‘, ‘key2‘: ‘value2‘}
ret = requests.get("http://httpbin.org/get", params=payload)

print(ret.url)
#輸出：http://httpbin.org/get?key1=value1&key2=value2
print(ret.text)
#輸出：{"args":{"key1":"value1","key2":"value2"},"headers":{"Accept":"*/*","Accept-Encoding":"gzip, deflate","Connection":"close","Host":"httpbin.org","User-Agent":"python-requests/2.18.4"},"origin":"182.48.111.194","url":"http://httpbin.org/get?key1=value1&key2=value2"}
 



#<2>、POST請求
#1、基本POST實例
payload={"key1":"value1","key2":"value2"}
ret=requests.post("http://httpbin.org/post",data=payload)

print (ret.text)
#輸出：{"args":{},"data":"","files":{},"form":{"key1":"value1","key2":"value2"},"headers":{"Accept":"*/*","Accept-Encoding":"gzip, deflate","Connection":"close","Content-Length":"23","Content-Type":"application/x-www-form-urlencoded","Host":"httpbin.org","User-Agent":"python-requests/2.18.4"},"json":null,"origin":"182.48.111.194","url":"http://httpbin.org/post"}


#2、POST請求
import json

url=‘https://api.github.com/some/endpoint‘
payload={"some":"data"}
headers={‘content-type‘: ‘application/json‘}

ret=requests.post(url,data=json.dumps(payload),headers=headers)

print (ret.text)
# 輸出：{"message":"Not Found","documentation_url":"https://developer.github.com/v3"}
print (ret.cookies)
#輸出：<RequestsCookieJar[]>

#3、其他請求
requests.get(url, params=None, **kwargs)
requests.post(url, data=None, json=None, **kwargs)
requests.put(url, data=None, **kwargs)
requests.head(url, **kwargs)
requests.delete(url, **kwargs)
requests.patch(url, data=None, **kwargs)
requests.options(url, **kwargs)

# 以上方法均是在此方法的基礎上構建
requests.request(method, url, **kwargs)

requests和BeautifulSoup