Python requests模組基礎使用方法例項及高階應用(自動登陸,抓取網頁原始碼)例項詳解

阿新 • • 發佈：2020-02-17

1、Python requests模組說明

requests是使用Apache2 licensed 許可證的HTTP庫。

用python編寫。

比urllib2模組更簡潔。

Request支援HTTP連線保持和連線池，支援使用cookie保持會話，支援檔案上傳，支援自動響應內容的編碼，支援國際化的URL和POST資料自動編碼。

在python內建模組的基礎上進行了高度的封裝，從而使得python進行網路請求時，變得人性化，使用Requests可以輕而易舉的完成瀏覽器可有的任何操作。

現代，國際化，友好。

requests會自動實現持久連線keep-alive

2、Python requests模組基礎入門

1）匯入模組

import requests

2）傳送請求的簡潔

示例程式碼：獲取一個網頁（個人github）

import requests
r = requests.get('https://github.com/Ranxf')    # 最基本的不帶引數的get請求
r1 = requests.get(url='http://dict.baidu.com/s',params={'wd': 'python'})   # 帶引數的get請求

我們還可以使用requests模組其它請求方法

1 requests.get(‘https://github.com/timeline.json') # GET請求

2 requests.post(“http://httpbin.org/post”) # POST請求

3 requests.put(“http://httpbin.org/put”) # PUT請求

4 requests.delete(“http://httpbin.org/delete”) # DELETE請求

5 requests.head(“http://httpbin.org/get”) # HEAD請求

6 requests.options(“http://httpbin.org/get” ) # OPTIONS請求

3）為url傳遞引數

>>> url_params = {'key':'value'}    #  字典傳遞引數，如果值為None的鍵不會被新增到url中
>>> r = requests.get('your url',params = url_params)
>>> print(r.url)

　　your url?key=value

4）響應的內容

r.encoding #獲取當前的編碼

r.encoding = 'utf-8' #設定編碼

r.text #以encoding解析返回內容。字串方式的響應體，會自動根據響應頭部的字元編碼進行解碼。

r.content #以位元組形式（二進位制）返回。位元組方式的響應體，會自動為你解碼 gzip 和 deflate 壓縮。

r.headers #以字典物件儲存伺服器響應頭，但是這個字典比較特殊，字典鍵不區分大小寫，若鍵不存在則返回None

r.status_code #響應狀態碼

r.raw #返回原始響應體，也就是 urllib 的 response 物件，使用 r.raw.read()

r.ok # 檢視r.ok的布林值便可以知道是否登陸成功

#*特殊方法*#

r.json() #Requests中內建的JSON解碼器，以json形式返回,前提返回的內容確保是json格式的，不然解析出錯會拋異常

r.raise_for_status() #失敗請求(非200響應)丟擲異常

post傳送json請求：

import requests
import json
 
r = requests.post('https://api.github.com/some/endpoint',data=json.dumps({'some': 'data'}))

print(r.json())

5）定製頭和cookie資訊

header = {'user-agent': 'my-app/0.0.1''}
cookie = {'key':'value'}
 r = requests.get/post('your url',headers=header,cookies=cookie) 
data = {'some': 'data'}
headers = {'content-type': 'application/json','User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:22.0) Gecko/20100101 Firefox/22.0'}
 
r = requests.post('https://api.github.com/some/endpoint',data=data,headers=headers)
print(r.text)

6）響應狀態碼

使用requests方法後，會返回一個response物件，其儲存了伺服器響應的內容，如上例項中已經提到的 r.text、r.status_code……

獲取文字方式的響應體例項：當你訪問 r.text 之時，會使用其響應的文字編碼進行解碼，並且你可以修改其編碼讓 r.text 使用自定義的編碼進行解碼。

r = requests.get('http://www.itwhy.org')
print(r.text,'\n{}\n'.format('*'*79),r.encoding)
r.encoding = 'GBK'
print(r.text,r.encoding)

示例程式碼：

import requests

r = requests.get('https://github.com/Ranxf')    # 最基本的不帶引數的get請求
print(r.status_code)                # 獲取返回狀態
r1 = requests.get(url='http://dict.baidu.com/s',params={'wd': 'python'})   # 帶引數的get請求
print(r1.url)
print(r1.text)    # 列印解碼後的返回資料

執行結果：

/usr/bin/python3.5 /home/rxf/python3_1000/1000/python3_server/python3_requests/demo1.py

200

http://dict.baidu.com/s?wd=python

…………

Process finished with exit code 0

r.status_code #如果不是200，可以使用 r.raise_for_status() 丟擲異常

7）響應

r.headers #返回字典型別,頭資訊

r.requests.headers #返回傳送到伺服器的頭資訊

r.cookies #返回cookie

r.history #返回重定向資訊,當然可以在請求是加上allow_redirects = false 阻止重定向

8）超時

r = requests.get('url',timeout=1)      #設定秒數超時，僅對於連線有效

9)會話物件，能夠跨請求保持某些引數

s = requests.Session()
s.auth = ('auth','passwd')
s.headers = {'key':'value'}
r = s.get('url')
r1 = s.get('url1')

10）代理

proxies = {'http':'ip1','https':'ip2' }
requests.get('url',proxies=proxies)

彙總：

# HTTP請求型別
# get型別
r = requests.get('https://github.com/timeline.json')
# post型別
r = requests.post("http://m.ctrip.com/post")
# put型別
r = requests.put("http://m.ctrip.com/put")
# delete型別
r = requests.delete("http://m.ctrip.com/delete")
# head型別
r = requests.head("http://m.ctrip.com/head")
# options型別
r = requests.options("http://m.ctrip.com/get")

# 獲取響應內容
print(r.content) #以位元組的方式去顯示，中文顯示為字元
print(r.text) #以文字的方式去顯示

#URL傳遞引數
payload = {'keyword': '香港','salecityid': '2'}
r = requests.get("http://m.ctrip.com/webapp/tourvisa/visa_list",params=payload) 
print（r.url） #示例為http://m.ctrip.com/webapp/tourvisa/visa_list?salecityid=2&keyword=香港

#獲取/修改網頁編碼
r = requests.get('https://github.com/timeline.json')
print （r.encoding）


#json處理
r = requests.get('https://github.com/timeline.json')
print（r.json()） # 需要先import json  

# 定製請求頭
url = 'http://m.ctrip.com'
headers = {'User-Agent' : 'Mozilla/5.0 (Linux; Android 4.2.1; en-us; Nexus 4 Build/JOP40D) AppleWebKit/535.19 (KHTML,like Gecko) Chrome/18.0.1025.166 Mobile Safari/535.19'}
r = requests.post(url,headers=headers)
print （r.request.headers)

#複雜post請求
url = 'http://m.ctrip.com'
payload = {'some': 'data'}
r = requests.post(url,data=json.dumps(payload)) #如果傳遞的payload是string而不是dict，需要先呼叫dumps方法格式化一下

# post多部分編碼檔案
url = 'http://m.ctrip.com'
files = {'file': open('report.xls','rb')}
r = requests.post(url,files=files)

# 響應狀態碼
r = requests.get('http://m.ctrip.com')
print(r.status_code)
  
# 響應頭
r = requests.get('http://m.ctrip.com')
print (r.headers)
print (r.headers['Content-Type'])
print (r.headers.get('content-type')) #訪問響應頭部分內容的兩種方式
  
# Cookies
url = 'http://example.com/some/cookie/setting/url'
r = requests.get(url)
r.cookies['example_cookie_name']  #讀取cookies
  
url = 'http://m.ctrip.com/cookies'
cookies = dict(cookies_are='working')
r = requests.get(url,cookies=cookies) #傳送cookies

#設定超時時間
r = requests.get('http://m.ctrip.com',timeout=0.001)

#設定訪問代理
proxies = {
      "http": "http://10.10.1.10:3128","https": "http://10.10.1.100:4444",}
r = requests.get('http://m.ctrip.com',proxies=proxies)


#如果代理需要使用者名稱和密碼，則需要這樣：
proxies = {
  "http": "http://user:[email protected]:3128/",}

# HTTP請求型別
# get型別
r = requests.get('https://github.com/timeline.json')
# post型別
r = requests.post("http://m.ctrip.com/post")
# put型別
r = requests.put("http://m.ctrip.com/put")
# delete型別
r = requests.delete("http://m.ctrip.com/delete")
# head型別
r = requests.head("http://m.ctrip.com/head")
# options型別
r = requests.options("http://m.ctrip.com/get")

# 獲取響應內容
print(r.content) #以位元組的方式去顯示，中文顯示為字元
print(r.text) #以文字的方式去顯示

#URL傳遞引數
payload = {'keyword': '香港',}

3、示例程式碼

GET請求

# 1、無引數例項
 
import requests
 
ret = requests.get('https://github.com/timeline.json')
 
print(ret.url)
print(ret.text)
 
 
 
# 2、有引數例項
 
import requests
 
payload = {'key1': 'value1','key2': 'value2'}
ret = requests.get("http://httpbin.org/get",params=payload)
 
print(ret.url)
print(ret.text)

POST請求

# 1、基本POST例項
 
import requests
 
payload = {'key1': 'value1','key2': 'value2'}
ret = requests.post("http://httpbin.org/post",data=payload)
 
print(ret.text)
 
 
# 2、傳送請求頭和資料例項
 
import requests
import json
 
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}
headers = {'content-type': 'application/json'}
 
ret = requests.post(url,data=json.dumps(payload),headers=headers)
 
print(ret.text)
print(ret.cookies)

請求引數

def request(method,url,**kwargs):

"""Constructs and sends a :class:`Request <Request>`.

:param method: method for the new :class:`Request` object.

:param url: URL for the new :class:`Request` object.

:param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.

:param data: (optional) Dictionary,bytes,or file-like object to send in the body of the :class:`Request`.

:param json: (optional) json data to send in the body of the :class:`Request`.

:param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`.

:param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`.

:param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload.

``file-tuple`` can be a 2-tuple ``('filename',fileobj)``,3-tuple ``('filename',fileobj,'content_type')``

or a 4-tuple ``('filename','content_type',custom_headers)``,where ``'content-type'`` is a string

defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers

to add for the file.

:param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth.

:param timeout: (optional) How long to wait for the server to send data

before giving up,as a float,or a :ref:`(connect timeout,read

timeout) <timeouts>` tuple.

:type timeout: float or tuple

:param allow_redirects: (optional) Boolean. Set to True if POST/PUT/DELETE redirect following is allowed.

:type allow_redirects: bool

:param proxies: (optional) Dictionary mapping protocol to the URL of the proxy.

:param verify: (optional) whether the SSL cert will be verified. A CA_BUNDLE path can also be provided. Defaults to ``True``.

:param stream: (optional) if ``False``,the response content will be immediately downloaded.

:param cert: (optional) if String,path to ssl client cert file (.pem). If Tuple,('cert','key') pair.

:return: :class:`Response <Response>` object

:rtype: requests.Response

Usage::

>>> import requests

>>> req = requests.request('GET','http://httpbin.org/get')

<Response [200]>

引數示例程式碼

def param_method_url():
  # requests.request(method='get',url='http://127.0.0.1:8000/test/')
  # requests.request(method='post',url='http://127.0.0.1:8000/test/')
  pass


def param_param():
  # - 可以是字典
  # - 可以是字串
  # - 可以是位元組（ascii編碼以內）

  # requests.request(method='get',# url='http://127.0.0.1:8000/test/',# params={'k1': 'v1','k2': '水電費'})

  # requests.request(method='get',# params="k1=v1&k2=水電費&k3=v3&k3=vv3")

  # requests.request(method='get',# params=bytes("k1=v1&k2=k2&k3=v3&k3=vv3",encoding='utf8'))

  # 錯誤
  # requests.request(method='get',# params=bytes("k1=v1&k2=水電費&k3=v3&k3=vv3",encoding='utf8'))
  pass


def param_data():
  # 可以是字典
  # 可以是字串
  # 可以是位元組
  # 可以是檔案物件

  # requests.request(method='POST',# data={'k1': 'v1','k2': '水電費'})

  # requests.request(method='POST',# data="k1=v1; k2=v2; k3=v3; k3=v4"
  # )

  # requests.request(method='POST',# data="k1=v1;k2=v2;k3=v3;k3=v4",# headers={'Content-Type': 'application/x-www-form-urlencoded'}
  # )

  # requests.request(method='POST',# data=open('data_file.py',mode='r',encoding='utf-8'),# 檔案內容是：k1=v1;k2=v2;k3=v3;k3=v4
  # headers={'Content-Type': 'application/x-www-form-urlencoded'}
  # )
  pass


def param_json():
  # 將json中對應的資料進行序列化成一個字串，json.dumps(...)
  # 然後傳送到伺服器端的body中，並且Content-Type是 {'Content-Type': 'application/json'}
  requests.request(method='POST',url='http://127.0.0.1:8000/test/',json={'k1': 'v1','k2': '水電費'})


def param_headers():
  # 傳送請求頭到伺服器端
  requests.request(method='POST','k2': '水電費'},headers={'Content-Type': 'application/x-www-form-urlencoded'}
           )


def param_cookies():
  # 傳送Cookie到伺服器端
  requests.request(method='POST',data={'k1': 'v1','k2': 'v2'},cookies={'cook1': 'value1'},)
  # 也可以使用CookieJar（字典形式就是在此基礎上封裝）
  from http.cookiejar import CookieJar
  from http.cookiejar import Cookie

  obj = CookieJar()
  obj.set_cookie(Cookie(version=0,name='c1',value='v1',port=None,domain='',path='/',secure=False,expires=None,discard=True,comment=None,comment_url=None,rest={'HttpOnly': None},rfc2109=False,port_specified=False,domain_specified=False,domain_initial_dot=False,path_specified=False)
          )
  requests.request(method='POST',cookies=obj)


def param_files():
  # 傳送檔案
  # file_dict = {
  # 'f1': open('readme','rb')
  # }
  # requests.request(method='POST',# files=file_dict)

  # 傳送檔案，定製檔名
  # file_dict = {
  # 'f1': ('test.txt',open('readme','rb'))
  # }
  # requests.request(method='POST',"hahsfaksfa9kasdjflaksdjf")
  # }
  # requests.request(method='POST',# files=file_dict)

  # 傳送檔案，定製檔名
  # file_dict = {
  #   'f1': ('test.txt',"hahsfaksfa9kasdjflaksdjf",'application/text',{'k1': '0'})
  # }
  # requests.request(method='POST',#         url='http://127.0.0.1:8000/test/',#         files=file_dict)

  pass


def param_auth():
  from requests.auth import HTTPBasicAuth,HTTPDigestAuth

  ret = requests.get('https://api.github.com/user',auth=HTTPBasicAuth('wupeiqi','sdfasdfasdf'))
  print(ret.text)

  # ret = requests.get('http://192.168.1.1',# auth=HTTPBasicAuth('admin','admin'))
  # ret.encoding = 'gbk'
  # print(ret.text)

  # ret = requests.get('http://httpbin.org/digest-auth/auth/user/pass',auth=HTTPDigestAuth('user','pass'))
  # print(ret)
  #


def param_timeout():
  # ret = requests.get('http://google.com/',timeout=1)
  # print(ret)

  # ret = requests.get('http://google.com/',timeout=(5,1))
  # print(ret)
  pass


def param_allow_redirects():
  ret = requests.get('http://127.0.0.1:8000/test/',allow_redirects=False)
  print(ret.text)


def param_proxies():
  # proxies = {
  # "http": "61.172.249.96:80",# "https": "http://61.185.219.126:3128",# }

  # proxies = {'http://10.20.1.128': 'http://10.10.1.10:5323'}

  # ret = requests.get("http://www.proxy360.cn/Proxy",proxies=proxies)
  # print(ret.headers)


  # from requests.auth import HTTPProxyAuth
  #
  # proxyDict = {
  # 'http': '77.75.105.165',# 'https': '77.75.105.165'
  # }
  # auth = HTTPProxyAuth('username','mypassword')
  #
  # r = requests.get("http://www.google.com",proxies=proxyDict,auth=auth)
  # print(r.text)

  pass


def param_stream():
  ret = requests.get('http://127.0.0.1:8000/test/',stream=True)
  print(ret.content)
  ret.close()

  # from contextlib import closing
  # with closing(requests.get('http://httpbin.org/get',stream=True)) as r:
  # # 在此處理響應。
  # for i in r.iter_content():
  # print(i)


def requests_session():
  import requests

  session = requests.Session()

  ### 1、首先登陸任何頁面，獲取cookie

  i1 = session.get(url="http://dig.chouti.com/help/service")

  ### 2、使用者登陸，攜帶上一次的cookie，後臺對cookie中的 gpsd 進行授權
  i2 = session.post(
    url="http://dig.chouti.com/login",data={
      'phone': "8615131255089",'password': "xxxxxx",'oneMonth': ""
    }
  )

  i3 = session.post(
    url="http://dig.chouti.com/link/vote?linksId=8589623",)
  print(i3.text)

json請求：

#! /usr/bin/python3
import requests
import json


class url_request():
  def __init__(self):
    ''' init '''

if __name__ == '__main__':
  heard = {'Content-Type': 'application/json'}
  payload = {'CountryName': '中國','ProvinceName': '四川省','L1CityName': 'chengdu','L2CityName': 'yibing','TownName': '','Longitude': '107.33393','Latitude': '33.157131','Language': 'CN'}
  r = requests.post("http://www.xxxxxx.com/CityLocation/json/LBSLocateCity",heards=heard,data=payload)
  data = r.json()
  if r.status_code!=200:
    print('LBSLocateCity API Error' + str(r.status_code))
  print(data['CityEntities'][0]['CityID']) # 列印返回json中的某個key的value
  print(data['ResponseStatus']['Ack'])
  print(json.dump(data,indent=4,sort_keys=True,ensure_ascii=False)) # 樹形列印json，ensure_ascii必須設為False否則中文會顯示為unicode

Xml請求：

#! /usr/bin/python3
import requests

class url_request():
  def __init__(self):
    """init"""

if __name__ == '__main__':
  heards = {'Content-type': 'text/xml'}
  XML = '<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><Request xmlns="http://tempuri.org/"><jme><JobClassFullName>WeChatJSTicket.JobWS.Job.JobRefreshTicket,WeChatJSTicket.JobWS</JobClassFullName><Action>RUN</Action><Param>1</Param><HostIP>127.0.0.1</HostIP><JobInfo>1</JobInfo><NeedParallel>false</NeedParallel></jme></Request></soap:Body></soap:Envelope>'
  url = 'http://jobws.push.mobile.xxxxxxxx.com/RefreshWeiXInTokenJob/RefreshService.asmx'
  r = requests.post(url=url,heards=heards,data=XML)
  data = r.text
  print(data)

狀態異常處理

import requests

URL = 'http://ip.taobao.com/service/getIpInfo.php' # 淘寶IP地址庫API
try:
  r = requests.get(URL,params={'ip': '8.8.8.8'},timeout=1)
  r.raise_for_status() # 如果響應狀態碼不是 200，就主動丟擲異常
except requests.RequestException as e:
  print(e)
else:
  result = r.json()
  print(type(result),result,sep='\n')

上傳檔案

使用request模組，也可以上傳檔案，檔案的型別會自動進行處理：

import requests
 
url = 'http://127.0.0.1:8080/upload'
files = {'file': open('/home/rxf/test.jpg','rb')}
#files = {'file': ('report.jpg',open('/home/lyb/sjzl.mpg','rb'))}   #顯式的設定檔名
 
r = requests.post(url,files=files)
print(r.text)

request更加方便的是，可以把字串當作檔案進行上傳：

import requests
 
url = 'http://127.0.0.1:8080/upload'
files = {'file': ('test.txt',b'Hello Requests.')}   #必需顯式的設定檔名
 
r = requests.post(url,files=files)
print(r.text)

身份驗證

基本身份認證(HTTP Basic Auth)

import requests
from requests.auth import HTTPBasicAuth
 
r = requests.get('https://httpbin.org/hidden-basic-auth/user/passwd',auth=HTTPBasicAuth('user','passwd'))
# r = requests.get('https://httpbin.org/hidden-basic-auth/user/passwd',auth=('user','passwd'))  # 簡寫
print(r.json())

另一種非常流行的HTTP身份認證形式是摘要式身份認證，Requests對它的支援也是開箱即可用的:

requests.get(URL,'pass')

Cookies與會話物件

如果某個響應中包含一些Cookie，你可以快速訪問它們：

import requests
 
r = requests.get('http://www.google.com.hk/')
print(r.cookies['NID'])
print(tuple(r.cookies))

要想傳送你的cookies到伺服器，可以使用 cookies 引數：

import requests
 
url = 'http://httpbin.org/cookies'
cookies = {'testCookies_1': 'Hello_Python3','testCookies_2': 'Hello_Requests'}
# 在Cookie Version 0中規定空格、方括號、圓括號、等於號、逗號、雙引號、斜槓、問號、@，冒號，分號等特殊符號都不能作為Cookie的內容。
r = requests.get(url,cookies=cookies)
print(r.json())

會話物件讓你能夠跨請求保持某些引數，最方便的是在同一個Session例項發出的所有請求之間保持cookies，且這些都是自動處理的，甚是方便。

下面就來一個真正的例項，如下是快盤簽到指令碼：

import requests
 
headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8','Accept-Encoding': 'gzip,deflate,compress','Accept-Language': 'en-us;q=0.5,en;q=0.3','Cache-Control': 'max-age=0','Connection': 'keep-alive','User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:22.0) Gecko/20100101 Firefox/22.0'}
 
s = requests.Session()
s.headers.update(headers)
# s.auth = ('superuser','123')
s.get('https://www.kuaipan.cn/account_login.htm')
 
_URL = 'http://www.kuaipan.cn/index.php'
s.post(_URL,params={'ac':'account','op':'login'},data={'username':'****@foxmail.com','userpwd':'********','isajax':'yes'})
r = s.get(_URL,params={'ac':'zone','op':'taskdetail'})
print(r.json())
s.get(_URL,params={'ac':'common','op':'usersign'})

requests模組抓取網頁原始碼並儲存到檔案示例

這是一個基本的檔案儲存操作，但這裡有幾個值得注意的問題：

1.安裝requests包，命令列輸入pip install requests即可自動安裝。很多人推薦使用requests，自帶的urllib.request也可以抓取網頁原始碼

2.open方法encoding引數設為utf-8，否則儲存的檔案會出現亂碼。

3.如果直接在cmd中輸出抓取的內容，會提示各種編碼錯誤，所以儲存到檔案檢視。

4.with open方法是更好的寫法，可以自動操作完畢後釋放資源

Python requests模組抽屜自動登入

#! /urs/bin/python3
import requests

'''requests模組抓取網頁原始碼並儲存到檔案示例'''
html = requests.get("http://www.baidu.com")
with open('test.txt','w',encoding='utf-8') as f:
  f.write(html.text)
  
'''讀取一個txt檔案，每次讀取一行，並儲存到另一個txt檔案中的示例'''
ff = open('testt.txt',encoding='utf-8')
with open('test.txt',encoding="utf-8") as f:
  for line in f:
    ff.write(line)
    ff.close()

因為在命令列中列印每次讀取一行的資料，中文會出現編碼錯誤，所以每次讀取一行並儲存到另一個檔案，這樣來測試讀取是否正常。（注意open的時候制定encoding編碼方式）

Python requests模組自動登陸例項：

#!/usr/bin/env python
# -*- coding:utf-8 -*-
import requests


# ############## 方式一 ##############
"""
# ## 1、首先登陸任何頁面，獲取cookie
i1 = requests.get(url="http://dig.chouti.com/help/service")
i1_cookies = i1.cookies.get_dict()

# ## 2、使用者登陸，攜帶上一次的cookie，後臺對cookie中的 gpsd 進行授權
i2 = requests.post(
  url="http://dig.chouti.com/login",data={
    'phone': "8615131255089",'password': "xxooxxoo",'oneMonth': ""
  },cookies=i1_cookies
)

# ## 3、點贊（只需要攜帶已經被授權的gpsd即可）
gpsd = i1_cookies['gpsd']
i3 = requests.post(
  url="http://dig.chouti.com/link/vote?linksId=8589523",cookies={'gpsd': gpsd}
)

print(i3.text)
"""


# ############## 方式二 ##############
"""
import requests

session = requests.Session()
i1 = session.get(url="http://dig.chouti.com/help/service")
i2 = session.post(
  url="http://dig.chouti.com/login",'oneMonth': ""
  }
)
i3 = session.post(
  url="http://dig.chouti.com/link/vote?linksId=8589523"
)
print(i3.text)

"""

Python requests模組github自動登入

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import requests
from bs4 import BeautifulSoup

# ############## 方式一 ##############
#
# # 1. 訪問登陸頁面，獲取 authenticity_token
# i1 = requests.get('https://github.com/login')
# soup1 = BeautifulSoup(i1.text,features='lxml')
# tag = soup1.find(name='input',attrs={'name': 'authenticity_token'})
# authenticity_token = tag.get('value')
# c1 = i1.cookies.get_dict()
# i1.close()
#
# # 1. 攜帶authenticity_token和使用者名稱密碼等資訊，傳送使用者驗證
# form_data = {
# "authenticity_token": authenticity_token,#   "utf8": "",#   "commit": "Sign in",#   "login": "[email protected]",#   'password': 'xxoo'
# }
#
# i2 = requests.post('https://github.com/session',data=form_data,cookies=c1)
# c2 = i2.cookies.get_dict()
# c1.update(c2)
# i3 = requests.get('https://github.com/settings/repositories',cookies=c1)
#
# soup3 = BeautifulSoup(i3.text,features='lxml')
# list_group = soup3.find(name='div',class_='listgroup')
#
# from bs4.element import Tag
#
# for child in list_group.children:
#   if isinstance(child,Tag):
#     project_tag = child.find(name='a',class_='mr-1')
#     size_tag = child.find(name='small')
#     temp = "專案:%s(%s); 專案路徑:%s" % (project_tag.get('href'),size_tag.string,project_tag.string,)
#     print(temp)



# ############## 方式二 ##############
# session = requests.Session()
# # 1. 訪問登陸頁面，獲取 authenticity_token
# i1 = session.get('https://github.com/login')
# soup1 = BeautifulSoup(i1.text,attrs={'name': 'authenticity_token'})
# authenticity_token = tag.get('value')
# c1 = i1.cookies.get_dict()
# i1.close()
#
# # 1. 攜帶authenticity_token和使用者名稱密碼等資訊，傳送使用者驗證
# form_data = {
#   "authenticity_token": authenticity_token,#   'password': 'xxoo'
# }
#
# i2 = session.post('https://github.com/session',data=form_data)
# c2 = i2.cookies.get_dict()
# c1.update(c2)
# i3 = session.get('https://github.com/settings/repositories')
#
# soup3 = BeautifulSoup(i3.text,)
#     print(temp)

Python requests模組知乎自動登入

#!/usr/bin/env python
# -*- coding:utf-8 -*-
import time

import requests
from bs4 import BeautifulSoup

session = requests.Session()

i1 = session.get(
  url='https://www.zhihu.com/#signin',headers={
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/54.0.2840.98 Safari/537.36',}
)

soup1 = BeautifulSoup(i1.text,'lxml')
xsrf_tag = soup1.find(name='input',attrs={'name': '_xsrf'})
xsrf = xsrf_tag.get('value')

current_time = time.time()
i2 = session.get(
  url='https://www.zhihu.com/captcha.gif',params={'r': current_time,'type': 'login'},})

with open('zhihu.gif','wb') as f:
  f.write(i2.content)

captcha = input('請開啟zhihu.gif檔案，檢視並輸入驗證碼：')
form_data = {
  "_xsrf": xsrf,'password': 'xxooxxoo',"captcha": 'captcha','email': '[email protected]'
}
i3 = session.post(
  url='https://www.zhihu.com/login/email',}
)

i4 = session.get(
  url='https://www.zhihu.com/settings/profile',}
)

soup4 = BeautifulSoup(i4.text,'lxml')
tag = soup4.find(id='rename-section')
nick_name = tag.find('span',class_='name').string
print(nick_name)

Python requests模組部落格園自動登入

#!/usr/bin/env python
# -*- coding:utf-8 -*-
import re
import json
import base64

import rsa
import requests


def js_encrypt(text):
  b64der = 'MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCp0wHYbg/NOPO3nzMD3dndwS0MccuMeXCHgVlGOoYyFwLdS24Im2e7YyhB0wrUsyYf0/nhzCzBK8ZC9eCWqd0aHbdgOQT6CuFQBMjbyGYvlVYU2ZP7kG9Ft6YV6oc9ambuO7nPZh+bvXH0zDKfi02prknrScAKC0XhadTHT3Al0QIDAQAB'
  der = base64.standard_b64decode(b64der)

  pk = rsa.PublicKey.load_pkcs1_openssl_der(der)
  v1 = rsa.encrypt(bytes(text,'utf8'),pk)
  value = base64.encodebytes(v1).replace(b'\n',b'')
  value = value.decode('utf8')

  return value


session = requests.Session()

i1 = session.get('https://passport.cnblogs.com/user/signin')
rep = re.compile("'VerificationToken': '(.*)'")
v = re.search(rep,i1.text)
verification_token = v.group(1)

form_data = {
  'input1': js_encrypt('wptawy'),'input2': js_encrypt('asdfasdf'),'remember': False
}

i2 = session.post(url='https://passport.cnblogs.com/user/signin',data=json.dumps(form_data),headers={
           'Content-Type': 'application/json; charset=UTF-8','X-Requested-With': 'XMLHttpRequest','VerificationToken': verification_token}
         )

i3 = session.get(url='https://i.cnblogs.com/EditDiary.aspx')

print(i3.text)

Python requests模組拉勾網自動登入

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import requests


# 第一步：訪問登陸頁,拿到X_Anti_Forge_Token，X_Anti_Forge_Code
# 1、請求url:https://passport.lagou.com/login/login.html
# 2、請求方法:GET
# 3、請求頭:
#  User-agent
r1 = requests.get('https://passport.lagou.com/login/login.html',headers={
           'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/63.0.3239.132 Safari/537.36',},)

X_Anti_Forge_Token = re.findall("X_Anti_Forge_Token = '(.*?)'",r1.text,re.S)[0]
X_Anti_Forge_Code = re.findall("X_Anti_Forge_Code = '(.*?)'",re.S)[0]
print(X_Anti_Forge_Token,X_Anti_Forge_Code)
# print(r1.cookies.get_dict())
# 第二步：登陸
# 1、請求url:https://passport.lagou.com/login/login.json
# 2、請求方法:POST
# 3、請求頭:
#  cookie
#  User-agent
#  Referer:https://passport.lagou.com/login/login.html
#  X-Anit-Forge-Code:53165984
#  X-Anit-Forge-Token:3b6a2f62-80f0-428b-8efb-ef72fc100d78
#  X-Requested-With:XMLHttpRequest
# 4、請求體：
# isValidate:true
# username:15131252215
# password:ab18d270d7126ea65915c50288c22c0d
# request_form_verifyCode:''
# submit:''
r2 = requests.post(
  'https://passport.lagou.com/login/login.json',headers={
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML,'Referer': 'https://passport.lagou.com/login/login.html','X-Anit-Forge-Code': X_Anti_Forge_Code,'X-Anit-Forge-Token': X_Anti_Forge_Token,'X-Requested-With': 'XMLHttpRequest'
  },data={
    "isValidate": True,'username': '15131255089','password': 'ab18d270d7126ea65915c50288c22c0d','request_form_verifyCode': '','submit': ''
  },cookies=r1.cookies.get_dict()
)
print(r2.text)

更多關於Python requests模組基礎使用方法請檢視下面的相關連結