資訊在json檔案時的爬取方法-Excel檔案寫入
阿新 • • 發佈:2020-12-02
程式開發資源庫-明日科技 https://zyk.mingrisoft.com/Develop/view/id/2555/type/7/cid/49.html
一、爬取資訊
分析網頁:開啟網易雲課堂-----輸入python搜尋-----點選全部,得到下面網頁
https://study.163.com/courses-search?keyword=python#/?ot=5
下面開始資料提取
import requests payload ={ "activityId": 0, "keyword": "python", "orderType": 5, "pageIndex": 1, "pageSize": 50, "priceType": -1, "qualityType": 0, "relativeOffset": 0, "searchTimeType": -1, } # 偽造頭部資訊 headers = { "accept":"application/json", "content-type":"application/json", "origin":"https://study.163.com", "user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36" } response = requests.post(url=url,json=payload,headers=headers) content = response.json() if content and content["code"]==0: # 如果content存在,並且content的code屬性的內容為0 content_list = content["result"]["list"] # 獲取content的result的list print(content_list) lesson_1 = content_list[0] print(lesson_1) lesson_1_name=lesson_1["productName"] # 獲取課程1的名字 print(lesson_1_name)
二、儲存到Excel
import xlsxwriter workbook_XDD = xlsxwriter.Workbook("網易雲課堂Python課程資料.xlsx") # 建立excel worksheet = workbook_XDD.add_worksheet("first_sheet") worksheet.write(0,0,'商品id') # 寫在第1行第1列 worksheet.write(0,1,'課程id') # 寫在第1行第2列 worksheet.write(0,2,'商品名稱') # 寫在第1行第2列 workbook_XDD.close()