價值10K+的Python面試題，珍藏已久，分享給大家！
阿新 • • 發佈：2022-03-15
之前一哥們在國外面試的時候，就是要他做的這個，直接給他說，做出來了給你15K，做不出來就拜拜~
大兄弟當時就不服了，這不是看不起我麼，分分鐘就給整完了~
嘮嘮叨叨
那我們直接開整，像我們練手的話，裝好Python和pycharm就OK了，沒安裝的話先安裝好，這裡我就不寫了。
1、主要知識點
- 爬蟲基本流程
- 非結構化資料解析
- 表格型別資料儲存
2、第三方庫安裝的幾種方法
requests
parsel
直接pip安裝即可
1)、cmd中使用pip命令安裝
win+r開啟搜尋出入cmd 按回車確認
然後在彈出來的命令提示符視窗輸入pip安裝指令加上模組，比如安裝requests , 輸入 pip install requests 按回車即可 。
2)、在pycharm設定裡安裝
3)、在pycharm使用pip命令安裝
輸入 pip install requests 按回車即可
程式碼部分
1、程式碼實現流程
- 傳送請求(訪問網站)
- 獲取資料(網頁原始碼)
- 解析資料(提取我們想要的資料內容 詳情頁連結)
- 傳送請求(訪問所有的詳情頁連結) 獲取資料
- 解析資料(提取到我們想要的資料內容)
- 儲存資料
2、程式碼展示
匯入模組
import requests import parsel import re import csv
偽裝
headers = {'cookie': 'TADCID=foOmU9bDp6JGIXg2ABQCFdpBzzOuRA-9xvCxaMyI12wTEaQSQ4euq_1sNSDmJybFCMezFLrAnKRGZ_uvGNNO_9cSzuJeK8RQlE4; TAUnique=%1%enc%3AHARC1EMLan58P07MI4ZMcqI%2BzHGWuLGBt6TE6zQDNwk%3D; TASSK=enc%3AAL%2Bm9xwFy7%2BjYONIRS%2F2kEbA%2FtOrlDbcW%2FwCSHs44XP9R3ddE%2BKJxi3FiDuozLe0Ov2ujtnFah8i0sN%2FRdUxZGis0TClwsaz7%2B7Uv8dh%2BvHM%2FfH9C%2FcEYLBYBtn1yLmBNg%3D%3D; ServerPool=A; PMC=V2*MS.2*MD.20220311*LD.20220311; TART=%1%enc%3AfD9OzCOGTHLKxR1qLNfmGZurd9xliidHT5bmQw2z505WnDQeBJdPDWc64WFlxikpNox8JbUSTxk%3D; TATravelInfo=V2*A.2*MG.-1*HP.2*FL.3*RS.1; TASID=9CCF4EA45B4141A8B5E4F03D36821474; ak_bmsc=31083286436C157F558D959D23D94849~000000000000000000000000000000~YAAQqF1kX6lPsVF/AQAAhTyqdw8F4+OoWZwjJCqsKUS/ykkFQHkXml5We7WY4q6KDUeIkm36a0Fs41jt7Jx6MFwnzloND2Iry1Iuwnj5I7oPxsI1RTjfGXSr408rscnzKPJHpRIXwuuiL+SNZxp233DOhrqrbTQ2cDTiGPk8qAYcLYq1OHpyOjLpc6L2zPbiSdvfDAuz2ujLUbWZV33YVrUd1UcmBMKJOSS/C12JeFdLCcjOihJvc4Zlu5HMYQUBdjTaV4zll3YO9YWxdm5pUT57vjI3WjxNhLwOXS93F3ogo/VOzmvk2n4rptCDH1vffz7Dpmp4yRn0dnX8RtiKiolFV00rBs0yC9Nxa67F0qPkJMMS6t6pNo+08PIre7VIiAIxQoWUNNiBiNDXeQ==; PAC=AHc5Ocqizh5jbN81AnjCtcF7k5P54vojrezhxeu8s4DdhkIZSMBuxXUioaVGVVo99Ysr_IbYXqNKjsddfzI8psluCp1NwuwQiBOvmdhP_r8ntVPeHXBc5u782Y8i4KrpV0a29aTnmykzihOxeEfilEfHZOGZxkWN8GRLwHay1MUpBazo7e4Pdtl3tndoYnNIDWcRtHzZJIDE9odWhqOzUE0%3D; TAReturnTo=%1%%2FRestaurants-g188590-Amsterdam_North_Holland_Province.html; roybatty=TNI1625!AJyUZ5ejQVombB9Jv3PVhqqhyMhwsanzT2C6omYz8l6mQNt%2FP5v6CLnnlymNXfhMwolnHznm%2BAmT81YSeygcVxnWHERn16eR747rX9fmWmeCMoris6ffxKTbJ6%2BjObZ6rmffv7I5wEGZ009WzKMlVA%2BXJAheGoIKHOD3gUDLVYlY%2C1; TATrkConsent=eyJvdXQiOiIiLCJpbiI6IkFMTCJ9; TASession=V2ID.9CCF4EA45B4141A8B5E4F03D36821474*SQ.9*LS.PageMoniker*GR.82*TCPAR.12*TBR.1*EXEX.98*ABTR.74*PHTB.27*FS.67*CPU.8*HS.recommended*ES.popularity*DS.5*SAS.popularity*FPS.oldFirst*LF.en*FA.1*DF.0*TRA.false*LD.188590*EAU._; TAUD=LA-1646980142821-1*RDD-1-2022_03_11*LG-863371-2.1.F.*LD-863372-.....; _pbjs_userid_consent_data=3524755945110770; _li_dcdm_c=.tripadvisor.com; _lc2_fpi=b140173de591--01fxvvhm5q52dte42gshbn1234; __gads=ID=887c76ae8964a5bc:T=1646981079:S=ALNI_MYwTZNsJPdidCGF3BTM3pOV79wAUg; _lr_sampling_rate=100; _lr_retry_request=true; _lr_env_src_ats=false; __li_idex_cache=%7B%7D; pbjs_li_nonid=%7B%7D; __vt=bI5Nl4_3wIiyQqd-ABQCIf6-ytF7QiW7ovfhqc-AvRvwyUuxl21BvNUgBcewLtYtxhD9pK8plYHHUPpFuGJQzlL9HjsNiQXGwLu0f-XidRXohA9m08ary-La12XkjuKCU2QeR3ijnhWjQ8bnjvOcAaUKoA; bm_sv=867C80B13B2E8AE707E1A411B950E849~HDnKV8jbSFu9eHNiLb/p3fK3KqcxdMjPpLXFMD9YvvwLoQEuDGPgZZwEDhQeezJZJhdrUxX02mvzmDqkV7615Fm508wASvLcLsXmW/6+1K9pDp2UuCDIYbuZgv/2m76YS7Og/SBcU6xkIVnHhMVqpxWfro/1T3kO1LdXuFuprhA=; OptanonConsent=isGpcEnabled=0&datestamp=Fri+Mar+11+2022+14%3A53%3A51+GMT%2B0800+(%E4%B8%AD%E5%9B%BD%E6%A0%87%E5%87%86%E6%97%B6%E9%97%B4)&version=6.30.0&isIABGlobal=false&hosts=&consentId=cc7e2f72-5007-428f-a72e-392f9741b69d&interactionCount=1&landingPath=https%3A%2F%2Fwww.tripadvisor.com%2FRestaurants-g188590-Amsterdam_North_Holland_Province.html&groups=C0001%3A1%2CC0002%3A1%2CC0003%3A1%2CC0004%3A1', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36', }
傳送請求(訪問網站)
response = requests.get(url, headers=headers) # Python學習交流群 815624229
獲取資料(網頁原始碼)
html_data = response.text
解析資料(提取我們想要的資料內容 詳情頁連結)
selector = parsel.Selector(html_data)
提取標籤的屬性內容 ::attr(href) 連結
link_list = selector.css('.bHGqj.Cj.b::attr(href)').getall() for link in link_list: link = 'tripadvisor/' + link
傳送請求(訪問所有的詳情頁連結) 獲取資料
detail_html = requests.get(link, headers=headers).text
解析資料
detail_selector = parsel.Selector(detail_html) store_name = detail_selector.css('.fHibz::text').get() comment_count = detail_selector.css('.eSAOV.H3:nth-child(2) .eBTWs::text').get() address = detail_selector.css('.eSAOV.H3:nth-child(3) .dyeJW.dUpPX:nth-child(1) .fhGHT::text').get() city = detail_selector.css('.breadcrumbs li:nth-child(4) span::text').get() phone = detail_selector.css('.eSAOV.H3:nth-child(3) .dyeJW.dUpPX:nth-child(2) .fhGHT a::text').get() score = detail_selector.css('.eEwDq .fdsdx::text').get() website = re.findall(',"website":"(http.*?)"', detail_html)[0] print(store_name, comment_count, city, address, phone, score, link, website)
儲存資料
with open('tripadvisor.csv', mode='a', newline='', encoding='utf-8') as f: csv_writer = csv.writer(f) csv_writer.writerow([store_name, comment_count, city, address, phone, score, link, website])
翻頁
for page in range(0, 131, 30): print(f'-------------------正在爬取第{page+1}頁-------------------') url = f'tripadvisor/RestaurantSearch?Action=PAGE&ajax=1&availSearchEnabled=true&sortOrder=popularity&geo=188590&itags=10591&eaterydate=2022_03_11&date=2022-03-12&time=20%3A00%3A00&people=2&o=a{page}'
地址我都遮蔽了，大家自己補全一下www. .com
兄弟們，幫我動動小手，點個贊+收藏，還可以順便評論一下，下次給大家弄個Python 30k 崗位的面試題補充一下點贊花掉的體力，嘿嘿~