bs4——BeautifulSoup模組：解析網頁
阿新 • • 發佈：2018-11-27
解析由requests模組請求到的網頁
1 import requests 2 from bs4 import BeautifulSoup 3 headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/\ 4 537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36', 5 'Host':'movie.douban.com'} 6 link = 'https://movie.douban.com/top250' 7 r = requests.get(link, headers=headers,timeout=2)8 soup = BeautifulSoup(r.text, 'lxml') #以lxml格式解析網頁文字
BeautifulSoup模組有兩個查詢方法：
一個是：find()，返回符合條件的第一條內容
1 import requests 2 from bs4 import BeautifulSoup 3 headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/\ 4 537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36', 5 'Host':'movie.douban.com'} 6 link = 'https://movie.douban.com/top250' 7 r = requests.get(link, headers=headers,timeout=2) 8 soup = BeautifulSoup(r.text, 'lxml') #以lxml格式解析網頁文字 9 find_result = soup.find('div', class_='hd') 10 print(find_result)
下面是解析後的結果：
C:\python3.5\python.exe C:/Users/MR/Desktop/test.py<div class="hd"> <a class="" href="https://movie.douban.com/subject/1292052/"> <span class="title">肖申克的救贖</span> <span class="title"> / The Shawshank Redemption</span> <span class="other"> / 月黑高飛(港) / 刺激1995(臺)</span> </a> <span class="playable">[可播放]</span> </div>
另一個是：find_all()，以列表格式返回符合條件的所有內容
1 import requests 2 from bs4 import BeautifulSoup 3 headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/\ 4 537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36', 5 'Host':'movie.douban.com'} 6 link = 'https://movie.douban.com/top250' 7 r = requests.get(link, headers=headers,timeout=2) 8 soup = BeautifulSoup(r.text, 'lxml') #以lxml格式解析網頁文字 9 find_result = soup.find_all('div', class_='hd') 10 print(find_result)
下面是解析後結果：
注意返回的是列表
C:\python3.5\python.exe C:/Users/MR/Desktop/test.py [<div class="hd"> <a class="" href="https://movie.douban.com/subject/1292052/"> <span class="title">肖申克的救贖</span> <span class="title"> / The Shawshank Redemption</span> <span class="other"> / 月黑高飛(港) / 刺激1995(臺)</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1291546/"> <span class="title">霸王別姬</span> <span class="other"> / 再見，我的妾 / Farewell My Concubine</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1295644/"> <span class="title">這個殺手不太冷</span> <span class="title"> / Léon</span> <span class="other"> / 殺手萊昂 / 終極追殺令(臺)</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1292720/"> <span class="title">阿甘正傳</span> <span class="title"> / Forrest Gump</span> <span class="other"> / 福雷斯特·岡普</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1292063/"> <span class="title">美麗人生</span> <span class="title"> / La vita è bella</span> <span class="other"> / 一個快樂的傳說(港) / Life Is Beautiful</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1292722/"> <span class="title">泰坦尼克號</span> <span class="title"> / Titanic</span> <span class="other"> / 鐵達尼號(港 / 臺)</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1291561/"> <span class="title">千與千尋</span> <span class="title"> / 千と千尋の神隠し</span> <span class="other"> / 神隱少女(臺) / Spirited Away</span> </a> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1295124/"> <span class="title">辛德勒的名單</span> <span class="title"> / Schindler's List</span> <span class="other"> / 舒特拉的名單(港) / 辛德勒名單</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/3541415/"> <span class="title">盜夢空間</span> <span class="title"> / Inception</span> <span class="other"> / 潛行凶間(港) / 全面啟動(臺)</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/2131459/"> <span class="title">機器人總動員</span> <span class="title"> / WALL·E</span> <span class="other"> / 瓦力(臺) / 太空奇兵·威E(港)</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/3011091/"> <span class="title">忠犬八公的故事</span> <span class="title"> / Hachi: A Dog's Tale</span> <span class="other"> / 忠犬小八(臺) / 秋田犬八千(港)</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/3793023/"> <span class="title">三傻大鬧寶萊塢</span> <span class="title"> / 3 Idiots</span> <span class="other"> / 三個傻瓜(臺) / 作死不離3兄弟(港)</span> </a> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1292001/"> <span class="title">海上鋼琴師</span> <span class="title"> / La leggenda del pianista sull'oceano</span> <span class="other"> / 聲光伴我飛(港) / 一九零零的傳奇</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1291549/"> <span class="title">放牛班的春天</span> <span class="title"> / Les choristes</span> <span class="other"> / 歌聲伴我心(港) / 唱詩班男孩</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1292213/"> <span class="title">大話西遊之大聖娶親</span> <span class="title"> / 西遊記大結局之仙履奇緣</span> <span class="other"> / 西遊記完結篇仙履奇緣 / 齊天大聖西遊記</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1292064/"> <span class="title">楚門的世界</span> <span class="title"> / The Truman Show</span> <span class="other"> / 真人Show(港) / 真人戲</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1291841/"> <span class="title">教父</span> <span class="title"> / The Godfather</span> <span class="other"> / Mario Puzo's The Godfather</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1889243/"> <span class="title">星際穿越</span> <span class="title"> / Interstellar</span> <span class="other"> / 星際啟示錄(港) / 星際效應(臺)</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1291560/"> <span class="title">龍貓</span> <span class="title"> / となりのトトロ</span> <span class="other"> / 鄰居託託羅 / 鄰家的豆豆龍</span> </a> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/5912992/"> <span class="title">熔爐</span> <span class="title"> / 도가니</span> <span class="other"> / 無聲吶喊(港) / 漩渦</span> </a> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1307914/"> <span class="title">無間道</span> <span class="title"> / 無間道</span> <span class="other"> / Infernal Affairs / Mou gaan dou</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1849031/"> <span class="title">當幸福來敲門</span> <span class="title"> / The Pursuit of Happyness</span> <span class="other"> / 尋找快樂的故事(港) / 追求快樂</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/6786002/"> <span class="title">觸不可及</span> <span class="title"> / Intouchables</span> <span class="other"> / 閃亮人生(港) / 逆轉人生(臺)</span> </a> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/3319755/"> <span class="title">怦然心動</span> <span class="title"> / Flipped</span> <span class="other"> / 萌動青春 / 青春萌動</span> </a> <span class="playable">[可播放]</span> </div>, <div class="hd"> <a class="" href="https://movie.douban.com/subject/1300267/"> <span class="title">亂世佳人</span> <span class="title"> / Gone with the Wind</span> <span class="other"> / 飄</span> </a> <span class="playable">[可播放]</span> </div>]