歌曲網站，教你爬取 mp3 和 lyric

阿新 • • 發佈：2020-11-30

從歌曲網站，獲取音訊和歌詞的流程：

1，輸入歌曲名，查詢網站中存在的歌曲 id
2，拿歌曲 id 下載歌詞 lyric

簡單的 url 拼接

3，拿歌曲 id 下載音訊 mp3

先用一個 POST 請求，拿 ID 取音訊資源路徑，

再用 GET 請求，拿到音訊資源

4 個網路請求，解決，

搜尋歌曲，獲取歌詞，獲取音訊資源路徑，獲取音訊資源

注意的是，4 個網路請求，都要模擬正常的瀏覽器請求，

GET 請求，需要配置請求頭，
POST 請求，需要配置請求頭和請求體

1，查詢網站的歌曲

先準備，模擬正常的瀏覽器請求

配置 Session，

有一個加解密，具體見 github repo.

def __init__(self, timeout=60, cookie_path='.'):
        self.headers = {
            'Accept': '*/*',
            'Accept-Encoding': 'gzip,deflate,sdch',
            'Accept-Language': 'zh-CN,zh;q=0.8,gl;q=0.6,zh-TW;q=0.4',
            'Connection': 'keep-alive',
            'Content-Type': 'application/x-www-form-urlencoded',
            'Host': 'music.x.com',
            'Referer': 'http://music.x.com/search/',
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
        }
        self.session = requests.Session()
        self.session.headers.update(self.headers)
        self.session.cookies = cookiejar.LWPCookieJar(cookie_path)
        self.download_session = requests.Session()
        self.timeout = timeout
        self.ep = Encrypyed()

封裝 Post 請求方法


    def post_request(self, url, params):
        """
        Post請求
        :return: 字典
        """

        data = self.ep.encrypted_request(params)
        resp = self.session.post(url, data=data, timeout=self.timeout)
        result = resp.json()
        if result['code'] != 200:
            click.echo('post_request error')
        else:
            return result

去搜索：

    def search(self, search_content, search_type, limit=9):
        """
        搜尋API
        :params search_content: 搜尋內容
        :params search_type: 搜尋型別
        :params limit: 返回結果數量
        :return: 字典.
        """

        url = 'http://music.x.com/weapi/xxx/get/web?csrf_token='
        params = {'s': search_content, 'type': search_type, 'offset': 0, 'sub': 'false', 'limit': limit}
        result = self.post_request(url, params)
        return result

拿到搜尋結果：


        result = self.search(song_name, search_type=1, limit=limit)

        if result['result']['songCount'] <= 0:
            click.echo('Song {} not existed.'.format(song_name))
        else:
            songs = result['result']['songs']
            if quiet:
                song_id, song_name = songs[0]['id'], songs[0]['name']
                song = Song(song_id=song_id, song_name=song_name, song_num=song_num)
                return song

下載歌詞

下載很簡單

        lyricUrl = 'http://music.x.com/api/song/lyric/?id={}&lv=-1&csrf_token={}'.format(song_id, csrf)
        lyricResponse = self.session.get(lyricUrl)

拿到一個 json ，獲取裡面的歌詞，

        lyricJSON = lyricResponse.json()
        lyrics = lyricJSON['lrc']['lyric'].split("\n")
        lyricList = []
        for word in lyrics:
            time = word[1:6]
            name = word[11:]
            p = Node(time, name)
            lyricList.append(p)
        json_string = json.dumps([node.__dict__ for node in lyricList], ensure_ascii = False, indent = 4)

寫入新建的本地檔案

        if not os.path.exists(folder):
            os.makedirs(folder)
        fpath = os.path.join(folder, str(song_num) + '_' + song_name + '.json')
        text_file = open(fpath, "w")
        n = text_file.write(json_string)
        text_file.close()

下載音訊分兩步

先拿到音訊資源路徑

        url = 'http://music.x.com/weapi/song/enhance/player/url?csrf_token='
        csrf = ''
        params = {'ids': [song_id], 'br': bit_rate, 'csrf_token': csrf}
        result = self.post_request(url, params)
         # 歌曲下載地址
        song_url = result['data'][0]['url']

        # 歌曲不存在
        if song_url is None:
            click.echo('Song {} is not available due to copyright issue.'.format(song_id))
        else:
            return song_url

再獲取音訊資源

        if not os.path.exists(fpath):
            resp = self.download_session.get(song_url, timeout=self.timeout, stream=True)
            length = int(resp.headers.get('content-length'))
            label = 'Downloading {} {}kb'.format(song_name, int(length/1024))

一邊下載，一邊看進度

           with click.progressbar(length=length, label=label) as progressbar:
                with open(fpath, 'wb') as song_file:
                    for chunk in resp.iter_content(chunk_size=1024):
                        if chunk:
                            song_file.write(chunk)
                            progressbar.update(1024)

交流基地：630390733