目标
这次要爬取的是qq音乐网站【排行榜】中所有的歌曲(共100首)
网址:https://y.qq/n/yqq/toplist/4.html#stat=y_new.top.pop.logout
废话不多说,直接开始吧!
接口分析
点击排行榜页面中的任意一首歌,进入到播放界面。
打开开发者工具,重新刷新网页,寻找歌曲下载接口
找呀找,找呀找
发现许多media类型的,选择size最大的那条,点击进去。
果然,找到了歌曲下载的url:
http://124.232.155.149/amobile.music.tc.qq/C400101A9o5D1EWzyd.m4a?guid=5300073565&vkey=30F12E95C394B1C0550AAAC8E946191E757BB8190B983FA1E13E7C0EEF6CE49AC1C051CB2922C371F7EBA77B8C36DA7F0BB3D85819525C75&uin=0&fromtag=66
然后,分析下这个url的构造:
和其他的歌曲对比,只有m4a文件名和vkey这两个参数不一致
所以,我们要找的就是这两个参数。
那么到底从何而来呢?
又找啊找,找啊找
终于发现了!找到了vkey!
接着找m4a文件名
找啊找,找啊找
咦!好像有新发现!
purl中好像已经包含了m4a文件名和vkey!
没错,就是purl了!
实际上,歌曲下载的url就是 http://124.232.155.149/amobile.music.tc.qq/+purl
接下来,就是要分析purl是怎么来的了
分析请求的url:
还是老办法,和其他的歌曲对比。(此处省略复杂繁琐对比。。。)
发现只有songmid这个参数不一致
所以,要分析的就是songmid是从哪个接口过来的。
再次回到排行榜页面
找啊找 找了很久 终于发现了!
最后,再分析下songmid的请求url:
https://c.y.qq/v8/fcg-bin/fcg_v8_toplist_cp.fcg?tpl=3&page=detail&date=2018-12-14&topid=4&type=top&song_begin=0&song_num=30&g_tk=5381&jsonpCallback=MusicJsonCallbacktoplist&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0
song_num是指定请求多少条数据。
由于我们要下载所有的歌曲(共100首),所以要将song_num改成100
完整代码
#目标:下载qq音乐排行榜前100首
#思路:
#1.歌曲下载链接url1:http://124.232.144.154/amobile.music.tc.qq/C400101A9o5D1EWzyd.m4a?guid=5300073565&vkey=B1A12B2E4190CB21134F652893BEEF69AE405D39E060DEEACF358A6F729541E160F926F5EADD391C68047B2ED0156690A97CA50AC99DA40D&uin=0&fromtag=66
#2.寻找下载链接中的参数filname:C400101A9o5D1EWzyd.m4a vkey
#3.filname与vkey的请求链接url2:https://u.y.qq/cgi-bin/musicu.fcg?callback=getplaysongvkey2904725697971273&g_tk=5381&jsonpCallback=getplaysongvkey2904725697971273&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0&data=%7B%22req%22%3A%7B%22module%22%3A%22CDN.SrfCdnDispatchServer%22%2C%22method%22%3A%22GetCdnDispatch%22%2C%22param%22%3A%7B%22guid%22%3A%225300073565%22%2C%22calltype%22%3A0%2C%22userip%22%3A%22%22%7D%7D%2C%22req_0%22%3A%7B%22module%22%3A%22vkey.GetVkeyServer%22%2C%22method%22%3A%22CgiGetVkey%22%2C%22param%22%3A%7B%22guid%22%3A%225300073565%22%2C%22songmid%22%3A%5B%22001A9o5D1EWzyd%22%5D%2C%22songtype%22%3A%5B0%5D%2C%22uin%22%3A%220%22%2C%22loginflag%22%3A1%2C%22platform%22%3A%2220%22%7D%7D%2C%22comm%22%3A%7B%22uin%22%3A0%2C%22format%22%3A%22json%22%2C%22ct%22%3A20%2C%22cv%22%3A0%7D%7D
#4.url2的构造:songmid
#5.寻找songmid:
# songmid请求的链接url3:https://c.y.qq/v8/fcg-bin/fcg_v8_toplist_cp.fcg?tpl=3&page=detail&date=2018-12-14&topid=4&type=top&song_begin=0&song_num=30&g_tk=5381&jsonpCallback=MusicJsonCallbacktoplist&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0
import requests
import re
import json
from multiprocessing.dummy import Pool as ThreadPool
#返回songmid和songname
def get_param():
url = 'https://c.y.qq/v8/fcg-bin/fcg_v8_toplist_cp.fcg?tpl=3&page=detail&date=2018-12-14&topid=4&type=top&song_begin=0&song_num=100&g_tk=5381&jsonpCallback=MusicJsonCallbacktoplist&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0'
response = requests.get(url).content.decode('utf-8')
purl_data = re.findall(r'\w+\((.*)',response)[0]
data_json = json.loads(purl_data)
songlist = data_json['songlist']
param_list = []
for item in songlist:
param_dict = {}
songmid = item['data']['songmid']
songname = item['data']['songname']
param_dict['songmid'] = songmid
param_dict['songname'] = songname
param_list.append(param_dict)
return param_list
#下载音乐
def download_music(param_dict):
songmid = param_dict['songmid']
songname = param_dict['songname']
data =json.dumps({"req": {"module": "CDN.SrfCdnDispatchServer","method": "GetCdnDispatch","param": {"guid": "5300073565","calltype": 0,"userip": ""}},"req_0": {"module": "vkey.GetVkeyServer","method": "CgiGetVkey","param": {"guid": "5300073565","songmid": [songmid],"songtype": [0],"uin": "0","loginflag": 1,"platform": "20"}},"comm": {"uin": 0,"format": "json","ct": 20,"cv": 0}})
url = 'https://u.y.qq/cgi-bin/musicu.fcg?callback=getplaysongvkey2904725697971273&g_tk=5381&jsonpCallback=getplaysongvkey2904725697971273&loginUin=0&hostUin=0&format=jsonp&inCharset=utf8&outCharset=utf-8¬ice=0&platform=yqq&needNewCode=0&data={}'.format(data)
response = requests.get(url).content.decode('utf-8')
purl_data = re.findall(r'\w+\((.*)\)',response)[0]
data_json = json.loads(purl_data)
purl = data_json['req_0']['data']['midurlinfo'][0]['purl']
download_url = 'http://124.232.144.154/amobile.music.tc.qq/{}'.format(purl)
print("=======正在下载:{}=======".format(songname))
music = requests.get(download_url)
with open("F:\BaiduNetdiskDownload\QQmusic\music\{}.m4a".format(re.sub(r'[\s+|@<>:\\"/]','',songname)),"wb") as m:
m.write(music.content)
if __name__ == '__main__':
param_list = get_param()
pool = ThreadPool(5)
pool.map(download_music,param_list)
pool.close()
pool.join()
运行效果
下载完的歌曲:
现在终于可以听歌了!
听着听着。。。发现有个别歌曲文件大小居然是0KB
原来,VIP音乐没下载下来。
更多推荐
python爬虫-批量下载qq音乐
发布评论