Scrape Center爬虫平台之spa7案例

爬虫平台之spa7案例"/>

Scrape Center爬虫平台之spa7案例

其实我并没有真懂，其实我觉得直接复制人家的main.js就得了。简单、干脆、直接，我也就打了几个断点观察一下就撤了。如果直接上无脑selenium，name会出现乱码。

import requests
import re
def getHTMLText(url):try:r=requests.get(url,timeout=30)r.raise_for_status()r.encoding='utf-8'return r.text[17:2000]except:pass
url="/js/main.js"
html=getHTMLText(url)
print(html)
pattern=r"(?<=name: ')(.+?)(?=',)"
name_list=re.findall(pattern,html)
print(name_list)

运行结果：
{
name: ‘凯文-杜兰特’,
image: ‘durant.png’,
birthday: ‘1988-09-29’,
height: ‘208cm’,
weight: ‘108.9KG’
}, {
name: ‘勒布朗-詹姆斯’,
image: ‘james.png’,
birthday: ‘1984-12-30’,
height: ‘206cm’,
weight: ‘113.4KG’
}, {
name: ‘斯蒂芬-库里’,
image: ‘curry.png’,
birthday: ‘1988-03-14’,
height: ‘191cm’,
weight: ‘83.9KG’
}, {
name: ‘詹姆斯-哈登’,
image: ‘harden.png’,
birthday: ‘1989-08-26’,
height: ‘196cm’,
weight: ‘99.8KG’
}, {
name: ‘扬尼斯-安特托昆博’,
image: ‘antetokounmpo.png’,
birthday: ‘1994-12-06’,
height: ‘211cm’,
weight: ‘109.8KG’
}, {
name: ‘拉塞尔-威斯布鲁克’,
image: ‘westbrook.png’,
birthday: ‘1988-11-12’,
height: ‘191cm’,
weight: ‘90.7KG’
}, {
name: ‘凯里-欧文’,
image: ‘irving.png’,
birthday: ‘1992-03-23’,
height: ‘188cm’,
weight: ‘88.5KG’
}, {
name: ‘安东尼-戴维斯’,
image: ‘davis.png’,
birthday: ‘1993-03-11’,
height: ‘208cm’,
weight: ‘114.8KG’
}, {
name: ‘乔尔-恩比德’,
image: ‘embiid.png’,
birthday: ‘1994-03-16’,
height: ‘213cm’,
weight: ‘127.0KG’
}, {
name: ‘克雷-汤普森’,
image: ‘thompson.png’,
birthday: ‘1990-02-08’,
height: ‘198cm’,
weight: ‘97.5KG’
}, {
name: ‘考瓦伊-莱昂纳德’,
image: ‘leonard.png’,
birthday: ‘1991-06-29’,
height: ‘201cm’,
weight: ‘102.1KG’
}, {
name: ‘达米安-利拉德’,
image: ‘lillard.png’,
birthday: ‘1990-07-15’,
height: ‘188cm’,
weight: ‘88.5KG’
}, {
name: ‘卡梅罗-安东尼’,
image: ‘anthony.png’,
birthday: ‘1984-05-29’,
height: ‘203cm’,
weight: ‘108KG’
}, {
name: ‘尼科拉-约基奇’,
image: ‘jokic.png’,
birthday: ‘1995-02-19’,
height: ‘213cm’,
weight: ‘128.8KG’
}, {
name: ‘卡尔-安东尼-唐斯’,
image: ‘towns.png’,
birthday: ‘1995-11-15’,
height: ‘211cm’,
weight: ‘112.5KG’
}, {
name: ‘克里斯-保罗’,
image: ‘paul.png’,
birthday: ‘1985-05-06’,
height: ‘185cm’,
weight: ‘79.4KG’
}
[‘凯文-杜兰特’, ‘勒布朗-詹姆斯’, ‘斯蒂芬-库里’, ‘詹姆斯-哈登’, ‘扬尼斯-安特托昆博’, ‘拉塞尔-威斯布鲁克’, ‘凯里-欧文’, ‘安东尼-戴维斯’, ‘乔尔-恩比德’, ‘克雷-汤普森’, ‘考瓦伊-莱昂纳德’, ‘达米安-利拉德’, ‘卡梅罗-安东尼’, ‘尼科拉-约基奇’, ‘卡尔-安东尼-唐斯’, ‘克里斯-保罗’]