无法使用beautifulsoup和请求进行网络抓取

本文介绍了无法使用beautifulsoup和请求进行网络抓取的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我正在尝试使用bs4以及来自此网站的请求来抓取前两个部分的值，即1 * 2和DOUBLECHANCE部分的值 web.bet9ja/Sport/SubEventDetail?SubEventID=76512106 我写的代码是:

I am trying to scrape the first two sections values i.e 1*2 and DOUBLECHANCE sections values using bs4 and requests from this website web.bet9ja/Sport/SubEventDetail?SubEventID=76512106 The code which I written is:

import bs4 as bs import urllib.request source = urllib.request.urlopen('web.bet9ja/Sport/SubEventDetail?SubEventID=76512106') soup = bs.BeautifulSoup(source,'lxml') for div in soup.find_all('div', class_='SEItem ng-scope'): print(div.text)

我跑步时什么也没得到，请任何人帮助我

when I run I am not getting anything please help me anyone

推荐答案

该页面是通过 JavaScript 加载的，因此您有2个选择.或使用 Selenium 或调用 Direct API .

The page is loaded via JavaScript, so you have 2 option. or to use selenium or to call the Direct API.

我直接使用 API 来获取所需的信息，而不是使用 Selenium .

Instead of using Selenium, I've called the API directly and got the required info.

有关 XHR&API <单击此处即可找到.

Further explanation about XHR & API < can be found once you click here.

import requests data = { 'IDGruppoQuota': '0', 'IDSottoEvento': '76512106' } def main(url): r = requests.post(url, json=data).json() count = 0 for item in r['d']['ClassiQuotaList']: count += 1 print(item['ClasseQuota'], [x['Quota'] for x in item['QuoteList']]) if count == 2: break main("web.bet9ja/Controls/ControlsWS.asmx/GetSubEventDetails")

输出:

1X2 ['3.60', '4.20', '1.87'] Double Chance ['1.83', '1.19', '1.25']