在Python中使用Web报废抓取td元素(Grabbing td element under web scrapping in Python)

在Python中使用Web报废抓取td元素(Grabbing td element under web scrapping in Python)

我试图用JS元素捕获元素，如下所示

<td align="right" valign="top" class="tabletext2" nowrap="nowrap"> <strong>Program Element Code(s):</strong></td>

网站是http://www.nsf.gov/awardsearch/showAward?AWD_ID=1227110&HistoricalAwards=false

python脚本如下所示

i=1300138; i=str(i); url= "http://www.nsf.gov/awardsearch/showAward?AWD_ID="+i+"&HistoricalAwards=false"; r = requests.get (url) #webbrowser.open(url,new =new ); soup = BeautifulSoup(urllib2.urlopen(url).read()) sp=BeautifulSoup(r.content) gd=sp.findAll('td',{'class':'tabletext2'},nowrap="nowrap") for item in gd: print item.text; if item.text=="Program Element Code(s):": print item.contents;

但是我无法让它发挥作用。 我需要在程序参考代码前面获取ID。任何帮助表示赞赏。 谢谢

Iam trying to catch the elements with a JS element as below

<td align="right" valign="top" class="tabletext2" nowrap="nowrap"> <strong>Program Element Code(s):</strong></td>

Website is http://www.nsf.gov/awardsearch/showAward?AWD_ID=1227110&HistoricalAwards=false

The python script looks like as below

i=1300138; i=str(i); url= "http://www.nsf.gov/awardsearch/showAward?AWD_ID="+i+"&HistoricalAwards=false"; r = requests.get (url) #webbrowser.open(url,new =new ); soup = BeautifulSoup(urllib2.urlopen(url).read()) sp=BeautifulSoup(r.content) gd=sp.findAll('td',{'class':'tabletext2'},nowrap="nowrap") for item in gd: print item.text; if item.text=="Program Element Code(s):": print item.contents;

But I cant get it to work . I need to grab the IDs in front of Program Reference Code(s) Any help is appreciated . Thanks

最满意答案

一种方法是在正确的"class":"tabletext2"之后获取下一个td "class":"tabletext2" ：

url= "http://www.nsf.gov/awardsearch/showAward?AWD_ID=1227110&HistoricalAwards=false" import requests from bs4 import BeautifulSoup r = requests.get(url) tds = BeautifulSoup(r.content).find_all("td",{"class":"tabletext2"}) print([td.find_next("td").text.strip() for td in tds if td.text.startswith("Program Reference Code(s)")]) [u'131E, 113E, 8048, 7433']

One way is to grab the next td after the correct "class":"tabletext2":

url= "http://www.nsf.gov/awardsearch/showAward?AWD_ID=1227110&HistoricalAwards=false" import requests from bs4 import BeautifulSoup r = requests.get(url) tds = BeautifulSoup(r.content).find_all("td",{"class":"tabletext2"}) print([td.find_next("td").text.strip() for td in tds if td.text.startswith("Program Reference Code(s)")]) [u'131E, 113E, 8048, 7433']