本文介绍了如何在scrapy中使用python请求?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我正在尝试使用 requests 来获取页面,然后将响应对象传递给解析器,但我遇到了一个问题:
def start_requests(self):产量 self.parse(requests.get(url))定义解析(自我,响应):#经过builtins.AttributeError: 'generator' 对象没有属性 'dont_filter'
解决方案您首先需要下载页面的响应,然后将该字符串转换为 HtmlResponse 对象
from scrapy.http import HtmlResponseresp = requests.get(url)response = HtmlResponse(url="", body=resp.text, encoding='utf-8')I am trying to use requests to fetch a page then pass the response object to a parser, but I ran into a problem:
def start_requests(self): yield self.parse(requests.get(url)) def parse(self, response): #pass
builtins.AttributeError: 'generator' object has no attribute 'dont_filter'
解决方案
You first need to download the page's resopnse and then convert that string to HtmlResponse object
from scrapy.http import HtmlResponse resp = requests.get(url) response = HtmlResponse(url="", body=resp.text, encoding='utf-8')
更多推荐
如何在scrapy中使用python请求?
发布评论