带有 Python 请求的异步请求

本文介绍了带有 Python 请求的异步请求的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我尝试了请求文档中提供的示例库用于python.

I tried the sample provided within the documentation of the requests library for python.

使用async.map(rs)，我得到了响应代码，但我想得到请求的每个页面的内容.例如，这不起作用:

With async.map(rs), I get the response codes, but I want to get the content of each page requested. This, for example, does not work:

out = async.map(rs) print out[0].content

推荐答案

注意

以下答案不适用于请求 v0.13.0+.编写此问题后，异步功能已移至 grequests.但是，您可以将 requests 替换为下面的 grequests 并且它应该可以工作.

Note

The below answer is not applicable to requests v0.13.0+. The asynchronous functionality was moved to grequests after this question was written. However, you could just replace requests with grequests below and it should work.

我保留这个答案是为了反映关于使用请求的原始问题 <v0.13.0.

I've left this answer as is to reflect the original question which was about using requests < v0.13.0.

要使用 async.map 异步 执行多项任务，您必须:

To do multiple tasks with async.map asynchronously you have to:

为您想对每个对象(您的任务)执行的操作定义一个函数

将该函数作为事件挂钩添加到您的请求中

在所有请求/操作的列表上调用 async.map

示例:

from requests import async # If using requests > v0.13.0, use # from grequests import async urls = [ 'python-requests', 'httpbin', 'python-guide', 'kennethreitz' ] # A simple task to do to each response object def do_something(response): print response.url # A list to hold our things to do via async async_list = [] for u in urls: # The "hooks = {..." part is where you define what you want to do # # Note the lack of parentheses following do_something, this is # because the response will be used as the first argument automatically action_item = async.get(u, hooks = {'response' : do_something}) # Add the task to our list of things to do via async async_list.append(action_item) # Do our list of things to do via async async.map(async_list)