我正在尝试集成 tqdm 进度条,以监视在Python 3.5中由 aiohttp 生成的POST请求。我有一个正在运行的进度栏,但似乎无法使用 as_completed()收集结果。
I'm attempting to integrate a tqdm progress bar to monitor POST requests generated with aiohttp in Python 3.5. I have a working progress bar but can't seem to gather results using as_completed(). Pointers gratefully received.
我发现的示例建议使用以下模式,该模式与Python 3.5 async def 定义:
Examples I've found suggest using the following pattern, which is incompatible with Python 3.5 async def definitions:
for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(coros)): yield from f工作(尽管已编辑)异步代码但没有进度条的b $ b收益:
Working (albeit redacted) async code without the progress bar:
def async_classify(records): async def fetch(session, name, sequence): url = 'app.example/api/v0/search' payload = {'sequence': str(sequence)} async with session.post(url, data=payload) as response: return name, await response.json() async def loop(): auth = aiohttp.BasicAuth(api_key) conn = aiohttp.TCPConnector(limit=100) with aiohttp.ClientSession(auth=auth, connector=conn) as session: tasks = [fetch(session, record.id, record.seq) for record in records] responses = await asyncio.gather(*tasks) return OrderedDict(responses)这是我修改 loop()的失败尝试:
This is my unsuccessful attempt at modifying loop():
async def loop(): auth = aiohttp.BasicAuth(api_key) conn = aiohttp.TCPConnector(limit=100) with aiohttp.ClientSession(auth=auth, connector=conn) as session: tasks = [fetch(session, record.id, record.seq) for record in records] for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks)): await f responses = await asyncio.gather(f) print(responses)推荐答案
await f 返回单回应。为什么将已经完成的未来传递给 asyncio.gather(f)尚不清楚。
await f returns a single response. Why would you pass an already completed Future to asyncio.gather(f) is unclear.
尝试:
responses = [] for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks)): responses.append(await f)
Python 3.6实现 PEP 530-异步理解:
responses = [await f for f in tqdm.tqdm(asyncio.as_completed(tasks), total=len(tasks))]它在 async def 现在开始起作用。
更多推荐
使用tqdm的asyncio aiohttp进度栏
发布评论