Celery Task中使用SocketIO进行服务器推送

本文介绍了Celery Task中使用SocketIO进行服务器推送的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我有一个 flask 应用程序，其中有许多长时间运行的异步任务(〜小时).将这些任务的状态与客户端进行通信很重要.

I have a flask application within which I have many long running asynchronous tasks (~hours). It's important that the state of these tasks is communicated with the client.

我使用 celery 来管理后台任务队列，并且我目前正尝试通过 socketIO 从每个后台线程向客户端广播更新.这可能吗?有没有更合适的策略可以实现我想要的目标?

I use celery to manage the background task queue, and I'm currently trying to broadcast updates to the client from each background thread via socketIO. Is this possible? Is there a better suited strategy to achieving what I would like?

推荐答案

您没有说，但是我想您打算使用Flask-SocketIO处理服务器端SocketIO，而不是官方的Node.js服务器，对吗?

You did not say, but I assume you plan on using Flask-SocketIO to handle the server-side SocketIO and not the official Node.js server, correct?

您可以做的事情是可以完成的，但是对于当前版本的Flask-SocketIO，问题在于托管Flask和Flask-SocketIO服务器的进程拥有与客户端的套接字连接，因此它是唯一的可以与他们交流的过程.目前，Flask-SocketIO在将数据从其他进程(例如Celery worker)发送到客户端方面没有提供任何帮助，您必须自己实现这一部分.专门针对Celery，您可以让长期运行的任务公开服务器进程可以提取并发送给客户端的进度信息.

What you want to do can be done, but with the current version of Flask-SocketIO, the problem is that the process that hosts the Flask and Flask-SocketIO server owns the socket connections with the clients, so it is the only process that can communicate with them. At this time, Flask-SocketIO does not offer any help in sending data to clients from other processes such as Celery workers, this part you have to implement yourself. Specifically for Celery, you can have your long running tasks expose progress information that the server process can pick up and send to the clients.

我目前正在对Flask-SocketIO进行改进，这将使任何进程都可以使用Redis发布/订阅后端将消息发送到已连接的客户端，以与Flask-SocketIO服务器进行通信.完成这项工作后，您将可以通过Celery进程透明地将数据写入任何客户端.

I am currently working on improvements to Flask-SocketIO that will enable any process to send messages to connected clients using a Redis pub/sub backend for communication to the Flask-SocketIO server. Once this work is completed you will be able to write data to any client transparently from your Celery process.

您还询问是否还有其他选择.您还应该考虑客户端可以轮询服务器的状态.如果不需要非常频繁地进行更新，则此选项将易于实施.客户端向服务器询问给定任务的状态，然后服务器向Celery任务询问.我在 Flask + Celery博客文章中展示了这种方法.

You also ask if there is another alternative. You should also consider that the client can poll the server for status. If the updates do not need to be very frequent, then this is an option that is going to be much easier to implement. The client asks the server for status for a given task, and the server in turn asks the Celery task. I showed this approach in my Flask+Celery blog article.