芹菜:卡在无限重复的超时中(超时等待UP消息)

本文介绍了芹菜:卡在无限重复的超时中(超时等待UP消息)的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我定义了一些时间为1200的任务:

I defined some tasks with a time limit of 1200:

@celery.task(time_limit=1200) def create_ne_list(text): c = Client() return c.create_ne_list(text)

每次新进程启动时，我还使用 worker_process_init 信号进行一些初始化:

I'm also using the worker_process_init signal to do some initialization, each time a new process starts:

@worker_process_init.connect def init(sender=None, conf=None, **kwargs): init_system(celery.conf) init_pdf(celery.conf)

此初始化功能需要花费几秒钟来执行.

This initialization function takes several seconds to execute.

除此之外，我正在使用以下配置:

Besides that, I'm using the following configuration:

CELERY_RESULT_SERIALIZER = 'json' CELERY_TASK_SERIALIZER = 'json' CELERY_ACCEPT_CONTENT = ['json'] BROKER_URL = 'amqp://' CELERY_RESULT_BACKEND = 'amqp://' CELERY_TIMEZONE = 'Europe/Berlin' CELERY_ENABLE_UTC = True

并使用以下命令启动我的工作者:

and start my worker with the following command:

celery -A isc worker -l info --concurrency=3

如预期的那样，启动工作程序会导致初始化函数被调用3次.现在，我可以发送任务，并且它们正在执行，一切似乎都运行顺利.

As expected, starting the worker results in the initialization function being called three times. Now, I can send tasks and they are being executed and everything seems to run smoothly.

但是:一旦任务超过其时间限制，工作人员就会陷入无限的产卵循环中，并由于超过时间限制而再次被杀死.

BUT: As soon as a tasks exceeds its time limit, the worker gets caught in an infinite loop of spawning and being killed off again because of exceeding the time limit.

[2014-06-13 09:46:18,978: ERROR/MainProcess] Timed out waiting for UP message from <Worker(Worker-20381, started daemon)> [2014-06-13 09:46:20,000: ERROR/MainProcess] Process 'Worker-20381' pid:18953 exited with 'signal 9 (SIGKILL)' // new worker 20382 getting started, initialization getting triggerd and soon after that --> [2014-06-13 09:46:18,978: ERROR/MainProcess] Timed out waiting for UP message from <Worker(Worker-20382, started daemon)> [2014-06-13 09:46:20,000: ERROR/MainProcess] Process 'Worker-20382' pid:18954 exited with 'signal 9 (SIGKILL)' // and so on....

有人知道为什么会这样吗?

Does anyone has an idea why this is happening?

推荐答案

答案似乎是信号 worker_process_init 要求处理程序阻塞的时间不要超过4秒.

The answer seems to be that the signal worker_process_init requires the handler to not be blocking for more than 4 seconds.

celery.readthedocs/zh_CN/latest/userguide/signals.html#worker-process-init

由于我的init函数执行需要更长的时间，因此工作程序将自动终止.之后，它自然地重新启动并再次触发init函数，然后导致工作线程再次终止，依此类推.

Because my init function takes longer to execute, the worker will be terminated automatically. After that it naturally restarts and triggers the init function again, which then results in the worker being terminated again and so on.