使用eventlet.GreenPool.spawn时，如何在主线程中引发异常(How can I raise exception in main thread when using eventlet

编程入门行业动态更新时间:2024-10-25 00:34:44

使用eventlet.GreenPool.spawn时，如何在主线程中引发异常(How can I raise exception in main thread when using eventlet.GreenPool.spawn)

我使用eventlet.GreenPool.spawn运行一些任务，然后等待所有greanthreads完成。我知道会引发异常 - 如何捕获该异常并将其抛入主线程？我很确定这很容易但是我完全错过了一些东西。

这是一个例子（失败了，我希望它成功）

import unittest
import eventlet


def broken_fetch(url):
    print " Raising exception "
    raise RuntimeError


class TestPool(unittest.TestCase):

    def test_error_is_bubbled_up(self):
        with self.assertRaises(RuntimeError):
            pool = eventlet.GreenPool(100)
            urls = ['http://google.com/', 'http://example.com/']
            for url in urls:
                pool.spawn(broken_fetch, url)
            pool.waitall()

if __name__ == '__main__':
    unittest.main()
 
 它的输出：  
> python errors.py
Raising exception 
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in fire_timers
    timer()
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
File "/usr/local/lib/python2.7/site-packages/eventlet/greenthread.py", line 192, in main
    result = function(*args, **kwargs)
File "errors.py", line 10, in broken_fetch
    raise RuntimeError
RuntimeError
Raising exception 
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in fire_timers
    timer()
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
File "/usr/local/lib/python2.7/site-packages/eventlet/greenthread.py", line 192, in main
    result = function(*args, **kwargs)
File "errors.py", line 10, in broken_fetch
    raise RuntimeError
RuntimeError
F
======================================================================
FAIL: test_error_is_bubbled_up (__main__.TestPool)
----------------------------------------------------------------------
Traceback (most recent call last):
File "errors.py", line 21, in test_error_is_bubbled_up
    pool.waitall()
AssertionError: RuntimeError not raised

----------------------------------------------------------------------
Ran 1 test in 0.003s

FAILED (failures=1)
I run some task using eventlet.GreenPool.spawn, then wait for all greanthreads to finish. I know there would be an exception raised - how can I catch that exception and throw it in a main thread? I am pretty sure it's easy however I'm totally missing something. 
Here is an example (which fails and I would like it to succeed) 
import unittest
import eventlet


def broken_fetch(url):
    print " Raising exception "
    raise RuntimeError


class TestPool(unittest.TestCase):

    def test_error_is_bubbled_up(self):
        with self.assertRaises(RuntimeError):
            pool = eventlet.GreenPool(100)
            urls = ['http://google.com/', 'http://example.com/']
            for url in urls:
                pool.spawn(broken_fetch, url)
            pool.waitall()

if __name__ == '__main__':
    unittest.main()
 
and it's output: 
> python errors.py
Raising exception 
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in fire_timers
    timer()
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
File "/usr/local/lib/python2.7/site-packages/eventlet/greenthread.py", line 192, in main
    result = function(*args, **kwargs)
File "errors.py", line 10, in broken_fetch
    raise RuntimeError
RuntimeError
Raising exception 
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 336, in fire_timers
    timer()
File "/usr/local/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 56, in __call__
    cb(*args, **kw)
File "/usr/local/lib/python2.7/site-packages/eventlet/greenthread.py", line 192, in main
    result = function(*args, **kwargs)
File "errors.py", line 10, in broken_fetch
    raise RuntimeError
RuntimeError
F
======================================================================
FAIL: test_error_is_bubbled_up (__main__.TestPool)
----------------------------------------------------------------------
Traceback (most recent call last):
File "errors.py", line 21, in test_error_is_bubbled_up
    pool.waitall()
AssertionError: RuntimeError not raised

----------------------------------------------------------------------
Ran 1 test in 0.003s

FAILED (failures=1)

                最满意答案
                
                    
                         除了KeyboardInterrupt和SystemExit之外，Eventlet的事件循环吞下所有异常。 检查各种eventlet 集线器的wait()实现。 以下摘录来自select hub的wait（） ：  
for listeners, events in ((readers, r), (writers, w)):
    for fileno in events:
        try:
            listeners.get(fileno, noop).cb(fileno)
        except self.SYSTEM_EXCEPTIONS:
            raise
        except:
            self.squelch_exception(fileno, sys.exc_info())
            clear_sys_exc_info()
 
 要解决此问题，您可以将异常信息作为值传递，然后在主线程中处理它。  
import unittest2
import sys
import eventlet


def broken_fetch(url):
    print " Raising exception "
    try:
        raise RuntimeError
    except:
        return sys.exc_info()

class TestPool(unittest2.TestCase):

    def test_error_is_bubbled_up(self):
        with self.assertRaises(RuntimeError):
            pool = eventlet.GreenPool(100)
            urls = ['http://google.com/', 'http://example.com/']
            for exc_info in pool.imap(broken_fetch, urls):
                if exc_info is not None:
                    exc_class, value, tb = exc_info
                    raise exc_class, value, tb

if __name__ == '__main__':
    unittest2.main()
 
 您可能希望关闭eventlet的DEBUG标志以防止eventlet打印吞下的异常。 当您想要在主线程中处理异常时，您可能不希望被重复的回溯打印混淆。  
import eventlet.debug
eventlet.debug.hub_exceptions(False)
Eventlet's event loop swallows all exceptions, barring KeyboardInterrupt and SystemExit. Check the wait() implementation of various eventlet hubs. The following excerpt is from select hub's wait(): 
for listeners, events in ((readers, r), (writers, w)):
    for fileno in events:
        try:
            listeners.get(fileno, noop).cb(fileno)
        except self.SYSTEM_EXCEPTIONS:
            raise
        except:
            self.squelch_exception(fileno, sys.exc_info())
            clear_sys_exc_info()
 
To work around this, you can pass the exception info as value and deal with it later in the main thread. 
import unittest2
import sys
import eventlet


def broken_fetch(url):
    print " Raising exception "
    try:
        raise RuntimeError
    except:
        return sys.exc_info()

class TestPool(unittest2.TestCase):

    def test_error_is_bubbled_up(self):
        with self.assertRaises(RuntimeError):
            pool = eventlet.GreenPool(100)
            urls = ['http://google.com/', 'http://example.com/']
            for exc_info in pool.imap(broken_fetch, urls):
                if exc_info is not None:
                    exc_class, value, tb = exc_info
                    raise exc_class, value, tb

if __name__ == '__main__':
    unittest2.main()
 
You may want to turn off eventlet's DEBUG flag to prevent eventlet from printing the swallowed exception. When you want to handle the exception in main thread, you probably don't want to get confused by the duplicated traceback print. 
import eventlet.debug
eventlet.debug.hub_exceptions(False)