在Cython中使用字典，尤其是在nogil内部

本文介绍了在Cython中使用字典，尤其是在nogil内部的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我正在词典，

my_dict = {'a':[1,2,3], 'b':[4,5] , 'c':[7,1,2])

我想在Cython nogil函数中使用此词典.因此，我试图将其声明为

I want to use this dictionary inside a Cython nogil function . So , I tried to declare it as

cdef dict cy_dict = my_dict

到目前为止，还可以.

现在，我需要遍历my_dict的键，如果值在列表中，则对其进行遍历.在Python中，非常简单，如下所示:

Now I need to iterate over the keys of my_dict and if the values are in list, iterate over it. In Python , it is quite easy as follows:

for key in my_dict: if isinstance(my_dict[key], (list, tuple)): ###### Iterate over the value of the list or tuple for value in list: ## Do some over operation.

但是，在Cython内部，我也想在nogil内部实现相同的功能.由于nogil内不允许使用python对象，所以我都被困在这里.

But, inside Cython, I want to implement the same that too inside nogil . As, python objects are not allowed inside nogil, I am all stuck up here.

with nogil: #### same implementation of the same in Cython

有人可以帮我吗?

推荐答案

没有GIL，就不能使用Python dict，因为您可以做的所有事情都涉及到操作Python对象.您最明智的选择是接受您需要的GIL. C ++映射也有一个不太明智的选择，但是可能很难针对您的特定情况进行申请.

You can't use Python dict without the GIL because everything you could do with it involves manipulating Python objects. You most sensible option is to accept that you need the GIL. There's a less sensible option too involving C++ maps, but it may be hard to apply for your specific case.

您可以使用with gil:重新获取GIL.这里显然存在开销(使用GIL的部分无法并行执行，并且可能会等待GIL的延迟).但是，如果字典操作只是较大的Cython代码中的一小部分，这可能还不错:

You can use with gil: to reacquire the GIL. There is obvious an overhead here (parts using the GIL can't be executed in parallel, and there may be a delay which it waits for the GIL). However, if the dictionary manipulation is a small chunk of a larger piece of Cython code this may not be too bad:

with nogil: # some large chunk of computationally intensive code goes here with gil: # your dictionary code # more computationally intensive stuff here

另一个不太明智的选择是使用C ++映射(以及其他C ++标准库数据类型). Cython可以包装这些文件并自动将其转换.根据您的示例数据给出一个简单的示例:

The other less sensible option is to use C++ maps (along side other C++ standard library data types). Cython can wrap these and automatically convert them. To give a trivial example based on your example data:

from libcpp.map cimport map from libcpp.string cimport string from libcpp.vector cimport vector from cython.operator cimport dereference, preincrement def f(): my_dict = {'a':[1,2,3], 'b':[4,5] , 'c':[7,1,2]} # the following conversion has an computational cost to it # and must be done with the GIL. Depending on your design # you might be able to ensure it's only done once so that the # cost doesn't matter much cdef map[string,vector[int]] m = my_dict # cdef statements can't go inside no gil, but much of the work can cdef map[string,vector[int]].iterator end = m.end() cdef map[string,vector[int]].iterator it = m.begin() cdef int total_length = 0 with nogil: # all this stuff can now go inside nogil while it != end: total_length += dereference(it).second.size() preincrement(it) print total_length

(您需要使用language='c++'进行编译).

(you need to compile this with language='c++').

一个明显的缺点是，必须预先知道字典中的数据类型(它不能是任意的Python对象).但是，由于您不能在nogil块内操作任意Python对象，因此无论如何您都受到了很大的限制.

The obvious disadvantage to this is that the data-types inside the dict must be known in advance (it can't be an arbitrary Python object). However, since you can't manipulate arbitrary Python objects inside a nogil block you're pretty restricted anyway.