如何将数组从C传递到嵌入式python脚本

本文介绍了如何将数组从C传递到嵌入式python脚本的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我遇到了一些问题，希望获得帮助.我有一个片段代码，用于嵌入python脚本.这个python脚本包含一个函数，该函数将期望接收一个数组作为参数(在这种情况下，我正在python脚本内使用numpy数组). 我想知道如何将数组从C传递给嵌入式python脚本，作为脚本内函数的参数.更具体地说，有人可以向我展示一个简单的例子.

I am running to some problems and would like some help. I have a piece code, which is used to embed a python script. This python script contains a function which will expect to receive an array as an argument (in this case I am using numpy array within the python script). I would like to know how can I pass an array from C to the embedded python script as an argument for the function within the script. More specifically can someone show me a simple example of this.

推荐答案

实际上，最好的答案可能是专门使用numpy数组，甚至从C代码中也可以.但是，如果那不可能，那么您将遇到与在C类型和Python类型之间共享数据的任何代码相同的问题.

Really, the best answer here is probably to use numpy arrays exclusively, even from your C code. But if that's not possible, then you have the same problem as any code that shares data between C types and Python types.

通常，至少有五个选项可用于在C和Python之间共享数据:

In general, there are at least five options for sharing data between C and Python:

创建要传递的Python list或其他对象.

使用与您在Python中为序列对象定义的相同方法(__getitem__等)定义新的Python类型(在C代码中)以包装并表示数组.

将指向数组的指针投射到intptr_t或显式的ctypes类型，或者不进行铸造；然后在Python端使用ctypes进行访问.

将指向数组的指针投射到const char *并将其作为str传递(或在Py3中为bytes)，并在Python端使用struct或ctypes进行访问./li>

创建一个符合buffer协议的对象，然后再次在Python端使用struct或ctypes.

Create a Python list or other object to pass.

Define a new Python type (in your C code) to wrap and represent the array, with the same methods you'd define for a sequence object in Python (__getitem__, etc.).

Cast the pointer to the array to intptr_t, or to explicit ctypes type, or just leave it un-cast; then use ctypes on the Python side to access it.

Cast the pointer to the array to const char * and pass it as a str (or, in Py3, bytes), and use struct or ctypes on the Python side to access it.

Create an object matching the buffer protocol, and again use struct or ctypes on the Python side.

在您的情况下，您想在Python中使用numpy.array.因此，一般情况变为:

In your case, you want to use numpy.arrays in Python. So, the general cases become:

创建要通过的numpy.array.

(可能不合适)

按原样将指针传递到数组，然后从Python中使用ctypes将其转换为numpy可以转换为数组的类型.

将指向数组的指针投射到const char *并将其作为str(或者在Py3中为bytes)传递，该类型已经是numpy可以转换为数组的类型.

创建一个与buffer协议匹配的对象，我相信numpy可以直接转换.

Create a numpy.array to pass.

(probably not appropriate)

Pass the pointer to the array as-is, and from Python, use ctypes to get it into a type that numpy can convert into an array.

Cast the pointer to the array to const char * and pass it as a str (or, in Py3, bytes), which is already a type that numpy can convert into an array.

Create an object matching the buffer protocol, and which again I believe numpy can convert directly.

对于1，下面是使用list的方法，只是因为这是一个非常简单的示例(我已经写过它了……):

For 1, here's how to do it with a list, just because it's a very simple example (and I already wrote it…):

PyObject *makelist(int array[], size_t size) { PyObject *l = PyList_New(size); for (size_t i = 0; i != size; ++i) { PyList_SET_ITEM(l, i, PyInt_FromLong(array[i])); } return l; }

这是numpy.array的等效项(假设您可以依靠C array不被删除-请参阅在文档中创建数组以了解有关此处选项的更多详细信息):

And here's the numpy.array equivalent (assuming you can rely on the C array not to be deleted—see Creating arrays in the docs for more details on your options here):

PyObject *makearray(int array[], size_t size) { npy_int dim = size; return PyArray_SimpleNewFromData(1, &dim, (void *)array); }

无论如何，无论如何，您最终都会得到类似于C的PyObject *(并且具有单个引用计数)，因此您可以将其作为函数参数传递，而在Python端看起来像numpy.array，list，bytes或其他合适的东西.

At any rate, however you do this, you will end up with something that looks like a PyObject * from C (and has a single refcount), so you can pass it as a function argument, while on the Python side it will look like a numpy.array, list, bytes, or whatever else is appropriate.

现在，您实际上如何传递函数自变量?好吧，您在注释中引用的纯嵌入中的示例代码显示如何做到这一点，但并没有真正解释发生了什么.实际上，与嵌入文档相比，扩展文档中的解释更多，特别是从C调用Python函数.另外，请记住，标准库源代码充满了以下示例这(尽管由于优化，或者只是因为尚未对其进行更新以利用新的简化C API功能而使它们不那么可读).

Now, how do you actually pass function arguments? Well, the sample code in Pure Embedding that you referenced in your comment shows how to do this, but doesn't really explain what's going on. There's actually more explanation in the extending docs than the embedding docs, specifically, Calling Python Functions from C. Also, keep in mind that the standard library source code is chock full of examples of this (although some of them aren't as readable as they could be, either because of optimization, or just because they haven't been updated to take advantage of new simplified C API features).

跳过有关从Python获取Python函数的第一个示例，因为大概您已经拥有了它.第二个示例(及其相应的段落)展示了一种简单的方法:使用 Py_BuildValue .因此，假设我们要调用存储在myfunc中的函数，并使用上面的makelist函数返回的列表mylist.这是您的工作:

Skip the first example about getting a Python function from Python, because presumably you already have that. The second example (and the paragraph right about it) shows the easy way to do it: Creating an argument tuple with Py_BuildValue. So, let's say we want to call a function you've got stored in myfunc with the list mylist returned by that makelist function above. Here's what you do:

if (!PyCallable_Check(myfunc)) { PyErr_SetString(PyExc_TypeError, "function is not callable?!"); return NULL; } PyObject *arglist = Py_BuildValue("(o)", mylist); PyObject *result = PyObject_CallObject(myfunc, arglist); Py_DECREF(arglist); return result;

当然，如果确定有有效的可调用对象，则可以跳过可调用检查. (通常，最好检查一下何时首次获取myfunc，因为这样可以提供更早和更好的错误反馈.)

You can skip the callable check if you're sure you've got a valid callable object, of course. (And it's usually better to check when you first get myfunc, if appropriate, because you can give both earlier and better error feedback that way.)

如果您想真正了解正在发生的事情，请尝试不使用Py_BuildValue的情况.正如文档所说，[PyObject_CallObject][6]的第二个参数是一个元组，而PyObject_CallObject(callable_object, args)等效于apply(callable_object, args)，它等效于callable_object(*args).因此，如果您想在Python中调用myfunc(mylist)，则必须将其有效地转换为myfunc(*(mylist,))，以便将其转换为C.您可以像这样构造tuple:

If you want to actually understand what's going on, try it without Py_BuildValue. As the docs say, the second argument to [PyObject_CallObject][6] is a tuple, and PyObject_CallObject(callable_object, args) is equivalent to apply(callable_object, args), which is equivalent to callable_object(*args). So, if you wanted to call myfunc(mylist) in Python, you have to turn that into, effectively, myfunc(*(mylist,)) so you can translate it to C. You can construct a tuple like this:

PyObject *arglist = PyTuple_Pack(1, mylist);

但通常，Py_BuildValue更容易(尤其是如果您还没有将所有内容打包为Python对象)，并且代码中的意图更加清晰(就像使用PyArg_ParseTuple相比使用显式tuple在另一个方向上起作用).

But usually, Py_BuildValue is easier (especially if you haven't already packed everything up as Python objects), and the intention in your code is clearer (just as using PyArg_ParseTuple is simpler and clearer than using explicit tuple functions in the other direction).

那么，您如何获得该myfunc?好吧，如果您是从嵌入代码创建函数的，则只需保持指针不变即可.如果您希望它从Python代码传入，那正是第一个示例所做的.例如，如果要从模块或其他上下文中按名称查找，则用于具体类型的API，例如 PyModule 和抽象类型，例如 PyMapping 非常简单，即使将结果转换为难看的样板，通常也很明显如何将Python代码转换为等效的C代码.

So, how do you get that myfunc? Well, if you've created the function from the embedding code, just keep the pointer around. If you want it passed in from the Python code, that's exactly what the first example does. If you want to, e.g., look it up by name from a module or other context, the APIs for concrete types like PyModule and abstract types like PyMapping are pretty simple, and it's generally obvious how to convert Python code into the equivalent C code, even if the result is mostly ugly boilerplate.

将所有内容放在一起，假设我有一个C整数数组，我想import mymodule并调用一个返回int的函数mymodule.myfunc(mylist).这是一个简化的示例(未经实际测试，也没有错误处理，但应显示所有部分):

Putting it all together, let's say I've got a C array of integers, and I want to import mymodule and call a function mymodule.myfunc(mylist) that returns an int. Here's a stripped-down example (not actually tested, and no error handling, but it should show all the parts):

int callModuleFunc(int array[], size_t size) { PyObject *mymodule = PyImport_ImportModule("mymodule"); PyObject *myfunc = PyObject_GetAttrString(mymodule, "myfunc"); PyObject *mylist = PyList_New(size); for (size_t i = 0; i != size; ++i) { PyList_SET_ITEM(l, i, PyInt_FromLong(array[i])); } PyObject *arglist = Py_BuildValue("(o)", mylist); PyObject *result = PyObject_CallObject(myfunc, arglist); int retval = (int)PyInt_AsLong(result); Py_DECREF(result); Py_DECREF(arglist); Py_DECREF(mylist); Py_DECREF(myfunc); Py_DECREF(mymodule); return retval; }

如果您使用的是C ++，则可能需要研究某种范围保护器/管理员/等.来处理所有这些Py_DECREF调用，尤其是当您开始进行适当的错误处理后(通常意味着早期的return NULL调用会通过该函数进行处理).如果您使用的是C ++ 11或Boost，则可能只需要unique_ptr<PyObject, Py_DecRef>.

If you're using C++, you probably want to look into some kind of scope-guard/janitor/etc. to handle all those Py_DECREF calls, especially once you start doing proper error handling (which usually means early return NULL calls peppered through the function). If you're using C++11 or Boost, unique_ptr<PyObject, Py_DecRef> may be all you need.

但是，实际上，如果您打算进行大量C&-Python通讯，那么减少所有这些丑陋模板的更好方法是查看所有旨在改善扩展Python的熟悉框架- Cython ， boost :: python 等.即使您正在嵌入，您实际上也在做与扩展相同的工作，因此它们可以以相同的方式提供帮助.

But really, a better way to reduce all that ugly boilerplate, if you plan to do a lot of C<->Python communication, is to look at all of the familiar frameworks designed for improving extending Python—Cython, boost::python, etc. Even though you're embedding, you're effectively doing the same work as extending, so they can help in the same ways.

为此，如果您在文档周围搜索，其中的一些 也具有帮助嵌入部分的工具.例如，您可以使用C代码和Python代码以及cython --embed在Cython中编写您的主程序.您可能需要用手指交叉和/或牺牲一些鸡肉，但是如果可以的话，它简直就是简单而高效. Boost并不是很简单，但是一旦完成所有工作，几乎所有事情都以您期望的方式完成，并且可以正常工作，对于嵌入扩展.依此类推.

For that matter, some of them also have tools to help the embedding part, if you search around the docs. For example, you can write your main program in Cython, using both C code and Python code, and cython --embed. You may want to cross your fingers and/or sacrifice some chickens, but if it works, it's amazingly simple and productive. Boost isn't nearly as trivial to get started, but once you've got things together, almost everything is done in exactly the way you'd expect, and just works, and that's just as true for embedding as extending. And so on.