Distutils 显然因(工作)SWIG 扩展而失败

本文介绍了Distutils 显然因(工作)SWIG 扩展而失败的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我通过 SWIG 将 C 库包装在一个 python 模块中，这里称为myExample".如果我编译:

$swig -python myExample.i$gcc -c myExample_wrap.c -I/usr/lib/python2.7 -fPIC -std=c99$ld -shared myExample_wrap.so -llapacke -o _myExample.so

我获得了一个完整的工作模块(我使用的某些功能需要 liblapacke).现在我想通过pip install"安装这个模块.

根据 distutils 部分(docs.python/2.7/distutils/setupscript.html)，我写了我的 setup.py 文件:

from distutils.core 导入设置，扩展设置(名称='我的例子'，版本='0.1'，ext_modules=[Extension('_myExample',['myExample.i'], libraries= ['lapacke'])] )

并以某种方式编辑 MANIFEST.in 以保留来源并避免出现类似本网站上类似问题的问题(即仅包含 myExample.h 和 myExample.c).然后我跑:

$python setup.py sdist

并通过pip install"获得可安装的包.它似乎完成了(没有错误，没有警告)，但是...它不起作用.在这个可安装的模块中(_myExample.so" - 注意下划线，它似乎是 distutils 所要求的 [也许它隐藏了答案?])有些方法不同，有些方法缺失等等......因此我决定每次一步.通过编译:

$python setup.py build_ext

我已经遇到了同样的问题:最终的模块与通过开头解释的通常编译获得的模块不同.

总结:给定一个 SWIG 接口，传统编译它或通过 distutils 编译它会产生不同的结果.怎么可能?我的 setup.py 错了吗?是否有另一种方法可以在不依赖 distutils 或 setuptools(产生相同问题)的情况下获取 pip 可安装模块?

Ps:包装代码很长，因此很遗憾我无法提供详细列表，但我完全可以在需要时添加更多内容.例如，手动编译的接口成功包含AdaptiveInterpolation"(工作正常)，而distutil 生产的接口有AdaptiveInterpolation_set"、AdaptiveInterpolation_get"，或者有很多以new_"开头的方法(在我的原版中没有)代码).

解决方案

实际上，分发此类包有两种选择:bdist_wheel 和 sdist.

让我们以 distutils.

example.h

int fact(int n);

example.c

#include "example.h"事实上(int n){如果(n == 0){返回 1；}别的 {返回 n * 事实(n - 1)；}}

example.i

%module 示例%{#define SWIG_FILE_WITH_INIT#include "example.h"%}事实上(int n)；

让我们构建一个 SWIG 接口，它默认生成 example_wrap.c.

swig -python example.i

轮子

Python Wheels 是一种现代分发格式，它简化了预构建包的分发(即包的用户不需要编译器、开发头文件等来安装它).要使用它，我们需要 setuptools 和 wheel(可以使用 pip 或从操作系统存储库安装，sudo apt-get installpython-wheel python-setuptools).

setup.py

from setuptools import setup, Extension设置(名称 = '例子',版本 = '0.1',作者 = 'SWIG 文档',description = '来自文档的简单 swig 示例',ext_modules = [扩展名('_example', sources = ['example_wrap.c', 'example.c'])],py_modules = ['示例'],package_data = {'': ['example.h']} # sdist 需要)

您可以使用以下方法构建轮子:

python setup.py bdist_wheel

在我的机器上它产生 example-0.1-cp27-cp27mu-linux_x86_64.whl 我可以用 pip 安装并像 python -c 'import 一样测试例子;打印(example.fact(5))'.注意文件名.它对兼容的 Python 版本、ABI 和平台进行编码.这是内容列表(unzip -l ...).

存档:example-0.1-cp27-cp27mu-linux_x86_64.whl长度日期时间名称--------- ---------- ----- ----2609 2018-02-24 11:06 示例.py70968 2018-02-24 13:31 _example.so10 2018-02-24 14:58 example-0.1.dist-info/DESCRIPTION.rst290 2018-02-24 14:58 示例-0.1.dist-info/metadata.json17 2018-02-24 14:58 示例-0.1.dist-info/top_level.txt105 2018-02-24 14:58 示例-0.1.dist-info/WHEEL193 2018-02-24 14:58 示例-0.1.dist-info/METADATA617 2018-02-24 14:58 示例-0.1.dist-info/RECORD------- -------74809 8 个文件

要构建兼容性更好的轮子，例如，您可以查看 manylinux.>来源

sdist 代表源代码分发，这意味着您的用户需要有一个编译器和相关的开发依赖项.源代码分发构建于:

python setup.py sdist

生成的 example-0.1.tar.gz 包含 (tar -tf ...):

example-0.1/示例-0.1/example.c示例-0.1/PKG-INFO示例-0.1/example.egg-info/示例-0.1/example.egg-info/PKG-INFOexample-0.1/example.egg-info/dependency_links.txt示例-0.1/example.egg-info/top_level.txt示例-0.1/example.egg-info/SOURCES.txt示例-0.1/setup.cfg示例-0.1/example.py示例-0.1/setup.py示例-0.1/example_wrap.c示例-0.1/example.h

I am wrapping via SWIG a C library in a python module, here called "myExample". If I compile:

$swig -python myExample.i $gcc -c myExample_wrap.c -I /usr/lib/python2.7 -fPIC -std=c99 $ld -shared myExample_wrap.so -llapacke -o _myExample.so

I obtain a full working module (liblapacke is necessary for some functions I used). Now I'd like to make this module installable via "pip install".

According to the distutils section (docs.python/2.7/distutils/setupscript.html), I wrote my setup.py file:

from distutils.core import setup, Extension setup(name='myExample', version='0.1', ext_modules=[Extension('_myExample',['myExample.i'], libraries= ['lapacke'])] )

and edited MANIFEST.in in a way to preserve the sources and avoid problems like in similar questions on this website (i.e. just including myExample.h and myExample.c). Then I run:

$python setup.py sdist

and obtained the package installable via "pip install". It seemed done (no errors, no warnings), but...it does not work. In this installable module ("_myExample.so" - notice the underscore, it seems to be required by distutils [maybe does it hide an answer?]) some methods are different, some are missing, etc... Consequently I decided to to one step per time. By just compiling:

$python setup.py build_ext

I already obtained the same problem: the final module is different from the one obtained via the usual compilation explained at the beginning.

Summing up: given a SWIG interface, compiling it traditionally or compiling it via distutils produces a different result. How could it be possible? Is my setup.py wrong? Is there maybe an alternative way for obtaining a pip-installable module without relying on distutils or setuptools(which produces the same problems)?

Ps: the wrap code is very long and so I cannot unfortunately give a detailed list, but I am fully available in adding more in case of need. For instance, the manually compiled interface successfully contains the "AdaptiveInterpolation" (working fine), while the distutil-produced one have "AdaptiveInterpolation_set", "AdaptiveInterpolation_get", or there are a lot of methods starting with "new_" (absent in my original code).

解决方案

Practically, there are two options for distribution of such package: bdist_wheel and sdist.

Let's take SWIG's example from the docs for distutils.

example.h

int fact(int n);

example.c

#include "example.h" int fact(int n) { if (n == 0) { return 1; } else { return n * fact(n - 1); } }

example.i

%module example %{ #define SWIG_FILE_WITH_INIT #include "example.h" %} int fact(int n);

Let's build a SWIG interface, which by default produces example_wrap.c.

swig -python example.i

Wheel

Python Wheels is a modern distribution format which simplifies distribution of pre-built packages (i.e. users of the package will not need a compiler, development headers and the like to install it). To use it we'll need setuptools and wheel (can be installed with pip or from OS repositories, sudo apt-get install python-wheel python-setuptools).

setup.py

from setuptools import setup, Extension setup( name = 'example', version = '0.1', author = 'SWIG Docs', description = 'Simple swig example from docs', ext_modules = [ Extension('_example', sources = ['example_wrap.c', 'example.c'])], py_modules = ['example'], package_data = {'': ['example.h']} # needed for sdist )

You can build the wheel with:

python setup.py bdist_wheel

On my machine it produces example-0.1-cp27-cp27mu-linux_x86_64.whl which I can install with pip and test like python -c 'import example; print(example.fact(5))'. Note the filename. It encodes compatible Python version, ABI and platform. Here's listing of the contents (unzip -l ...).

Archive: example-0.1-cp27-cp27mu-linux_x86_64.whl Length Date Time Name --------- ---------- ----- ---- 2609 2018-02-24 11:06 example.py 70968 2018-02-24 13:31 _example.so 10 2018-02-24 14:58 example-0.1.dist-info/DESCRIPTION.rst 290 2018-02-24 14:58 example-0.1.dist-info/metadata.json 17 2018-02-24 14:58 example-0.1.dist-info/top_level.txt 105 2018-02-24 14:58 example-0.1.dist-info/WHEEL 193 2018-02-24 14:58 example-0.1.dist-info/METADATA 617 2018-02-24 14:58 example-0.1.dist-info/RECORD --------- ------- 74809 8 files

To build wheels with better compatibility, you can, for instance, look at manylinux.

Source

sdist stands for source distribution, which means your users will need to have a compiler and relevant development dependencies. Source distribution is built with:

python setup.py sdist

Produced example-0.1.tar.gz contains (tar -tf ...):

example-0.1/ example-0.1/example.c example-0.1/PKG-INFO example-0.1/example.egg-info/ example-0.1/example.egg-info/PKG-INFO example-0.1/example.egg-info/dependency_links.txt example-0.1/example.egg-info/top_level.txt example-0.1/example.egg-info/SOURCES.txt example-0.1/setup.cfg example-0.1/example.py example-0.1/setup.py example-0.1/example_wrap.c example-0.1/example.h