1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
|
<h1>Extending/Embedding FAQ</h1> <ul> <li><a class="reference internal" href="#can-i-create-my-own-functions-in-c" id="id3">Can I create my own functions in C?</a></li> <li><a class="reference internal" href="#id1" id="id4">Can I create my own functions in C++?</a></li> <li><a class="reference internal" href="#writing-c-is-hard-are-there-any-alternatives" id="id5">Writing C is hard; are there any alternatives?</a></li> <li><a class="reference internal" href="#how-can-i-execute-arbitrary-python-statements-from-c" id="id6">How can I execute arbitrary Python statements from C?</a></li> <li><a class="reference internal" href="#how-can-i-evaluate-an-arbitrary-python-expression-from-c" id="id7">How can I evaluate an arbitrary Python expression from C?</a></li> <li><a class="reference internal" href="#how-do-i-extract-c-values-from-a-python-object" id="id8">How do I extract C values from a Python object?</a></li> <li><a class="reference internal" href="#how-do-i-use-py-buildvalue-to-create-a-tuple-of-arbitrary-length" id="id9">How do I use Py_BuildValue() to create a tuple of arbitrary length?</a></li> <li><a class="reference internal" href="#how-do-i-call-an-object-s-method-from-c" id="id10">How do I call an object’s method from C?</a></li> <li><a class="reference internal" href="#how-do-i-catch-the-output-from-pyerr-print-or-anything-that-prints-to-stdout-stderr" id="id11">How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)?</a></li> <li><a class="reference internal" href="#how-do-i-access-a-module-written-in-python-from-c" id="id12">How do I access a module written in Python from C?</a></li> <li><a class="reference internal" href="#how-do-i-interface-to-c-objects-from-python" id="id13">How do I interface to C++ objects from Python?</a></li> <li><a class="reference internal" href="#i-added-a-module-using-the-setup-file-and-the-make-fails-why" id="id14">I added a module using the Setup file and the make fails; why?</a></li> <li><a class="reference internal" href="#how-do-i-debug-an-extension" id="id15">How do I debug an extension?</a></li> <li><a class="reference internal" href="#i-want-to-compile-a-python-module-on-my-linux-system-but-some-files-are-missing-why" id="id16">I want to compile a Python module on my Linux system, but some files are missing. Why?</a></li> <li><a class="reference internal" href="#how-do-i-tell-incomplete-input-from-invalid-input" id="id17">How do I tell “incomplete input” from “invalid input”?</a></li> <li><a class="reference internal" href="#how-do-i-find-undefined-g-symbols-builtin-new-or-pure-virtual" id="id18">How do I find undefined g++ symbols __builtin_new or __pure_virtual?</a></li> <li><a class="reference internal" href="#can-i-create-an-object-class-with-some-methods-implemented-in-c-and-others-in-python-e-g-through-inheritance" id="id19">Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)?</a></li> </ul>
<ul class="simple"> </ul> <section id="can-i-create-my-own-functions-in-c"> <h2>Can I create my own functions in C?</h2> <p>Yes, you can create built-in modules containing functions, variables, exceptions and even new types in C. This is explained in the document <a class="reference internal" href="../extending/index#extending-index"><span class="std std-ref">Extending and Embedding the Python Interpreter</span></a>.</p> <p>Most intermediate or advanced Python books will also cover this topic.</p> </section> <section id="id1"> <h2>Can I create my own functions in C++?</h2> <p>Yes, using the C compatibility features found in C++. Place <code>extern "C" {
... }</code> around the Python include files and put <code>extern "C"</code> before each function that is going to be called by the Python interpreter. Global or static C++ objects with constructors are probably not a good idea.</p> </section> <section id="writing-c-is-hard-are-there-any-alternatives"> <span id="c-wrapper-software"></span><h2>Writing C is hard; are there any alternatives?</h2> <p>There are a number of alternatives to writing your own C extensions, depending on what you’re trying to do.</p> <p><a class="reference external" href="https://cython.org">Cython</a> and its relative <a class="reference external" href="https://www.csse.canterbury.ac.nz/greg.ewing/python/Pyrex/">Pyrex</a> are compilers that accept a slightly modified form of Python and generate the corresponding C code. Cython and Pyrex make it possible to write an extension without having to learn Python’s C API.</p> <p>If you need to interface to some C or C++ library for which no Python extension currently exists, you can try wrapping the library’s data types and functions with a tool such as <a class="reference external" href="https://www.swig.org">SWIG</a>. <a class="reference external" href="https://riverbankcomputing.com/software/sip/intro">SIP</a>, <a class="reference external" href="https://cxx.sourceforge.net/">CXX</a> <a class="reference external" href="https://www.boost.org/libs/python/doc/index.html">Boost</a>, or <a class="reference external" href="https://github.com/scipy/weave">Weave</a> are also alternatives for wrapping C++ libraries.</p> </section> <section id="how-can-i-execute-arbitrary-python-statements-from-c"> <h2>How can I execute arbitrary Python statements from C?</h2> <p>The highest-level function to do this is <a class="reference internal" href="../c-api/veryhigh#c.PyRun_SimpleString" title="PyRun_SimpleString"><code>PyRun_SimpleString()</code></a> which takes a single string argument to be executed in the context of the module <code>__main__</code> and returns <code>0</code> for success and <code>-1</code> when an exception occurred (including <a class="reference internal" href="../library/exceptions#SyntaxError" title="SyntaxError"><code>SyntaxError</code></a>). If you want more control, use <a class="reference internal" href="../c-api/veryhigh#c.PyRun_String" title="PyRun_String"><code>PyRun_String()</code></a>; see the source for <a class="reference internal" href="../c-api/veryhigh#c.PyRun_SimpleString" title="PyRun_SimpleString"><code>PyRun_SimpleString()</code></a> in <code>Python/pythonrun.c</code>.</p> </section> <section id="how-can-i-evaluate-an-arbitrary-python-expression-from-c"> <h2>How can I evaluate an arbitrary Python expression from C?</h2> <p>Call the function <a class="reference internal" href="../c-api/veryhigh#c.PyRun_String" title="PyRun_String"><code>PyRun_String()</code></a> from the previous question with the start symbol <a class="reference internal" href="../c-api/veryhigh#c.Py_eval_input" title="Py_eval_input"><code>Py_eval_input</code></a>; it parses an expression, evaluates it and returns its value.</p> </section> <section id="how-do-i-extract-c-values-from-a-python-object"> <h2>How do I extract C values from a Python object?</h2> <p>That depends on the object’s type. If it’s a tuple, <a class="reference internal" href="../c-api/tuple#c.PyTuple_Size" title="PyTuple_Size"><code>PyTuple_Size()</code></a> returns its length and <a class="reference internal" href="../c-api/tuple#c.PyTuple_GetItem" title="PyTuple_GetItem"><code>PyTuple_GetItem()</code></a> returns the item at a specified index. Lists have similar functions, <a class="reference internal" href="../c-api/list#c.PyList_Size" title="PyList_Size"><code>PyList_Size()</code></a> and <a class="reference internal" href="../c-api/list#c.PyList_GetItem" title="PyList_GetItem"><code>PyList_GetItem()</code></a>.</p> <p>For bytes, <a class="reference internal" href="../c-api/bytes#c.PyBytes_Size" title="PyBytes_Size"><code>PyBytes_Size()</code></a> returns its length and <a class="reference internal" href="../c-api/bytes#c.PyBytes_AsStringAndSize" title="PyBytes_AsStringAndSize"><code>PyBytes_AsStringAndSize()</code></a> provides a pointer to its value and its length. Note that Python bytes objects may contain null bytes so C’s <code>strlen()</code> should not be used.</p> <p>To test the type of an object, first make sure it isn’t <code>NULL</code>, and then use <a class="reference internal" href="../c-api/bytes#c.PyBytes_Check" title="PyBytes_Check"><code>PyBytes_Check()</code></a>, <a class="reference internal" href="../c-api/tuple#c.PyTuple_Check" title="PyTuple_Check"><code>PyTuple_Check()</code></a>, <a class="reference internal" href="../c-api/list#c.PyList_Check" title="PyList_Check"><code>PyList_Check()</code></a>, etc.</p> <p>There is also a high-level API to Python objects which is provided by the so-called ‘abstract’ interface – read <code>Include/abstract.h</code> for further details. It allows interfacing with any kind of Python sequence using calls like <a class="reference internal" href="../c-api/sequence#c.PySequence_Length" title="PySequence_Length"><code>PySequence_Length()</code></a>, <a class="reference internal" href="../c-api/sequence#c.PySequence_GetItem" title="PySequence_GetItem"><code>PySequence_GetItem()</code></a>, etc. as well as many other useful protocols such as numbers (<a class="reference internal" href="../c-api/number#c.PyNumber_Index" title="PyNumber_Index"><code>PyNumber_Index()</code></a> et al.) and mappings in the PyMapping APIs.</p> </section> <section id="how-do-i-use-py-buildvalue-to-create-a-tuple-of-arbitrary-length"> <h2>How do I use Py_BuildValue() to create a tuple of arbitrary length?</h2> <p>You can’t. Use <a class="reference internal" href="../c-api/tuple#c.PyTuple_Pack" title="PyTuple_Pack"><code>PyTuple_Pack()</code></a> instead.</p> </section> <section id="how-do-i-call-an-object-s-method-from-c"> <h2>How do I call an object’s method from C?</h2> <p>The <a class="reference internal" href="../c-api/call#c.PyObject_CallMethod" title="PyObject_CallMethod"><code>PyObject_CallMethod()</code></a> function can be used to call an arbitrary method of an object. The parameters are the object, the name of the method to call, a format string like that used with <a class="reference internal" href="../c-api/arg#c.Py_BuildValue" title="Py_BuildValue"><code>Py_BuildValue()</code></a>, and the argument values:</p> <pre data-language="c">PyObject *
PyObject_CallMethod(PyObject *object, const char *method_name,
const char *arg_format, ...);
</pre> <p>This works for any object that has methods – whether built-in or user-defined. You are responsible for eventually <a class="reference internal" href="../c-api/refcounting#c.Py_DECREF" title="Py_DECREF"><code>Py_DECREF()</code></a>‘ing the return value.</p> <p>To call, e.g., a file object’s “seek” method with arguments 10, 0 (assuming the file object pointer is “f”):</p> <pre data-language="c">res = PyObject_CallMethod(f, "seek", "(ii)", 10, 0);
if (res == NULL) {
... an exception occurred ...
}
else {
Py_DECREF(res);
}
</pre> <p>Note that since <a class="reference internal" href="../c-api/call#c.PyObject_CallObject" title="PyObject_CallObject"><code>PyObject_CallObject()</code></a> <em>always</em> wants a tuple for the argument list, to call a function without arguments, pass “()” for the format, and to call a function with one argument, surround the argument in parentheses, e.g. “(i)”.</p> </section> <section id="how-do-i-catch-the-output-from-pyerr-print-or-anything-that-prints-to-stdout-stderr"> <h2>How do I catch the output from PyErr_Print() (or anything that prints to stdout/stderr)?</h2> <p>In Python code, define an object that supports the <code>write()</code> method. Assign this object to <a class="reference internal" href="../library/sys#sys.stdout" title="sys.stdout"><code>sys.stdout</code></a> and <a class="reference internal" href="../library/sys#sys.stderr" title="sys.stderr"><code>sys.stderr</code></a>. Call print_error, or just allow the standard traceback mechanism to work. Then, the output will go wherever your <code>write()</code> method sends it.</p> <p>The easiest way to do this is to use the <a class="reference internal" href="../library/io#io.StringIO" title="io.StringIO"><code>io.StringIO</code></a> class:</p> <pre data-language="pycon">>>> import io, sys
>>> sys.stdout = io.StringIO()
>>> print('foo')
>>> print('hello world!')
>>> sys.stderr.write(sys.stdout.getvalue())
foo
hello world!
</pre> <p>A custom object to do the same would look like this:</p> <pre data-language="pycon">>>> import io, sys
>>> class StdoutCatcher(io.TextIOBase):
... def __init__(self):
... self.data = []
... def write(self, stuff):
... self.data.append(stuff)
...
>>> import sys
>>> sys.stdout = StdoutCatcher()
>>> print('foo')
>>> print('hello world!')
>>> sys.stderr.write(''.join(sys.stdout.data))
foo
hello world!
</pre> </section> <section id="how-do-i-access-a-module-written-in-python-from-c"> <h2>How do I access a module written in Python from C?</h2> <p>You can get a pointer to the module object as follows:</p> <pre data-language="c">module = PyImport_ImportModule("<modulename>");
</pre> <p>If the module hasn’t been imported yet (i.e. it is not yet present in <a class="reference internal" href="../library/sys#sys.modules" title="sys.modules"><code>sys.modules</code></a>), this initializes the module; otherwise it simply returns the value of <code>sys.modules["<modulename>"]</code>. Note that it doesn’t enter the module into any namespace – it only ensures it has been initialized and is stored in <a class="reference internal" href="../library/sys#sys.modules" title="sys.modules"><code>sys.modules</code></a>.</p> <p>You can then access the module’s attributes (i.e. any name defined in the module) as follows:</p> <pre data-language="c">attr = PyObject_GetAttrString(module, "<attrname>");
</pre> <p>Calling <a class="reference internal" href="../c-api/object#c.PyObject_SetAttrString" title="PyObject_SetAttrString"><code>PyObject_SetAttrString()</code></a> to assign to variables in the module also works.</p> </section> <section id="how-do-i-interface-to-c-objects-from-python"> <h2>How do I interface to C++ objects from Python?</h2> <p>Depending on your requirements, there are many approaches. To do this manually, begin by reading <a class="reference internal" href="../extending/index#extending-index"><span class="std std-ref">the “Extending and Embedding” document</span></a>. Realize that for the Python run-time system, there isn’t a whole lot of difference between C and C++ – so the strategy of building a new Python type around a C structure (pointer) type will also work for C++ objects.</p> <p>For C++ libraries, see <a class="reference internal" href="#c-wrapper-software"><span class="std std-ref">Writing C is hard; are there any alternatives?</span></a>.</p> </section> <section id="i-added-a-module-using-the-setup-file-and-the-make-fails-why"> <h2>I added a module using the Setup file and the make fails; why?</h2> <p>Setup must end in a newline, if there is no newline there, the build process fails. (Fixing this requires some ugly shell script hackery, and this bug is so minor that it doesn’t seem worth the effort.)</p> </section> <section id="how-do-i-debug-an-extension"> <h2>How do I debug an extension?</h2> <p>When using GDB with dynamically loaded extensions, you can’t set a breakpoint in your extension until your extension is loaded.</p> <p>In your <code>.gdbinit</code> file (or interactively), add the command:</p> <pre data-language="none">br _PyImport_LoadDynamicModule
</pre> <p>Then, when you run GDB:</p> <pre data-language="shell">$ gdb /local/bin/python
gdb) run myscript.py
gdb) continue # repeat until your extension is loaded
gdb) finish # so that your extension is loaded
gdb) br myfunction.c:50
gdb) continue
</pre> </section> <section id="i-want-to-compile-a-python-module-on-my-linux-system-but-some-files-are-missing-why"> <h2>I want to compile a Python module on my Linux system, but some files are missing. Why?</h2> <p>Most packaged versions of Python don’t include the <code>/usr/lib/python2.<em>x</em>/config/</code> directory, which contains various files required for compiling Python extensions.</p> <p>For Red Hat, install the python-devel RPM to get the necessary files.</p> <p>For Debian, run <code>apt-get install python-dev</code>.</p> </section> <section id="how-do-i-tell-incomplete-input-from-invalid-input"> <h2>How do I tell “incomplete input” from “invalid input”?</h2> <p>Sometimes you want to emulate the Python interactive interpreter’s behavior, where it gives you a continuation prompt when the input is incomplete (e.g. you typed the start of an “if” statement or you didn’t close your parentheses or triple string quotes), but it gives you a syntax error message immediately when the input is invalid.</p> <p>In Python you can use the <a class="reference internal" href="../library/codeop#module-codeop" title="codeop: Compile (possibly incomplete) Python code."><code>codeop</code></a> module, which approximates the parser’s behavior sufficiently. IDLE uses this, for example.</p> <p>The easiest way to do it in C is to call <a class="reference internal" href="../c-api/veryhigh#c.PyRun_InteractiveLoop" title="PyRun_InteractiveLoop"><code>PyRun_InteractiveLoop()</code></a> (perhaps in a separate thread) and let the Python interpreter handle the input for you. You can also set the <a class="reference internal" href="../c-api/veryhigh#c.PyOS_ReadlineFunctionPointer" title="PyOS_ReadlineFunctionPointer"><code>PyOS_ReadlineFunctionPointer()</code></a> to point at your custom input function. See <code>Modules/readline.c</code> and <code>Parser/myreadline.c</code> for more hints.</p> </section> <section id="how-do-i-find-undefined-g-symbols-builtin-new-or-pure-virtual"> <h2>How do I find undefined g++ symbols __builtin_new or __pure_virtual?</h2> <p>To dynamically load g++ extension modules, you must recompile Python, relink it using g++ (change LINKCC in the Python Modules Makefile), and link your extension module using g++ (e.g., <code>g++ -shared -o mymodule.so mymodule.o</code>).</p> </section> <section id="can-i-create-an-object-class-with-some-methods-implemented-in-c-and-others-in-python-e-g-through-inheritance"> <h2>Can I create an object class with some methods implemented in C and others in Python (e.g. through inheritance)?</h2> <p>Yes, you can inherit from built-in classes such as <a class="reference internal" href="../library/functions#int" title="int"><code>int</code></a>, <a class="reference internal" href="../library/stdtypes#list" title="list"><code>list</code></a>, <a class="reference internal" href="../library/stdtypes#dict" title="dict"><code>dict</code></a>, etc.</p> <p>The Boost Python Library (BPL, <a class="reference external" href="https://www.boost.org/libs/python/doc/index.html">https://www.boost.org/libs/python/doc/index.html</a>) provides a way of doing this from C++ (i.e. you can inherit from an extension class written in C++ using the BPL).</p> </section> <div class="_attribution">
<p class="_attribution-p">
© 2001–2023 Python Software Foundation<br>Licensed under the PSF License.<br>
<a href="https://docs.python.org/3.12/faq/extending.html" class="_attribution-link">https://docs.python.org/3.12/faq/extending.html</a>
</p>
</div>
|
