How to write a Python to Cpp converter - danecross/PyNA62Analysis GitHub Wiki

In essence, there are two main components to making a C++ extension:

1. the C++ module code to make the module
2. the setup.py to export the module

I will walk through the ConversionExample in the main directory. It was basically ripped off of the Python Docs, but there are a few notes that are relevant to our case, and the information is a little spread out.

The example from the docs is pretty straightforward: we are wrapping a built-in C function "system" which basically calls a command line argument. The end goal is to be able to compile example-import.py without errors and get the correct output, which looks like a verbose ls.

This thing is a little nebulous, but once it's set up, I think it will be easy to add functions. This is all done in one file (per Module -- if we need more modules we will cross that bridge).

NOTE: this is only for functions. If we need to build classes, go here

#define PY_SIZE_T_CLEAN
#include <Python.h> # <-- this is accessible only in Python 3. More on this later. 

There are four main components to the Module code:

1. Actual Method Wrapper
2. Method Table
3. Module Definition
4. Initialization Function

Let's go down the list:

This is where the Meat^TM is. It handles errors, calls the function, and returns values. In our example code, this is what it looks like:

static PyObject* spam_system(PyObject *self, PyObject *args){
        const char *command; // 1
        int sts; // 2


        // error handling ---- 3
        if ( !PyArg_ParseTuple(args, "s", &command) ){
                return NULL; //this is the error code for this function
        }

        // function calling  ----- 4
        sts = system(command);
        return PyLong_FromLong(sts); // 5
}

Notes on this:

1. *command is apart of the Python.h library
2. sts is where we will be storing our result. This will change with what the method returns
3. error handling is difficult and I made another wiki on how to deal with it. She's a complicated creature but also super necessary.
4. Finally, we actually call the function
5. PyLong and C/C++ long are different, thankfully the Python.h library has a conversion method.

This acts as a list of methods that we can call with our module. Whenever you make a new method wrapper, you have to add it to the method table. Here is the code:

static PyMethodDef SpamMethods[] = {
        {"system", spam_system, METH_VARARGS, "execute shell command."}, // 1
        {NULL, NULL, 0, NULL} // 2
};

Notes:

1. These four arguments are what you need to register the method. They are:
   • "system" -- this is will be the method you call after you import the module. In this case, we call "spam.system(..)"
   • spam_system -- this is the wrapper method name.
   • METH_VARARGS -- this tells PyArg_ParseTuple() how to parse the arguments. We can also use METH_NOARGS.
   • "execute shell command." -- this is for the documentation. Just give a short description of the method and what it does.
2. This is in every implementation that I have seen. Not sure what it does, don't really care.

This puts the whole thing into a Module with titles and everything!

static struct PyModuleDef spammodule = {

        PyModuleDef_HEAD_INIT,
        "spam",                  // Name of Module
        NULL,                    // Module Documentation
        -1,                      // 1
        SpamMethods              // This is the thing we made in the previous step

};

Notes:

1. From the Python Docs:

Size of the per-interpreter state if the Module. -1 if module keeps state in Global Variables.

From what I can understand this will probably always be -1.

This is where library magic comes in and makes us an executable without anymore thinking. In this function we call another function and badabing we have a shiny new Module.

It is important to note that the naming here is very important. The function name must always have the signature PyMODINIT_FUNC PyInit_name(void), where name is the name of the module. This function should also be the only non-static item in the module file. Here's the code:

PyMODINIT_FUNC PyInit_spam(void){
        return PyModule_Create(&spammodule);
}

The above is pretty self-explanitory.

Now we need a mechanism that will call the PyInit_ function to make it an importable module. This is where source distributions come in handy. There is really only one thing to understand here: the setup.py file. After that it's just a few command line calls and you've got yourself a package.

This gives the sdist compiler all it needs to understand where to put the module code and information. Because all of our code is C++ extension (completely not pure if you will), I will only talk about how to put Extensions into the file.

from distutils.core import setup, Extension

spam_module = Extension('spam', sources=['spammodule.cpp'], language='C++', )

setup(name='spam',
      version='1.0',
      ext_modules=[spam_module],
      )

And save the setup.py

Almost Done

So now everything is set up, all we have to do is type the following commands into the terminal:

python3 setup.py sdist             # 1
cd dist/                           # 2
cp spam-1.0.tar.gz ~/              # 3
cd
tar -xvf spam-1.0.tar.gz           # 3
cd spam-1.0/
python3 setup.py install --user    # 4
cd 

rm spam-1.0.tar.gz
rm -r spam-1.0/                    # 5

Notes:

1. This calls the PyInit_ function and creates a dist/ directory and a MANIFEST file.
2. The dist/ directory contains our tar file, which we can unpack anywhere and import spam will work.
3. Unpack tar file in the home directory, for tidiness purposes
4. Run the install. NOTE that this is being installed using python3, not python and we are installing in the user directory, because install permissions.
5. clean up

So you don't have to type these commands in every time, cd into the SummerProject/ConversionExample/MakeSpamDist/ directory and type in the following:

source compile_spam.sh

As a quick check that everything is correct, run the following:

python3 

import spam
spam.system("ls -l")
exit()

You should get something that looks exactly like what you get when you type in ls -l on your command line.