GNC Autocoding Functions and Wrapping in CircuitPython - spacecraft-design-lab-2019/documentation GitHub Wiki

This page gives an introduction to converting algorithms in MATLAB into CircuitPython for use onboard the spacecraft. The process goes as follows:

  1. Preparing your MATLAB code
  2. Using MATLAB Autocoder to generate C code
  3. Making your C code work with MicroPython
  4. Making your MicroPython code work with CircuitPython

1) Preparing your MATLAB code

  • A helpful link with general advice for getting started.
  • Matlab Coder can deal with a lot of MATLAB features but not everything is supported. Here is a link detailing the Matlab language features, functions and objects supported by Matlab coder.
  • Helpfully, if there is an issue with your MATLAB code which the auto-coder can't resolve, MATLAB will tell you (reasonably) clearly what that issue is when you try to perform the auto-coding operation. It is important to note that the requirements for your code will differ depending on which settings you choose in the auto-coder. Generally when you are coding for an embedded target (like we are) you are going to have the most restrictions on your code.

2) Using MATLAB Autocoder to Generate C Code

Packages to install

  • Matlab Coder - The main auto-coding application
  • Embedded Coder - Adds additional settings to Coder for memory management etc. Also allows you to generate code for specific hardware (Arm Cortex-M etc.)
  • Fixed-Point Designer - Adds an option in Coder to convert all double precision operations to single precision. This is not required for Arm Cortex-M4, but it drastically reduces the computation time. Some functions have been found to be inappropriate for conversion to single precision, most likely due to variables hitting the datatype max in some context.

Using the MATLAB coder graphical interface

  1. While in MATLAB, open the folder containing the project you want to convert to C. Click the Apps tab at the top, then click MATLAB Coder.
  2. Type the name of the function you want to autocode into the box then press enter.
  3. You should see a drop-down menu for "Numeric Conversion". If you want to convert to single precision, select "convert to single precision", then click "Next". While the MCU can accept both 32 and 64 bit, it is much faster to use single precision when speed is at a premium. Here is a more complete tutorial showing how to ensure your code is generated with single (float) precision.
  4. Define your functions input types/sizes manually to ensure they are correct. If you want singles, make sure to use "single" for floating point data rather than "double". Click "Next".
  5. You're now going to check for run time issues by generating a MATLAB .mex file for your function. Enter the name of a simple script which calls your function (you might need to write this), then click "check for issues", this might take a while. This step could fail and MATLAB should tell you what you need to change in your code. You might also receive a warning, if you click "view Matlab line execution counts" you'll get more details on the potential issue. A common warning is due to using MATLAB built-in functions which use double precision. These might still be generated using doubles and you will have to change them to floats later. Click "Next"
  6. On the generate code screen, click "More Settings"

C code generation settings

Make sure you set the code generation parameters as follows:

  • PATHS - Leave as default. Starting from the directory containing the MATLAB function being auto-coded, you will find the C code in /codegen/lib/<your_function_name>
  • SPEED - De-select "support non-finite numbers" unless specifically needed eg. if your code uses inf, or NaN. (this will avoid extra C header files and dependencies)
  • MEMORY - Ensure "Enable variable sizing" is selected, change "Dynamic memory allocation" to "never". "Max Stack usage" should be set significantly lower than the total stack size of the MCU, while still allowing enough memory for your function to run. The default stack size allocated on the SAMD51 by cirucit-python is 44KB (0xC000). You can find this in the linker-script circuitpython/ports/atmel-samd/asf4/samd51/gcc/gcc/samd51g19a_sram.ld. For functions using many arrays and/or calling many other sub-functions you will want to allocate a reasonable amount of stack space. It's difficult to know how much is needed/how much the MCU can afford to give at any one time. More testing is needed with the hardware so for now aim for less than 35% (>15KB) as a ballpark guess. Leave the other settings as their defaults (column-major etc.).
  • CODE APPEARANCE - Change "Generated file partitioning method" to "Generate all functions into a single file", un-tick "MATLAB function help text", un-tick "Preserve extern keyword".
  • DEBUGGING - Tick "Report differences from MATLAB" and "Highlight potential data type issues"
  • CUSTOM CODE - Make sure "C99(ISO)" is selected for "standard math library"
  • HARDWARE - "Hardware board" should be "MATLAB Host computer". For "toolchain" select "GNU gcc/g++ | gmake" if this doesn't show up it means you likely don't have the GNU toolchain on your path. Select "Faster runs" for "build configuration".
  • ALL SETTINGS - Change the following settings, everything else should be left as the default:
  • Set "Enable run time recursion" to "No"
  • Set "Initialize function required" and "Terminate function required" to "No", unless you used NaN or Inf types in which case you will need to enable "Initialize function required" to "Yes"
  • Set "preserve variable names" to "user names" (useful for debug)
  • Set "Generate Makefile" to "No"
  • Click "close"

Generate your Code

  • Click "Generate"
  • If code generation fails, look at the errors and adjust your MATLAB code accordingly then continue the process.
  • If code generation succeeds, a little box should pop up, click "view report" then go through the tabs and read the messages to ensure there are no potential issues.

3) Making your C code work with MicroPython

In order to fully load your newly minted C-function onto a CircuitPython board, it will have to be converted to a Micropython format, and then integrated into CircuitPython. These formats have added difficulties, so it will likely cause errors for you somewhere along the way when you're tying it all together. You should be proficient at building CircuitPython (tutorial here: https://learn.adafruit.com/building-circuitpython), and you should have pulled a circuitpython fork (for example, ours: https://github.com/spacecraft-design-lab-2019/circuitpython). There are many idiosyncrasies to building custom functions in CircuitPython, especially with modifying the makefiles and the C-Python base.

Moving your C-function into circuitpython

There are many ways to create a custom function in CircuitPython, but this is the way that we have chosen to do it for our purposes. This is by no means the most elegant way, and it ignores a good bit of CircuitPython etiquette, so don't tell Adafruit.

  • First, you'll need to move your autocoded .c and .h file from their folders into the shared-bindings (as an example for our ulab module: /CircuitPython/circuitpython/shared-bindings/ulab).
  • Add a bunch of include statements (shown in the picture below). If you're working with ndarrays or the linear algebra package, you'll need to add that as well.

Changing data types/functions

This section will heavily reference several files which contains the first successful wrapping of MATLAB autocoded functions in CircuitPython. Namely:

Basically, CircuitPython is a fork of MicroPython, which uses a whole bunch of its own data types/functions that you'll have to change in your C code. Below is a list of changes that will have to made in order for the MATLAB code to work with Micro/CircuitPython. This list is not comprehensive, and these were all found by repeatedly compiling and reading the error messages, so you may also have to do that for your specific problem. Known necessary changes:

  • Find and replace all instances of double with mp_float_t
  • MATLAB likes to make its own boolean type. Find and replace boolean_T with bool
  • If basic trig functions are throwing an error upon compiling, look for any sqrt, sin, or cos functions in your code. Replace them with MICROPY_FLOAT_C_FUN(sqrt), MICROPY_FLOAT_C_FUN(sin), MICROPY_FLOAT_C_FUN(cos), respectively
  • MATLAB may make a bnorm function for doing vector norms. Replace any scale parameter included with scale = 3.3121686421112381E-30 to meet floating point specs.

Writing your MicroPython function

  • Next, you'll need to write a wrapper that takes in ulab arrays from the top level CircuitPython and accesses the underlying C arrays in order to pass those to the autocoded MATLAB. An example can be found in ulab_controller_MEKFstepC here. *This function takes in a state and covariance to be updated, x_k_input and P_k_input, along with a bunch of measurements/parameters/etc. The state and covariance are to be updated in place (which is why all other inputs have the qualifier const); it is highly recommended that your function takes in arrays from top-level CircuitPython to be modified in place in order to avoid memory allocation issues. *Next, you must extract the pointer to the underlying ndarray object from the input object. Example here
ulab_ndarray_obj_t *your_variable_obj = MP_OBJ_TO_PTR(your_variable_input);
  • Next, you'll need to extract the pointer to the underlying C array from the ndarray object. Example here
mp_float_t *your_variable_data = (mp_float_t *)your_variable_obj->array->items;
  • Call your C code using your extracted C arrays. Example here
yourFunction(your_variable_data, your_output_variable_data);

At the end of the day, it should look something like this.

4) Making your MicroPython code work with CircuitPython

*Next, you'll need write some declarations for your MicroPython function in the corresponding __init__.c file. Example declarations can be found here.

1) Define a function object using Micropython's MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN function, Example:

MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(ulab_<your package>_<your function>_obj, <# of arguments>, <# of arguments>, ulab_<your package>_<your function>);

2) Add the new function object to the package's dictionary of known functions. For ulab, this can be done either as a local function or a global function. i.e:

  • local:
import ulab as np
A = np.array(<your data here>)
A.do()
  • global:
import ulab as np
A = np.array(<your data here>)
np.do(A)
  • For integration with ulab, local functions will go in the ulab_ndarray_locals_dict_table
  • Global functions go in the ulab_globals_table
  • For both local and global functions, you'll then have to define a QSTR for your MicroPython function within one of the two tables (from what I've gathered, QSTRs are ways of defining constant strings in CircuitPython such that they're defined to be a constant string in the CircuitPython source code, thus linking the string to your function). An example QSTR definition can be found here.
{ MP_ROM_QSTR(MP_QSTR_<your_function>), MP_ROM_PTR(&ulab_<your_package>_<your_function>_obj) },

Adding new files to C-Python and shared-module

One important thing to remember is that a file in the shared-bindings folder always needs a comparable file in the shared-module folder. These are pretty much blank, but contain important links to tie everything together. They can pretty much be copied from existing files where the names are just replaced. For the file referenced previously (controller.c), the shared-module file is at https://github.com/spacecraft-design-lab-2019/circuitpython/blob/master/shared-module/ulab/controller.c and https://github.com/spacecraft-design-lab-2019/circuitpython/blob/master/shared-module/ulab/controller.h.

One last thing: if you're attempting to add a new .c file (rather than modify an existing one) in the shared-bindings, you'll also need to modify a line in the makefile for the module (in this case, ulab) to make sure that it builds correctly. Currently, this exists right here: https://github.com/spacecraft-design-lab-2019/circuitpython/blob/master/py/circuitpy_defns.mk#L378, so you should be able to add your line in right after

	ulab/__init__.c \
	ulab/ndarray.c \
	ulab/linalg.c \
	ulab/controller.c \

in the SRC_SHARED_MODULE_ALL variable.