Python on VxWorks - kuhlenough/cpython-for-VxWorks GitHub Wiki
VxWorks is the leading commercial RTOS (Real Time Operating System). It is used in more types of devices or things than any other OS. More specifically VxWorks is typically used in applications where the device must meet some sort of safety or security certification or criteria. While there are instances where VxWorks is used in consumer devices like SLR cameras, printers, and home routers; the RTOS has been displaced in these sort of applications by embedded Linux. Today you will find VxWorks in planes and trains; in factories in robots and PLCs; in medical equipment like MRIs and ultrasound equipment; in military equipment like tanks, drones and missiles; in power generation equipment like wind turbines; in telecom equipment and on mars.
Current releases of VxWorks 7 supports 32bit and 64bit compilation on ARM, PowerPC, and Intel processors. Currently the clang compiler from llvm.org is used for ARM and Intel. GNU GCC is used for PowerPC.
The safe and secure embedded market where VxWorks is the leader is evolving, and devices that use to be made up of multiple processors running multiple operating systems are now being consolidated. Device security and IoT trends mean a device must be more configurable and upgradeable, and a better host for portable code. In this evolving space Wind River's customers have asked for Python support on VxWorks.
VxWorks has gradually evolved to be more POSIX like, but is not a UNIX operating system. VxWorks utilizes UNIX interface to CPython, but does not support all the APIs or capabilities needed for all modules, the following CPython modules are not supported; spwd, grp, nis, termios, resource, tkinter, curses, dbm, fpectl, readline.
VxWorks supports configurations without network, filesystem, MMU, and process support. Use of cpython on minimalistic VxWorks is not feasible, and an appropriate alternative like lua or micropython should be explored. In the era of cheap RAM and FLASH, requirements for an RTOS to be small to reduce hardware costs have largely disappeared. The cost of safety certification is the primary driver of minimalistic RTOS configurations.
The following are configurable components are required on a VxWorks system hosting all supported Python functionality:
· Full MMU and RTP (process) support
· A Read/Write filesystem
· Shared Library support
· Real Time Clock
· Network and socket support
· Unix pipes
· The /dev device
· The /dev/random device
· A /tmp directory
· Syslog support
· Open SSL library
· User management
· UNIX compatibility library
The UNIX compatibility library provides APIs like lstat() and symlink(), (even though there are no file links in VxWorks). It is for porting 3rd party code to VxWorks and simplifies hosting generic UNIX code on VxWorks.
Building Python for VxWorks is done by configuring automake for cross compilation. The VxWorks configuration captured by ./configure is propagated to setup.py and only supported interfaces are built. Small changes are required to Python for this to work correctly. Larger changes are required for VxWorks filepath handling and subprocess support.
VxWorks uses forward slash like POSIX to separate files and directories in a filepath. An absolute path is always preceded by a device name. In POSIX semantics, all mount points are at the root, and the mount point is always the device name. So an absolute path takes the form:
<device_name>/<dir0>/../<dirN>/<filename>
Unlike POSIX, device names may contain a forward slash or colon. Like Windows, VxWorks uses semicolon to separate paths in environment variables.
VxWorks does not support any POSIX APIs for starting a child process. Submodule requires a custom implementation using the VxWorks specific API rtpSpawn(). (VxWorks specific APIs are camel case with the library name as the prefix.)
Wind River plans to provide its customers with a build wrapper to build Python as part of standard product. The information below describes what this build wrapper does. It is intended as a reference for Python developers and anyone attempting to port Python to an earlier version of VxWorks.
GNU Automake (which is used to build Python,) has support for cross compilation. VxWorks does not have a native compiler, all binaries are compiled on Windows or Linux and then transferred to the target system running VxWorks. Those familiar with cross-building Linux will be familiar with a sysroot, similar information is found in VxWorks in the VSB (VxWorks Source Build) project. Specifically the compiler toolchain settings, the system headers, and system libraries. Typically this information is provided to automake twice, once for compilation, and again for installation.
Automake nomenclature is non-standard, it refers to the “target system” as the execution “host”. Where most embedded engineers use “host” to refer to the compilation or build “host”. Specifying a host as an argument to configure informs the script you are cross compiling. Automake accepts the following host triplet arguments for VxWorks:
· x86-wrs-vxworks
· ppc-wrs-vxworks
· x86_64-wrs-vxworks
· arm-wrs-vxworks
Successful cross compilation requires:
-
An installed build-host version of Python in the Path of identical version to the Python being cross-built
-
Setting the environment variables for the cross build environment. Specifically CC, CPP, CFLAGS, CPPFLAGS, LD, LDSHARED, LDFLAGS, LIBS These need to be harvested from the VSB Makefiles
-
Set CONFIG_SITE to point to a site.config shell file specifying default values for configure tests that do not produce the correct results for cross compilation
-
Invoke configure with arguments specifying the cross build environment for post build install
A typical invocation for VxWorks cross compilation:
./configure --host=arm-wrs-vxworks
--prefix=$(VSB_DIR)/usr/root \
--bindir=$(VSB_DIR)/usr/root/llvm/bin \
--includedir=$(VSB_DIR)/usr/h/public \
--libdir=$(VSB_DIR)/usr/lib/common --with-libm=no \
--with-ensurepip=no --with-suffix=.vxe`
Ideally a configure script is maintained to adapt to cross compilation, in reality most projects additions to automakes do not consider cross compilation, and often produce erroneous results. Python is no exception. The pragmatic workaround is to provide cached values in site configuration file, the erroneous test is skipped and the cached value is used. The current site.config file used for Python on VxWorks is:
#defined in UNIX layer, but just a wrapper to select()
ac_cv_func_poll=no
# forces define of PY_FORMAT_LONG_LONG (Python)
ac_cv_have_long_long_format=yes
#Python wants these explicit when cross compiling
ac_cv_file__dev_ptmx=no
ac_cv_file__dev_ptc=no
ac_cv_buggy_getaddrinfo=no
ac_cv_func_gettimeofday=yes
#ignore empty header in UNIX layer
ac_cv_header_langinfo_h=no
#test failing though header is in UNIX layer
#perhaps include paths truncated in configure?
ac_cv_header_sys_times_h=yes
#avoid not finding pthread in various extra libs,
ac_cv_pthread_is_default=yes
#gcc fails to compile endian test
case $host in
ppc*-vxworks )
ac_cv_c_bigendian=yes ;;
arm*-vxworks )
ac_cv_c_bigendian=no ;;
x86*-vxworks )
ac_cv_c_bigendian=no ;;
esac
Ideally CPython will be modified to be more cross-compile friendly by incremental PRs and the requirement of using a site.config file gradually removed.
https://github.com/Wind-River/cpython incremental PRs may come from other users.