NEP2 Extension Modules - ghewgill/neon-lang GitHub Wiki

Currently, adding a new standard library module to Neon involves integrating it into the existing source tree, and rebuilding everything from scratch. In addition, everything is statically linked into the Neon executor so this will inevitably produce conflicts between static libraries (symbols or versions).

In order to make it easy to add new modules to Neon, this proposal outlines how such extension modules might be developed and deployed.

Development

To develop an extension module, a developer would create a .neon file which declares all the extension functions (declaration syntax TBD). Additionally, the devloper would create a shared library module (.dll, .so, .dylib, etc as appropriate) which contains the implementation of the extension functions. This shared library may either contain the extension code statically, or it may link to further shared libraries that may be installed elsewhere.

The interface required of the developer to implement the extension functions would be a new simple C interface, not the existing internal Neon executor classes which require C++11. In this way there are no specific build dependencies on creating an extension library, making such libraries easy to build independently. A single C header file would define the interface to the Neon runtime system.

These extension libraries would be straightforward to build on platforms where compilation from source is commonplace (such as Posix). They can be distributed as source, or as compiled binaries if desired.

Deployment

A .neon interface file and a shared library file (and any other supporting files) would be packaged together in a zip file. This zip file would be extracted into its own directory during installation. This directory would normally be located under an existing Neon library directory, either the system one or a user-specific location.

For example, if an extension module is called foobar, then the directory structure might look like:

- lib/
  - foobar/
    - foobar.neon
    - neon_foobar.dll

In this way, an IMPORT foobar statement would look in the lib directory for either a foobar.neon file (same as today) or a foobar directory. In the case of a foobar directory, the loader would expect to find a file named foobar.neon. If this foobar.neon file declares extension functions, then a shared library in the same directory as foobar.neon would be loaded with a name determined in a platform-specific way. For example:

• neon_foobar.dll on Windows
• libneon_foobar.so on Posix
• neon_foobar.dylib on macOS

The neon_ prefix would be required to make a distinct name that wouldn't be likely to conflict with other libraries that might exist or be required.

Other Runtimes

The above applies to the "native" Neon bytecode backend. Other ways of building extension modules would be defined for other backends such as JVM or CLI, which are appropriate for that backend.