I currently have a microkernel architecture that allows code blobs to run and message each other. But it's hard for anybody else to see or use this. I want a novice accessible way to express the state of my microkernel; to define:

1. what components are running
2. properties of those components
3. what is wired to what

I want something that is text based, simple, and with some general utility - that I can load, that sets up the kernel state, and can even be used by child components to define their own state. Typically people invent a "novice accessible" glue language or grammar for these use cases. Here we want something that defines how kernel modules or apps relate to each other, what they are made of, their permissions and "manifests" (since an app is more than a single wasm blob). This effectively defines a standard of what the kernel "is" and exists at a higher level than the kernel itself. The kernel is just an implementation of the standard. In a sense we care less about the implementation and care more about the formalism. For example WebGPU is a standard and wgpu_rs is an implementation. HTML is a standard and spidermonkey is an implementation. This can all be thought of as attaching a control harness, or basically a kind of glorified steering wheel to the system.

A "mature" system would have some kind of GUI or fancy way of specifying what is loaded, what permissions that piece has, and what it is wired to - but I don't have time for that - so a text file will have to do. By way of example we can imagine a trivial app that it should be possible for an utter novice to be able to express - such as this hypothetical app that a user may want to build:

 [Camera] <---------> [Segment out faces] <--------> [Paint camera and segmented faces to a display].

A formal grammar for this can be something as simple as:

Camera { permissions: "camera device", }

Segmenter { cpu: "medium" }

Display { permissions: "display device", }

Camera -> Display

Segmenter <- Camera

Segmenter -> Display

Previously the kernel consisted of moderate weight threadlets or modules; where one of them had a special job to manage the rest. That special one I called a "broker" and it kept a lookup list of the rest to allow dynamic messaging and binding between them all. The benefit of this was that the kernel was truly tiny, and agnostic (it does not tell users how to do things and it has to be recompiled rarely).

But I think I want more. I want to have some kind of way to easily and dynamically specify wires between modules. And also I wonder if modules can be lighter weight than threads; and also not merely be threads, but more like manifests or random arbitrary objects. I feel like this has to be pushed down to the kernel possibly therefore; which does make the kernel heavier.

There are various approaches for making the kernel dynamically load and wire things together. It already does have an API for loading and wiring things - that I currently drive from main() with hardcoded rust commands. But it could also parse a grammar description. And also, we may want to formalize a richer concept of what the artifacts are that a kernel manages. One way to do this would be to promote the idea of a scenegraph dag ideas out to global scope, basically have the whole kernel manage arbitrary objects. So a jpeg, a snippet of code, a layout capability - this would all become top level concepts in the kernel. The kernel basically provides access to what is kind of like a file system in a way. And this is a single consistent interface for all services... and this makes an idea of tiny fragments of execution simpler, and it also clarifies an idea of what the atoms are. I would have to expose an api to let anybody manipulate the graph; but this is nice because it makes the display module lighter, and makes other modules lighter also. On the down side it can be tricky to manage and run tiny fragments of code and sandbox them safely.

Let's sketch what changes a grammar could require:

1. Frags / Modules / Handlers / Data / Block

   • there is some argument to not just load "modules" (wasm blobs) but "anything"; a grammar implies a richer set of capabilities
   • has a type; code, data other - probably will use mime types; types can be inferred; may also have a filename or url
   • holds any piece of data or code; can be very small; nothing, a single line of code, a single image, can be a wasm blob
   • can act as a manifest to bootstrap a number of related pieces of a larger 'package' or collection of related frags
   • has perms that can be altered such as access to file system, networks, other fragments; perms percolate to children
   • code an ability to send and receive messages, this is the primary way it communicates and is triggered to do work
   • code is typically run as a separate thread / threadlet
   • code does not support inter-frag direct calls to each others functions right now; this needs study
   • we may want to identify if a frag is a singleton
   • are organized in a dag; effectively a file system; pieces can have a parent

2. Broker / Pool / Kernel Fragment Manager

   • this is a currently existing special module that is currently loaded up at start by hardcoded logic in main()
   • it could be enhanced to track persistent state of system over reboots of hardware - by reading in the proposed script file
   • it currently acts like a scene graph manager or a database; and this role would be enhanced
   • API
   • currently can load and unload other frags in a DAG from network or filesystem
   • currently can enumerate / search for frags
   • currently can start and stop runnable frags
   • currently can change frag perms
   • currently wire or unwire frags to each other
   • Execution
   • this thing could run wasm modules itself or a wasm module runner can be a separate module
   • it's easy enough to also run larger js modules as a separate thread
   • but what about tiny javascript fragments; are they run here synchronously? how are they permission restricted and time throttled?
   • what scope is exposed to a js fragment? what methods, powers, capabilities?
   • see also https://github.com/denoland/rusty_v8 https://duktape.org/ https://bellard.org/quickjs/

3. Fragment: A Camera Device

   • currently is native rust for now as a test
   • currently is addable to pool like any fragment
   • gets cpu time and spits frames out to any listener

4. Fragment: A Face Segmenter

   • currently is native rust, later ideally written as a wasm blob to exercise dynamic loading
   • gets wired up to camera device; processes frames, spits out segmentation details

5. Fragment: A Display

   • currently is using makepad ; this may have to change because i need fully featured scriptable display elements
   • other fragments can fiddle with children fragments here that represent displayable elements
   • paints part of scene graph to hardware
   • similar to HTML DOM

6. Fragment: Layout

   • does not exist yet
   • can do layout of a set of children pieces
   • rather than decorating children with properties such as alignment i'd prefer code to adjust relative position; is that possible?

7. Fragment: Text

   • does not exist yet
   • html like text printing engine to print into a specified region

8. Fragment: 3d geometry, Camera, Lights, Materials...

   • does not exist yet

I do think that domain specific languages, or even perhaps a slightly stronger term “grammars” are the right succinct way to express “intent”. That is to say, that although I’ve been using a term “computation” as a catch all; the more specific requirement or feature is not just being able to compute in the general case but to be able to interpret and perform an action at an arms length from an abstract hand wavy description. And really that is what computation is. One expresses a hand wavy idea and the remote hardware does it. We think of code as so precise, so exact, but really, it is an abstraction in itself; a way to access hardware in a predictable way and to not get bogged down on the minutae (even if sometimes programming seems like it is dealing with minutae it still is abstract compared to the hardware).

Grammars can range from declarative to procedural and can be incredibly pendantic (such as say Rust), fairly forgiving (such as say Javascript) or highly forgiving such as say Scratch or some of the visual grammars such as say Unreal Blueprints. I lean towards the latter for a description language for describing modules that a kernel loads and related services.

Ultimately the goal isn't a grammar or DSL per se; the goal is to express "intent" - by whatever means necessary. The grammar should be "right sized" to best express intentions; not to allow literal manipulation of a system in a granular way. The difference is that a wrong sized grammar can say "place a cube at xyz=23,453,23" but a right sized grammar can say "place a cube on the ground".

Rust is the wrong language to express high level abstract intent. It's unusable by novices, it is overly pedantic. We want an expression that is neutral, portable, small, fast, novice friendly.

Typical users for Orbital are going to be people who want a kind of ROS for embeddable systems. This could be hardware AR glass vendors. It could also be more traditional computer vision customers such as farmers who use field robots to prune their fields - and who want to write their own apps.

There are a few interesting links that touch on related topics:

https://github.com/denoland/rusty_v8 https://duktape.org/ https://bellard.org/quickjs/ -> Rather than writing a new grammar I could just use javascript... This is a common pattern. I still lean towards my own grammar but it is noteworthy. I could also do both; use json {} with say javascript embedded methods. This is a technique I used in https://github.com/anselm/blox and https://github.com/anselm/blox2 quite successfully. Here is an example largely declarative grammar that defines an interactive cherry tree shedding blossoms in 3d : https://github.com/anselm/blox/blob/master/public/examples/cherry_blossoms.js

https://github.com/WAVM/WAVM -> proposes to be a VM basically and it could possibly replace my kernel.

https://www.fastly.com/blog/announcing-lucet-fastly-native-webassembly-compiler-runtime -> This has been around for a while but I raise it again as an example of something that must have some kind of formal scheme for permissioning code fragments and what they respond to. I have not looked at it closely enough to see what grammar they use and how formal it is. I am sure it is not Rust itself however but some kind of sysadmin style easy to edit notation.

https://blog.stackblitz.com/posts/introducing-webcontainers/ -> This also has to solve similar challenges as my kernel - in terms of defining what is loaded, how they talk to each other and so on. I haven't had time to look more closely.

https://www.oreilly.com/library/view/mastering-ros-for/9781783551798/ch11s03.html https://www.researchgate.net/figure/Car-software-architecture-ROS-Robot-Operating-System-GUI-graphical-user-interface_fig1_338600698 https://sites.google.com/site/mrsdproject201213teamb/home/project-status/recent-updates/march21st-userinterfaceusingqtrosandopencv -> ROS systems as a whole tackle similar families of problems being computer vision computing platforms running many modules.

https://developers.google.com/web/updates/2018/09/inside-browser-part1 -> A general article talking about how browsers mediate interaction with computational hardware and sandbox execution safely. Similar challenges to what I am facing.

https://developer.magicleap.com/en-us/learn/guides/web-platform-overview -> A good example of what not to do. Effectively proposes extensions to CSS, similar to the AFRAME proposed extensions to HTML. This is the wrong direction because the weight of these existing grammars burdens down what are almost unrelated spaces. For a little benefit there is a lot of pain.

https://github.com/PistonDevelopers/piston -> A modular 3d engine that in some ways also could benefit from the grammar I am proposing. If my grammar could act as a highest level glue for describing what pieces are loaded and exist. However normally module selection is done by tree shaking - rather than by human agency.

Deno / NodeJS https://deno.land/[email protected]/getting_started/installation -> It's worth noting that Deno and NodeJS (and Python and many other tools) are in fact a collection of "power tools" glued together crudely by a lightweight scripting language. This is similar to what I am proposing; however I am leaning more towards something more declarative, and less procedural; or at least having procedures nestled into declarative blocks - because I want something that a novice can work with that is "composable" and that ideally could be painted in a graphical UX similar to say Unreal Blueprints or Max/MSP.

https://arxiv.org/pdf/1709.00084.pdf -> This is a study of how to express work for a robot at the right level of abstraction. Also very similar to what I'm proposing. Especially relevant is that I absolutely want to avoid a literal transliteration of raw source code to some kind of visual blocky wire diagram language. A visual diagram language has to be abstracted, simple, easier to use. It has to infer what is actually meant by a wire between two objects - not be literally "source code".

https://github.com/OpenArCloud/sparcl -> In memory representations of shared state between modules will probably end up having something similar to this in that there will be some kind of datastore, one can query it for features and also add to it. I do think the grammar that I load up should produce effectively a DAG that will embed dynamically discovered state ("this is a wall", "this is a floor") in a way that can be queried.

https://blockbench.net -> a cute example app I'd love to be able to support, I see rich apps like this as good use cases for this engine.

Toit -> https://docs.toit.io/language/language -> a good example of a microkernel architecture that insulates execution from iOT capabilities. This almost perfectly articulates the main thesis that I've been exploring in general around computational sandboxes.

https://without.boats/blog/revisiting-a-smaller-rust/ -> Here is an exploration of grammars. I notice a tension between precise grammars such as Rust and a desire for lighter faster grammars. I think the author could go further towards declarative grammars as well.

https://github.com/PistonDevelopers/dyon -> Here we see yet another grammar; again trying to solve for a tension which is that no novice will ever program in Rust.