Understanding the challenges better

March 22nd 2021

General Observations

We are effectively building a small rust kernel in userland. This can also be seen as a microservice.

Here's a good talk on these topics from a high level perspective by Josh Triplett:

https://www.reddit.com/r/rust/comments/cupayl/intel_and_rust_the_future_of_systems_programming/

Major Kernel Parts

There are two perspectives on the app - a consumer facing perspective and a technical perspective.

From the consumer perspective this is an app that allows users to manage, load and run rich persistent apps over the web such as accounting software, games, music players and so on. It should allow both typical web apps (HTML/CSS/JS) with weak models of PWA, but also we want to allow richer 'native' experiences and granular permissions management.

Then, there's the technical perspective. We see that we can roughly break it into these pieces:

1. Modules -> The design is around an idea of modules of code or "units of computation".

2. Module Registry -> Under the hood is a registry of all apps and fine grained permissions controls over which apps have which capabilities. Modules can depend on other modules, and there are built-in system modules for input and output to various devices. Support for WASM/WASI modules is a core concept; we expect "unsafe" modules to be arriving over the net.

3. Messaging -> Modules can send messages to each other. The plan is just pubsub for now and to later support shared memory.

4. Graphics -> Graphics would normally simply be "just another computational unit" or module - but there are special considerations. We want to invert some of the models of control for rendering to reduce state transfer while avoiding DSLs.

We will drill down on some of these pieces below.

NO DSL

It's important to clarify what we are NOT doing. We are NOT specifically providing any kind of DSL grammar. Often when there is a narrow pipeline between two services (notably between say a video game engine and rendering hardware) there is a strong temptation to define abstract semantics to transport intention between services. We don't do that. We're just interested in making a computational soup where modules can run and focusing on an ability for them to talk to each other efficiently. Our focus is that they can talk - and not at all on what they say to each other. Ie: There MAY be DSL's - we do not inhibit that - but we want to avoid endorsing one.

NO NANOPROCESS

We are NOT doing a WASM nanoprocess style service. The WASI design is where there is a set of capabilities described by say a POSIX API and then permissions are trickled down to WASM modules that are able to then talk to those POSIX interfaces if they have the appropriate permissions. This is an architectural model where there is a highly capable "outer environment" and it is charged with managing possibly poorly behaved "inner processeses". The outer environment permissions the inner processes. We do not do this. Instead we are interested in heterogenous modules that are all equal, and allowing them to talk to each other. Security, sandboxing and permissions are critical, but are defined around dynamic interfaces. If there is any outer environment that is defining permissions, it is largely the user, and the module registry - which will define which modules are "trusted". But all modules and messages are "equal".

NO ECS

We are NOT doing an Entity Component System. We are concerned with the scale and scope of "applications". Each of our threads, libraries or "code blobs" is expected to be either an entire service (a game, a user experience or app) or an entire tool (a tensorflow package, a device driver). Individual apps can of course implement ECS internally. Right now at this scope we are not doing so.

NO HIERARCHICAL THREADING, DAGS, SCENE-GRAPHS

We are NOT supporting a DAG or hierarchy of behaviors arranged in a scene-graph. We are only supporting a flat namespace of top level "larger" services or applications. Of course developers can implement DAGS, scene-graphs and more complex structures inside of their applications. This also touches on DSLs (which we are NOT providing or writing).

Note that for now because WASM does not support multithreading we don't allow modules to have sub-threads either. This is expected to change as WASI adds support for such capabilities via POSIX.

KERNEL MESSAGING

How will our computational units talk to each other? We need a late-binding solution that works with WASM. Finding ways to bind computational units of work is a long standing focus of computer science. For example see For a survey of UNIX techniques see https://homepage.cs.uri.edu/~thenry/resources/unix_art/ch07s02.html . One solution for late-binding services for us us https://zeromq.org/get-started/ . But in our case we're dealing not with full blown separate processes (possibly running on separate machines) but threads all running in a shared memory space. So simpler approaches like crossbeam and shared_memory seem good.

Examples of the kinds of traffic between computational units is state such as:

1. user input
2. a database query
3. an audio stream
4. a framebuffer

Examples of kinds of communication patterns:

1. BOUND METHODS. In typical operating systems a module may express an API wall that is directly bound or statically linked against another module and direct functions (an API wall) is permitted. We have some bias toward non-blocking but a mature architecture supports diverse needs - so this won't do.

2. PIPES. A useful kernel IPC (inter-process-communication) pattern in UNIX is a pipe. Two executables can be dynamically wired together using spontaneously manufactured pipes. Pipes are usually TCP sockets with serialization. This helps users have a mental model and express intent as well. See https://ocaml.github.io/ocamlunix/pipes.html. This conceptual pattern is good but we're inside threads, not between processes - so we want more of a "TPC".

3. PUBSUB. Another useful pattern is pubsub - or basically "events". Here one thread can publish an event or message that other threads can choose to listen to. See https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern . Also see https://en.wikipedia.org/wiki/Message-oriented_middleware . Also see https://docs.rs/flo_stream/0.6.0/flo_stream/ . This is the pattern we implemented. Right now we have a global pubsub broker but arguably we should support threads directly binding to each other (permissioned by a broker).

4. SHARED MEMORY. Another pattern is shared memory. This is probably mandatory for dealing with large buffers moving between threads such as a camera stream capture buffer thread and a tensorflow image analysis thread. The challenge here is less technical and more around formalizing patterns for sharing memory. See https://crates.io/crates/shared_memory . We will do this later.

5. RUST PATTERNS. Rust has some design thinking around messaging. See https://medium.com/@polyglot_factotum/rust-concurrency-patterns-communicate-by-sharing-your-sender-11a496ce7791 and https://doc.rust-lang.org/book/ch16-02-message-passing.html .

KERNEL THREADS

We are managing "heterogenous" threads - each different. Computation can (in abstract) be arranged on a spectrum from say functionlets to large apps - and it's important for us to get the sizing right for the computational units we are managing. We're seeing units that are effectively smaller applications, relying on shared libraries. Approaches include:

1. Rust based thread runner. This is the pattern we have been exploring. It gives us full granular control over threads without any particular limits (Rust has full access to any system capabilities). There is some work to make sure that any inter-thread communication can be thrown over the wall between threads efficiently. Note that we do not use "processes" but rather simply shared memory threads. We are relying on WASM sandboxing for isolation between modules.

2. WASI Nanoprocesses? This was rejected as an approach. See https://www.infoq.com/news/2020/05/webassembly-security-nanoprocess/ and https://icwe2020.webengineering.org/wp-content/uploads/2020/06/ICWE2020_keynote-David_Bryant.pdf . The criticism I have of this design pattern right now it is presumes a bulky "kernel" to which the nanoprocess "gets permission" to have access to "capabilities". But in our architecture there is no broadly reaching set of "kernel capabilities". Most capabilities are provided by OTHER modules.

3. ROS and RTOS techniques should be reviewed as well (see solutions like https://www.ros.org/ ).

4. ASYNC. This was rejected. We can't do cooperative multi-tasking for possibly non-consensual apps. Developers may want async, promise and so on (in Javascript it's an almost universal pattern at this point) but at our level we wont' deal with it for now. https://rust-lang.github.io/async-book/01_getting_started/02_why_async.html .

THREAD MODULE REGISTRY

We note that there will need to be a formal registry of the modules that any given instance has downloaded. This will enumerate permissions as well on a per module basis. This existed previously but now simply grows in importance. That said, there may be emerging standards (such as WAPM) that may be useful. Registry services include:

1. Only allows one copy of a thing to run? This would be the default (our modules are typically 'applications').
2. Message interfaces
3. Any install or uninstall behavior?
4. Other module dependencies (with version numbers)
5. Current version number
6. Cryptographic keys for trust graph
7. Trusted/Untrusted
8. Permissions/Capabilities
9. Native or WASM

EXPOSED KERNEL CAPABILITIES

The kernel itself formalizes services that MAY be visible to the modules that have appropriate permissions. We can formalize the set of kernel capabilities as services like so:

1. Enumerate Modules available / Module Search
2. Module load
3. Send message to module
4. Start / stop thread
5. Create / destroy a pipe etc
6. Whats running currently

GRAPHICS

Graphics is unfortunately special. It's worth looking at the completely fresh vision that people are using now: https://www.youtube.com/watch?v=CvWWcCvhV3w . This still needs to be documented more clearly and we see significant effort focused on this in the next while.