ORBITAL DESIGN AND CODE DOCUMENT FOR M1/M2/M3 - This is R1 of the general vision.

Feb 01 2021

The goal of this document is to answer questions like: “What are we doing next?”, “What’s the actual plan?”, “Why are we doing this now?”.

This is the first revision of thoughts (R1). Each successive revision of thoughts will re-iterate and re-arrange thinking based on new evidence, ideas, input and test results. So the "plan" will evolve as the thinking evolves; focus will shift to different topics as they are deemed more important or less important or are solved.

Today we seek to first lay out a general roadmap broken into M1,M2,M3 (Milestone 1, Milestone 2, Milestone 3). Milestones are broader than revisions. M1/M2/M3 reflect the "big picture" and are stable. Revisions however are smaller and more frequent. There can be R1,R2,R3,R4,R5 to R-INFINITY to attempt to achieve that picture. R numbers do not correspond to M numbers. R numbers are just a way of defining any "epochs" or "plateaus" of comprehension as ideas come together in my mind (and where I am often playing catch-up with to where other minds have already reached).

Our schedule is something like this:

M1: Define / Get this document up, get a kernel running, and get one example on that kernel such as a camera cv demo M2: Exercise / Rev initial code prototype for broader usability (I think some kind of lightweight 3d user editable world is a good goal). M3: Polish / work on smoothing out how computation moves around and is sandboxed; bolster existing demos.

M1 (Milestone 1): Prototyping support for persistent apps

Our M1 technical goal (this goal) is a kernel or foundation for dynamically loadable durable persistent apps. In some ways this can be thought of as similar to what an app store needs, or a desktop operating system needs. Apps exist "somewhere" and are durable and persistent on the client, and sandboxed in a way that they cannot interfere with each other or with other user files and assets; and can do useful work.

M1 is to "cause to exist" an absolutely bare bones microkernel with one bare bones app example on top of that. I would prefer to avoid writing this but I will do it if I have to. There are many wasm module runners out on the net (such as Fastly) but we cannot find anybody who has written a wasm module runner where modules talk to each other. All the module runners treat the modules as second class citizens that are privileged and permissioned to access an exterior set of services. The conceptual model is that there is a previously existing substrate that has "all the powers" and the modules are given access to some of those powers. Our model is different. In our model there is nothing else except modules, and modules have to be permissioned to speak to each other. We are a bit like docker, a kind of hypervisor or a virtualized operating system inside an operating system; providing a consistent safe sandboxed environment for microthreads with pre-emptive scheduling (rather than voluntary scheduling). I'd love to see even tiny threads such as say individual avatars in a video game being represented by our kernel threads even.

We may even want to throw this away after we are done and use a third party service once somebody invents it. We're not attached to "owning" this piece, but we have to write it because nobody else has written it yet. Ultimately we seek to stand on the shoulders of giants, leveraging Servo, Rust and other tools. Rust itself already provides a crates like concept which is a good starting point. But we will want to carefully sandbox anything that we’re fetching from remote and allowing to run locally.

Our work is ultimately user facing - so although this rev is a bare bones microkernel we do want to hypothetically sketch out a user facing interface for how users would interact with rich apps (SLAM algorithms, video games, rss news readers) on that kernel (seeing them, setting permissions, starting and stopping). We will want to imagine (but do not write at this time) a user visible "app store" or place where they can see apps they could fetch to their device and manage them. From a user perspective we position ourselves as something somewhere between a "web browser" (which typically provides only extremely weak concepts of persistence such as PWA) and an "operating system" (which has very strong concepts of persistent apps and permissions.

Approach:

1. Understand Servo threads; can we just use the Servo threads capability to manage our persistent apps? Is Servo good enough?
2. Define whatever it takes to have some kind of sandboxed computational environment with loadable and unloadable apps.
3. Expose an API that allows userland scripts to talk to this sandbox.
4. Write a userland web app that demonstrates this. A good first example could be an app that uses the video camera to identify faces.
5. Target only a single piece of hardware for first version (MacOSX)

Validation:

1. Can we fetch an "app" at runtime and run it (such as say a camera app that uses the video camera to identify faces).
2. Are apps persistent?
3. Are apps sandboxed?
4. Can we throttle CPU on an app?
5. What would a user interface for browsing apps look like? Will it look like some kind of desktop? (Just draw a picture).
6. Can apps talk to devices (such as say a camera or a display?)
7. Can apps get user events and interaction?

Challenges:

1. Can we use Servo or do we need a separate microkernel? I'm finding it hard to find a microkernel that delivers what we want. I do see some ideas such as https://github.com/tomaka/redshirt and there are many many others as well.
2. Is WASI the right way to sandbox filesystem access? Are all apps therefore ultimately WASM modules or not?
3. Who manages threads? See https://github.com/unrelentingtech/rusty-sandbox . Also see https://github.com/servo/gaol .
4. Should we just support the whole browser stack? Javascript, WebXR, WebGPU and so on?

M2 (Milestone 2): Flexing Capabilities and new Hardware support

Our M2 goal is to look more at real future apps. It’s almost a given that typical media experiences on the web will shift away from look-at style text interfaces that block the world to look-through style blended interfaces that add digital buttons or verbs to the real world. Even for people without eyesight or hearing, there may be a ‘situated in place’ or ‘embodied’ representation of actions they can take. To support this we also want to explore having the browser act as a user agent to filter between competing inputs and reduce noise.

For M2 our goals are to be more user facing. Can ordinary uses use this app? Are we able to leverage other tools (such as say other common browser features / capabilities that people take for granted?

Goals:

1. Build any simple multiplayer VR world demo as an example “rich app”; basically something that has eye appeal.
2. Incorporate an ordinary web browser as an application; Servo would be a good example.
3. Test build on new hardware; be on more than one piece of hardware.

Challenges:

1. Test allowing multiple apps to write to the same view and scene graph looks increasingly critical.
2. Which new hardware is best? Ideally would prefer AR glasses or something truly new.
3. Android may be a good target for new hardware.

M3 (Milestone 3): Feature Matrix and automated builds

Our M3 goal is to look at 'computational seas'. Compute wants to move around easily; from CPU to GPU, from device to device, from the cloud to cell tower compute, to device compute. We want to express "intent" and move "intent" or high level expressions of work (aka computation) around easily anywhere. We’re especially interested in portability and ease of compilation to hardware that does not exist yet. New hardware will emerge increasingly rapidly. It should be straightforward for hardware vendors to compile this browser and have access to the entire browser ecosystem quickly even if their hardware did not exist when we wrote this.

Approach:

1. Formalize a build environment for third party contribution; anybody should be able to download test build with demos.
2. Document
3. Using Jenkins we’ll automate build with unit tests across all platforms
4. All pass/fails will be reported to a feature matrix visible on the website
5. Pick a third platform and port
6. Write up each piece of this research and publish on web; encourage feedback and participation.