See Orbital - Project Europa for source code. See Orbital Demo for a demo.

Orbital Europa Architecture (January 2022 ~ June 2022)

This is the first pass testbed architecture for our work in 2022. The goal of this testbed is to exercise ideas around an SDL, to identify the concepts, objects and principles that we will then refine over time.

The example parts consist of these pieces:

• Bootstrap Support
• Services Hypervisor
• View Service
• Flo Service
• Manifest Example
• Server Key-Value Store / Websockets

Bootstrap

The bring-up of the system itself is done from user-land, and so is typically succinct. In the sample main entry point we order the Services layer to fetch an application manifest and produce and run that application (to do whatever the manifest dictates). The rest is taken care of by the internals:

Its instructive to see just how succinct our expression of work is - especially since this is effectively the grammar we are exposing to novice users:

SERVICES.channel({ service: "*:/services/flo", load: "*:/apps/metaverse1/metaverse1" })

See https://github.com/orbitalweb/europa/blob/main/public/main.js for the raw main entry point to the entire application.

Hypervisor / Service Manager Concept

This year we have the concept of a services microkernel or hypervisor (similar to last year). We essentially continue to have a consensus that Orbital is an operating system, or an "application runner" where apps and services interact.

To deliver on this, the heart of Orbital is a recognition that there needs to be some kind of message router that can route messages to services on demand, but that also has an added responsibility of producing those service off disk (or from a remote end point) if needed. This seems to be a constantly emerging core requirement for every architectural variation we explore.

This touches on several related patterns:

• We find similarities here to Erlang and the Actor Model.
• We see a pub-sub messaging model emerge.
• We see an NPM, DENO or Rust Crates like package managing requirement, with a high need for secure packages to be fetched over the net and persistently cached. Package Managers are a common concept - we see them in Unity, in C++, in Unix. Here we have specific risks such as dynamically loading at runtime into client environments, versioning, entry points, permissions, dependencies, provisioning and so on. See https://en.wikipedia.org/wiki/Package_manager for more details. We only implement the absolute barest bones in our version.

See our minimal services handler here (https://github.com/orbitalweb/europa/blob/main/public/services/services.js)

View Service

One of the significant frustration points we ran into last year was the nascent state of Rust graphics rendering. Modern rich app ecosystems have extremely rich tooling for rendering images, fonts, boxes, circles, geometry, shadows, 2d and 3d objects as well as for dealing with word-wrap, buttons and other typical visual elements. Although our focus is NOT visual graphics specifically, it was difficult to tell a story about use-cases without having to re-invent and re-integrate significantly heavy libraries and tooling, that themselves simply were not mature.

This year we avoid these issues by carefully writing an abstraction layer that talks to Babylon3D. We pretend we are talking to a graphics engine at a high degree of remove, avoiding any direct dependencies other than math libraries, and effectively "throw" all of our requests over the fence to View Service. This lets us have high quality visuals and lets us exercise the rest of the ideas rather than getting hung up on visualization.

The plan is at the end of this year to switch to Bevy3D - taking our mature abstraction layer and porting that to Bevy.

See (https://github.com/orbitalweb/europa/blob/main/public/services/view.js)

FLO Service

Last year we noticed that index.html in a traditional web browser is effectively a manifest that orders the browser to produce a number of assets and to ultimately produce an application. Although html tags are still used, most of the time browsers are no longer used to only produce declarative layouts but rather to stand up and run interactive applications as specified by the manifest. Our engine has a similar pattern.

Flo is our application manifest parser. It is a service loaded and managed by the kernel. It is brought up on demand by the user. A typical command to Flo is to order it to produce an application from a manifest. A manifest is our application abstraction. It is a document that describes services to load, how to wire them to each other, and what state to pass them initially.

From our goals for this year:

The Flow Language allows for a high level, formal, and consistent definition of software components and their relationships to each other. It has a specific emphasis on granular security with the intent of allowing heterogeneous components written by different developers to co-exist in the same computational sandbox without stomping on each other. This solves a problem in that historically grammars for expressing scenarios are fragmented and developers are forced to either switch grammars or (more often) express their highest level concepts in raw procedural code - which makes re-composing and re-purposing scenarios harder, and inaccessible to novices. There’s value in a declarative “manifest” that defines the components of a system, application or scenario in a durable, portable, simple consistent way and we found that as we worked on Orbital that this grammar emerged as a consistent organizing principle.

See (https://github.com/orbitalweb/europa/blob/main/public/services/flo.js)

One discovery this year is that we found is that it is much more convenient to have a lightweight scripting language capability rather than a purely declarative grammar. As such Flo can parse and run a scripting language (in this case Javascript for convenience but any scripting language will do). The reason for this is that it is highly convenient to be able to decorate service message handling with small on-the-spot procedural event handlers, rather than having to refer to a separate file or document. A typical example of this is to wire the output of one service to the input of another.

Manifests and SDLS

As mentioned above "A manifest is our application abstraction. It is a document that describes services to load, how to wire them to each other, and what state to pass them initially."

In our test application we begin to define SDLS - which is the heart of our actual focus this year. From our design goals: SDLs are common, but often domain specific. For example https://carla.org/ has a grammar for describing vehicle related scenarios, as does https://www.safetypool.ai/database . However typically in consumer applications larger systems are expressed procedurally in code (that is then often compiled) and high level components and their relationships are not easy to rearrange dynamically, or by novices.

We're extremely interested in finding ways to succinctly express complex ideas in a right sized grammar that is accessible to everybody but that isn't heavy, coercive or otherwise unusable. We should be able to use it to describe both entire applications and also to describe state to send to services.

Here is the application manifest for our bootstrap demo:

(https://github.com/orbitalweb/europa/blob/main/public/apps/metaverse1/metaverse1.js)

In this manifest we begin by defining a pile of 3d objects

let myground = {
	uuid:"/myground",
	kind:"box",
	network:"static",
	created:0,
	updated:0,
	physics:{shape:"box",mass:0},
	receivesShadows: true,
	xyz:[0,-0.1,0],
	whd:[3,0.1,3],
	mat: {
		alpha: 0.5,
		heightmap: "./apps/metaverse1/textures/heightMap.png",
		art: "./apps/metaverse1/textures/ground.jpg",
	}
}

Our grammar is procedural, but in this case we're using a procedural grammar to make a bunch of declarations.

These declarations can be thought of as our "scenario". And the notation can be considered an "SDL" or a scenario definition language.

Ultimately we order the Services hypervisor to manufacture a couple of services - that will chew through the SDL and produce whatever they say:

let view = await SERVICES.channel({ service:"*:/services/view", observe:"*" })

let net = await SERVICES.channel({ service:"*:/services/net" })

Philosophically what's going on here circumvents some serious design flaws in traditional application engineering. Because we are passing everything as messages, at a high level, at the right degree of abstraction, this document remains succinct and "appropriate". We do however pay some costs. This is a message based event model and it means it lacks the performance of natively bound raw functions that directly invoke the requisite library. We believe that services themselves should be tightly bound code modules (written in WASM) but that at a high level an application should necessarily be composed out of dynamically fetched services bound together - rather than as monolithic binary blobs.

Server Key-Value Store and Websockets

We do have one special dedicated component for our test environment which is a dedicated key-value store server that acts as a multicast hub to echo traffic to client instances. This is stand-in code with the idea that later this would be swapped out for a higher performance key-value store such as Anna. In our current implementation the server also serves the web pages that drive the app. See:

(https://github.com/orbitalweb/europa/blob/main/server.js)