THEORY

Review 2021

Last year we saw a need to be able to express richer concepts than browsers typically allow. Browsers normally focus on layout, and have hacked on support for loading and running javascript. We saw instead a need for dynamically loading apps, made out of many parts, in durable ways, that can run in the background, are secure, lightweight, late binding - effectively more of an operating system or app runner rather than a browser as we know it. We felt that the best use of time this year was to define a grammar for describing complex yet typical scenarios beyond merely layout.

Overview 2022

The goal of this years work on Orbital is to define and exercise an SDL (scenario definition language) for expressing complex software systems, such as applications, made out of reusable parts with an emphasis on lightweight composability and formal security policies to allow such systems to run in third party sandboxes with some degree of trust. We are looking for a 'right sized' high level grammar for describing these systems that is accessible to novices - anybody should be able to wire together a simple app out of parts and share it with their peers

Tensions with classical apps

Lean towards static content or limited user creativity. We all create text, images, video and other 'static' content but very few of us create interactive content. The reasons for this are several fold. One of the main reasons is that software programming tools are not exposed to consumers. Another reason is that safe computation is not mature; sandboxing is not a mature art yet. It is true that writing interactive experiences can be harder, but it also provides a way to more succinctly describe larger topics. We believe a future browser reduces these barriers.

Overly restrictive security. Often when apps are shipped to devices such as the iphone or android there is a special emphasis on security and sandboxing perms and privs of those apps. They have to specially request access to services such as the camera or user storage or the network. Beyond this as well throttling their CPU access is important so that they don't saturate local computational resources. Security as a whole is poorly expressed industry wide, and instead we see a walled garden approach with a careful curation of app stores to prevent bad actors from publishing apps. This human intervention model however restricts healthy conversations on the net by censoring apps not just for safety and security but for the whims of the current walled garden vested interests.

Poor persistence Apps that run on the web have a further problem in that they don't have persistence.

No background Agents. And, on mobile, they cannot run in the background in the sense of doing real work such as scanning a scene with a camera.

Modal. And apps as a whole have a design defect in that they are modal - which is a pattern that reflects single focus devices but that doesn't reflect upcoming Augmented Reality interfaces where apps will instead be "decorators" that enhance or augment existing objects, or augment other apps, rather than being fully in control of a view.

Complicated to write. Stepping back, an app is a bundle of content, libraries and event handling and scripting glue all wired together. There is a common pattern of a 'manifest' that describes an app - and we see this pattern in many tools such as rust crates, javascript npm modules, and Unity and Unreal app packages and apps that run in the App Store. Even web pages have a similar pattern where the index.html describes assets including scripting to load and run. We propose to push this pattern into the core of our browser more formally. This implies that our browser takes on a more clearly defined role as an 'app runner' including concepts such as a hypervisor to manage threads, inter-app messaging, app loading and unloading. There are typical patterns we need to comply with; operating systems often use cryptography to validate dependencies for example - and we need to do similar things to prevent bad actors. But this shouldn't mean that apps are an impossibly complex thing to create or write. None of this is extremely hard at a high level.

Shared views or synthesis. Apps today do not augment existing shared views nor use shared state - rather they own the view and ignore each other.

To move to the next generation of apps, where apps can be granular, composed from existing pieces, distributed in a computational soup, where they can be trusted without relying on a walled garden as a trust broker - a much more formal model of what an application is is needed.

Age of Synthesis

We live in a time where the real and the virtual are merging back together. Although we are not strictly focused on "3d visualization" the fact is that our customers do live in the real world, and augmented reality headsets will mean there's some emphasis on 3d understanding, annotation and "information in place".

The real challenge of the upcoming metaverse is not to merely decorate an augmented reality world with 3d geometry, art and other assets however. Static assets are somewhat meaningless - the real world is not merely a "3d wiki".

The challenge is more to allow participants to describe and share agents or behavior that can run in a durable and safe way, that others can view and interact with, and that doesn't overwhelm their view or in other ways provide a bad experience. If applications are expressed more formally they can take advantage of networks to fetch pieces, they can show up on devices and be reasonably scoped as to local permissions and powers, and can be generally easily distributed without a walled garden. App stores are not needed if apps can be more formally defined.

Outcomes

If we can make it easier to share apps then this helps humanity be truly creative; to express and share ideas more richly, not with just static images. We could share living sketches of our ideas; vignettes, stories, games, architectural visions, procedurally generated music, art and other experiences that right now are relegated only to expert programers. If we can scope what apps are better then we give people powerful ways to speak and communicate ideas - not just one essay or one image or one idea - but a space of ideas that is an interactive exploration of a topic space. The kinds of apps we can imagine include:

• a store agent that takes your order at a store.
• an app that warns you about a pedestrian traffic closure near you
• or that lets you and a group of friends socially organize and curate a bunch of photos of an event
• mark up the real world with post it notes or verbs
• tools for telling stories with rich visual media
• creating a social ar musical instrument or a game in a park
• or a farmer specifying the navigation route for their farmbot
• or even play-testing the impacts of a new law on a community with a local digital model

Basically the real world needs to be programmable; participatory, and open.