Deep Dive on WebXR

I decided to throw out all this. It's just an incredible waste of time. OpenXR is probably the better solution. So these notes are legacy:

What is WebXR? How is it implemented?

At a design level WebXR is intended to provide a wrapper for VR/AR devices and inputs. This helps programmers write to one consistent interface.

https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API/Fundamentals

There is an implementation in Rust that we can arguably leverage. The rust version provides what appears to be an ordinary rust set of method calls (an API) to handle device inputs such as hand controllers. It claims to be independent of the web. It may also be able to bring up a display (although that isn't clear because the people who wrote it provided zero examples of how to use it). The earlier WebVR version apparently can bring up a display but it has suffered bitrot and no longer is maintained or runs. https://github.com/servo/webxr ...

As far as examples go this is it ( from https://paper.dropbox.com/doc/Rust-webxr-plans--BOlqqibXsSO9RUVgkooGWSs_Ag-xmqpUtCzy8yNMGWwUUxsz ):

	struct MyDevice {..}
	impl XRDevice for MyDevice {..}
	struct MyDiscovery;
	impl XRDiscovery for MyDiscovery {
	    fn supports_session(&self, mode: XRSessionMode) -> bool {..}
	    fn request_session(&mut self, mode: XRSessionMode, xr: XRSessionBuilder)
	        -> Result<XRSession, XRError>
	    {
	        if !self.supports_session(mode) { return Err(XRError::NoMatchingDevice) }
	        xr.spawn(|| {
	            // Device initialization
	            MyDevice {..}
	        })
	    }
	}

There's pseudo documentation on WebXR from JS: https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API/Lifecycle . Also see the spec at https://www.w3.org/TR/webxr/

There's no real documentation or support. It is not not totally clear how to drive it or what it does even. Are there examples? Does this thing actually open displays? Talk directly to remote devices? Can I hook up controllers and get input state from them?

https://servo.zulipchat.com/# <- this group apparently is where discussions happen... I've asked.

Binding WebXR to Javascript via WebIDL?

From some points of view a browser has become a javascript sandbox. Can we wire up spidermonkey?

https://medium.com/programming-servo/programming-servo-my-own-private-runtime-8a5ba74c63c8. <- some good advice https://medium.com/programming-servo/programming-servo-just-hanging-around-d8f660c33df2

See https://github.com/servo/rust-mozjs which wraps https://github.com/servo/mozjs .

Apparently Mozilla is using a super duper old version of XCode : https://firefox-source-docs.mozilla.org/setup/macos_build.html#macos-sdk-is-unsupported ! I did go and fetch this version - and hard coded a link to it in the makefile.cargo ... still it terrifies me that it is this old.

WASM-Bindgen and WebIDL notes

To effectively bind WebXR to javascript there's this idea of a WebIDL. In fact any WASM logic has to leap up to JS and then back down to native code apparently? Which seems kind of insane? Or are WASM module methods bound directly to native API's exposed by WebIDL?

https://developer.mozilla.org/en-US/docs/Web/API https://developer.mozilla.org/en-US/docs/WebAssembly/Concepts

"By itself, WebAssembly cannot currently directly access the DOM; it can only call JavaScript, passing in integer and floating point primitive data types. Thus, to access any Web API, WebAssembly needs to call out to JavaScript, which then makes the Web API call. Emscripten therefore creates the HTML and JavaScript glue code needed to achieve this."

See https://hacks.mozilla.org/2019/08/webassembly-interface-types

https://rustwasm.github.io/wasm-bindgen/examples/webgl.html

It looks like browsers as a whole defines the WebIDL: https://heycam.github.io/webidl/

Then there is some kind of web-sys crate: https://rustwasm.github.io/2018/09/26/announcing-web-sys.html

It’s super unclear to me if this just translate RUST -> javascript. There’s no other way to like “get a canvas” or “talk to webgl” right?

Also this Gloo thing is unclear https://rustwasm.github.io/2019/03/12/lets-build-gloo-together.html

Mozilla can load and run WebAssembly obviously… but does it expose API methods to WebAssembly? https://developer.mozilla.org/en-US/docs/WebAssembly

Some people say WebAssembly cannot talk to the DOM -> and they wrote whole bindings -> https://github.com/mbasso/asm-dom

Many people say this … https://sudonull.com/post/62869-WebAssembly-and-DOM-manipulation

Here is another crate that does this too : https://github.com/koute/stdweb

https://www.reddit.com/r/rust/comments/9fnjss/the_relationship_between_stdweb_and_wasmbindgen/

https://github.com/rustwasm/wasm-bindgen

How about Unity3d - how do Unity blobs talk to WebGL? It must be through the same wasm-bindgen scheme as well - so through javascript?

https://docs.unity3d.com/2019.3/Documentation/Manual/webgl-gettingstarted.html

https://www.chromium.org/developers/web-idl-interfaces

Questions and notes

There's something called a Servo Script crate (which presumably "supports" WebXR although WebXR is just a wrapper itself and doesn't actually DO anything). However like most of Rust and most of the community it seems allergic to documentation. There's not even any clear documentation on what it "is". https://summerofcode.withgoogle.com/archive/2018/projects/5111490306637824/ . Is this scripting? Or is that separate? See also : https://github.com/servo/servo/issues/1799

Note that in my case I could write my own bindings for any blob of rust code I intend to compile to WASM. Depending on the maturity of WebIDL I can see a case where a blob of rust code can statically compile to WASM against some bindings (such as for display rendering) and then call what feel like ordinary functions. However since the display is actually a separate thread then messages are required anyway... so there's not a huge win for me to allow rust code to pretend it is calling native methods in some external module.