frame_assembly_and_compositor_spec - mark-ik/graphshell GitHub Wiki
Date: 2026-02-28 Status: Canonical interaction contract Priority: Active (Stages 1β4 complete; Stage 4b implemented)
Related:
ASPECT_RENDER.md../../../archive_docs/checkpoint_2026-03-22/graphshell_docs/implementation_strategy/aspect_render/2026-02-20_embedder_decomposition_plan.md-
../PLANNING_REGISTER.mdΒ§0, Β§0.10 -
../workbench/workbench_layout_policy_spec.mdΒ§3.2, Β§3.4 ../viewer/viewer_presentation_and_fallback_spec.md-
../../TERMINOLOGY.mdβCompositorAdapter,TileRenderMode,Composition Pass,Surface Composition Contract
Adopted standards (see 2026-03-04_standards_alignment_report.md Β§Β§3.6, 3.7)):
- OpenTelemetry Semantic Conventions β diagnostics channels (GL state violation, chaos mode, frame timing) follow OTel naming and severity conventions
-
OSGi R8 β
TileRenderModeresolution andCompositorAdaptercapability dispatch follow OSGi capability vocabulary
This spec defines the canonical contracts for:
- Composition pass ordering β the three-pass frame model and ownership rules.
- TileRenderMode β the render pipeline classification for each tile.
- CompositorAdapter β lifecycle, GL state isolation, callback ordering.
-
Embedder decomposition seams β
EmbedderCorevsRunningAppStateboundary. - Frame loop coordination β begin/layout/paint ownership.
This spec covers the current render architecture and what is being actively built. The egui_glow β egui_wgpu UI-backend cut has already landed; the remaining renderer work is the deeper WebRender/runtime bridge path rather than the old egui backend swap.
Historical note: the earlier composited-viewer contract note has been archived. Any still-relevant future-work ideas from its Appendix A are tracked in ../PLANNING_REGISTER.md Β§0.10 rather than in a separate active render-contract doc.
Every node viewer pane tile frame is composed in three ordered passes:
| Pass | Name | Owner | Content |
|---|---|---|---|
| 1 | UI Chrome Pass | Render aspect | Tab bar, badge overlay, focus/hover rings, tile chrome |
| 2 | Content Pass | Viewer (via CompositorAdapter) | Web content, native content, or fallback surface |
| 3 | Overlay Affordance Pass | Render aspect | Selection rings, diagnostic affordances, pointer hit targets |
Invariant: Pass ordering is Graphshell-owned sequencing and must not rely on incidental egui layer behavior. The three-pass order is always Chrome β Content β Overlay. No pass may mutate the output of a preceding pass.
Invariant: The CompositedTexture TileRenderMode renders Overlay Affordance Pass content over the composited texture in the pipeline; the NativeOverlay mode renders affordances in tile chrome/gutter regions because native content owns its own window region.
TileRenderMode is the runtime-authoritative render pipeline classification for a node viewer pane tile. It is resolved from ViewerRegistry at viewer attachment time.
TileRenderMode =
| CompositedTexture -- Servo GL texture composited into egui frame
| NativeOverlay -- native window overlay (e.g. Wry); owns its own region
| EmbeddedEgui -- egui-native content (native viewers, metadata card)
| Placeholder -- no viewer attached; fallback surface
Invariant: Every NodePaneState must have a TileRenderMode set at viewer attachment time. A tile must never enter the compositor dispatch without a resolved TileRenderMode.
Resolution path: ViewerRegistry::resolve_mode(viewer_id) -> TileRenderMode. The resolved value is stored on NodePaneState and updated only when the viewer changes.
| Mode | Content Pass behavior | Overlay Pass behavior |
|---|---|---|
CompositedTexture |
Invoke CompositorAdapter GL callback; render Servo texture into tile rect |
Render affordances over texture in tile rect |
NativeOverlay |
No GL callback; native content owns its region | Render affordances in chrome/gutter only (not over native content) |
EmbeddedEgui |
Render via normal egui widget tree | Render affordances as egui overlays |
Placeholder |
Render fallback surface (loading indicator, error state, empty state) | Render affordances over fallback |
Workbench layout policy may derive visible navigation geometry that differs from the logical navigation-region remainder when overlay-form Navigator hosts occlude part of the workbench surface.
Current runtime rule:
- Runtime consumers that make visibility or placement decisions should honor the full visible navigation rect set rather than collapsing immediately to one fallback rect.
- This includes viewport culling, diagnostics geometry summaries, and floating overlay placement.
- Graph/input consumers follow the same rule via the runtime-carried typed visible-region contract.
- The remaining follow-on work is promotion of visible navigation geometry into a first-class pane/render contract so the canonical render model itself no longer speaks in single-rect terms where a derived visible region set is the real authority.
See workbench_layout_policy_spec.md Β§3.4 for the authoritative definition of
logical navigation region versus visible navigation geometry.
CompositorAdapter wraps backend-specific content callbacks. It owns:
- Callback ordering within the Content Pass.
- GL state isolation (save/restore GL state around callbacks; the callback must not leak GL state into the egui render path).
- Clipping and viewport contracts (the callback renders only within the tile rect).
- The post-content overlay hook (called after content callback, before the Overlay Affordance Pass).
Every CompositorAdapter callback must:
- Save GL state before invoking the content callback.
- Restore GL state after the callback returns, regardless of whether the callback succeeded or panicked.
- The egui render path must observe consistent GL state before and after a compositor callback.
GL state diagnostics: Violations (leaked GL state) must be observable via the diagnostics channel schema. A GL_STATE_MISMATCH channel event is emitted when state restoration fails.
Content callbacks are registered at viewer attachment time. A callback is a function of type:
fn render_content(tile_rect: Rect, clip_rect: Rect, gl_state: &mut GlStateGuard)
Callbacks are unregistered at viewer detachment time. A tile with no registered callback falls back to Placeholder rendering.
The historical RunningAppState monolith conflated embedder and app-layer responsibilities. The decomposition boundary is:
EmbedderCore (embedder responsibility) |
RunningAppState (app-layer responsibility) |
|---|---|
| Servo instance |
EmbedderCore (owned) |
Windows map (HashMap<EmbedderWindowId, EmbedderWindow>) |
AppPreferences |
| Event loop waker |
Gui handle |
WebViewDelegate + ServoDelegate trait impls |
Intent queues |
| WebDriver channels | Gamepad provider (app-level routing) |
EmbedderCore invariant: EmbedderCore must not hold references to graphshell app state (preferences, intent queues, graph data). It communicates via GraphSemanticEvent emissions only.
GraphSemanticEvent is the clean boundary: All semantic information crossing from the embedder layer into the graphshell graph/app layer must pass through GraphSemanticEvent. No direct embedderβgraph calls.
| Stage | Status | Description |
|---|---|---|
| 1 | Complete | Semantic bridge extraction (semantic_event_pipeline.rs) |
| 2 | Complete | Toolbar decomposition (7 focused submodules) |
| 3 | Complete |
CompositorAdapter extraction (wraps rendering paths EmbedderCore exposes) |
| 4a | β Complete |
shell/desktop/ui/gui.rs frame orchestration isolated from workbench layout driving |
| 4b | β Complete |
EmbedderCore/RunningAppState boundary closure plus host-runtime service extraction (WebDriverRuntime, GamepadRuntime, EmbedderWindow internal service splits) |
Historical Stage 4 sequencing note: Compositor pass-order correctness and GL-state diagnostics hardening landed before the Stage 4b decomposition follow-through. See viewer/2026-02-26_composited_viewer_pass_contract.md Appendix A for the sequencing rationale.
The egui frame loop coordinates three phases:
-
Begin frame β
ctx.begin_frame(input): owned by the Render aspect entry point. - Layout pass β widget tree construction and layout computation: owned by Workbench (tile tree traversal) + Viewer (per-tile content).
-
Paint β
ctx.end_frame()+ GPU surface present: owned by the Render aspect.
Invariant: Workbench must not call ctx.begin_frame or ctx.end_frame. It participates in the layout pass only. Frame start/end are Render aspect responsibilities.
Invariant: The compositor dispatch (CompositorAdapter callbacks) runs within the layout pass, after the tile rect is known and before end_frame. It does not run between begin/end frame boundaries.
| Criterion | Verification |
|---|---|
| Pass ordering is always Chrome β Content β Overlay | Test: instrument pass entry points β verify ordering across 100 frames |
| GL state is identical before and after a compositor callback | Test: capture GL state before/after callback β no differences |
| GL state mismatch emits diagnostics channel event | Test: inject a leaking callback β GL_STATE_MISMATCH event in diagnostics |
Every NodePaneState has a resolved TileRenderMode
|
Test: attach viewer β tile_render_mode field is non-null |
NativeOverlay affordances render in chrome/gutter only |
Test: NativeOverlay mode β no affordance draw calls inside native content rect |
EmbedderCore emits no direct graph mutations |
Architecture invariant: no graph_app.* calls from EmbedderCore module |
GraphSemanticEvent is the only crossing point |
Architecture invariant: all embedderβapp communication passes through GraphSemanticEvent
|
| Compositor callback is unregistered on viewer detach | Test: detach viewer β callback list is empty; tile falls back to Placeholder
|
The following section records the design analysis for eliminating redundant render pass open/close cycles per frame (C4 in the compositor optimization sequence).
C3 batched command encoding β all draw calls now share a single CommandEncoder per frame
via pending_encoder. But each draw_instanced() call still creates its own
begin_render_pass() / drop cycle. A typical frame has 50-200 draw calls across ~10-20
unique render targets, meaning 50-200 render pass cycles where ~10-20 would suffice.
Render pass creation is expensive: the GPU driver must resolve load/store ops, flush tile
memory (on tiled GPUs), and validate attachments. Sharing one render pass per target
eliminates this overhead.
RenderPass<'a> borrows the CommandEncoder mutably. If we stored both the encoder and
the pass on WgpuDevice, we'd have a self-referential borrow. Rust forbids this.
A scoped closure approach also fails: while the render pass borrows the encoder, we also
need to call methods on wgpu_dev (get_pipeline, create_bind_groups, etc.) β and those
require &mut self. If the encoder is still on wgpu_dev, we can't take &mut self.
A draw-command-recording approach (collect Vec<DrawCmd> per target, replay in one pass)
avoids lifetimes but adds per-frame allocation and borrow complexity from
TextureBindings<'_>.
The take/put-back encoder is the most pragmatic solution:
- Encoder is an
Option<CommandEncoder>field onWgpuDevice. - Renderer calls
wgpu_dev.take_encoder()β takes encoder out, makingwgpu_devfreely usable for&selfand&mut selfmethods. - Renderer creates render pass locally from the taken encoder, issues N draws.
- Render pass is dropped (closing the pass on the GPU).
- Renderer calls
wgpu_dev.return_encoder(encoder)β puts encoder back.
The key insight: while the encoder is taken out, wgpu_dev is fully accessible for
pipeline lookups, buffer creation, bind group creation. The render pass is a local
variable in the renderer β clean lifetime.
// renderer code:
let mut encoder = wgpu_dev.take_encoder();
let target_uniforms = wgpu_dev.create_target_uniforms(target_w, target_h);
{
let mut pass = encoder.begin_render_pass(&desc);
for batch in batches {
let pipeline = wgpu_dev.ensure_pipeline(variant, blend, depth, fmt);
let (bg0, bg1) = wgpu_dev.create_bind_groups(..., &target_uniforms);
let ibuf = wgpu_dev.create_instance_buffer(data);
pass.set_pipeline(pipeline);
pass.set_bind_group(0, &bg0, &[]);
pass.set_bind_group(1, &bg1, &[]);
pass.set_vertex_buffer(0, wgpu_dev.unit_quad_vb.slice(..));
pass.set_vertex_buffer(1, ibuf.slice(..));
pass.set_index_buffer(wgpu_dev.unit_quad_ib.slice(..), Uint16);
pass.draw_indexed(0..6, 0, 0..count);
}
} // pass dropped here
wgpu_dev.return_encoder(encoder);impl WgpuDevice {
pub fn take_encoder(&mut self) -> wgpu::CommandEncoder { ... }
pub fn return_encoder(&mut self, encoder: wgpu::CommandEncoder) { ... }
pub fn create_target_uniforms(&self, width: u32, height: u32)
-> (wgpu::Buffer, wgpu::Buffer) { ... }
pub fn ensure_pipeline(&mut self, variant: WgpuShaderVariant,
blend_mode: WgpuBlendMode, depth_state: WgpuDepthState,
target_format: wgpu::TextureFormat) -> Option<&wgpu::RenderPipeline> { ... }
pub fn record_draw<'a>(&mut self, pass: &mut wgpu::RenderPass<'a>, ...) { ... }
}-
draw_passes_wgpu()picture cache loop: take encoder βcreate_target_uniformsβ create render pass β loop batches (prepare bind groups + instance buffer + record draw) β drop pass β return encoder. -
draw_cache_target_tasks_wgpu(),draw_clip_batch_list_wgpu(),draw_quad_batches_wgpu(): same take/put-back pattern per target. - Texture cache targets: the per-target loop in
draw_passes_wgpucreates ONE pass and passes it to each helper function. - Scissor reset: when switching from a scissored draw to non-scissored within the same
pass, explicitly
set_scissor_rect(0, 0, w, h)to reset.
- Add
take_encoder(),return_encoder(),create_target_uniforms(),ensure_pipeline()toWgpuDevice. Keep existingdraw_instanced()working. - Migrate picture cache rendering in
draw_passes_wgpu(). - Migrate texture cache target rendering (one pass per target across all helpers).
- Optionally unify composite path (
render_composite_instances_to_view()). - Remove old
draw_instanced()if all callers migrated.
-
Medium: Render pass must be created with correct depth attachment upfront β
has_opaqueis computed before the batch loop, so this info is available. - Medium: Scissor rect state persists within a pass; must explicitly reset.
- Low: Pipeline/bind group changes within a pass are standard wgpu operations.