Renderer

The central renderer is RendererSystem. It is both an ECS system and a render pass, and it contributes frame graph information to telemetry. It owns renderer state and delegates work to specialized subsystems.

This page is the subsystem overview. For the exact per-frame sequence, see Renderer Frame Flow. Platform and GPU Support is the canonical platform and Vulkan requirement reference; Renderer Feature Paths describes runtime path selection. For CSG and transparent rendering, see CSG and Transparency.

Subsystems

RendererSystem coordinates:

• RendererShaderSystem
• RendererMeshSystem
• RendererMaterialSystem
• RendererCsgSystem
• RendererDeferredSystem
• RendererAvboitSystem
• RendererRayTracingSystem
• RendererReflectionSystem

The renderer validates and recreates frame targets when dimensions, sample count, or presentation framebuffer compatibility changes. Pipeline caches must include framebuffer/render-target compatibility, not only material or shader identity.

The implementation is organized by renderer domain under impl/ecs_render: kernel, shared, shader, mesh, material, deferred, csg, avboit, optics, raytrace, and reflection. Public renderer state is also split into per-concern state groups so the subsystem code keeps ownership local while RendererSystem remains the coordinator.

State groups owned by RendererSystem include:

• mesh state
• material state
• draw state
• CSG state
• deferred state
• AVBOIT state
• ray tracing state, including scene BVH, hardware shadow, software shadow, caustic, and surfel GI concerns
• prepared CSG frame state
• the shared deferred GPU task graph, its analysis/queue assignments/compiled packets, and the accepted-submission transaction
• validated targets, pipelines, and prepared task recording state
• accepted state seeds and temporal-history state used when a new graph imports prior work
• prepared shadow visibility state

RendererSystem declares read access to active camera, transforms, cameras, renderer components, material instances, CSG receivers, and CSG cutters.

Each frame builds a bounded GpuTaskGraph. Renderer code declares semantic work, external completions, and resource uses; the compiler owns inferred hazards, physical-queue assignment, compatible packet formation, barriers, and submission dependencies. Recording is normally native, and accepted packet callbacks own temporal commits and recovery. See GPU Task Graph and Renderer Frame Flow.

Mesh and Material Pass

The material pass consumes ECS draw state from mesh/model/skinning components and material assets. Materials expose only the mesh and pixel stages for the ECS renderer. In the normal generated path, the material pixel stage comes from the authored .surface hook, and the mesh stage is the shared engine mesh shader.

The render path choice is material/device dependent:

• MeshShader + PS when the material has a mesh stage and the device supports mesh shaders.
• CS + PS through compute emulation when the device does not support mesh shaders.
• VertexIndexed persistent indexed raster on the fixed engine mesh program: cached object-space vertices plus a trailing index region, decoded once and reused across ordinary instance/view changes with no steady-state dispatches.

If native mesh shaders are unavailable, the renderer derives compute-emulation resources from the mesh shader internally, or reuses the persistent indexed cache when the fixed engine program and draw set qualify. The authored material and shader remain the source of truth. Windows ARM64 selects this portable path by default; a qualified adapter can opt into native mesh shaders only before graphics instance creation. See Platform and GPU Support.

Material pass work includes:

• resolving renderer-visible components
• loading material and shader assets
• creating material pipelines
• creating or reusing material and mesh heap-slot resources
• gathering draw items
• partitioning regular, CSG, and CSG receiver surface draws
• building instance GPU data
• gathering typed material bytes
• uploading draw buffers
• rendering draw partitions

The material system also prunes mutable material instance cache data during ECS update.

Deferred Rendering

The deferred subsystem creates frame targets for scene rendering, lighting, and final composition. It owns deferred lighting, compute composition, and Graphics presentation pipelines.

Deferred lighting is material-driven through BXDF dispatch. The geometry pass packs a material shading-model id into the G-buffer, and the deferred lighting shader uses the cook-generated BXDF dispatch module to call the .bxdf selected by the owning material.

The BXDF surface contract is shared with AVBOIT accumulate shaders. Opaque lighting reconstructs the surface from the G-buffer; transparent accumulate draws build it directly from the material surface hook, then dispatch the same material BXDF.

Scene camera and light data are uploaded through renderer-owned buffers. Camera state comes from scene ECS components, not from input/controller state.

Compute composition combines the renderer's accumulated outputs into linear compositeColor. The final Graphics presentation pass applies display mapping, consumes the optional Scene UI output layer, and writes to the presentation framebuffer.

Deferred targets are recreated when the frame size changes. Final presentation pipeline creation depends on presentation framebuffer compatibility.

HDR10 Presentation

The effective presentation mode is selected per swapchain. Testbed requests HDR10 before device creation; when the active surface supports the HDR10/PQ format, deferred/AVBOIT RGB remains linear until the final presentation pass converts it to Rec.2020/PQ. Otherwise the renderer retains its SDR/sRGB presentation path. An optional scene UI layer blends with mapped scene color in the final presentation pass before the single SDR or HDR10 encoding; standalone UI presents its layer over black. See HDR10 Presentation.

CSG Rendering

CSG support is integrated into the renderer. Receiver components define which meshes can be affected, and cutter components define active shape operations.

Renderer CSG work includes:

• CSG frame-state building
• interval peel resources
• interval sample state upload
• receiver span building
• interval combining
• cap rendering
• opaque and transparent pass integration

CSG assets and shader includes are cooked so renderer shaders can evaluate shape modules consistently.

Opaque and transparent CSG are both represented in frame state. The renderer can build CSG work regions so interval and clear work can be restricted to affected screen regions.

Transparency

Transparent rendering uses AVBOIT-style passes. The renderer owns AVBOIT targets and heap-slot resource contexts for:

• low-resolution raster/coverage
• depth warp
• extinction accumulation
• integration/transmittance
• transparent accumulation
• final composite integration with deferred targets

Transparent rendering stays material-driven and shares scene lighting data with the deferred path.

Surface-authored transparent materials carry cook-generated AVBOIT occupancy, extinction, and accumulate pixel shaders. Those shaders evaluate the same material .surface hook, and the accumulate shader dispatches the material BXDF for the transparent fragment before applying AVBOIT accumulation. Transparent or refractive materials require a project surface hook; explicit shaders are allowed only for opaque, non-refractive materials.

The renderer clears AVBOIT targets when transparent renderers exist or when stale transparent target data needs clearing after a previous transparent frame.

Transparent materials with a finite surface IOR above 1.0 also participate in camera-visible refraction (nwbRefractionIorEligible in impl/assets/graphics/refraction/ior.slangi). A capture pass retains one primary refractive instance per pixel and removes it from AVBOIT accumulation; a separate resolve owns its transmission through a bounded hardware TLAS trace or a screen-space depth march. See CSG and Transparency.

Reflections

Reflections combine hierarchical screen-space tracing with a bounded hardware ray queue under one graph domain. RendererSystem::setReflectionSettings applies ReflectionSettings: the Disabled, ScreenSpace, Hardware, and Hybrid (default) trace modes, temporal and spatial filtering, budgets, and analytic environment colors for exterior misses. The domain publishes opaque and glass radiance targets from accepted packets only, exposes completed-frame traversal counters through tryGetLatestReflectionStatistics, and degrades to a prepared fallback pipeline when hardware preparation is not ready. See Reflections.

Ray Tracing and Shadows

Hardware ray tracing is optional. The renderer queries these graphics features independently:

• acceleration structures
• ray tracing pipeline
• ray query

When acceleration structures and RayQuery are available, runtime ray tracing builds BLAS/TLAS resources. Shadow visibility uses inline RayQuery compute for opaque occluders; ray tracing pipeline support remains a separate capability. Without the hardware pair, the renderer builds GPU software BVHs for shadows, caustics, and surfel GI. See Platform and GPU Support.

Each shadow-slot light can carry a physical soft-shadow source size. Directional lights use LightComponent::angularRadius, while point and spot lights use LightComponent::sourceRadius. A zero source size collapses to the hard-shadow path.

The current hardware shadow path uses acceleration structures plus inline RayQuery from compute shaders. It builds BLAS/TLAS resources and, when soft-shadow resources are ready, runs a hardware soft opaque trace into the same half-resolution shadowSoftHalfA target used by the software path. The shared geometry downsample, optional temporal reproject-merge, scalar a-trous denoise, and bilateral upsample chain then writes the full-resolution shadow visibility image. If the hardware soft chain is unavailable for a frame, the renderer falls back to the direct full-resolution RayQuery trace.

Transparent shadows use staged hardware ray queries on RT hardware (hardware_transparent_shadow.cpp: gather, evaluate, and overflow-continuation compute passes). The staged passes collect per-pixel transparent crossings and evaluate RGB transmittance; when the scene contains transparent occluders but the staged path cannot run, the renderer falls back to software shadow BVHs. If soft transparent resources are ready, the staged transparent trace produces a colored half-resolution signal, the RGB resolve filters it into HalfA while the opaque wavelet output stays in HalfB, and one combined upsample resolves both with a shared float4 guided fit. If that fold cannot run, the renderer falls back to a software traversal pass that multiplies colored transparent transmittance onto the hardware-written opaque shadow mask. Opaque-only RT scenes skip this transparent work. The combined-resolve preparation is shared by both backends (prepareSoftCombinedResolvePipelines runs in the hardware and software preflight branches alike), so software soft-transparent frames use the same HalfA/HalfB fusion.

When the hardware RayQuery shadow path is unavailable, the renderer can use software/GPU-BVH shadow visibility paths built from available mesh/runtime data. The standalone fallback starts with a full-resolution opaque pre-pass so every shadow slot has a valid hard mask. The no-RT path can additionally use light-space depth-resolved shadows: RendererSystem::setSoftwareShadowSettings selects Automatic/SoftwareTrace/LightSpace (spots are skipped, only directional and point lights are admitted, over-budget lights are dropped), depth renders into a D32 view array, and capture/shade passes resolve visibility with the snapshot carried on the shadow graph plan. See Renderer Feature Paths.

When soft-shadow resources are ready, both the hardware soft path and the no-RT software path can feed a half-resolution cone-jittered soft opaque trace into a shared resolve sequence: half-resolution geometry downsample, optional temporal reproject-merge, one scalar a-trous wavelet pass, and a bilateral upsample into the full-resolution visibility target. If the temporal merge is unavailable, the path still runs the spatial resolve; if the soft path is unavailable, the hardware direct trace or software hard pre-pass remains the fallback.

The shared soft-opaque trace takes two source samples per half-resolution pixel per frame. Temporal reprojection supplies broader source-area coverage over time, while the current-frame samples stabilize freshly disoccluded or history-clamped regions. The colored transparent trace uses TransparentShadowSampling (ReferenceThree by default, TemporalOne after accepted filtered history via TransparentShadowSamplingHistory): bootstrap and reset frames use three samples, steady-state temporal frames use one sample, resolved through ResolveTransparentShadowSampleCount. Shadow sampling keeps finite float precision for R2 indices, hash offsets, and jitter-cone blends (shadow_sampling.slangi): unnormalized indices and hash values up to 65536 exceed finite half range, and 1-cos(theta) needs float precision for small finite source angles.

On the software path, the soft transparent signal is parallel: transparent occluders are traced into a separate half-resolution colored transmittance buffer, denoised with the RGB variant of the shadow resolve, and multiplied onto the soft opaque visibility only during the final upsample. If that soft transparent fold does not run, the renderer uses the adaptive transparent fallback fold (coarse transmittance feeding the adaptive resolve modes, or the full-resolution adaptive fallback).

Software shadows, GI, and caustics use shared per-instance material records. The hardware opaque-shadow fast path instead marks non-transparent TLAS instances FORCE_OPAQUE and accepts the first opaque RayQuery hit, so it neither binds nor reads the hardware shadow material table. Staged hardware transparent evaluation and software traversal evaluate the material surface hook through the cook-generated surface dispatch. Both integrate Fresnel transmission and Beer-Lambert absorption over geometric crossings to produce RGB transmittance. Opaque occluders zero transmittance and terminate standalone software traversal, while the hardware opaque path treats any opaque hit as fully blocked.

Transparent shadow integration uses geometric face normals for interval pairing and keeps the material hook's shading normal input stable, which preserves hard normals across transparent occluders.

The adaptive software economizer is transparent-only. It traces a coarse transparent layer, classifies full-resolution edge pixels, and can launch only the compacted edge list through an indirect pass. The opaque software pre-pass stays full-resolution and hard; soft opaque shadows are handled by the separate half-resolution jittered trace and resolve path.

The deferred lighting pass samples the same visibility image either way. When no shadow backend writes it for a frame, the image is cleared to an all-lit white state so lighting does not read stale data.

GPU task-graph scheduling

Queue transport is frozen at centralized scheduler admission (GpuTaskScheduler::scheduleGraph in core/task/gpu/scheduler_compilation.cpp, invoked once per frame from renderer_frame_pipeline_graph_schedule.cpp) rather than a renderer-owned lane schedule. The shared deferred graph declares shadow visibility, software caustics, surfel GI, hardware dispatchRays caustics, AVBOIT, lighting, composite, and presentation as semantic work. The compiler uses the actual device topology to place compatible tasks on Graphics, Compute, or Transfer queues, generate packet waits/barriers, and preserve the declared overlap opportunities.

On a machine without a dedicated Compute queue, compatible tasks route to Graphics. On a dedicated topology, independent hardware-caustic and Compute-ray-effect work can overlap, AVBOIT's raster and compute phases retain their declared order, and the optional lagged-lighting path consumes only an accepted prior-history completion. Packet boundaries are compiler results derived from the declared graph. See GPU Task Graph for queue, ownership, and recovery rules.

Refractive Caustics

The ray tracing subsystem also owns refractive caustic work. Materials opt in with the refractive metadata flag, while shader-side .surface output carries the actual refraction and transmission values.

Caustic work uses two backend-specific photon producers:

• hardware ray tracing over the scene TLAS
• software compute traversal over the software scene BVH and per-mesh BVHs

Both producers splat into the same RGB integer accumulator. A shared compute resolve converts that accumulator into a screen-space caustic irradiance texture through a half-resolution path: a geometry downsample pre-pass builds the receiver cache, a prepare pass converts full-resolution accumulator flux into half-resolution irradiance, five spatial edge-aware a-trous wavelet passes denoise it, and a final edge-aware upsample writes the full-resolution irradiance target. Deferred lighting samples the resolved irradiance additively before tone mapping.

Both caustic producers use 2x temporal-reuse checkerboard emission. gridSide remains the full photon-grid side, but each frame emits only the parity-selected half of that full grid and advances frameIndex; the next frame emits the complementary cells. Per-photon jitter is keyed to the full-grid cell, while per-frame photon count is halved and flux is correspondingly normalized so the splat-space temporal accumulator recombines the two-frame union without reducing steady-state energy.

The caustic irradiance target is cleared to black every frame. The fixed-point accumulator uses the renderer's internal 0.85 temporal decay, persists across accepted frames, decays before each new photon splat, and is normalized in the resolve so static brightness does not change. If no caustic light, refractive caster, or traceable scene exists, the black irradiance target is the additive no-op.

Skinned refractors use the mesh skinning system's per-frame normal repack so refractive shadow and caustic rays use live deformed normals.

Global Illumination

Diffuse indirect lighting uses a persistent, screen-spawned surfel field owned by the ray tracing subsystem. The field is stored in renderer ray tracing state so window resizes do not reset convergence.

Surfel GI can trace through either the software scene BVH or a hardware inline RayQuery twin over the TLAS. Both producers write the same surfel pool and share the same material/light contract. Per-frame work is:

• age-free stale surfels into a persistent free list
• clear and rebuild the world-hash cell heads from live surfels
• spawn one surfel per empty hash bucket from G-buffer world position and normal samples
• build indirect trace dispatch arguments from the live surfel counter
• trace live surfels through the selected software-BVH or hardware-RayQuery backend
• resolve the surfel field into a half-resolution irradiance target
• upsample that target into the full-resolution surfelIrradiance texture through a surface-gated joint-bilinear filter

The trace stores directional irradiance as a 2-band spherical-harmonic field in each surfel record, plus a live-only 16-cell piecewise-constant directional guide (4 azimuth x 4 height bins, equal solid angle, four float4 lanes per surfel in a separate guide buffer that is never snapshotted). Fresh surfels train all 16 cells with 64 rays; mature surfels (8+ samples) keep 4 uniform explorer rays (odd-stride cell coverage with a frame/surfel phase) and up to 8 guided rays sampled from the guide CDF with MIS against the uniform PDF. The guide trains on the first mature update and then once per four surfel visits (frame/decimator staggered), blending observed-cell luminance (clamped) at 0.35 and decaying unobserved cells at 0.125 from a 0.1 floor. Guide hashing and random-number generation live in surfel_guide.slangi. It also snapshots the previous frame's surfel pool and cell-head table before mutation so surfel-to-surfel bounce gathering reads a stable frame-start field. The trace dispatch is indirect so dead pool slots do not launch workgroups. The deferred lighting pixel shader samples the resolved full-resolution texture rather than the writable surfel pool; if surfel GI is disabled or no surfel covers a pixel, the alpha-zero resolve result falls back to the existing hemi-ambient path.

The field is progressive rather than a single-frame result. A stable scene improves as surfels seed and their bounded running means accumulate, but no fixed capture frame guarantees convergence. If GI resources are unavailable, waiting does not replace the hemi-ambient fallback. See Platform and GPU Support.

Feature-Gated Vulkan Paths

The graphics layer exposes active feature queries for meshlets, ray query, acceleration structures, ray tracing pipelines, variable rate shading, shader specializations, subgroup lane counts, cooperative vector support, and advanced ray tracing extensions.

Windowed Vulkan initialization rejects CPU-type devices (VK_PHYSICAL_DEVICE_TYPE_CPU), which is how software rasterizers such as lavapipe are excluded; there is no by-name check. Native mesh-shader mode remains disabled on devices where the required hardware features are unavailable.

Material Binding

Material and shader parameter binding is typed-only at binding 6 using generated typed .bind helpers. Shaders that include a generated material bind require NWB_MATERIAL_TYPED_BINDING == 1.

In generated materials, the pixel shader is the stage that reads typed material data through the .surface hook. The shared mesh shader stays generic and interface-free.

The renderer enforces material or shader interface-name matches with stable authored instance identifiers. Typed material data uses binding 6; string fallback material lookup is unsupported.

Push-constant layouts and heap handles are prepared before draw command recording. Draw code uses the prepared heap-slot contexts and asserts typed upload invariants instead of creating descriptor state inside the draw loop.

Renderer Inputs and Outputs

Main inputs:

• scene camera and light components
• transform components
• mesh/model/skinning ECS components
• material assets
• shader assets
• material instance overrides
• CSG receiver/cutter components
• cooked shader archive resolver
• graphics feature queries

Main outputs:

• deferred frame targets
• full-resolution shadow visibility image
• caustic irradiance image
• surfel GI irradiance image
• AVBOIT targets
• presentation framebuffer composite
• GPU timing scopes
• frame graph contribution

Testbed Render Pass Order

Testbed registers render passes in this order:

1. MeshSkinningSystem
2. RendererSystem
3. UiLayerSystem

The renderer receives skinned runtime mesh data after the skinning pass has updated it. Testbed's scene UI prepares an independent layer; graph dependencies join it with mapped scene color in the renderer's terminal presentation pass.