GPU Task Graph - limztudio/NWBLot GitHub Wiki

GPU Task Graph

Core::GpuTaskGraph is the renderer's scheduling and resource-state model. Renderer code declares semantic work and resource use; the graph compiler derives dependencies, physical-queue placement, native packets, barriers, ownership transfers, and submission waits. Core::GpuTaskScheduler records and submits compiled packets. It lives in core/task/gpu beside the graph compiler and is a peer of CpuTaskScheduler; GraphicsRuntime owns the device and presentation lifecycle and borrows both schedulers. See CPU Task Scheduler.

GPU task code uses neutral RHI contracts and the narrow headers under core/graphics/backend_selection/ to call the selected device and command list directly. Native handles, flags, and validation remain inside the provider. Vulkan is the default and only implemented provider; the Metal selection is reserved and rejected until implemented. The selection boundary adds no runtime dispatch or per-resource storage. See Architecture.

Task Contract

Each task declares:

  • a stable semantic identity
  • required command capabilities owned by the task implementation (graph callers do not select a queue; the compiler derives legal placement from command requirements, resource sharing/ownership, dependency crossings, and overlap opportunity)
  • explicit task or external-completion dependencies
  • imported textures, buffers, acceleration structures, pipelines, or conservative hazard domains
  • read/write access, required resource state, texture subresource or buffer byte range, and queue-sharing policy
  • immutable resource sets for prepared material and trace work
  • optional overlap, queue-crossing, and packet-boundary hints
  • accepted and discarded outcome callbacks when CPU-visible state depends on submission

Typed imports retain the referenced engine objects through recording. Graph-owned uploads retain immutable CPU data, so asynchronous recording never depends on caller-owned temporary memory.

Prepared draw and trace paths declare the exact bindless resource sets they may access. Graph construction fails when a prepared path cannot provide a complete declaration. Resource-set membership is validated with reusable owned scratch rather than repeated renderer-side scans; imported resources are indexed by the graph so feature paths resolve the same identity without duplicating lookup tables.

Typed payload tasks use the exact signature TaskT::Record(const Payload&, CommandList&, const GpuTaskRecordContext&) -> bool and can optionally implement the exact signatures Accepted(Payload&, const QueueSubmissionToken&) -> void and Discarded(Payload&) -> void. Default arguments, implicitly convertible parameters, and differently qualified payloads do not satisfy those contracts. One payload lifecycle belongs to one graph-owned recording attempt; a retry begins only after the prior unaccepted attempt is terminal. An unexpected callback exception terminalizes the exact lifecycle state it reached and then continues toward the application-entry exception boundary.

Produced graph results

Graph builders return Expected<Result> aggregates or scalar resource/task identities, while the graph and explicit frame-lived callback state remain mutable inputs. IGpuTaskGraphOutputLayerContributor::declareTaskGraphOutputLayer(graph) returns Expected<GpuTaskGraphOutputLayer>: successful empty or upload-only layers retain their existing meaning; an unexpected result rejects graph construction. GraphicsRuntime::submitStandaloneTaskGraph returns Expected<QueueSubmissionToken>, and Device::executeCommandLists returns Expected<QueueSubmissionReceipt> carrying the accepted token and emitted timeline-wait count. Consume tokens and commit state only after admission. Task Record/accepted/discarded callback signatures remain the contracts above. Borrowed declaration bytes from uploadBlobData(blob) use Expected<BinaryByteView> and remain valid only for the declaration view lifetime.

Declaration Lifetime

Graph declaration writers and compiler readers use one declaration-admission gate. A compiler entry observes a coherent declaration, while a writer is rejected during compile access, teardown, or an active recording plan. Every successful compile-relevant mutation advances declarationRevision, so a compiled graph is valid only for the exact graph generation and declaration revision it analyzed.

The first native recording claim freezes the declaration for that compiled plan. Imports, tasks, built-in operations, uploads, resource sets, pipelines, and presentation endpoints cannot mutate while any packet in the attempt is recording, recorded, submitting, accepting, or awaiting terminal discard. GpuTaskGraph::tryReset() likewise refuses concurrent declaration/compile access, discard notification, an unresolved submission binding, or an in-flight packet. A retry or different compiled plan begins only after every task in the prior attempt reaches its required terminal state.

Compile, Record, Submit

flowchart LR
    Declare["Declare tasks and resource uses"]
    Analyze["Infer hazards and dependencies"]
    Compile["Assign queues, packets, and barriers"]
    Record["Record ready native packets"]
    Submit["Submit with accepted timeline waits"]
    Commit["Commit accepted state or recover"]

    Declare --> Analyze --> Compile --> Record --> Submit --> Commit
Loading

GpuTaskScheduler::scheduleGraph (in core/task/gpu/scheduler_compilation.cpp) is the single execution-admission entry: it runs deterministic analysis, freezes queue assignment against a current physical-queue pressure snapshot, and initializes the caller-owned runtime artifacts for the resulting immutable plan. Explicit queue loads in GpuTaskGraphCompileOptions override automatic timeline sampling for deterministic tooling and tests. The renderer calls it once per frame from renderer_frame_pipeline_graph_schedule.cpp with FrontierSafe packetization and a packet-timing envelope spanning the shadow-prepare task through the frame-timing-end task. GpuTaskGraphCompiler performs the deterministic analysis step and produces the immutable compiled graph. It:

  1. combines explicit dependencies with resource hazards
  2. assigns compatible physical Graphics, Compute, or Transfer queues
  3. groups compatible tasks into native packets
  4. emits resource-state transitions
  5. emits paired release/acquire barriers for exclusive cross-family ownership
  6. records the timeline dependencies required between physical queues

Independent packets can record from a ready frontier on worker-affined command-buffer pools. Ready packet selection still follows the deterministic compiled plan, but ordinary packets for independent native queues may enter Vulkan submission concurrently. The graph serializes only the irreversible CPU publication tail—timing-ticket resolution, lifecycle callbacks, completion tokens, and accepted-frontier updates—so graph state remains coherent while native queue work overlaps. Packets with a pre-submit hook or an accepted-frontier join take an exclusive submission boundary. Only accepted submissions produce usable completion tokens and commit application-visible state.

Native Command-List Publication

A graph recorder owns an exact (CommandList*, recording lease serial) capability on its thread-local capability stack. While that capability is active, ordinary public command-list mutation and foreign-thread inspection are denied. Successful recording closes the native command buffer, completes retained-resource publication, and only then changes the list from Recording to Recorded.

A Recorded graph list is immutable. Diagnostic access takes a temporary exclusive Reading lease; submission and revocation wait for that read to finish. The exact graph submit path moves Recorded to Submitting. Native acceptance consumes that publication and returns the list to Unowned, while a rejected native attempt returns the same publication to Recorded so transaction policy can discard or retry it. Reset/rejection can revoke only the matching still-unsubmitted recording serial; a stale capability cannot revoke a reused command list or consume a later recording.

Command-list destruction is non-throwing and callback-free. It aborts an unfinished native attempt, releases retained recording claims and unsubmitted upload chunks without polling, clears active marker nesting, and removes the exact publication. Device-owned crash-marker history is deliberately not erased.

GpuRecordedGraph has a separate TLS-aware shared/exclusive operation gate around its packet storage, timing tickets, state seeds, and retained command lists. Recording and reset take exclusive ownership. Submission takes shared ownership, so independent packet submissions may overlap while reset cannot invalidate their borrowed artifact. Public record and submit operations use non-blocking admission: a conflicting reader or writer makes the operation fail instead of wait. Compatible same-artifact nesting reuses the active TLS lease. An unsafe shared-to-exclusive upgrade refuses, while cross-artifact or transaction reentry requests non-blocking exclusive admission and refuses on conflict. Submission always acquires the recorded-artifact gate before the transaction gate.

GpuRecordedGraph::tryReset() is also non-blocking: it returns false without mutation when the current thread already borrows the artifact or another live recording/submission owns the gate. The assertion-style reset() is only for externally serialized callers. Destruction rejects same-thread destruction from an active artifact operation, then takes the sole blocking exclusive join before revoking retained publications. A successful reset prepares all replacement packet, timing, state-seed, and scratch storage before publication, then revokes only the exact old unsubmitted command-list serials and installs the replacement with non-throwing moves. Allocation failure therefore leaves the recorded artifact and its publications intact.

Planning And Import Guarantees

The compiler keeps all declared and inferred dependency reasons for diagnostics, then uses a stable transitive reduction for scheduling. Redundant edges must not create extra packet dependencies, physical-queue waits, or producer signal frontiers.

Conditional recovery tasks contribute zero estimated normal overlap and queue load because they execute only after normal work fails. Their declarations, legal queue requirements, and accepted-frontier joins still apply.

Resource-range planning can complete first-use selection when the first indexed predecessor fully covers the requested range, after validating the selected history bounds. It preserves source and fragment order and rejects malformed selected prefixes rather than scanning past them to find coverage. Covered terminal states do not add redundant bounds to the result. Scratch storage follows the operation lifetime, including fragmented fan-in.

Queue placement is deterministic for the current device topology. Capability requirements remain mandatory; direct dependency affinity, queue load, and ownership-transfer cost are scoring inputs rather than alternate synchronization rules. A compiled or recorded graph is bound to one device generation, so a retired or unavailable physical queue cannot be reused after device recreation.

Packetization preserves an explicitly requested merge when it is legal. A merge across a consumer frontier requires an explicit immediate-successor opt-in; consumers then wait for the complete packet, while resource transitions and timing remain per task. The optional frontier-scored policy may merge a cheap immediate same-queue successor only when doing so does not hide a cross-queue consumer signal point. This lets a dependent queue begin after its true producer instead of waiting for unrelated same-queue tail work.

Imported exclusive-owner state is validated twice. Compilation verifies that each declared external completion token names the exact source physical queue and reaches the required timeline value. Recording then requires an immutable handoff snapshot that covers the exact texture range or declared buffer byte interval, or the full acceleration-structure backing allocation, with the expected source owner and release destination. A pending queue-family release must be acquired with its original byte interval; handoff subset selection rejects clipping a pending release. A stale, partial, or mismatched handoff fails before native commands are recorded; the graph never guesses an import state from resource creation defaults.

Command-list resource-state selection (CommandListResourceSelection) is operation-owned membership for whole-resource filtering. Finish adding identities before any other scratch allocations, and destroy the selection before reusing that scratch for another operation such as state fan-in. Vulkan command buffers retain GPU-visible resources through CommandBufferResourceReferences, which indexes membership instead of scanning retained lists on every retain.

Recording telemetry distinguishes the sum of packet recording spans from ready-frontier wall time. It also records observed packet overlap and worker utilization, so parallel-recording reports are not mistaken for operating-system CPU time.

Physical Queues And Ownership

Tasks request capabilities; they do not choose a renderer-wide lane.

  • Compatible work resolves to the primary Graphics queue when no dedicated queue is available.
  • A distinct physical queue always requires the compiled execution dependency.
  • A same-family queue change needs accepted timeline ordering but no Vulkan queue-family transfer; a prior accepted wait on the consuming queue may already cover it.
  • An exclusive resource crossing queue families receives paired release/acquire barriers.
  • A concurrently shared resource keeps its sharing contract and never receives an ownership transfer.
  • Windowed presentation terminates on the primary Graphics queue.

CommandListResourceStateHandoff is a state seed at graph or recording boundaries. It does not synchronize queues or retain resources. Execution ordering comes from accepted queue submission tokens or same-queue order. Persistent return-state caches publish the actual accepted native final state regardless of the producer's queue class; a legal queue reassignment must not leave an older snapshot for the next consumer.

Compatible reads of a concurrently shared resource can reuse one immutable packet-final state snapshot instead of chaining through the preceding reader. The anchor must cover the consumer range, admit every participating queue, and preserve its entire range through the remaining source packet. Overlapping writes, changed source states, and outgoing ownership barriers prevent reuse. Each consumer keeps its exact declared state and the dependency on the source packet; later incompatible transitions and writes still join overlapping readers. A consumer that later writes or changes to an incompatible read state on an overlapping range inside the same task joins outstanding readers at task entry instead of borrowing an earlier shared-read anchor.

Texture anchors require native close to preserve their state; compatible shader/depth reads share the selected provider's read-only layout. Buffer and acceleration-structure backing-buffer anchors may restore a known creation state on close, with an explicit consumer transition from the actual imported snapshot state. Terminal exports continue to use the latest tracked state. Native barrier lowering normalizes consumed private arrays in place while preserving layouts, ranges, ownership, barrier order, and rendering-scope handling.

Reader fan-in can stop at a covering writer that already ordered older readers, or at a covering incompatible read epoch established by another task. Partial boundaries and internal state changes prevent the read-epoch cutoff.

The submitter retains every declared dependency token for validation and lifecycle admission. A prior accepted native wait covering all commands on the exact consuming queue may cover a later wait on the same current-device producer timeline at no greater value. Rejected submissions publish no coverage, and binary or external synchronization remains explicit. Runtime and telemetry distinguish emitted timeline waits, same-queue elisions, duplicate merges, and inheritedTimelineWaitElisionCount, preserving exact planned-token conservation.

The Vulkan backend creates scheduler queues from one validated canonical native-queue registry. Each registered queue has a unique Vulkan family/index and native handle, and every active queue class has an explicit primary route. This keeps physical queue identities, timeline tokens, and presentation routing unambiguous.

External multi-producer texture handoffs are range-exact. Every imported mip/slice range must be covered by the frozen native snapshot with the declared source owner and release destination, and the handoff remains invalid until all terminal producer packets are accepted. Broad, partial, overlapping, or ownership-incompatible sources are rejected instead of being merged speculatively.

A completion binding may describe sources from only one physical producer queue; ranges produced on different queues require distinct completion bindings.

Renderer Integration

RendererSystem builds one deferred graph containing:

  • shadow preparation
  • per-frame and mesh uploads
  • G-buffer, material, and CSG work
  • AVBOIT
  • shadow visibility
  • caustics
  • surfel GI
  • deferred lighting and composite
  • presentation
  • optional readback or frame-lagged-history tails
  • failure recovery

Packet count and queue placement are compiler results. A dedicated topology can overlap independent work, while a single compatible queue executes the same graph serially.

Output-layer contributors declare independent raster work and an exact produced color version consumed by the final output consumer. Presentation contributors append final Graphics work before the terminal presentation endpoint. Accepted final consumption retires the exact CPU output-layer generation. Acceptance is not GPU completion: output texture, descriptor, and backing versions remain retained through the final-consumer token until GPU completion. Slot reuse also requires completion of every accepted upload, clear, and raster prefix. A rejected or unsubmitted final consumer leaves the immutable CPU generation pending for retry. Timing is packet-owned, so telemetry follows the compiled packet identity and actual queue route. Queue-assignment scoring terms stay independently visible in timing_feedback.h so they can be inspected without becoming part of graph correctness. Duration histories remain route-specific, while a Graphics-to-Compute-to-Graphics sequence updates one shared switch timeline.

Acceptance And Recovery

A graph transaction binds one exact graph generation, compiled plan, recording-attempt generation, and transaction reset generation. Every packet in that attempt must become terminal as Accepted or Rejected before the binding can resolve or the transaction can reset. Transaction reset builds replacement packet, accepted-frontier, and external-handoff storage off to the side, then publishes it through non-throwing moves and sets validity last; allocation failure therefore leaves the prior transaction intact.

Pre-submit validation failures leave an unaccepted packet available for the caller's declared retry/discard policy. Once Vulkan accepts a packet, its completion token and native effects are irreversible. Typed accepted observers run before that acceptance becomes visible to the rest of the transaction; if an observer throws, the transaction commits the accepted token first and rethrows, so the observer is never replayed as though the packet had not submitted.

Exception cleanup is failure-atomic. A submission-attempt scope promotes the transaction gate to exclusive ownership, abandons every unresolved timing ticket and unaccepted packet without invoking discard observers, resolves the complete binding, and lets the original exception continue to the application-entry boundary. If a discard observer itself throws, its exact packet/lease is consumed before rethrow and the remaining attempt is terminalized callback-free. Exceptions are not converted into ordinary false results so that execution can continue with ambiguous GPU state.

If a later packet rejects after earlier work was accepted, graph-owned recovery waits on the latest accepted physical-queue frontiers and restores resources and timestamp state needed for safe reuse. If required recovery cannot be accepted, rendering remains suspended until resource or device recreation.

Compiled and recorded graphs are device-generation-bound. Device recreation invalidates their command buffers, pipelines, state seeds, and completion tokens.

Buffer Byte Ranges

GPU tasks synchronize the byte intervals declared in GpuTaskResourceUse::range.bufferRange:

const GpuTaskResourceUse use{
    .resource = bufferResource,
    .range = GpuTaskResourceRange{ .bufferRange = BufferRange(256u, 128u) },
    .requiredState = ResourceStates::UnorderedAccess,
    .access = GpuTaskResourceAccess::Write,
};

This declaration covers bytes [256, 384). Disjoint intervals do not create resource hazards; overlapping reads and writes retain the required ordering. State transitions, packet state seeds, terminal exports, and queue-family ownership transfers preserve the affected intervals, including when a consumer spans several producers.

Omitting the range keeps the whole-buffer default. BufferRange(offset, BufferRange::s_AllBytes) covers the remaining bytes from that offset. Empty, overflowing, and out-of-bounds task ranges are rejected. Acceleration structures retain whole-allocation synchronization.

The declaration must cover every byte touched by the task's commands and internal state transitions. For explicit native transitions, pass the same range as the last argument to CommandList::setBufferState(buffer, state, forceMemoryDependency, range). Whole-buffer operations, including implicit vertex/index and graphics/mesh indirect binding state transitions, still require whole-buffer declarations. Compute indirect dispatch transitions only its argument structure at the dispatched offset. Built-in uploads and copies declare and transition their exact transfer intervals; buffer clears cover the whole buffer.

A pending queue-family release must be acquired with its original byte interval. Handoff subset selection rejects clipping a pending release. Release separate intervals on the producer when consumers need independent ownership ranges.

Built-In Operations

The graph provides owned helpers for buffer copies, texture transfers, uploads, clears, and packet-lifecycle work. These helpers participate in the same hazard, lifetime, queue, and acceptance model as renderer-defined tasks.

Combined depth/stencil uploads select exactly one Depth or Stencil aspect and use that plane's byte layout. Automatic is valid for unambiguous color, depth-only, or stencil-only formats, but is rejected for combined depth/stencil formats. Depth/stencil copies require a Graphics-capable task on the current Vulkan backend.

Source Map

  • core/task/gpu/scheduler.h plus scheduler_compilation.cpp: GPU scheduler attach, centralized execution admission (scheduleGraph/compileGraph), record, submit, and device wait
  • core/task/gpu/task_graph.h: public graph declaration API
  • core/task/gpu/task_desc.h: task and resource-use descriptions
  • core/task/gpu/task_contract.h: typed payload Record/Accepted/Discarded requirements
  • core/task/gpu/compiler*.cpp: analysis, queue policy, packetization, resource versions, ranges, and barriers
  • core/task/gpu/compiled_graph.h plus compiled_graph.cpp, compiled_graph_read_view.cpp, and compiled_graph_attempt_binding.cpp: immutable compile result, read-view queries, and recording-attempt binding
  • core/task/gpu/task_graph_storage.cpp plus task_graph_teardown.cpp: graph storage with teardown queries in the teardown unit
  • core/task/gpu/task_graph_recording_lifecycle.cpp plus task_graph_reset.cpp, task_graph_accessors.cpp, and task_graph_declaration_view.cpp: recording-attempt/submission lifecycle with reset and accessor plus declaration-view queries in their focused units
  • core/task/gpu/task_graph_packet_lifecycle.cpp plus task_graph_packet_discard.cpp: packet lifecycle with discard handling in the discard unit
  • core/task/gpu/packet_runtime*.cpp and packet_runtime_internal.h: preflight, recording, concurrent submission, transactions, and rejection, with the exception finalizer in packet_runtime_exception_finalizer.cpp; artifact operations live in packet_runtime_artifact_operation.cpp, the recording/submission gate split across packet_runtime_submission_gate.cpp and packet_runtime_recorded_graph.cpp
  • core/task/gpu/persistent_state.*: graph state carried across frames; the accepted snapshot supports in-place same-arena self-filtering via replaceResourceSubset (retain new live handles, then compact the snapshot — no cross-cache copy)
  • core/task/gpu/timing_feedback.h: queue-assignment scores and route-specific timing history
  • core/task/gpu/README.md: buffer byte-range contract; the partial-DAG placement cases in tests/unit/task/gpu/task_graph_queue_partial_dag_tests.cpp cover multiword score preservation across queue moves and rebuilds, sparse-row scratch bounds, relation-state reset on rebuild, assignment exclusion without local assignment, and mask-allocation avoidance for ordered/unused reachability, with enabled unit-test fixtures kept under tests/
  • core/graphics/backend_selection/backend.h and the narrow selected concern headers: concrete device and resource operations behind compile-time selection
  • core/graphics/rhi/resource.h: BufferRange
  • core/graphics/rhi/resource_state_selection.h: operation-owned resource membership
  • core/graphics/vulkan/command_buffer_resource_references.h: retained command-buffer resource indexes
  • core/graphics/vulkan/backend_context_device.cpp, device.cpp, device_queue.cpp, and device_queue_topology.cpp: native and physical queue registration, with queue-topology getters in the topology unit
  • core/filesystem/volume_io.cpp plus volume_segment_compact.cpp: segmented volume I/O with segment compaction in the compact unit
  • impl/ecs_render/renderer_frame_pipeline_graph*.cpp plus the focused stage builders under impl/ecs_render/deferred/, impl/ecs_render/csg/, impl/ecs_render/avboit/, and impl/ecs_render/raytrace/: renderer graph construction
  • impl/ecs_render/**/task_graph_*.cpp: subsystem tasks
  • tests/unit/task/gpu/: nwb_gpu_task_tests, including task_graph_buffer_range_tests.cpp

See Renderer Frame Flow for the frame sequence and Diagnostics and Telemetry for timing and failure reporting.

⚠️ **GitHub.com Fallback** ⚠️