Code Reading Guide - limztudio/NWBLot GitHub Wiki

Code Reading Guide

This order starts at runtime ownership, follows data into the renderer, and enters Vulkan only after the engine-facing contracts are clear.

1. Loader And Project Boundary

Read:

  • loader/main.cpp
  • loader/project_entry.h
  • loader/project_entry.cpp
  • global/terminal_entry.h
  • core/common/application_entry.h
  • engine_namespace.h and the engine-domain global.h umbrellas
  • CoolStuff/Testbed/namespace.h
  • CoolStuff/Testbed/entry.cpp
  • CoolStuff/Testbed/project.h
  • CoolStuff/Testbed/runtime.cpp

Root engine_namespace.h owns the engine namespace wrappers; Testbed's independent local namespace header owns its application wrappers. The entry adapter supplies the required NWB loader functions while ordinary application code belongs to Testbed.

These files show crash/log setup, ConfigureProjectRuntime, graphics initialization, cooked-volume mounting through IFilesystem, asset registration, ProjectRuntimeContext, project callbacks, world creation, onUpdate, and shutdown.

2. Frame Ownership

Read:

  • core/frame/module.h
  • core/frame/module.cpp
  • core/frame/win32.cpp
  • core/frame/linux.cpp

Follow platform events into the project update callback and registered render passes. The project owns and ticks its ECS world; the frame owns the platform loop, graphics lifetime, the shared CPU and GPU task schedulers, timing, and presentation.

3. CPU And GPU Execution

Read:

  • core/task/README.md
  • core/task/cpu/scheduler.h
  • core/task/cpu/types.h
  • core/task/gpu/scheduler.h plus scheduler_compilation.cpp (centralized scheduleGraph admission), compiler_internal.h, and packet_runtime_internal.h
  • docs/cpu_task_scheduler.md

Frame owns both schedulers. GraphicsRuntime borrows them and attaches the GPU scheduler to the live device. ECS worlds and the loader own CpuTaskScope lifetimes. Tests live under tests/unit/task/cpu (nwb_cpu_task_tests) and tests/unit/task/gpu (nwb_gpu_task_tests).

4. Runtime Assets And Cooked Volumes

Read runtime loading first:

  • core/assets/manager.*
  • core/assets/registry.*
  • core/assets/ref.h
  • core/assets/auto_registration.*
  • core/filesystem/filesystem.h
  • core/filesystem/factory.h
  • core/filesystem/volume_file_system.h

Then read cook and publication:

  • pipeline/readme.md
  • pipeline/launch.py
  • pipeline/command_line.*
  • pipeline/asset_builder/build.cpp
  • pipeline/asset_gatherer/gather.cpp
  • core/assets/cook_entry_registry.*
  • core/assets/volume/
  • core/filesystem/volume_build.cpp
  • core/filesystem/volume_staging.cpp

Concrete codecs and cookers live under impl/assets_shader, impl/assets_material, impl/assets_texture, impl/assets_mesh, impl/assets_skeleton, and impl/assets_model.

5. ECS And Project Systems

Read core/ecs/ for entity lifetime, component pools, systems, queries, message buffers, and scheduling. Then follow the content systems under:

  • impl/ecs_scene/
  • impl/ecs_mesh/ (skinning accepted-geometry reuse and current-pose local/optical bounds in skinning/deformation_state.* and skinning/local_bounds.*)
  • impl/ecs_model/
  • impl/ecs_model_renderer/
  • impl/ecs_render/
  • impl/ecs_ui/ (input routing contracts in impl/ecs_ui/toolkit/input/README.md: identified pointer/contact providers, command press-arm/release-submit, bindings, hover, gestures, and popup policy)

6. Renderer Entry And Frame Pipeline

Start with:

  • impl/ecs_render/module.h
  • impl/ecs_render/module.cpp
  • impl/ecs_render/renderer_frame_pipeline.h
  • impl/ecs_render/renderer_frame_pipeline.cpp
  • impl/ecs_render/renderer_frame_pipeline_resources.cpp (resetFrameTaskState lives here)
  • impl/ecs_render/renderer_frame_pipeline_graph.cpp (the declaration orchestrator; per-domain task bodies moved to impl/ecs_render/graph/: frame_graph_avboit_occupancy.*, frame_graph_avboit_extinction.*, frame_graph_avboit_accumulation.*, frame_graph_reflection_resolve.*, frame_graph_software_bvh_build_state.*, and frame_graph_transparent_csg_tasks.*) plus the focused stage builders (impl/ecs_render/deferred/*_builder.* including the deferred frame-tail builder, impl/ecs_render/csg/transparent_csg_interval_builder.*, impl/ecs_render/avboit/clear_chain_builder.*, avboit_pass_upload_helper.*, material/geometry-preparation builders, and compute-emulation capture helper, impl/ecs_render/raytrace/hardware_caustics_stage_builder.* and task_graph_optical_scene_upload.*), the scheduler-admission unit renderer_frame_pipeline_graph_schedule.cpp (single scheduleGraph call with FrontierSafe packetization), the packet-metrics unit renderer_frame_pipeline_graph_packet_metrics.cpp, and the execute-side impl/ecs_render/execute/frame_execute_lifecycle.* (12 prepare/accept methods) with per-domain merge validators (graphics_prefix_timing_resolver.*, opaque_emulation_merge_validator.*, shadow_prepare_packet_validator.*, shadow_visibility_merge_validator.*, surfel_caustics_merge_validator.*)
  • impl/ecs_render/renderer_frame_pipeline_execute.cpp plus the execute-side focused units (impl/ecs_render/execute/frame_execute_cpu_restore.*, frame_execute_history_copy.*, and frame_execute_surfel_readback.*)

RendererSystem is declared in impl/ecs_render/module.h. The frame-pipeline files show validation, immutable frame snapshots, graph declaration, native execution, and accepted-state updates.

Read feature domains next:

  • impl/ecs_render/material/: material assets, typed bytes, resource resolution, pipelines, and draws (persistent indexed raster and accepted object-space geometry cache in object_geometry_pass.cpp / task_graph_object_geometry_cache.*)
  • impl/ecs_render/mesh/: mesh resources and compute-emulated/native mesh work
  • impl/ecs_render/deferred/: G-buffer, lighting, composite, and presentation
  • impl/ecs_render/csg/: CSG resources and interval work
  • impl/ecs_render/avboit/: transparency targets and tasks (generated-geometry reuse across compatible passes in generated_geometry_reuse.*)
  • impl/ecs_render/raytrace/: shadows, caustics, software BVHs, hardware RT, refraction resolve, and surfel GI (staged hardware transparent shadows in hardware_transparent_shadow.*, per-stage shadow resolve in soft_shadow_resolve_state.h, staged caustic resolve in caustic_resolve_state.h)
  • impl/ecs_render/optics/: coincident refractive-volume resolution shared by render paths
  • impl/ecs_render/reflection/: reflection settings, graph domain, postprocess, feedback, history, sampling, and statistics

impl/ecs_render/raytrace/raytracing_system.cpp coordinates ray-effect feature paths (frame CPU-state snapshots live on renderer_raytracing_state.*; shared snapshot/prepared-build ABIs live in raytrace/graph_snapshots.h and raytrace/prepared_builds.h; accepted optical metadata uploads and their residency live in raytrace/optical_scene_upload.* with graph wiring in raytrace/task_graph_optical_scene_upload.*); impl/ecs_render/reflection/reflection_system.cpp and task_graph_reflection.cpp own the reflection domain. Reflection snapshots expose hasHardwareWork() so zero-budget frames declare no hardware consumer. The shared shader-side contracts live beside the compute shaders under impl/assets/graphics/reflection/ and impl/assets/graphics/raytrace/.

7. GPU Task Graph

Read:

  • core/task/gpu/task_graph.h
  • core/task/gpu/task_desc.h
  • core/task/gpu/task_contract.h
  • core/task/gpu/compiler.cpp
  • core/task/gpu/compiled_graph.h
  • core/task/gpu/packet_runtime.h
  • core/task/gpu/packet_runtime_internal.h
  • core/task/gpu/persistent_state.h
  • core/task/gpu/timing_feedback.h
  • core/task/gpu/scheduler.h plus scheduler_compilation.cpp (centralized scheduleGraph admission), compiler_internal.h, and packet_runtime_internal.h
  • core/graphics/rhi/resource_state_selection.h
  • core/graphics/gpu_timing.h

Then follow the focused compiler and packet-runtime translation units for queue assignment, resource versions, packetization, recording frontiers, concurrent submission, and rejection. task_contract.h defines the typed payload Record/Accepted/Discarded shape. The GPU timing implementation is split among gpu_timing_accumulator.cpp, gpu_timing_listener.cpp, and gpu_timing_submission.cpp.

Renderer task definitions are named task_graph_*.cpp within their feature domains.

8. Graphics API, Backend Selection, And Vulkan

Read the engine-facing API before backend implementation:

  • core/graphics/api.h
  • core/graphics/runtime/runtime.h
  • core/graphics/backend_contract.h
  • core/graphics/rhi/
  • core/graphics/backend_selection/configuration.h
  • core/graphics/backend_selection/backend.h and the narrow selected concern headers
  • core/graphics/backend_selection.h for the full context

Consumer code and graphics tests reach public selected backend definitions through these headers; they do not include native Vulkan headers directly. Selection happens at compile time and adds no runtime dispatch. Vulkan is the default and only implemented provider; the future Metal branch is explicitly rejected until implemented. Tests, fixtures, and test-only helpers live under tests/. Graphics tests exercise existing public contracts after normal backend selection; Vulkan-, Metal-, and other provider-specific tests, private provider fixtures, native SDK assertions, and production test hooks are prohibited.

Then enter the current Vulkan provider:

  • core/graphics/vulkan/backend_context_instance.cpp
  • core/graphics/vulkan/backend_context_adapter.cpp
  • core/graphics/vulkan/backend_context_device.cpp
  • core/graphics/vulkan/backend_context_presentation.cpp
  • core/graphics/vulkan/device.cpp
  • core/graphics/vulkan/device_submission_lifecycle.cpp
  • core/graphics/vulkan/queue_submission.cpp
  • core/graphics/vulkan/tracked_command_buffer.cpp
  • core/graphics/vulkan/swap_chain_presentation.h
  • core/graphics/vulkan/gpu_descriptor_heap*.cpp
  • core/graphics/vulkan/resource_bindings*.cpp
  • core/graphics/vulkan/command_buffer_resource_references.h

Use Platform and GPU Support while reading adapter admission and Bindless Descriptor Heap while reading binding code. Follow the focused backend context, queue, submission, and command-buffer files before treating device.cpp as the only Vulkan lifetime owner.

9. Materials And Shader ABI

For material cooking, read:

  • impl/assets_material/material_parse.cpp
  • impl/assets_material/bind_parse.cpp
  • impl/assets_material/bind_typed_layout.cpp
  • impl/assets_material/material_bind_codegen.cpp
  • impl/assets_material/material_generated_shaders.cpp
  • impl/assets_material/material_dispatch_codegen.cpp
  • impl/assets_material/cook.cpp
  • impl/assets_material/runtime.cpp

Read shared CPU/GPU ABI headers and Slang includes together under impl/assets/graphics/. Binding slots, resource layouts, and struct fields must match on both sides.

Project material interfaces, surfaces, and BXDFs are under project asset roots such as CoolStuff/Testbed/assets and tests/smoke/assets.

10. Tests As Contracts

Unit tests exercise concrete edge cases and regressions; routine success checks and source-text wiring assertions fall outside that scope. Integration/smoke suites retain actual CLI and runtime workflows, and standalone benchmark executables remain explicit performance workloads under tests/performance/ and tests/smoke/. .helper/standard.md section 20 defines the scope.

Useful entry points:

  • tests/unit/task/cpu/: CPU scheduler, scopes, ranges, waits, cancellation, MPMC submission, and profiling
  • tests/unit/task/gpu/: GPU task graph, buffer ranges, packets, and compiler policy
  • tests/unit/graphics/: selected public presentation receipts, device-creation and cooperative-vector bounds, resource-sharing input validation, crash markers, and GPU timing
  • tests/unit/ecs_graphics/: executable renderer ownership, stale-generation, aliasing, cancellation, and failure-recovery regressions
  • tests/smoke/REFLECTION.md and tests/smoke/reflection_smoke.py: reflection capture contracts, routes, filters, and optical fixture checks
  • tests/smoke/renderer_ab_benchmark.py and tests/integration/renderer_ab_benchmark_tests.py: frozen paired workloads, timing admission, completed-sample normalization, and control-equivalence checks
  • tests/unit/filesystem/: IFilesystem and factory mounts
  • tests/integration/pipeline_cli_tests.py: pipeline launcher and tool CLI
  • tests/integration/assets_graphics/: material, texture, shader, model, and cook contracts
  • tests/smoke/: windowed renderer and capture paths
  • .helper/standard.md section 18: source ownership and focused domain boundaries

First Complete Pass

  1. loader/main.cpp
  2. CoolStuff/Testbed/runtime.cpp
  3. core/frame/module.cpp
  4. core/assets/manager.cpp
  5. pipeline/asset_builder/build.cpp
  6. core/ecs/world.cpp
  7. core/task/cpu/scheduler.h
  8. impl/ecs_render/module.h
  9. impl/ecs_render/renderer_frame_pipeline.cpp
  10. core/task/gpu/compiler.cpp
  11. core/graphics/vulkan/device.cpp