Documentation reviewed from source revision b22a87c8c. This records the documented contract at that revision; it is not a new runtime validation. API and usage · All crates.

The consumer surface is the root facade; system, loom, and backend/* are private and lib.rs picks one combination at compile time. flash decorates the selected primitive backend and never exposes a second backend path; wasm32 compiles its own tree and ignores these features. Exception: the process-wide flash engine, its async task gate, and the diagnostics registry always take bookkeeping locks from the private system::lock::Mutex, because they outlive Loom model permutations.

Features are all off by default. flash, loom, and no-block are native and test-only and no shipping crate may reference them; loom composes independently with flash.

Arc<T>, Weak<T>, and OnceLock<T> route through the system backend on every lane — one ownership type workspace-wide, with Loom modelling only the coordination primitives stored inside that ownership graph. Consumers import every synchronization type from this crate, never from std.

Third-party crates that block through parking_lot_core — dashmap shards are the workspace's only such consumer on wasm — are not covered by those wrappers. parking_lot_core picks its wasm thread parker at compile time and, unless its nightly feature is on, selects the one whose every method panics; +atomics alone is not enough, so a contended shard aborts the worker that touched it. This crate turns that feature on for cfg(target_arch = "wasm32") and holds the reference in wasm/thread.rs, which makes every wasm build nightly-only — the wasm target already is, since +atomics forces -Z build-std. The resulting parker calls memory.atomic.wait32, which is legal in a Worker and traps on the browser main thread; wasm playback owns its work in Workers for that reason.

Mutex and RwLock are flash-blind by contract: they guard bounded critical sections whose holder always runs and never parks while held, so virtual time can never advance with the lock held. Flash-awareness lives only in the wait primitives, and flash::sync::{Mutex, RwLock} are plain re-exports.

sync::Retired<T> (common/retire.rs) is the one writer-side reclamation mechanism for ArcSwap values read on a no-free path. swap and rcu pay every reader debt on the displaced pointer before they return, so a displaced value owned only by the list has no reader guard and can never gain one; retire frees every such quiesced value, then holds the new one. It has no lock of its own: each writer keeps it behind the writer's existing serialization, which a no-free reader must never take. ThreadGate, the HLS layout frames and resident sessions, and the assets and storage availability snapshots all retire through it.

ThreadGate (common/gate.rs) has ONE waiter. Same-thread registration performs no publication; a changed waiter publishes through ArcSwap and keeps displaced handles in a writer-only Retired list that signal never touches, so signalling stays lock-free. An RT signal thread must pre-warm arc_swap's per-thread debt node; the audio source does so in warm_up. Every WaitGate caller snapshots current() BEFORE checking its predicate, so a racing signal is never lost.

• Model tests opt in with #[kithara::test(loom)], which renames them loom_model_<name>.
• just test run --loom=on selects the lane, filtering on that prefix, so unmarked tests never run under Loom. Plain just test leaves the feature off and a marked test runs once on the system backend; --flash=on runs the same models through the flash decorator.
• Loom has no timed waits: every timed wait panics in the loom-only lane, so a model needing virtual deadlines runs under flash,loom.
• Arc / Weak / ArcSwap operations stay outside the model. A model asserts a small deterministic coordination contract, never a network, decoder, runtime, or wall-clock test.

On wasm32 a thread::spawn is a Web Worker and the thread-affinity assertions (assert_main_thread / assert_not_main_thread) actually panic; natively they are no-ops, so an affinity bug stays invisible until it runs in a browser.

tokio is re-exported as an ENUMERATED subset: no glob, no root spawn / spawn_blocking, no time / fs / process. This crate is the sole sanctioned holder of a direct tokio dependency (arch.tokio_dep_quarantine); consumers route through the platform spawn chokepoints.

A thread asks for its own scheduling class. thread::set_current_class takes a ThreadClass and applies it to the calling thread only, so the code that knows what a thread feeds decides its class, and no thread changes another's. Normal requests nothing. AudioFeed is for a thread that keeps an audio output buffer filled ahead of the device, at the priority each platform gives its own media playback threads: nice -16 on Linux and Android (Android's THREAD_PRIORITY_AUDIO), the user-interactive QoS class on Apple, the MMCSS "Audio" task on Windows, and nothing on wasm, where browsers expose no thread scheduling. The OS answer is returned, never swallowed. Linux lowers a nice value only within the process's RLIMIT_NICE (20 minus the lowest nice it allows, so -16 needs at least 36) or with CAP_SYS_NICE; a refused thread keeps the class it started with. Thread class contract: source revision deaafda65.

time::{sleep, timeout, Instant, Duration, SystemTime} is the single clock chokepoint, and under flash it is the virtual clock. web_time is internal to this crate and std::time::{Instant, Duration} is banned outside it. Gates: arch.no-direct-time, arch.no-direct-thread-wait, and arch.no-implicit-clock (hidden clock reads in library code).

flash replaces the wall clock with a process-global virtual timeline so warm-cache offline playback tests run at CPU speed. Off the feature every flash macro is an identity no-op.

time::Instant follows that virtual timeline and its arithmetic saturates. time::WallInstant is the explicit real monotonic clock for external host APIs — a window event loop whose deadline type cannot be virtual — and application timers must keep using Instant.

Flash is opt-in per region, not a global mode flip. Two flags of the per-thread Mode (in flash/ctx.rs) decide virtual vs real; both default to false ⇒ REAL by default.

• ambient — per-test gate, set by #[kithara::test(flash(bool))] and propagated across every spawn form (per poll for async). Read by flash_ambient().
• active — dynamic gate, pushed by a production #[kithara::flash(bool)] region and itself gated by ambient, so a dynamic flash is inert outside an ambient flash test. Read by flash_enabled().

They govern disjoint primitive classes:

• flash_enabled() — the stateless time primitives (time::sleep / time::timeout, Instant::now, and the thread time helpers).
• flash_ambient() — the stateful sync primitives (Condvar, Notify, mpsc, oneshot), each latching the gate ONCE at construction, plus tokio::task::yield_now, which consults ambient per call (a yield has no signal partner, so no latch). Wait and signal run on different threads; a per-callstack gate would let the two sides target different engines and hang.

Annotation scope:

• Production #[kithara::flash(true|false)] is dynamic: it pushes active through the region's call stack and across its spawns.
• Test #[kithara::test(flash(true|false))] is lexical: it rewrites the test BODY's direct time calls to unconditional virtual entry points and sets ambient for the whole test graph. So a prod fn the body calls keeps its stateless time real unless that fn is itself #[flash], and since the rewriter keys on the last two path segments the body must use a qualified path (time::sleep) — a bare single-segment call is a compile error, not a silent real-clock call. Default under the feature is flash(true); real-socket suites are flash(false), whose graph is identical to production.

thread::spawn_named threads are dedicated virtual-time pacers: the child sets active = ambient for its whole callstack, so its Instant::now reads and its waits share one clock domain despite being keyed on different flags.

The clock is quiescence-driven, not additive: it advances only when every participating root is parked, and only ever jumps to the earliest registered deadline. Clock values are a pure function of the pending-deadline multiset, independent of scheduling: runs are deterministic and every timed wait collapses to zero real time.

• Participant accounting is intrinsic to the wrapped primitives: there is no registration API and no consumer registers anything. A thread earns Credit lazily on its first wrapped wait, and until then is invisible to the engine — so a busy-spin thread that never waits cannot stall the clock, and neither can a test body that never waits.
• The platform spawn forms reset the credit on entry (a reused pool thread must not inherit a stale credit) and settle its slot via an RAII participant after the closure returns or unwinds, so a wrapped wait on a blocking pool thread must be spawned through platform task::spawn_blocking. They reserve the child's slot synchronously on the spawning thread, before the OS thread exists or a pooled closure is queued. At pooled exit the thread's credit is cleared and its previous dedicated-pacer flag restored, so a reused thread cannot pace later unrelated work.
• Every peer a timed waiter depends on must therefore be spawned (its slot reserved) BEFORE the waiter can park: a waiter parked in a spawn→spawn gap is the only credit, and the clock jumps its whole backstop before the peer exists (thread_gate_refreshes_waiter_after_thread_handoff).
• A wrapped wait does deadline registration, wait accounting, and advance-rule evaluation in ONE critical section under the engine lock, closing the wake-to-re-register race. Lock order is always domain → engine; wakes fire only after the engine lock is released.
• Async tasks are counted separately in active_async, owned by the spawn poll-wrapper rather than the firer. On completion a counted task moves its slot from active_async to active under the engine lock; it stays counted until that slot is released. Quiescence requires no pinning participant slots or granted waiters; a task or grant whose owner cannot be polled is exempt under the rules below.
• When a spawned async task or pooled blocking closure finishes while a poll of its JoinHandle is waiting, the task and handle's join handoff retains the existing slot. The runtime wakes the waiting joiner before releasing the finished work's slot, so the clock cannot advance during the completion-to-wake gap. Work with no waiting handle poll releases immediately, even if its handle still exists. A ready or dropped JoinHandle releases any slot retained for that poll. The handle charges Tokio's task budget itself and polls the runtime's handle unconstrained, so a poll the budget turns away yields with the caller's waker and retains nothing.
• Every granted async waiter retains one separate credit from grant publication until its unique receipt is consumed or dropped. Entering a poll does not settle that credit. The existing task record owns known-task grants; grants without an existing task record remain conservative active credits. A completed task record survives until its last grant settles.
• Grant publication, cancellation, and settlement share the engine lock. Each receipt identifies its creating task and engine, so cancellation before grant removes the waiter without releasing a credit. Cancellation after grant releases exactly one credit while the creating engine remains alive. Wakes run after unlock. Receipts do not keep a stopped engine alive or settle against a later engine.

Virtualized waits:

• thread::park_timeout(d) is unparkable: it wakes when the clock crosses now + d or when a peer thread::unpark wakes it, whereas thread::sleep(d) has no early wake. A flash-active unpark fires BOTH the engine entry and the OS park slot (park mode is decided by the TARGET's flags), and an unpark arriving before the park is remembered.
• thread::paced_backoff(d) is the backoff for a synchronous poll loop whose data comes from another engine-visible thread: a real sleep(d) throttle off the sim path, and under flash a deadline-less cooperative yield that cannot inflate the clock and re-polls in lockstep with the producer. Use it instead of sleep in every such loop.
• thread::paced_backoff and thread::yield_now take the engine yield on a flash callstack and on any dedicated participant, even one with no active flash region. A pooled spawn_blocking closure is such a participant: it inherits the ambient gate and a dedicated credit, and that credit holds the clock, so a real sleep there would freeze the clock for the whole backoff.
• A sync::Condvar waiter registers its engine entry before releasing the domain guard, so a predicate change plus notify is serialized after the entry — no lost wake. The caller re-checks its predicate after re-acquiring.
• tokio::task::yield_now() is engine-backed only inside an ambient Flash test and is the cooperative wait of a loop that has no work until another participant progresses. Outside ambient Flash it is an ordinary scheduler yield. Its sibling tokio::task::yield_runnable() hands back one scheduler turn while the task still has work: it self-wakes before returning Pending, so the task gate retains its async slot across the re-poll. thread::yield_runnable() is the synchronous fairness yield and retains the dedicated thread's active credit. The runnable forms do not register engine yield waiters or release participant credit. Real I/O still follows the separate paced-clock rule below.

A cooperative yield re-checks at the current instant when every timed waiter is a ParkRole::Backstop under an event edge, or no timed waiter exists: crossing a backstop would reveal no new work. A required ParkRole::Deadline instead owes the clock a stop, including a thread gate's poll deadline. The normal advance rule reaches the selected deadline and wakes the yielders with it; unpaced selection prefers required deadlines, while real I/O pacing uses the earliest timed waiter and preserves its real-time debt. Each granted waiter retains its receipt credit, so a pollable grant stops further advance until consumed or dropped. A fairness yield with work in hand uses yield_runnable and retains its participant slot instead of registering a quiescent yield waiter.

Runnable-yield and deadline contracts: source revision a2fca2332.

Async counterpart: a task no live thread can poll must not pin the clock. A synchronous wrapped wait inside a poll (a BRIDGED wait) releases that poll's own slot. A runnable task last driven by that same sole runtime thread is also unpollable until the bridge returns. Driver and single-poller facts come from the runtime that actually enters each poll; another runtime's spawn handle does not determine pollability. Tasks that another worker can poll keep pinning.

Retained grants use the same actual-driver evidence. A parked or runnable task's grants cease pinning only behind its bridged sole driver. A Running or RunningNotified task's grants cease pinning only when its own async slot is absent and its actual driver is bridged: another worker cannot concurrently enter that same poll. A running task that still owns its slot keeps pinning even if old driver metadata names a bridged thread. No driver evidence or a completed task means pin conservatively. Exemption retains every credit; driver resume restores pinning under the engine lock before the poll continues.

When a selected grant is already unpollable, the engine repeats its existing deadline and pace rule until a retained pin, no eligible waiter, or unpaid real I/O pace debt stops it. Each step removes existing waiters, so this drain is finite under the engine lock. Equal deadlines remain one batch, and all collected wakes run after unlock; a queued wake is not required to reach the bridge's timer.

Async waiter grant contract source: revision b01a2e7e.

flash::reset() clears a drained unit-test or Loom run. No waiter, grant receipt, or participant from the old run may remain reachable. Nextest's per-test process isolation handles product-test state; reset is not a live-engine restart path.

A task awaiting a real socket parks and releases its quiescence slot, so the engine cannot observe real transit. Corrective invariant, extending the pooled-participant one that covers an ambient spawn_blocking closure: virtual time must never outrun real time while real I/O is in flight.

• flash::real_io() brackets ONE real I/O operation; consumers use #[kithara::flash(io)], and kithara-net applies it around request establishment, full-body reads, and streaming body pending windows.
• While at least one scope is live the clock is paced, not pinned: the first scope anchors a real instant to a virtual one and the advance rule refuses any jump past that anchor plus the real time elapsed since. Pacing rather than pinning keeps a virtually-delayed peer live — a fixture server sleeping on a virtual delay still progresses at 1:1 real pace. A pollable or unknown-owner grant still pins the clock in both paced and unpaced modes; real I/O pacing cannot bypass its receipt lifetime.
• The pacer is a raw std thread, spawned lazily on first entry and deliberately INVISIBLE to the engine (a platform spawn_named would count it as a participant and pin the clock it exists to advance). It is event-driven: it parks until the earliest paced deadline and re-runs the advance rule on wake, and never unparks itself (that would arm its own token and busy-spin).
• When the last scope drops the anchor clears and full-speed collapse resumes immediately.

flash::hang_dump(context) is pure, so the caller routes it anywhere; the test harness records it in the hang artifact before its timeout panic. The sync-primitive registry it reads is gated at runtime by KITHARA_FLASH_SYNC_TRACE (default off - a wrapped primitive then pays only a null check), and KITHARA_FLASH_SYNC_BT=1 adds backtraces. Holder and waiter evidence otherwise comes from registered identities and static source locations, not from live stacks.

Deadline-less (indef) waiters are the exception: nothing but a matching signal frees one, so the parking site is the whole diagnosis, and a dump's own backtrace - taken by the watchdog, not by the code that parked - names the wrong thread. Each indef entry therefore records where it parked, BEFORE taking the engine lock (a stack walk under it would hold the whole engine for its duration). Timed waiters record nothing - their deadline already names the give-up authority.

The dump marks an indef waiter pins_clock when it is the reason the clock is stuck: an async waiter whose task still holds an active_async slot, or a sync waiter parked mid-poll on a bridged thread. The marker needs no env var, so every lane carries it; the recorded stack prints only for a marked waiter, since a healthy test parks hundreds of unmarked ones.

Ground truth is two runs compared: the default real-time run, which catches concurrency and timing bugs, and the fast flash run. Divergence in sample-count positions or PCM flags that virtualization distorted something; the FILE phase-continuity cluster is the first such oracle.

Poll-scoped async blocking detection, test lanes only: enabled by just test run --no-block=on and kept ON by just ci gate lane 1 (flash ON + no-block ON; lane 2 is flash OFF). Like flash it is an inert twin swapped in at lib.rs, so prod builds stay byte-identical.

Two levels, both scoped by the Watched per-poll combinator, installed automatically at every #[kithara::test] async root and at both spawn chokepoints:

1. Chokepoint (forbid) — deterministic, timing-independent. The native blocking primitives (thread parks, Condvar waits, blocking mpsc receives) refuse to run inside a poll, attributing the caller via an unbroken #[track_caller] chain. Mutex/RwLock locks, spawn_blocking, and wake operations are deliberately NOT intercepted (short locks in async are legal; long waits are level 2's job).
2. Budget — wall/CPU timing of each poll on the REAL clock (std::time, never the virtualized platform clock); thread CPU comes from a cached snapshot, so a poll can include a little pre-poll CPU. A lenient blanket budget applies everywhere (KITHARA_NO_BLOCK_BUDGET_MS overrides) and the strict tier is opt-in via #[kithara::no_block]. Reports classify CPU-spin vs blocked-wait from the thread-CPU/wall ratio: in panic mode the blanket tier panics ONLY on CPU-spin, still emitting census lines for the rest, while the strict tier panics on every over-budget class.

Modes via KITHARA_NO_BLOCK: panic (default), census, off. A census observation reaches one sink: KITHARA_NO_BLOCK_LOG=<file> when it names one, tracing otherwise. Never both — a profile storing the output of passing tests would carry the whole census into its JUnit as well. A configured census sink is evidence: an open or write failure panics the attempt instead of silently dropping a record.

Escape: #[kithara::allow_block] suppresses level 1 AND pauses the level-2 timer. Its guards are !Send: they mutate thread-local state in Drop and must not cross .await.

Flash interop runs one way — flash calls no_block, never the reverse. A BRIDGED wait (a wrapped sync wait inside an async poll) is reported BEFORE the engine wait, while the engine's own wait runs under a permit, so engine coordination neither trips level 1 nor counts against the budget.

just test rtsan-async compiles every watched poll as an RTSan nonblocking context (nightly, --cfg rtsan); suppressions live in .config/rtsan/async-suppressions.txt. arch.no-raw-no-block bans hand-calling the expansion targets outside their owner files.

CancelToken is the single cancellation token type across the workspace — no crate depends on an async-runtime cancel crate (tokio_util). It is built on a private propagate-down Node tree (common/cancel/node.rs): a flag, a cold list of Weak children, and a waker registry. A node holds its parent Arc without ever reading it — the walk is always down — purely to keep an intermediate node alive for its Weak children. is_cancelled() is a single Acquire load, lock-free and wait-free, hence RT-safe on the audio produce-core.

The tree uses std::sync only (no parking_lot, no flash variant), so it compiles unchanged on wasm32. Every guarded section is a linear structural edit and wakers fire outside the lock, so a poisoned lock is recovered rather than propagated.

• cancel() is idempotent (a repeat neither re-drains nor re-recurses): it swaps the flag, drains this node's wakers, then recurses down through live Weak children. A master cancel reaches every descendant by writing their flags, and the flag store happens before any waker fires, so a thread observing a wake is guaranteed to see the flag.
• child() derives a new node under the parent's children-lock, so a concurrent cancel() either includes it in the snapshot or has already set the parent flag — in which case the child is born cancelled and a future or waker on it never parks. Cancelling a child or sibling never marks the parent. Clone keeps the same node: cancelling a clone is observed by the original.
• cancelled() is cancel-safe in tokio::select! (dropping it unregisters its slot). on_cancel() registers a synchronous waker — the counterpart for a thread parked on a flash-aware Condvar/Notify — and its guard unregisters on drop. The waker runs on the cancelling thread: keep it cheap, non-blocking, idempotent. A registration arriving after the drain fires immediately (the fired latch).

CancelToken::root() (an owning master a consumer-crate top cancel()s on teardown) and CancelToken::never() (a sentinel where a token is structurally required but no cancellation source exists) mint a fresh subtree root instead of deriving from a parent, and are restricted to owner/sentinel sites — see docs/guides/cancel-policy.md, enforced by just lint arch (cancel_root_sites). Everywhere else, derive with .child() or take a CancelToken from your caller. Dropping a root is passive: it does not cancel its subtree.

CancelScope::new(Option<CancelToken>) is the seam between composed and standalone subsystems and the canonical replacement for a cancel.unwrap_or_default() fallback: Some(parent) makes the scope's token a child, so a master cancel reaches it, while None mints a fresh root. token() vends clones of that one node, and Drop is passive — a composed scope never cancels a token handed from above.

CancelGroup is a read-only OR-combinator over several tokens, composable with |: is_cancelled() is true once any source is cancelled, cancelled() parks a slot per source (dropping it unregisters every slot), and equality is source-array identity (Arc::ptr_eq). on_cancel() registers one shared once-gated wake across all sources. An empty group never resolves.