kithara hls - zvuk/kithara GitHub Wiki
Documentation reviewed from source revision 19ca073f2. This records the documented contract at that revision; it is not a new runtime validation. API and usage ยท All crates.
Cross-file contracts for kithara-hls: facts no single file shows, that a wrong design would violate. Field semantics, variant lists and playlist tags are documented where they are declared; repo-wide rules belong to AGENTS.md, and encoded/container media types plus the file/segment source contract to kithara-stream.
This crate owns HlsEvent and DrmEvent. HlsFailure is the event payload for
reader-side protocol failures; HlsError remains the crate's operational error
type returned by HLS APIs. Event seek fields use kithara-stream::SeekEpoch.
HlsConfig<S> -> Hls<S> (StreamType) -> HlsCoord<S>, which owns
SessionSlots<S> (active + optional incoming HlsSession<S> -> HlsVariant<S>),
PlaylistCache<S>, and KeyStore<S>. HlsPeer<S> (impl dl::Peer) drives the
coord and emits FetchCmd batches to kithara-stream::dl::Downloader ->
AssetStore<S>, which performs AES-128-CBC decryption (kithara-drm).
HlsSource<S> (impl Source) reads through the coord. None of these types is
public contract.
store: AssetStore<S> and pools: PoolRegion<S> share one closed schema
S: HasPool<u8> and one hard overall budget: HTTP client, playlist cache, key
store, segment storage, and temporary byte buffers all draw on it. There is no
global or per-component fallback pool โ do not add one. Hls::create wraps the
master cancel token in a CancelScope (see config.rs); dropping HlsSource
cancels the scope and tears the peer down.
HlsConfig<S> is this crate's one configuration struct โ tunables and per-call
wiring together โ and #[derive(Patch)] generates HlsConfigPatch beside it,
what a document's hls: section may say. Eight knobs travel: size_probe_method,
download_batch_size, acquire_attempt_budget, the three ephemeral-cache bounds,
event_channel_capacity, and look_ahead_bytes.
net_options is the one tunable deliberately skipped, so hls.net_options is
refused by name rather than parsed and dropped. An embedder holding a configuration
document already spells those options somewhere โ kithara-app spells them in its
own top-level net: section โ and two spellings of one value, one of them dead, is
what a document exists to prevent. The rest of the skipped fields are per-resource
identity (url, base_url, discriminator, headers, initial_abr_mode) or a
live handle only code can hand over (store, pools, keys, bus, cancel,
downloader).
One authoritative active session plus at most one incoming session for a
pending ABR decision, published through ArcSwap<ResidentSessions>. active()
resolves the published ABR variant index and retries when the selector moves
under it, so a resolution never mixes snapshots.
The audio layer drives VariantControl. plan_variant_reader(landing) opens
nothing โ only prepare_variant_reader(plan, profile) opens the incoming
session, builds an OpenedVariantReader and publishes both resident;
take_prepared_variant_reader hands it over, promote_variant / abort_variant
resolves.
VariantTransition identity is (abr ticket, seek epoch). Ordinary transitions
retain the outgoing source; only the exact UpSwitch { reason: EscapeStalled }
claim marks it abandoned, and prepare re-validates that claim before audio may act
on it. Any mismatch discards the incoming session. Promotion runs inside
commit_if_seek_epoch, so a seek landing mid-promotion rolls the outgoing session
back instead of half-switching; it is deferred while the reader is untaken or the
ABR claim is Locked.
Lock cycle: HlsSession::is_ready is a pure question asked under the transition
lock and must not wake the peer โ wake_peer takes the peer state lock while
cancel_incoming_for_seek takes the transition lock. Wake via
wake_peer_for_readiness after dropping the lock.
No read fence: no generation counter, read gate, or decoder acknowledgement.
A switch becomes visible on promotion, and its only published fact is the active
variant's MediaInfo. Reads are never short-circuited โ read_at / wait_range
/ phase_at go through HlsCoord::variant_serving, which prefers the active
variant and otherwise falls back to any shrunk (served_from > 0 or
served_until < num_segments) variant still serving its pre-switch byte range โ
never an idle one at default served bounds. The audio layer detects a switch as any
format change (kithara-audio pipeline::decode::format::detect,
pipeline::rebuild::policy::superseded). Do not reintroduce a fence: it
duplicates a fact MediaInfo already carries.
Owed fetches are tagged RequestPriority::High and drain first, so untagged
fetches queue behind every tagged one. Owed = the segment the audible session's
reader is stopped at, or every planned fetch of a not-yet-audible session.
HlsPeer::poll_next serves the active session first, reserves one slot for the
incoming session, and returns leftover budget to whichever still wants work; at
prefetch_budget == 1 they alternate (SessionTurns).
dispatch_constructing caps an incoming session at construction_segment_end,
derived from the same window reader_is_ready waits on so the two cannot
disagree. dispatch_from anchors the look-ahead on
max(session position, prefetch anchor) and stops at the first segment past
look_ahead_bytes or look_ahead_segments (Init exempt). A
construction_segment_end bound disables both caps: the bound names a debt that
must land in full, look-ahead only trims optional prefetch, and an option must
never cut a debt.
dispatch_from publishes prefetch_resume_at, the cursor byte at which the
deferred segment enters the window, so Source::advance (take_prefetch_resume)
wakes the peer once per deferred segment, not once per read: the window is
cursor-anchored, so only the reader's consumption can make it stale and only that
consumption may re-open it. Popped-but-undispatchable non-terminal entries are
pushed back to the queue front, so an orphaned Downloading slot is re-claimed,
never dropped.
While a transition builds, the outgoing session yields the downloader to it. An
audible session's prefetches past the owed window ride a rotating lookahead
cancel token โ a child of the session's fetch token, so a seek's rearm burns both.
Installing the incoming slot (prepare_planned_variant_reader) retires that token:
queued packs deliver cancelled and in-flight ones abort, freeing capacity the
construction is otherwise starved of โ those bytes lie past the cut the transition
latches and are dead to the splice anyway.
Until the transition resolves, dispatch_active holds the audible session to the
owed window (dispatch_owed), which is everything the splice still consumes โ
pinned by dispatch_owed_reaches_the_transition_latch. A seek can park the
outgoing decoder's reads segments past the byte cursor; starving those fetches
parks the decoder forever while the incoming prime waits on its frontier just as
long. A retire that burned a fetch the wider window still owes is recovered by the
cancelled-settle requeue.
Retired and recovering fetches re-enter in plan order (requeue_planned is an
ordered insert): dispatch caps read the queue head, and a far look-ahead entry
parked at the front would wall off every nearer segment behind it. Pinned by the
variant/flow/plan_queue.rs unit tests.
Priority is stamped at emit and the Downloading claim dedupes re-emission, so a
prefetch stamped Low can never be re-issued as High once the reader catches up;
escalation goes through the slot instead. wait_range is the single filing site of
a parked read (RT try_read / probe_wait and the off-RT Read park all land
there): it calls note_reader_demand() on every claimed slot the read still needs
bytes from โ the whole span, not the first gap, so the downloader serves them
concurrently. build_cmd's live demand probe reads it back and reschedule walks
the command past later-stamped urgent work. Without this an urgent down-switch
starves the audible variant behind the whole construction window and the ring
drains dry.
Two sites must NOT file demand. phase_at stays a pure query โ tracing, error
formatting and the transition's incoming-session probe observe it and must not arm
scheduling. The readiness poll of a session under construction
(reader_is_ready โ poll_range) parks the same way, wait_end filing included,
but files no demand: it is not a read, the incoming variant's fetches are owed
already, and escalating its leftover look-ahead would put it back in front of the
audible variant.
Demand is filed by the same site as ReaderRuntime::wait_end but owned per slot:
a byte-space end alone cannot say which fetch a read waits on, and the escalation
must outlive the wait that filed it. A stale note surviving onto the next claim
is the tolerable side โ clearing on claim would drop a legitimate one.
A fetch cancelled in flight settles back toward the plan it was popped from, gated
by the plan revision, which supersedes on every rebuild โ including a
short-circuit rebuild that changes nothing, because it still claims plan ownership.
settle_cancelled re-inserts only when the revision captured at pop is still
current and the entry is absent. A look-ahead retire rebuilds nothing, so its
cancels re-enter and the reader never waits on work nobody holds; a seek's rearm
rebuilds, so its cancels fail the compare and drop โ the rebuilt plan owns the
re-dispatch, and a stale prefix must not resurrect behind the target.
When the downloader's soft_timeout marks an in-flight slot stalled it wakes the
peer immediately so reconcile_escape can move off that variant; a stalled reader
produces no progress wake. Reader progress reaches the peer through one forwarding
task delivering at most one wake before yield_runnable() hands back a scheduler
turn. A reader edge stored during that turn is still work in hand, so Flash retains
the task's participant slot across the re-poll. The task releases its slot when
the next notification wait is pending and no work remains. Outside Flash the yield
is the ordinary Tokio hand-off.
Reader-wake fairness contract: source revision a2fca2332.
HlsVariant::build_init_entry keys the init slot on the playlist #EXT-X-MAP URL
โ a static fact โ never on the init's known byte size; variant/io/init.rs
carries the failure mode. The init is enqueued at the queue front so the demuxer
has the container header before any media segment.
Init and media use the same Segment acquisition and slot-settlement
algorithm. The existing attempt budget settles a permanent local acquisition
obstruction as Failed and wakes readiness; a successful acquisition resets
its counter, and a live TmpClaimed writer keeps its existing exemption.
Loaded and Failed slots stay out of the fetch plan. Init and owed-media
priorities, plan revisions, and cancellation admission remain owned by that
same fetch plan.
While declared but unsized (init_size() == 0), the fresh-activation frame
(served_from() == 0) reserves the init prefix for the declared init. Reads
remain pending until commit supplies its size. A terminally failed init retains
ownership of its prefix whether sized or unsized: raw reads remain pending
without returning media bytes, while range polling and waiting report
SegmentUnavailable. Loaded media must not satisfy the prefix or turn failure
into successful EOF. The same init ownership governs reads, readiness, failure
coverage and reader demand. A switched-in
variant's init is orphaned in natural space, so its reads are not gated here.
Init acquisition and reader contract source: revision c21f986c24d1.
HlsVariant::header_byte_range (alias format_change_segment_range) is the one
answer to "what does a demuxer re-read to re-establish container state after a
decoder-recreate": the virtual init range with an #EXT-X-MAP, otherwise segment
0's natural byte range for implicit-framing containers to re-scan, and
Err(SourceError::FormatChangeNotApplicable) on a shrunk variant.
rebuild_with_decoder_probe also enqueues seg 0 when from_seg > 0: the decoder
factory's Symphonia probe needs the container header even with a separate init.
HlsTrackState::apply_boundary_crossing calls it only on the aligned rescue path
(active variant changed since the last poll, reader physically resolved to a
segment, served_from() == 0). A landing-anchored reader from
HlsVariant::prepare_reader gets no seg-0 probe and instead backs off one segment
behind the landing (forward_segment - 1).
Layout (variant/map/offsets.rs) is the single coherent owner of every
cross-variant coordinate; its Frame / FrameSnapshot doc comments own the
publication rules. The caller-side rule: every layout read goes through
try_published, which returns None while a writer was in flight or the
publication changed, and a None means keep your gate closed and retry next tick
โ never spin, never fall back to a partial read.
layout_seek_invariant is the one formula for "a layout reset would change nothing
worth a re-mint": canonical single-variant full-range geometry
(is_canonical_complete, every served size exact) with nothing parked in
deferred_prefix. A live seek tail alone does not force the reset โ against a
canonical table it freezes nothing and only helps the EOF gate, so fully-cached
segment-aware seeks keep skipping the O(N) rebuild. It gates both
HlsCoord::prepare_for_seek's reset call (a fully-cached seek must not touch the
layout at all โ pinned by stress_seek_audio's reset-counter assertion) and the
re-mint inside reset_layout_to_full_range; ABR invalidation and the reader wake
stay unconditional. HlsCoord::reset_for_seek is layout-only โ it cancels no
in-flight body fetch.
While a segment-aware seek tail is live (VariantSeek::segment_aware_tail != NO_SEEK_TAIL, only for containers not needing exact byte sizes), the reader, the
demuxer's byte map and the peer's cursor all hold bytes minted on the frame the
seek anchored. A settle for a media segment behind the tail must not re-key
that space: HlsVariant::apply_commit parks its size in
VariantSeek::deferred_prefix, and the next re-mint
(reset_layout_to_full_range) retires the tail and applies the parked sizes
atomically with the fresh frame. While parked the real size lives only in
deferred_prefix and the size atom still reads placeholder โ a transitional split
of truth that ends at that re-mint. Freeze and drain both run inside the Layout
write lock, so a settle can never park behind a tail whose space is already gone
and a parked size can never be lost or applied twice.
The init is never parked: init reads gate on its size being exact, so a parked
init starves every consumer waiting for its bytes (the demuxer probe of an ABR
pending variant deadlocks before activation could drain it). That same gate makes
the byte-space shift an init settle causes unobservable to a post-seek reader โ
init_read_at / init_contains refuse until the size is exact, so the demuxer
cannot advance off the seek alias' anchor first. Optimistic init reads would
silently reopen the shift window. Pinned by
post_seek_prefix_settle_does_not_move_the_frame and neighbours in
variant/tests.rs.
prepare_for_seek is the crate's SeekPrepare impl and the only site that
rebuilds the byte space for a seek. The control thread runs it once per seek
before the epoch is minted (SeekHandle::begin). Both halves take a lock โ
cancel_incoming_for_seek the transition lock, Layout::reset the write lock plus
a copy of the offset table โ so neither may run on the produce core, where the
reader resolves its anchor. Because the rebuild precedes the epoch, every observer
of that epoch already sees a matching layout and ByteMap::anchor_at_time stays a
pure query. Pinned by the rtsan-hls lane.
Byte EOF is minted only when total_bytes() > 0, the offset is at or past it, and
eof_ready() holds; suppressed while a seek alias covers range.start or the
timeline is flushing. It is never inferred for an in-range segment whose body has
not arrived, nor while a served segment's size is unknown โ either yields
WaitBudgetExceeded so the reader holds. An immediate seek before size estimation
completes can leave a served segment at size 0, where a premature Eof latches the
audio consumer into AtEof and skips the track. Pinned by
tests/tests/kithara_queue/early_seek_size_withheld_advance.rs.
sizes_complete() is MEDIA-only, so exact-seek short-circuits gate on
all_sizes_complete(); a media-only short-circuit would skip SizeDemand::Init and
the complete_exact_seek_if_ready anchor correction. A lazy size probe is allowed
only for a SegmentContent::Plain slot with a non-exact size โ an encrypted
segment's transport length is not its plaintext length โ so encrypted demands fall
back to a full body fetch pushed to the queue front.
The seek alias (variant/flow/seqlock.rs) is a lock-free base + resolved-exact
anchor pair under a generation tag: the produce core reads it on every
find_at_offset, off-RT resolvers publish the exact anchor under the base
generation so a stale resolver cannot attach to a newer alias, and retirement is
generation- or position-checked. HlsVariant::seek_point_at_time is the sole
time โ segment mapping, so a variant switch and a plain seek to the same time
cannot diverge.
The fetch plan (variant/flow/plan_queue.rs) follows the same split: its
fetch_is_planned, reached from phase_at on the produce core, reads only the
lock-free membership mirror its module docs own.
Source::wait_range(start..end, timeout) has two modes selected by timeout, and
HlsVariant::wait_range ignores the timeout value:
-
Some(_)โ wake-free probe (RT worker /Stream::probe_read): one non-blocking check that never sleeps, answering from the current virtual layout and otherwise returningWaitBudgetExceededimmediately. -
Noneโ event-driven blocking wait (off-RTStream::read/prime_seek_range):HlsCoord::wait_range_blockingparks on the shared readiness gate until the probe resolves, a covering segment fails, or cancel fires. No wall-clock data poll.
HlsSessionReader picks the blocking mode while the session's ConstructionGate is
armed (an off-RT decoder factory is building against it), the wake-free
Some(Duration::ZERO) probe otherwise.
SizeSignal (src/signal.rs) owns the readiness gate and the wakes; its doc
comments own the fire sites. Facts living nowhere else: the gate is single-waiter
(the one off-RT wait_range(_, None) reader) and a lock-free ThreadGate, not a
condvar, because RT-reachable edges signal it on the produce core; and for DRM the
decrypt gate opens only at commit, so FetchSlot::settle is the load-bearing wake,
not the byte writes. The reader's park is bounded (READER_REAIM_INTERVAL) and is
not a data poll โ it returns WaitBudgetExceeded so an off-RT reader can re-aim a
mis-aimed peer. WAIT_HANG_TIMEOUT must exceed the kithara-net per-fetch total
timeout, so a stalled upstream fails terminally before the watchdog trips.
KeyStore resolves the #EXT-X-KEY URL and asks KeyProcessorRegistry for an
optional PreparedKeyRequest. The registry holds opaque resolvers and knows
nothing about domains or providers โ domain matching and request shaping belong to
kithara-play policy. #EXT-X-KEY URIs resolve relative to the segment URL,
not the media-playlist URL. The original key URL owns memory and persistent-cache
identity; a prepared URL is used only for the wire request, and resolver
preparation happens inside the per-resource transaction after the cache rechecks,
so a cache hit never creates fresh salt or processor state. Session memory owns the
validated key that synchronous segment construction needs, so a cache persistence
failure does not turn a successful key fetch into a playback failure.
Decryption is part of the resource lifecycle, not a read-side step:
DecryptProcessor wraps a DecryptContext as a ResourceProcessor (identity =
key||iv), encrypted segments are acquired with acquire_resource_with_ctx, and
the store decrypts during the writer's commit before a reader becomes ready. Commit
must pass Some(final_len) โ None silently skips decryption. PKCS7 unpad shrinks
the committed length below the announced size, so HEAD-based estimates are upper
bounds and the settle path adopts the committed final_len.
Every playlist, init/media segment, and encryption key is an AssetResource::Url
in the Hls asset scope โ playlists are cache resources like anything else, not a
side channel. Path and root naming belong to the layout registered for the
Hls<S> marker (kithara-assets DefaultLayout by default); this crate does not
own it, and there is no separate playlist or analysis layout.
Playlist and key bytes are one validated cache transaction
(AtomicFetch::fetch_validated under AssetStore::with_resource_transaction). A
cached body is parsed before acceptance; a validation failure or an empty committed
resource removes it via AssetStore::remove_resource and performs one network
fetch. Network bytes are validated before write-back, so an invalid response
never becomes a persistent cache hit. A valid body stays usable when persistence
fails โ PlaylistCache keeps the parsed value for the rest of that instance.
VariantSegments holds the store's reader in exactly two slots keyed by
ResourceKey โ init prefix and media segment, which is what one read_at walks โ
so a decoder consuming a segment in small buffers opens the store once, not once
per buffer. Two slots is the bound because a held reader pins its asset; a reader
whose resource went Failed or Cancelled is dropped and reopened.
Eviction routes through an EvictionSubscription guard held by HlsSource:
HlsCoord::broadcast_eviction marks the lost key Missing on every variant that
owned it and rebuilds the active variant's queue from the reader's segment.
Non-active variants stay relaxed and pick their Missing entries up on the next
activation; HlsVariant::on_evict also releases the held resource.
Composes with kithara-audio as Audio<Stream<Hls<S>>>; emits HlsEvent /
DrmEvent through a DeferredBus on the shared EventBus. Throughput estimation
and ABR decision policy live in kithara-abr; this crate only publishes
claims/commits and reports Abr::progress.