Three tiers, one ladder. Same mutator shape at every rung — climbing is a config change, never a rewrite.

As of 2026-08. Stable API — semver from here (Upgrading). Tiers 1–2 ship. Tier 3 (local-first) has not shipped, As of 2026-08.

@ultimat3/realtime ships two entry points, and which one a name lives in is a mechanical fact rather than a convention — the barrels are disjoint, and a test asserts it.

import { useQuery } from '@ultimat3/realtime';             // island
import { syncNode } from '@ultimat3/realtime/server'; // sync node

A server render is not a missing install. With no DOM, every read hook answers pending with no subscription, and useMutation() refuses with X_LIVE_SERVER_RENDER. hasPageSocket() answers false there, every time, so it is not a guard that makes a page-body read safe: a browser-only read in code no island imports is X_LIVE_ROUTE_NO_ISLAND at the budgets step, hasPageSocket() included. In a browser, a hook in a bundle that never called installRealtime() is X_REALTIME_UNINSTALLED.

A file that needs both writes both imports. Nothing was deleted in the split — if an import stops resolving after upgrading to 8.0.0, the name moved to ./server.

Why it is two entries and not one: a single barrel carried useLive beside openNatsClient, so bun build --target=browser on an entry importing only the hook failed with "Browser build cannot require() Node.js builtin: stream/web", out of nats — the island this page tells you to write could not be bundled at all. The package's sideEffects array is what lets the bundler shake the bus out; the split is what makes "the client entry cannot reach the bus" something a build cannot quietly undo, since a namespace import or an export * defeats tree-shaking.

Tier 1 for presence, typing indicators, toasts, cursors. Tier 2 for "the list updates when someone else edits". Tier 3 for offline-capable apps.

Row 3 is on the entity, not the query. persist: true is an entity() option in 21.0.0, read by realtime's IndexedDB persister and one outbox. Read the row's own section before writing anything against it.

Tier 1's events are at-most-once, by construction; its records frames are repaired. A dropped events frame is counted and logged, never resent. A dropped records frame is marked on the server, which sends the socket a replay-gap frame once it drains, and the client re-runs the channel's catchUp read (21.0.0; below). State that must arrive belongs on a live query or a records channel; ephemera belong on events.

Tier 2 covers what people almost always mean by "make it realtime": the list updates without a refresh, and my own click feels instant. It delivers both with no client database, no client schema versioning, no conflict-resolution UX, and no offline-write semantics to design. Tier 3 buys exactly one additional property — writes that survive being offline — and costs a durable local store, a rebase log, client migrations, and a conflict story per mutator. Charging every app for that is how realtime frameworks become slow frameworks.

// mutator (action + optimistic local twin)
export const likePost = mutator({
  // REQUIRED (`X_MUTATOR_NOT_IDEMPOTENT`): a replay under the same Idempotency-Key answers the
  // first result instead of running `server` again.
  idempotent: true,
  // Convergent, not incremental: `local` replays on every rebase, so applying it N times has to
  // equal applying it once — `likedByMe` is what makes the second application a no-op.
  local(tx, { postId }) {
    tx.posts.update(postId, (p) =>
      p.likedByMe ? {} : { likedByMe: true, likeCount: p.likeCount + 1 });
  },
  async server(ctx, { postId }) { return ctx.posts.like(postId); },
  conflict: 'server-wins', // | 'last-write-wins' | custom(merge)
});

A mutation the server refuses is rolled back, not retried. The optimistic write goes, and so does every write made after it — undone newest first, then replayed without it, which is sound only because local is pure. The refused intent is dropped from the rebase log: a denial is a decision about that intent, and retrying it would put the write the server refused back on the screen. A refused key stays in the queue as failed for the UI to render, and re-issuing the same idempotency key is treated as a new intent with a new sequence at the back of the queue, not a collapse onto the denial.

local is a pure function of (tx, input), therefore replayable. Hence the rule: no I/O, no Date.now(), no Math.random() inside local. Same input, same patch, every replay.

interface Mutator extends Action — a mutator is an action, so every action member is already on it (likePost.openapi(), .client({ baseUrl }), .job(), .contract(), .as(), .describe(), .named(), and the lifted .input .output .policy .mcp). The three names it was authored with — .local, .server, .conflict — project unchanged, plus a brand and its own descriptor. Both halves are reached through the mutator itself, likePost.local(tx, input), never a declaration object. A mutator has no .def.

.server() routes through the action's own callable, never the declared server. It calls base(input, { ctx }), and that callable is invoke — so the authoritative half cannot skip the input parse, the policy or the output parse. Reaching the declaration from there would be a second execution path, which is the one thing @ultimat3/action exists to prevent. Hence the guarantee this page rests on: the offline/realtime half of a mutator gets exactly the same actor gate → parse → policy → handle → parse core as an HTTP call, an MCP tool call and a job run, because it is that call. A denied .server() never reaches the declared half, and neither does one whose input fails the schema — X_UNAUTHENTICATED, X_FORBIDDEN and X_INPUT_INVALID come back from the same core that serves the HTTP route.

The parsed-vs-raw asymmetry between .local() and .server() is deliberate, not an inconsistency. local replays on every rebase and must stay a pure function of (tx, input) — no I/O, no clock, no randomness — so re-parsing on each replay would be wasted work on input that was already validated once.

.local() is the name. No alias: the old .applyLocal is gone, not deprecated.

.named() rewraps rather than dropping the twin — a renamed mutator keeps both halves, its conflict, and every inherited action member.

Opt-in per entity, and shipped in 21.0.0 (CHANGELOG.md). The 20.x shape (store, queue and log on LiveClientOptions, MemoryLocalStore, a createOpfsLocalStore() that threw X_NOT_IMPLEMENTED) is deleted with LiveClient. What replaces it:

The boot runs as one deferred classic script per page (/_x/assets/page-boot/<hash>.js, from @ultimat3/realtime/boot). It is rendered only on a scope-tagged (private) document that emitted an island reaching @ultimat3/realtime. It wipes other scopes, restores this principal's rows, and opens the outbox, so a reload that opens no live hook and writes nothing still replays what the previous load queued.

Trade-off: two principals in two tabs of one browser. The newer page's boot wipes the other principal's rows and queued writes from disk. The older tab keeps its records and its queue in memory, and re-persists them on its next write; if it closes first, what it had queued offline is gone.

Written As of 2026-09-05 because four agents building one app each polled a query from an island instead of subscribing. Rewritten for 21.0.0, which deletes the app's socket adapter, its sync URL and its LiveClient: the page's one socket is the framework's.

// app/feed/feed.island.tsx — the only module of /feed a browser downloads
import { useConnection, useQuery } from '@ultimat3/realtime';
import { For, Match, Switch } from 'solid-js';
import { render } from 'solid-js/web';

type FeedRow = { id: string; title: string };
const LIVE_FEED = { name: 'liveFeed', live: true } as const;

export function mount(el: HTMLElement, props: { orgId: string }) {
  el.textContent = '';                       // Solid's render APPENDS; the server's loading shell goes first
  render(() => <Feed orgId={props.orgId} />, el);
}

function Feed(props: { orgId: string }) {
  const feed = useQuery<FeedRow>(LIVE_FEED, { orgId: props.orgId }); // subscribe, once
  const connection = useConnection();
  return (
    <Switch>
      <Match when={feed().status === 'pending'}><p>loading…</p></Match>
      <Match when={feed().status === 'failed'}><p>could not load the feed</p></Match>
      <Match when={'data' in feed()}>
        {connection.offline && <p>offline — showing the last rows</p>}
        <ul><For each={(feed() as { data: readonly FeedRow[] }).data}>{(row) => <li>{row.title}</li>}</For></ul>
      </Match>
    </Switch>
  );
}

feed() is an AsyncState: pending, then ready (or refreshing) carrying the rows, or failed. It re-renders on every patch the sync node sends, with no timer and no refetch. The list holds ids; the rows come from the page's one RecordStore, so a write to a record through any path re-renders every list showing it. A non-live query uses the same hook: drop live, and name its entity ({ name, entity: 'post' }) so its rows are store records too.

Two limits, As of 2026-09-22. x build sees realtime only in the island's own graph. An island that reaches it through a package, rather than importing it, calls installRealtime({ signal: createSignal }) in mount itself, or throws X_REALTIME_UNINSTALLED. And live and entity on the ref are stated by hand, not derived from the query declaration. Both are in the plan-101 DX ledger.

As of 2026-10. On a signed-in page with a sync node, the page boot restores persisted records and the queued writes from disk. Server markup rendered before a queued write would paint the old value first, so an island that can show such a value is held: hidden (visibility:hidden, its box kept) until it mounts over the restored store.

As of 2026-09-05. The sync node's changes come from Postgres logical replication — a real DATABASE_URL and the replicator role (x dev --role replicator). The embedded database has no walsender, so under a plain x dev a subscription took its snapshot and then heard nothing: every --live query in every scaffolded app was dead in development, which is where an author first tries one. x dev now installs the in-process bridge — startLiveReplicator, on @ultimat3/realtime/server since 2026-09-23 (@ultimat3/testing no longer exports it), the row observer the framework's own live tests run on — whenever the database is embedded, and the ready line says so:

Never both: with a real database the decoder already carries this process's own writes, and a bridge beside it would deliver each of them twice. --json carries the same fact as liveFeed.

An entity the app publishes itself (recordPublisher, for a deployment with no replicator) is the bridge's one exception: it stops carrying that entity's writes to channels, so each row reaches a channel member once — from the app's publish, as in production. Live queries still hear the bridge.

Postgres (logical replication slot)
    │  WAL decode, per-table
    ▼
replicator role  ──► change feed (ordered, per-table, with LSN)
    │
    ▼
incremental matcher  ──► for each registered live query: does this row enter/leave/update the result?
    │                      (predicate + order + limit evaluated against the changed row only)
    ▼
NATS subject per query-hash + tenant  ──► fanout
    │
    ▼
sync role (stateless)  ──► WS frame: {qid, op: insert|update|delete, row, lsn}
    │
    ▼
Solid signal patch — fine-grained, no re-render of the list

The matcher is why this is affordable: membership is decided from the changed row plus the query's predicate, order, and limit — never by re-running the query. Which is exactly why live: true requires a deterministic, bounded sql (Queries and live queries).

The page holds one RecordStore per tab, shared by every island through a globalThis handle. It is an identity-mapped store: a record is one value per record type (the entity name) and record key, however many lists, islands and paths hold it. It shipped in 21.0.0 (packages/realtime/src/record-store.ts).

The key comes from the server, on every path. A browser cannot compute a key without importing the app's entity() declarations, so it never does.

So an entity with a composite primary key can be live. A non-live useQuery lists its records in the envelope's records[type] key order, which rowsOf fills in data order. An answer with no envelope holds its rows itself, and they are not records. A live window whose node named no type keeps its rows under ?query:<name> (unnamedType), so two unnamed windows never merge two entities' rows.

As of 2026-09-22, 21.0.0. A channel is a declaration, never a string. The hub API that took raw topics (hub.guard, subscribe(socket, topic), publish(topic), publishFrame, channelFrame, TopicGuard*, the nodeId option, and a { kind: 'topic' } subscribe target) is deleted.

A channel is written once, in two halves on one ref, the same split as a query's QueryRef: an island holds the browser half and never imports an entity or a database package.

// app/feed/channel-ref.ts — the BROWSER half: name, params, catch-up read. No entity, no policy.
import { channelRef } from '@ultimat3/realtime';

export const ORG_FEED = channelRef('org-feed', {
  params: ['orgId'],
  catchUp: { name: 'orgFeed' }, // the query re-read on replay-gap or a new epoch
});

// app/feed/channels.ts — the SERVER half, on the same ref. Registers on import.
import { channel } from '@ultimat3/realtime';
import { Post } from '../posts/entity';
import { ORG_FEED } from './channel-ref';
import { feedRead } from './policies';

export const orgFeed = channel(ORG_FEED, {
  records: [Post],  // every param must be a column of each listed entity, or refused
  policy: feedRead, // decided at subscribe, with the params as input
  events: true,     // ephemeral payloads: typing, presence
});

// in an island: the ref, never the server half (which would bundle the entity and its driver)
import { useChannel } from '@ultimat3/realtime';
import { ORG_FEED } from './channel-ref';

declare const orgId: string;
const feed = useChannel(ORG_FEED, { orgId }, { onEvent: (event) => console.debug(event) });

channel(name, { params, catchUp, policy, … }) in one call still exists for a channel no browser holds. policy is required on either form: a declaration without one is refused (X_CHANNEL_DECLARATION_INVALID), and a public channel says so with policy: allow('public'). The reference app's idiom is examples/dummy/apps/web/app/posts/channel-ref.ts beside channels.ts. The catch-up query's name is written by hand on the ref, like a QueryRef's; x build emitting refs is plan-101 DX ledger #21.

Under realtime: { transport: 'nats' } every role holds a connection, and only two cannot work without one (As of 2026-10). The role decides, not the transport: a role that only publishes sends channel events and cache invalidations, and both mean "re-read".

x dev is one process running several roles, and it never goes down for the bus. With sync in the set (the default) it waits 3 s for NATS (DEV_BUS_WAIT_MS), then boots without it: the refusal a ROLE=sync container would have exited on is logged once at warn (ultimate bus is away: x dev serves without live until it answers, with its code, cause and fix), the dial goes on in the background, pages, jobs and tasks run, and the sync node stays mounted and not ready — an upgrade is shed with a 503 and a retry delay, so an open page keeps asking. When NATS answers the node starts (sync node started: the bus answered) and those pages go live with no restart. checks.transport is degraded there, never failing, so a bus lost under a running x dev leaves /readyz at 200. With --role web,worker nothing waits at all. A container is unchanged: ROLE=sync and the replicator keep the 15 s refusal, because they have nothing else to serve. realtime.enabled: false starts neither realtime role, so no process of such an app waits for the bus — the connection is still dialled in the background, because cache invalidation between replicas crosses it.

A page that is open when NATS goes away keeps its socket: the sync node does not close it, and nothing is reloaded. What the page reads, As of 2026-10:

Until 2026-10-10 the refusal marked the membership failed and nothing cleared it: events flowed again and the page read failed until a reload, while a new tab was live at once.

One outage is one fact. Each condition below is written on its 1st, 2nd, 4th, 8th, … failure and once more when it ends (outageLog, isOutageMilestone, on @ultimat3/core), at a level chosen by what the process can still do: warn for one that is degraded and serving, error for one that cannot do its job.

Until 2026-10-10 a three-minute outage wrote presence.sweep failed at error every 10 s on every sync node (19 lines on one), sent the monitor one event per refused beat per socket, and wrote cache.invalidate.partial once per bust on every publisher. A failure that is not the bus being away — a policy that threw inside a presence leave — is still written every time, as <operation> failed.

Every frame carries an LSN. The client's last-seen LSN is what makes reconnect a delta instead of a refetch.

A frame is routed as soon as it arrives, so two frames from one socket can be in flight at once. What is ordered is narrow and deliberate As of 2026-08 — a lane per socket would put every frame behind the slowest one, and the slowest one is a subscribe's snapshot read.

Ordering is not what bounds the caps. N sequential subscribes still pass a check-then-act limit N times, so every refusal a subscribe can answer with — the sid claim, maxPerSocket, maxPerTenant, maxTopicsPerSocket, maxTopicsPerNode — is decided synchronously, before the first await, against a count that already includes the subscribes still in flight. One WebSocket write carrying N subscribe frames used to pass each cap N times.

A dead TCP connection that was never closed fires no close event. Only the client can end one, so it beats.

A reconnect re-announces everything, one frame at a time. hello, then one subscribe per registration carrying that registration's cursor, then one per topic. hello itself carries neither cursors nor topic membership — resume is decided per subscription, by the frame that also names the query and its input, and topic membership is state on the node's socket. Without the topic half a channel stayed silent from the first reconnect onwards while its handler was still installed, and its presence membership was swept.

Writes never ride the socket, As of 21.0.0. The socket mutate path is deleted: useMutation posts the mutator's action over HTTP through clientTransport, with an idempotency key, so a resend after a lost answer is served from the action's idempotency store. Its answer's records are adopted before the overlay goes, so nothing flickers.

Reconnect is the expensive part of any sync engine, and it is where naive designs fall over. A deploy or a network blip drops N sockets at once; each client then asks "what changed since LSN X?" If the answer requires replaying arbitrary WAL history or re-running every query, a rolling restart becomes a self-inflicted thundering herd that outlasts the deploy.

One harness, two runs, two questions. Reachability: how long until a killed node's clients are back and receiving. Delivery: how many patches were lost getting there. A first-delivery timer answers the first and is blind to the second, which is why there are two.

scripts/bench/restart-bench.ts produced both, each with its own transcript beside it in scripts/bench/results/.

# reachability, 50,000 clients — 2026-08-11
bun run scripts/bench/restart-bench.ts --clients 50000 \
  --out scripts/bench/results/50k-restart.json

# delivery, 10,000 clients, a probe every 200ms — 2026-08-17
bun run scripts/bench/restart-bench.ts --clients 10000 --probe-interval-ms 200 \
  --out scripts/bench/results/10k-restart-seq.json

Per client, the first channel patch received on the reconnected socket: reconnect and resubscribe and one delivery. It is not a consistency metric, and cannot be one — see below.

The timings are unchanged and still stand. Only the name was wrong: this metric was published as "time-to-consistent" until 2026-08, and it never measured consistency. The harness recorded each client's last-seen sequence number and read it nowhere, so a patch the node dropped was invisible to it by construction. Nothing here is retracted — a number that timed reachability is now called reachability.

Every client counts observed sequence gaps in the probe stream it received, per connection. An observed gap is a break between two frames one connection actually received — the publisher numbers every probe, so a missing number between two arrivals is a frame that was published to a subscriber and never came.

Zero observed gaps is a lower bound on loss, not a proof of zero loss. The counter can only see a hole with a received frame on each side of it, so three losses are invisible to it by construction:

So the honest claim is "no client observed a lost channel frame", not "no channel frame was lost". Every other statement of this result on the wiki is shorthand for this paragraph.

Not evidence about 50,000. The 50,000-client run predates the counter and carries no delivery number; As of 2026-08 the 10,000-client run is the only one with delivery accounting, and it does not extrapolate.

A dropped records channel frame is now repaired, As of 2026-09-22 (21.0.0). SocketRegistry.deliver still counts every refusal (the series channel_frames_dropped_total, the warn line channel.frames_dropped carrying { topic, dropped, total }, and node.sockets.droppedChannelFrames for a test that cannot scrape). It also marks the (socket, topic) as gapped, and once the socket drains it sends one replay-gap frame, counted in channel_replay_gaps_total. That series counts gaps announced, by design: the repair happens in the browser. The client (client-channels.ts) keeps a cursor per channel. On replay-gap, or a new epoch, it re-runs the channel's catchUp query and applies the frames that arrived meanwhile after it. A dropped events frame is ephemeral by design and is only counted. The live-query path already repairs the same drop, as it has since 2026-08: the subscriber is marked desynced, and the next change re-snapshots it.

What it is not: a multi-node result — neither run crossed NATS, so this is per-node recovery, not fanout. Not a throughput figure either: no requests/sec, no message rate, no sustained-load number. Per-node socket capacity in the tables above this section is still a target derived from Bun's native WebSocket implementation, not a benchmark result. Long-running Bun processes are also less battle-proven than Node's; sustained-socket memory profiling is explicit roadmap work.

sync holds no durable state, so a restart is only dangerous in aggregate. Closing 50,000 sockets at once means 50,000 simultaneous reconnects, all resubscribing, all asking "what changed since my LSN?" — a self-inflicted DDoS landing during a deploy, when capacity is already reduced. Worse, it is fractal: surviving nodes overload, drop connections, and the herd re-forms.

So the drain is server-directed:

{ "type": "reconnect", "v": 3, "afterMs": 1830, "reason": "drain" }

ReconnectFrame (packages/realtime/src/sync-protocol.ts) has those four fields and no cursor: v is PROTOCOL_VERSION, reason one of drain, overload, rebalance.

Full drain sequence per role: Deployment.

PROTOCOL_VERSION is 3 As of 2026-09-22 (21.0.0; it was 2 from 2026-08-24) (packages/realtime/src/sync-protocol.ts). A mismatch is X_PROTOCOL_VERSION on the frame, with one instruction, rather than a per-field decode error nobody can act on.

Clients and sync nodes must be redeployed together across this bump. A cursor rides in BOTH directions — the client's subscribe and the node's snapshot — so a skew breaks resume from either side.

The version guards incompatibility, never novelty. Two earlier realtime changes deliberately did not move it: an additive optional field (snapshot.entity) and a removed field read through list() (hello.resume) are both readable in either direction, because decode builds a whitelist object and list() answers [] for an absent key. cursor() is the other kind of reader — str/num throw — so removing a field it reads is the one shape that needs the number.

A client on build A connecting to a sync node on build B is accepted, then sent an update-available frame carrying the node's buildId; the socket is not killed. Skew is the build the client claims against the node's own: ?build= on the dial, and the hello frame's buildId, which the node records on every hello and which is the later word — a dial without ?build= starts as "not skewed" and the first hello settles it. Neither side's build moves while a socket is open, so the answer is a property of that socket for its lifetime — a client learns about a deploy on the socket it opens against the new node, never on one it is already holding. Every hello on that socket re-reports the same answer, the heartbeat's included. The client's AppUpdateAvailable signal flips and the app renders its own update affordance. See PWA and offline.

Verbatim shapes: packages/realtime/src/errors.ts. Full index: Error codes.

• Truth is always the server. A client is never the merge authority.
• One authz system — the query's policy, re-evaluated per delivered row.
• sync is stateless. Any state that must survive a restart lives in Postgres or NATS.
• A live query must be deterministic and bounded (orderBy + limit), or x verify rejects it.
• Presence and typing indicators are tier 1 forever — never model ephemeral state as rows.
• local does no I/O, no Date.now(), no Math.random(). It must be replayable.
• Offline writes go through the tier-3 mutator queue, never Background Sync guesswork.
• Climb the ladder for a property you need, not for a tier you like. Tier 3 costs a durable client store forever.