Writing a Transport

A transport is an async generator that yields string chunks. That is the entire contract.

async function* myTransport(prompt, { config }) {
  yield 'Hello';
  yield ', world';
}

Three transports ship: echo (repeats what you said, for building UI before a model exists), local (a stub for a browser-resident model), and remote (a stub for a server). The stubs are stubs on purpose — see Why nothing here calls a model.

Because the interface is uniform, no component knows where its text came from. That is what makes offline-first a configuration rather than a rewrite.

A hosted model

async function* hosted(prompt, { config }) {
  const response = await fetch(config.endpoint, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ model: config.model, prompt, stream: true })
  });

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  for (;;) {
    const { done, value } = await reader.read();
    if (done) break;
    yield decoder.decode(value, { stream: true });
  }
}

Never put an API key in page JavaScript. config.endpoint should point at your own server, which holds the credential and talks to the provider. Anything in the page is readable by anyone who opens devtools.

Server-sent events

Most providers stream SSE rather than raw text, so unwrap it:

async function* sse(prompt, { config }) {
  const response = await fetch(config.endpoint, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ prompt, stream: true })
  });

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  for (;;) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });

    // Lines arrive split across chunks; keep the incomplete tail.
    const lines = buffer.split('\n');
    buffer = lines.pop();

    for (const line of lines) {
      if (!line.startsWith('data:')) continue;
      const data = line.slice(5).trim();
      if (data === '[DONE]') return;
      const text = JSON.parse(data).delta?.text;
      if (text) yield text;
    }
  }
}

A model in the user agent

This is the case the package exists to make possible. With WebLLM, the shape is the same:

import * as webllm from '@mlc-ai/web-llm';

let engine;

async function* local(prompt, { config }) {
  engine ??= await webllm.CreateMLCEngine(config.model || 'Llama-3.2-1B-Instruct-q4f32_1-MLC', {
    initProgressCallback: ({ text }) => {
      // The first load downloads a model measured in hundreds of megabytes.
      // Saying nothing for two minutes reads as a broken page.
      services.conversation.update(pendingId, { text });
    }
  });

  const stream = await engine.chat.completions.create({
    messages: [{ role: 'user', content: prompt }],
    stream: true
  });

  for await (const part of stream) {
    const text = part.choices[0]?.delta?.content;
    if (text) yield text;
  }
}

Two things to handle that a hosted model does not have:

First load is slow and large. Report progress into the pending record, or into a system turn. Cache it — WebLLM uses the Cache API, so a second visit is fast.

Not every device can run it. Check WebGPU and fall back rather than failing:

const canRunLocally = 'gpu' in navigator && await navigator.gpu?.requestAdapter();
services.config.transport = canRunLocally ? 'local' : 'remote';

Why this matters commercially

A model in the browser has no per-token cost. The conversation also never leaves the device, which is a privacy property you cannot buy from a hosted provider at any price.

Situation	With a hosted model	With a local one
Long-tail support	Cost per conversation can exceed the deflection's value	Zero marginal cost
Kiosk / in-store	Unmetered usage by passers-by	Unmetered costs nothing
Classroom	Per-seat spend nobody budgeted	One download, any number of sessions
Internal tooling	Hard to justify a line item	No line item
Field application	Needs connectivity	Works offline
Sensitive intake	Conversation leaves the device	It does not

The honest caveat: a 1–3B model in a browser is not a frontier model. It is good at classification, extraction, rephrasing, triage and scripted guidance — which is most of what a product chat surface actually does. Route the hard questions out; keep the volume in.

Registering your transport

const services = document.querySelector('machvive-chat-syncopation-services');

services.registerTransport('anthropic', async function* (prompt, { config }) {
  // …yield chunks…
});

registerTransport selects what it registers, so that one call is enough. Pass { select: false } to register without switching, and set services.config.transport later to choose between several:

services.registerTransport('local', localTransport, { select: false });
services.registerTransport('hosted', hostedTransport);

// Route by device capability rather than by guessing.
const canRunLocally = 'gpu' in navigator && await navigator.gpu?.requestAdapter();
services.config.transport = canRunLocally ? 'local' : 'hosted';

services.daemon.transports lists every selectable name, built-ins included. Registering a name that already exists replaces it, which is how you override remote rather than inventing a parallel one.

Prefer this over driving the conversation yourself

You can add records directly, and for a one-off fixture that is fine. But the daemon coordinates everything around a turn, and a page that bypasses it loses all of it:

	Registered transport	Hand-driven
Refuses a second concurrent send	yes	you implement it
stop() keeps the partial text	yes	you implement it
A throwing transport becomes a visible error record	yes	you implement it
daemon:idle fires	yes	no — the composer's Stop never turns back into Send

That last row is the one that bites. A surface wired by hand looks correct until someone sends a message and the button stays on Stop for the rest of the session.

If you still want to drive it directly:

const { conversation } = services;

async function ask(text) {
  conversation.add({ role: 'user', text });
  const reply = conversation.add({ role: 'assistant', text: '', status: 'pending' });

  try {
    for await (const chunk of myTransport(text, { config: services.config })) {
      conversation.append(reply.id, chunk);
    }
    conversation.update(reply.id, { status: 'complete' });
  } catch (error) {
    // Make the failure visible. A silent drop leaves the user staring at a
    // prompt that appears to have done nothing.
    conversation.update(reply.id, { status: 'error', meta: { 'mcs:error': error.message } });
  }
}

Recording what a turn cost

import { META } from '@machfivetechchicago/machvive-chat-syncopation-ai';

conversation.update(reply.id, {
  status: 'complete',
  meta: {
    [META.MODEL]: 'claude-opus-5',
    [META.TOKENS]: usage.output_tokens,
    [META.LATENCY]: Date.now() - startedAt,
    [META.SOURCE]: 'remote'
  }
});

META.SOURCE is worth setting honestly — 'local' means no token cost and no data egress, and that is a distinction a dashboard will want later.

Why nothing here calls a model

Three reasons, in order of how much they matter:

1. Stack agnosticism is the point. The moment this package has an opinion about a provider, it has an opinion about your architecture, your billing and your data residency. A chat bubble should not.
2. Credentials belong on a server. A transport bundled here would invite someone to pass an API key to a web component.
3. The interesting transport is the local one, and that is a large dependency with its own release cadence and device requirements. Making it a seam rather than a dependency keeps this package installable in seconds.

The suite enforces this: a test fails if any module gains a fetch, XMLHttpRequest, WebSocket or an external URL. A claim of absence needs a test asserting that absence, or it goes false one feature later with nothing failing.

Next: Theming.