en Incremental Sugar - chiba233/yume-dsl-token-walker GitHub Wiki

Incremental Sugar

Structural Slice | Exports | Home

Span-based helpers that bridge SourceSpan positions with yume-dsl-rich-text's incremental session API. While parseSlice reparses a region from scratch, an incremental session reuses previous parse state and only updates the changed portion.

When to use:

  • An editor user types keystroke-by-keystroke — the session carries forward previous parse state
  • You need to track whether each edit was truly incremental or fell back to a full reparse
  • You want a stateful session that you applyEdit to repeatedly, not a one-shot slice

When to use parseSlice instead:

  • One-shot region extraction (e.g., render a single node)
  • You don't need to track incremental vs. full-fallback mode
  • Stateless — no session to manage

Demo: keystroke-level editing

import { createSimpleInlineHandlers } from "yume-dsl-rich-text";
import { createSliceSession, applyIncrementalEditBySpan, toSliceEdit, replaceSliceText } from "yume-dsl-token-walker";
import type { SourceSpan } from "yume-dsl-rich-text";

const handlers = createSimpleInlineHandlers(["bold", "italic"]);
const source = "head $$bold(world)$$ tail";

// 1. Create a session — parses the initial source
const session = createSliceSession(source, { handlers });

// 2. Locate the region to edit (e.g., from nodeAtOffset or editor selection)
const start = source.indexOf("world");
const span: SourceSpan = {
    start: { offset: start, line: 1, column: start + 1 },
    end: { offset: start + 5, line: 1, column: start + 6 },
};

// 3. Apply the edit — session updates its internal document
const result = applyIncrementalEditBySpan(session, span, "DSL", { handlers });

console.log(result.doc.source); // "head $$bold(DSL)$$ tail"
console.log(result.mode);       // "incremental" or "full-fallback"
console.log(session.getDocument().source); // same — session is stateful

Standalone helpers

You can use the low-level helpers independently, without a session:

import { toSliceEdit, replaceSliceText } from "yume-dsl-token-walker";

const span: SourceSpan = {
    start: { offset: 3, line: 1, column: 4 },
    end: { offset: 7, line: 1, column: 8 },
};

// Convert span to IncrementalEdit payload
const edit = toSliceEdit(span, "XYZ");
// { startOffset: 3, oldEndOffset: 7, newText: "XYZ" }

// Pure string replacement by span offsets
const next = replaceSliceText("abcdefgh", span, "XYZ");
// "abcXYZgh"

API reference

toSliceEdit(span, newText)

const toSliceEdit: (span: SourceSpan, newText: string) => IncrementalEdit;

Convert a SourceSpan + replacement text to an IncrementalEdit payload. Maps span.start.offsetstartOffset and span.end.offsetoldEndOffset.

replaceSliceText(source, span, newText)

const replaceSliceText: (source: string, span: SourceSpan, newText: string) => string;

Pure string helper. Slices source at span offsets and inserts newText. No parsing involved.

createSliceSession(source, options?, sessionOptions?)

const createSliceSession: (
    source: string,
    options?: IncrementalParseOptions,
    sessionOptions?: IncrementalSessionOptions,
) => SliceSession;

Create an incremental session. Thin alias for createIncrementalSession from yume-dsl-rich-text, named to sit naturally near parseSlice workflows.

Param Description
source Initial source text
options Parse options — handlers, allowForms, etc.
sessionOptions Session options forwarded to createIncrementalSession

applyIncrementalEditBySpan(session, span, newText, options?)

const applyIncrementalEditBySpan: (
    session: SliceSession,
    span: SourceSpan,
    newText: string,
    options?: IncrementalParseOptions,
) => SliceSessionApplyResult;

High-level convenience: reads current source from session.getDocument(), builds next source via replaceSliceText, converts span to edit via toSliceEdit, then delegates to session.applyEdit(...).

Param Description
session A session from createSliceSession
span The region to replace
newText Replacement text
options Parse options for re-parsing

Types

SliceSession

interface SliceSession {
    getDocument: () => IncrementalDocument;
    applyEdit: (
        edit: IncrementalEdit,
        newSource: string,
        options?: IncrementalParseOptions,
    ) => SliceSessionApplyResult;
    rebuild: (newSource: string, options?: IncrementalParseOptions) => IncrementalDocument;
}

SliceSessionApplyResult

interface SliceSessionApplyResult {
    doc: IncrementalDocument;
    mode: "incremental" | "full-fallback";
    fallbackReason?: string;
}
Field Description
doc The updated document after the edit
mode "incremental" if the edit reused previous state; "full-fallback" if a full reparse was needed
fallbackReason When mode is "full-fallback", explains why incremental failed