en Getting Started - chiba233/yumeDSL GitHub Wiki

Getting Started

Install

npm install yume-dsl-rich-text   # or pnpm add / yarn add

Three steps

1. Create a parser

import {
    createParser,
    createSimpleInlineHandlers,
    createSimpleBlockHandlers,
    createSimpleRawHandlers,
    declareMultilineTags,
} from "yume-dsl-rich-text";

const dsl = createParser({
    handlers: {
        ...createSimpleInlineHandlers(["bold", "italic", "underline", "strike"]),
        ...createSimpleBlockHandlers(["info", "warning"]),
        ...createSimpleRawHandlers(["code"]),
    },
    blockTags: declareMultilineTags(["info", "warning", "code"]),
});

If you only want to declare a few tag names at near-zero cost, empty handler objects are also a standard pattern:

const dsl = createParser({
    handlers: {
        bold: {},
        italic: {},
        ...createSimpleBlockHandlers(["info", "warning"]),
        ...createSimpleRawHandlers(["code"]),
    },
    blockTags: declareMultilineTags(["info", "warning", "code"]),
});

bold: {} / italic: {} mean "register this tag name and leave the final output to the parser's default materialization / fallback"
this empty-object form is a stable zero-cost declaration syntax
if you want a fixed explicit output shape such as { type, value }, use createSimpleInlineHandlers(...)

Actual behaviour:

parseRichText("$$bold(world)$$", {
    handlers: {bold: {}},
});
// => [{ type: "bold", value: [{ type: "text", value: "world" }] }]

parseRichText("$$code(js)%\nconst x = 1;\n%end$$", {
    handlers: {code: {}},
});
// => [{ type: "text", value: "$$code(js)%\nconst x = 1;\n%end$$" }]

parseRichText("$$info(note)*\nhello\n*end$$", {
    handlers: {info: {}},
});
// => [{ type: "text", value: "$$info(note)*\nhello\n*end$$" }]

empty-object fallback only gives you the default inline output path
raw / block are not auto-enabled by {}; without raw / block methods they still degrade back to source text
if the same tag should support inline plus raw/block, write inline + raw / block explicitly

What each helper does:

Helper	Creates	Tag form
`createSimpleInlineHandlers`	Inline handlers (pass through children)	`$$tag(content)$$`
`createSimpleBlockHandlers`	Block handlers (recursively parse content)	`$$tag(arg)* ... *end$$`
`createSimpleRawHandlers`	Raw handlers (capture content verbatim)	`$$tag(arg)% ... %end$$`
`declareMultilineTags`	Block-level newline normalization	raw / block / inline

2. Parse

const tokens = dsl.parse("Hello $$bold(world)$$!");

Result:

[
    { type: "text", value: "Hello ", id: "rt-0" },
    {
        type: "bold",
        value: [{ type: "text", value: "world", id: "rt-1" }],
        id: "rt-2",
    },
    { type: "text", value: "!", id: "rt-3" },
]

Each token has:

type — "text" for plain text, or the type returned by your handler (usually the tag name)
value — string for text tokens, child token array for inline/block tags
id — unique within a parse

About ctx: You'll see handler callbacks take a ctx parameter in the examples. You don't need to know what it is — the parser passes it to you, just include it. See DslContext if you're curious.

3. Strip to plain text

const plain = dsl.strip("Hello $$bold(world)$$!");
// "Hello world!"

Useful for search indexing, plain-text previews, or accessibility fallbacks.

4. Use shorthand syntax (since 1.3)

Inside inline tag arguments, you can omit the $$...$$ wrapper and use tag(...) shorthand:

const dsl = createParser({
    handlers: {
        ...createSimpleInlineHandlers(["bold", "italic"]),
    },
    implicitInlineShorthand: true,   // or ["bold", "italic"] for allowlist
});

const tokens = dsl.parse("$$bold(Hello italic(world))$$");
// equivalent to "$$bold(Hello $$italic(world)$$)$$"
// shorthand only works inside inline args, root level still needs full syntax

Shorthand only activates inside inline argument context, not at root level
Full DSL syntax ($$tag(...)$$) always takes priority over shorthand
Literal parentheses use $ / $ escape
Only inline tags registered in handlers are recognized

See DSL Syntax — Implicit inline shorthand and ParseOptions — implicitInlineShorthand.

5. Incremental parsing for large documents

When your document is large (tens of KB+) and you need per-keystroke structural parsing, use an incremental session to avoid re-scanning from scratch every time.

The incremental session maintains the structural layer (StructuralNode[] + Zone[]), not TextToken[]. Its value is that after each edit you immediately get an updated structural tree for highlighting, outline, lint, and other structure-level features — without rebuilding from zero. If you also need TextToken[] for final rendering, call dsl.parse() on the updated text.

import {
    createParser,
    createIncrementalSession,
    createSimpleInlineHandlers,
    createSimpleBlockHandlers,
    createSimpleRawHandlers,
} from "yume-dsl-rich-text";

// ── 1. Prepare handlers and parser ──

const handlers = {
    ...createSimpleInlineHandlers(["bold", "italic", "underline"]),
    ...createSimpleBlockHandlers(["info", "warning"]),
    ...createSimpleRawHandlers(["code"]),
};

const dsl = createParser({handlers});

// ── 2. Initial document text (e.g. when the editor loads) ──

let currentSource = "Hello $$bold(world)$$!\n$$code()%console.log('hi')%end$$";

// ── 3. Create incremental session + first render ──

const session = createIncrementalSession(currentSource, {handlers});

// Structural tree: for highlighting / outline / lint
let tree = session.getDocument().tree;
updateHighlighting(tree);
updateOutline(tree);

// TextToken[]: for final rendering
let tokens = dsl.parse(currentSource);
render(tokens);

// ── 4. User edits: change "world" to "everyone" ──
//    Old text: "Hello $$bold(world)$$!..."
//                          ^^^^^
//    startOffset = 14, oldEndOffset = 19, newText = "everyone"

const startOffset = 14;
const oldEndOffset = 19;
const newText = "everyone";

// Build the full updated text
currentSource =
    currentSource.slice(0, startOffset) +
    newText +
    currentSource.slice(oldEndOffset);

// ── 5. Advance incremental session + re-render ──

const result = session.applyEdit(
    {startOffset, oldEndOffset, newText},
    currentSource,
);

// result.mode: "incremental" | "full-fallback"

// Structural tree is incrementally updated (several to 10× faster on large docs)
tree = result.doc.tree;
updateHighlighting(tree);
updateOutline(tree);

// TextToken[] still uses dsl.parse() (it's fast on its own)
tokens = dsl.parse(currentSource);
render(tokens);
// [
//   { type: "text", value: "Hello " },
//   { type: "bold", value: [{ type: "text", value: "everyone" }] },
//   { type: "text", value: "!\n" },
//   { type: "code", value: "console.log('hi')" },
// ]

The session automatically chooses incremental or full rebuild — incremental when safe, silent fallback to full otherwise.
If you also need to know exactly which structures changed, use session.applyEditWithDiff(...) instead — its return value includes a diff field.
Call session.rebuild(newSource) any time you want to force a full rebuild.

See Incremental Parsing for details.