en Structural Query - chiba233/yume-dsl-token-walker GitHub Wiki

Structural Query

Interpret | Lint | Home

Search and locate nodes in a StructuralNode[] tree. These operate on the structural parse tree from parseStructural, not on TextToken[].

When to use:

  • Find a specific tag by name or position
  • Collect all tags of a certain type
  • Walk the entire tree with parent/depth context
  • Figure out "which tag is the cursor inside" in an editor

Demo: editor "cursor in which tag"

The most common real-world use case — given a cursor offset in source text, find which tag encloses it:

import {parseStructural} from "yume-dsl-rich-text";
import {enclosingNode} from "yume-dsl-token-walker";

const source = "Hello $$bold($$italic(world)$$)$$!";
const tree = parseStructural(source, {trackPositions: true});

// User's cursor is at offset 22 — inside "world"
const tag = enclosingNode(tree, 22);

if (tag) {
    console.log(tag.type); // "inline"
    console.log(tag.tag);  // "italic"  ← the deepest enclosing tag
}

enclosingNode skips text/escape/separator nodes and returns only tag nodes (inline / raw / block) — you can access .tag, .children, .args directly without extra type guards. When implicit inline shorthand is enabled upstream (implicitInlineShorthand in yume-dsl-rich-text), inline nodes marked with implicitInlineShorthand: true are skipped as enclosing targets, so cursor targeting prefers independently sliceable outer tags.


findFirst — first match

Depth-first pre-order search. Returns the first node matching the predicate, or undefined. Stops immediately when a match is found (early-exit).

import {parseStructural} from "yume-dsl-rich-text";
import {findFirst} from "yume-dsl-token-walker";

const tree = parseStructural("Hello $$bold($$italic(world)$$)$$");

const italic = findFirst(tree, (node) =>
    node.type === "inline" && node.tag === "italic",
);
// italic.type === "inline", italic.tag === "italic"

The predicate receives (node, ctx) where ctx has parent, depth, and index:

// Find the first text node that lives inside a bold tag
const textInBold = findFirst(tree, (node, ctx) =>
    node.type === "text" &&
    ctx.parent?.type === "inline" &&
    (ctx.parent as { tag: string }).tag === "bold",
);

findAll — all matches

Same traversal order as findFirst, but collects all matches:

import {parseStructural} from "yume-dsl-rich-text";
import {findAll} from "yume-dsl-token-walker";

const tree = parseStructural("$$bold(a)$$ then $$bold(b)$$");
const bolds = findAll(tree, (node) =>
    node.type === "inline" && node.tag === "bold",
);
// bolds.length === 2

nodeAtOffset — deepest node at position

Find the deepest (most specific) node whose source span contains a byte offset. Requires trackPositions: true.

import {parseStructural} from "yume-dsl-rich-text";
import {nodeAtOffset} from "yume-dsl-token-walker";

const input = "Hello $$bold(world)$$";
const tree = parseStructural(input, {trackPositions: true});

const node = nodeAtOffset(tree, 14); // offset 14 is inside "world"
// node.type === "text", node.value === "world"

Returns undefined if no node contains the offset or if positions were not tracked.

Difference from enclosingNode

nodeAtOffset enclosingNode
Returns Any node (text, escape, separator, inline, raw, block) Only tag nodes (inline, raw, block)
Use case "What exact thing is at this offset?" "Which tag wraps this offset?"
Return type StructuralNode | undefined StructuralTagNode | undefined

nodePathAtOffset — full path to deepest node

Returns the full path from the outermost node to the deepest node containing the given offset, as StructuralNode[]. The last element is the same node nodeAtOffset would return.

Useful for editor breadcrumbs, context-aware completion, and nesting level display.

import {parseStructural} from "yume-dsl-rich-text";
import {nodePathAtOffset} from "yume-dsl-token-walker";

const source = "Hello $$bold($$italic(world)$$)$$!";
const tree = parseStructural(source, {trackPositions: true});

const path = nodePathAtOffset(tree, 22);
// path[0].type === "inline", path[0].tag === "bold"   — outermost
// path[1].type === "inline", path[1].tag === "italic"  — middle
// path[2].type === "text",   path[2].value === "world" — deepest

Returns an empty array if no node contains the offset or positions are absent.


enclosingNode — deepest tag at position

Find the deepest tag node (inline / raw / block) enclosing a source offset. Skips text, escape, and separator nodes. Also skips inline nodes flagged as implicit shorthand (implicitInlineShorthand === true).

Return type is StructuralTagNode | undefined — already narrowed, so .tag, .children, .args are directly accessible without type guards.

import {parseStructural} from "yume-dsl-rich-text";
import {enclosingNode} from "yume-dsl-token-walker";

const input = "Hello $$bold(world)$$";
const tree = parseStructural(input, {trackPositions: true});

const tag = enclosingNode(tree, 14);
if (tag) {
    console.log(tag.tag); // "bold"
    // tag.children is accessible — no type guard needed
}

Returns undefined if the offset is not inside any tag, or if positions were not tracked.

Offset semantics: the offset must be a string index into the original source text passed to parseStructural — not an index into rendered, printed, or display text. This also applies to nodeAtOffset.


walkStructural — visit every node

Pure side-effect traversal. Unlike findFirst/findAll, it doesn't collect or return anything — use it when you need to inspect every node with full context.

import {parseStructural} from "yume-dsl-rich-text";
import {walkStructural} from "yume-dsl-token-walker";

const tree = parseStructural("$$bold(hello $$italic(world)$$)$$");

walkStructural(tree, (node, ctx) => {
    console.log(`${"  ".repeat(ctx.depth)}${node.type}`);
});
// inline
//   text
//   inline
//     text

StructuralVisitContext

Every predicate and visitor callback receives this context:

interface StructuralVisitContext {
    parent: StructuralNode | null;  // null for top-level nodes
    depth: number;                   // 0 for top-level
    index: number;                   // position within parent's child array
}

How children are traversed

Internally, all query functions share a single DFS engine. Which children get traversed depends on the node type:

Node type Children traversed
text none
escape none
separator none
inline children
raw args (content is a raw string, not traversed)
block args, then children

This means findFirst/findAll/walkStructural will find nodes inside args of raw/block tags, but will not descend into raw content strings.


Type reference

Type Description
StructuralPredicate (node: StructuralNode, ctx: StructuralVisitContext) => boolean
StructuralVisitor (node: StructuralNode, ctx: StructuralVisitContext) => void
StructuralTagNode Extract<StructuralNode, { type: "inline" | "raw" | "block" }>
StructuralVisitContext { parent, depth, index }