yume-dsl-token-walker Wiki

Zero-dependency operation layer for yume-dsl-rich-text. Parser gives you trees — this package interprets, queries, lints, and slices them.

yume-dsl-rich-text 的零依赖操作层。 Parser 给你树——这个包负责解释、查询、lint、切片。

200 KB benchmark (Kunpeng 920 / Node v24.14.0): full parseRichText ~24 ms, parseStructural ~21 ms — already fast. nodeAtOffset + parseSlice ~0.17 ms — only reparses the touched region. Interpret 10,000 tokens → HTML string in ~2 ms. Lint 50 rules against a 200 KB document in ~45 ms.

Thanks to Claude and Codex for their tireless patience in iterating on the design and implementation with me.

感谢 Claude 和 Codex 在设计与实现过程中不厌其烦地陪我反复打磨方案。

---

English

API

Page	Description
Interpret	Turn TextToken[] into HTML, VNodes, or any output you define. Sync, lazy generators, recursion-safe
Async Interpret	Same as Interpret but handlers can await — for fetch, database, async renderers
Structural Query	Search and locate nodes in a StructuralNode[] tree — findFirst, findAll, nodeAtOffset, nodePathAtOffset, enclosingNode
Lint	Validate DSL source with custom rules, report diagnostics, auto-fix with atomic edit application
Structural Slice	Re-parse only a region of a large document — incremental editing pipeline with full position mapping

Tutorials

Page	Description
Tutorial: Blog Renderer	From zero to a working DSL → HTML blog renderer — step by step
Tutorial: Game Dialogue	Build shake / color / wait commands for a visual novel typewriter
Tutorial: Editor Lint + Auto-fix	Custom lint rules, diagnostics, atomic auto-fix — CI and editor ready

Reference

Page	Description
Error Handling & Safety	onError observer, error phases, recursion detection, circular reference safety
Exports	Complete export reference — all functions, helpers, and types

---

中文

API

页面	说明
解释 API	把 TextToken[] 变成 HTML、VNode 或任何你定义的输出。同步，惰性 generator，递归安全
异步解释 API	和解释 API 相同，但 handler 可以 await——适用于 fetch、数据库、异步渲染器
结构查询	在 StructuralNode[] 树里搜索和定位节点——findFirst、findAll、nodeAtOffset、nodePathAtOffset、enclosingNode
Lint	用自定义规则校验 DSL 源码，上报诊断结果，原子化自动修复
结构切片	只重新解析大文档中的一个区域——增量编辑管线，完整位置映射

实战教程

页面	说明
教程：博客渲染器	从零到一个可用的 DSL → HTML 博客渲染器，逐步讲解
教程：游戏对话引擎	为视觉小说打字机构建 shake / color / wait 指令
教程：编辑器 Lint + 自动修复	自定义 lint 规则、诊断、原子化自动修复——CI 和编辑器就绪

参考

页面	说明
错误处理与安全性	onError 观察者、错误阶段、递归检测、循环引用安全
导出一览	完整导出参考——所有函数、辅助工具、类型

---

Ecosystem / 生态

text ──▶ yume-dsl-rich-text (parse) ──▶ TextToken[] / StructuralNode[]
                                              │
                                  yume-dsl-token-walker
                                   ├─ interpret  (TextToken[] → TNode[])
                                   ├─ query      (StructuralNode[] search)
                                   ├─ lint       (StructuralNode[] validation)
                                   └─ slice      (region re-parse)

Package	Role
yume-dsl-rich-text	Parser — text to token tree
yume-dsl-token-walker	Operations — interpret, query, lint, slice (this package)
yume-dsl-shiki-highlight	Syntax highlighting — tokens or TextMate grammar
yume-dsl-markdown-it	markdown-it plugin — DSL tags inside Markdown