CREATIVE CONTROL EVALS - SyberLabs/RISE GitHub Wiki
What this is for: measuring whether RISE's guidance and library let a model write readings that work, by something that runs rather than by taste alone (Creative Control design §11). Each case is a prompt a reader might give and a reference Current a model following the guide should be able to write for it, in one style.
docs/evals/creative-control/
premium-educational/<id>.json { "prompt": "...", "current": { ...rise.current.v2... } }
open-field/<id>.json
streamed/<id>.json { "prompt": "...", "lines": [ "@say ...", "@hold 1500", ... ] }
The worked Currents each style's guidance teaches
(src/live/guide/styles/*.js, served as ui://rise/guide/<style>) are in
the corpus unchanged; src/test/eval-creative.test.js fails if they drift
apart. Every other case is the corpus's own.
A streamed case (added 2026-10-10, RISE Live stage 3) is the answer a model writes as it thinks, in beats, one per line (the line format, "Beats streamed"). It is read as the Live venue reads one: through the text-stream parser into the reducer, whole and again cut a character at a time, in a provider's uneven deltas, and in pairs. It passes only if nothing is dropped or refused (a scene the admission refuses included), the same Current comes out at every cut, and that Current then passes everything below.
npm run eval:creativeIt runs on demand and is not part of CI. src/test/eval-creative.test.js
runs the same evaluation over the corpus as a unit test, so a broken case
fails the unit suite on main.
For each case, in style and file order:
-
Acceptance: the Current goes through the Worker's own
rise_presentdoor (dispatchinworker/mcp-server.mjs): the MCP size limit, the strict validator, and the static admission of every code scene and every figure (an SVG scene,src/core/svg-admission.js). A refusal is printed in the Worker's words, the text a model would read. -
Code scenes, headless: every admitted code scene is run through the
scene worker's own protocol (
attachSceneWorker) over a recording 2D context that draws no pixels (src/test/fake-canvas-context.js), with the reading's theme and its style's library defaults, three times:- played: the dry frame, 60 frames at 16.7 ms, then each cue its beats name, animated, with frames until its tweens land;
- seeked: every cue with
instant: true, as a replay delivers them; - reduced motion: every cue animated, and no tween may still be moving before the next frame.
- Figures: every figure is admitted again by the same function the card runs before it draws one, and reported with its size. It is not drawn: the browser draws it as an image, in the page only. Every style has at least one case with a figure.
- Report: cues answered out of cues sent, frames drawn, the slowest frame, frames over the soft budget (24 ms), whether the card's kill switch would have ended the scene, and every error with its phase and line.
It exits non-zero on any refusal, any exception a scene throws (on load, init, a frame or a cue, a rejected tween included), or a tween reduced motion left moving.
- Pixels. Nothing is drawn; a scene that draws the wrong picture, or nothing, passes. Looking at a reading is still a person's job.
- The card's frame times. Times are measured in Node against a context that does no work, so they are a proxy: a scene slow here is slow in the card, but one fast here may not be fast there.
- Native engines. A native scene is checked against its engine's manifest by the validator; the engines draw in the page and are not run.
-
WebGL. A scene that calls
rise.use("webgl2")fails here (Node has no WebGL) and must be judged in a browser. - Flashing. The flash gate samples real pixels; there are none here.
- Isolation. The scene code runs in this Node process, not a sandbox. Only add cases you have read.
- Write the prompt and the Current a model should be able to write for it,
in the style's directory, as
<current id>.json. - Run
npm run eval:creativeand read the whole report, not only the last line. - Prefer a case that exercises something the corpus does not yet: a new
rise.libcall, a cue on a hold withmaxMs, a scene that mixesrise.ctxwith the library, a style's typography under alook.
Published from docs/evals/creative-control/CREATIVE-CONTROL-EVALS.md. Edits made here are overwritten on the next push to main.