Prototypes - MatthewFeroz/openeval GitHub Wiki
Prototypes
Current UI reference, captured October 21, 2026 against public source 6bd47be. The September 30 review and its original images remain archived. These current captures replace it as the reference for setup, reports and recovery.
What these images establish
The homepage, chart, Clerk form and unavailable-link screen were captured from production without signing in. Setup, evaluation and Studio captures use the real public components with simulated authentication and Convex responses. The report is synthetic. Those images establish layout and controls, not real account acceptance or a worker result.
The separately measured Oracle result came from Harbor/Docker on isolated Linux state. Fresh Clerk signup and real editor/viewer/removal/download acceptance remain open. No model call ran for this review.
Home and run database


The database contains the approved sanitized 260-trial snapshot from two published studies. Select a study, filter outcomes and models, then select two runs for an A/B comparison. Arrow keys move between chart dots; Enter selects. Cost provenance and missing values remain visible. These small studies do not establish a general model ranking.
Sign-in and setup



/login uses Clerk. Runner setup has Machine, Connection, Provider and First evaluation steps. The standalone Oracle worker check can run before connecting a model provider. The setup-enabled source CLI and the registry package can both report 0.2.0 while requiring different Harbor versions. Follow the first-evaluation guide and record the actual artifact, not just its version string.

Saved report and publication

The saved report opens from setup, Evaluations or Report library. Edit chart in Studio changes presentation, not outcomes. The owner creates a public link, explicitly publishes later saved changes and can revoke the link. Editor and viewer invitations do not grant worker controls. Membership removal and link revocation affect different access paths. Previously downloaded copies cannot be recalled.

This image verifies incorrect-link guidance only. The dated hosted check records earlier publication and revocation observations. It does not complete real teammate-role acceptance.
Studio and exports

Studio offers Analysis and Presentation. Social images choose a question and White or Black style. Saving and publication are separate. Export history survives reload. The browser check exercises authenticated download transport with simulated completion; the local Node/Chromium renderer separately generated real PNG and ZIP output. This is not a claim that a real Clerk session downloaded an export.
Recovery


An offline worker disables Start experiment and links Runner setup. An incompatible selection explains what to change. Missing-profile and unhealthy setup states offer worker-check/provider recovery; the browser scripts exercise those states. Closing the browser does not stop worker execution.
Navigation and capture method
Home opens /evaluations, /machines handles setup, /reports lists saved reports, and /studio?report=REPORT_ID edits a saved report. Public readers use /share#TOKEN; tokens are omitted from this documentation. See How an evaluation works for execution order.
Capture manifest records routes, state, source revision, dimensions and hashes. Playwright Chromium automated the captures after the collaborative preview twice returned Preview automation snapshot timed out after 15000ms. Images were visually inspected. Static evidence uses reduced motion; capture-only hooks wait for chart SVG and prevent a fixed header from covering long forms. Source UI and styles were unchanged.
Scripts: setup, evaluation, exports. The export script needed a functional assertion fix because neutral themes omit model-logo markup. Acceptance review records the check results and remaining limits.