Prototypes - MatthewFeroz/openeval GitHub Wiki

Prototypes

Current UI reference, captured October 21, 2026 against public source 6bd47be. The September 30 review and its original images remain archived. These current captures replace it as the reference for setup, reports and recovery.

What these images establish

The homepage, chart, Clerk form and unavailable-link screen were captured from production without signing in. Setup, evaluation and Studio captures use the real public components with simulated authentication and Convex responses. The report is synthetic. Those images establish layout and controls, not real account acceptance or a worker result.

The separately measured Oracle result came from Harbor/Docker on isolated Linux state. Fresh Clerk signup and real editor/viewer/removal/download acceptance remain open. No model call ran for this review.

Home and run database

Current homepage

Current cost versus agent-time chart

The database contains the approved sanitized 260-trial snapshot from two published studies. Select a study, filter outcomes and models, then select two runs for an A/B comparison. Arrow keys move between chart dots; Enter selects. Cost provenance and missing values remain visible. These small studies do not establish a general model ranking.

Sign-in and setup

Actual Clerk sign-in form

Current installation step, simulated account

Standalone worker check before model credentials, simulated worker

/login uses Clerk. Runner setup has Machine, Connection, Provider and First evaluation steps. The standalone Oracle worker check can run before connecting a model provider. The setup-enabled source CLI and the registry package can both report 0.2.0 while requiring different Harbor versions. Follow the first-evaluation guide and record the actual artifact, not just its version string.

Installation at 390 CSS pixels, simulated account

Saved report and publication

Synthetic saved report and owner-only sharing controls

The saved report opens from setup, Evaluations or Report library. Edit chart in Studio changes presentation, not outcomes. The owner creates a public link, explicitly publishes later saved changes and can revoke the link. Editor and viewer invitations do not grant worker controls. Membership removal and link revocation affect different access paths. Previously downloaded copies cannot be recalled.

Unavailable link, actual anonymous production request without a token

This image verifies incorrect-link guidance only. The dated hosted check records earlier publication and revocation observations. It does not complete real teammate-role acceptance.

Studio and exports

Neutral Studio, synthetic data and simulated export history

Studio offers Analysis and Presentation. Social images choose a question and White or Black style. Saving and publication are separate. Export history survives reload. The browser check exercises authenticated download transport with simulated completion; the local Node/Chromium renderer separately generated real PNG and ZIP output. This is not a claim that a real Clerk session downloaded an export.

Recovery

Offline guidance with a simulated worker

Incompatible-selection guidance with simulated profiles

An offline worker disables Start experiment and links Runner setup. An incompatible selection explains what to change. Missing-profile and unhealthy setup states offer worker-check/provider recovery; the browser scripts exercise those states. Closing the browser does not stop worker execution.

Navigation and capture method

Home opens /evaluations, /machines handles setup, /reports lists saved reports, and /studio?report=REPORT_ID edits a saved report. Public readers use /share#TOKEN; tokens are omitted from this documentation. See How an evaluation works for execution order.

Capture manifest records routes, state, source revision, dimensions and hashes. Playwright Chromium automated the captures after the collaborative preview twice returned Preview automation snapshot timed out after 15000ms. Images were visually inspected. Static evidence uses reduced motion; capture-only hooks wait for chart SVG and prevent a fixed header from covering long forms. Source UI and styles were unchanged.

Scripts: setup, evaluation, exports. The export script needed a functional assertion fix because neutral themes omit model-logo markup. Acceptance review records the check results and remaining limits.