Development Pilot EN - teamswyg/laya-tools GitHub Wiki
Progress, completion-report and question displays: first shadow-pilot draft
Status: design draft; zero live Riido pilot executions. Go warm-start training/publication and a separate four-case synthetic CLI smoke are complete. Live-reader/UI integration and100/400/2400-case review below remain unexecuted. The next checks use public synthetic cases and explicit test doubles.
The user's selected first scope is progress-report, completion-report and question displays on development-work messages. completion_report means a sentence claiming that work was completed. It does not prove execution success, successful CI, verified completion of the entire assignment or approval. The display should say “Completion report,” rather than “Completed.”
Implemented and missing
| Category | Current verified scope |
|---|---|
| Implemented: classification/training | An eight-intent Go classifier and rule controls. V2 actually fitted and evaluated320 separately authored synthetic training cases and40validation/40calibration/40test cases. |
| Implemented: selected model | v1_warm_start,1,024bins,32,960bytes,1,520→1,920steps,temperature0.9. The two2,048-bin controls were compared but not selected. No new three-class model was trained for this pilot. |
| Implemented: public bridge | SnapshotReader.Read and Adapter.Propose, opaque references, explicit current active-label intent bindings, rejection/filtering of duplicate/missing/inactive/group entries, canonical-version checks and shadow output. |
| Implemented: boundary tests | Explicit test doubles cover read scope, ambiguous status candidates, missing versions, duplicate displays and absence of source text in responses. These are not actual native-reader/catalog calls. |
| Missing | Live Riido reader, current native catalog discovery, the exact message/work placement target, content-revision binding, revision-event validation, real UI integration, a new pilot harness and complete cost measurements. |
| Outside this first scope | Work-state writes, real reactions, notifications, cancellation execution, model/reasoning-level adjustment and use of actual source messages for external training/publication. |
V2's new test result was34/40, with17/20in each language. Completion reports scored5/5and questions4/5, but only five cases represented each intent. The17accepted non-unclear predictions across all eight intents do not establish these three displays' operational accuracy or100%precision. Current native labels and team emoji configuration are unknown. This draft does not claim completed live integration or verification of2,400actual work cases.
Actual CLI check with the redownloaded model
The model from published revision 81335eadd9f2753d3b86c932e9b00c636714e3b4 processed four original public synthetic messages. Intents were4/4correct, but only the question exceeded the fixed0.9confidence gate and proposed a label/emoji.
| Message intent | Confidence(rounded) | Actual shadow output |
|---|---|---|
| Progress report | 0.858 | Withheld below the gate |
| Completion report | 0.821 | Withheld below the gate |
| Question | 0.971 | Label/emoji proposal |
| Planned | 0.815 | Withheld below the gate; outside the three-display scope |
All mutation/state-change flags were false and no actual reader was connected. Peak OS RSS for this macOS process handling these four short inputs was5,488,640bytes(about5.23MiB). This observation includes model loading, classification, JSONL and proposal handling in that process; it does not measure real-catalog reads, UI placement cost or throughput. Keep “intent4/4,” “display proposal1/4” and “actual adapter pilot0runs” separate. Do not lower the threshold to admit the withheld messages.
Sources: V2 results/limits, training rubric, catalog bridge. The separate smoke receipt is publication/DEVELOPMENT-SMOKE.json in this private cache.
Display candidates and the current-label allowlist
| Classifier intent | Pilot display | Draft emoji | Canonical code | Work-state change |
|---|---|---|---|---|
progress |
Progress report | 🛠️ | 1f6e0-fe0f |
None |
completion_report |
Completion report | ✅ | 2705 |
None |
question |
Question | ❓ | 2753 |
None |
Names and emoji are configurable drafts, not claims that native entries exist. Do not force blocker, reference, cancel_request, planned or unclear into these displays. Retain the full eight-intent probabilities: do not renormalize over three intents or substitute a lower-ranked allowed intent.
Build the label allowlist on each read as the intersection of entries that:
- Came from the application's trusted reader for the current scope.
- Are currently active labels, not group containers.
- Are explicitly bound by user configuration to one of the three intents through opaque references.
- Have no duplicate or conflicting reference/intent binding.
Missing, removed or inactive bindings produce no label proposal. Do not infer meaning from names, colors or order, or create new labels. Native IDs, names, credentials and the opaque-reference lookup remain within the private reader. An existing matching label/emoji is an annotation no-op. Supply only a confirmed emoji allowlist; do not automatically promote the draft defaults to native configuration.
Keep the catalog bridge's confidence0.9and margin0.05. The preceding CLI smoke used margin0, distinct from the next catalog pilot. Guarded input, out-of-scope intents, ambiguity and scores below the gate abstain. A missing model artifact must not silently switch the pilot to unlearned rules. Rules are a separately identified comparison; their scores are not calibrated probabilities.
Proposal flow and work-revision events
The intended flow is already-available message → current opaque snapshot read → locked-model classification → three-display allowlist → revision binding → private shadow preview/observation. Readers need not fetch extra source text or documents. The existing bridge takes caller-held Request.Text and sends only explicitly requested opaque Work references to its metadata reader.
The pilot harness must set StatePlanningAvailable=false and supply no lifecycle events. Assert mutation_executed:false and state_change:false for every row. CandidateStatusRef is not completed-work evidence or permission to change status. The bridge has no verified-event or user-JSON trusted-flag input API.
Distinguish revision evidence for display placement from work-lifecycle transition evidence.
| Evidence | Future required binding | Current implementation |
|---|---|---|
| Revision observation receipt/event | Provenance from the authenticated application reader, owner scope, opaque Work reference, original canonical work revision, target content reference/revision and observation/change event reference | Not yet connected to a new pilot harness or live reader |
| Progress/completion lifecycle fact | Independently verified started/completed fact from the work system, target work/scope/command, expected/current revision, actual work outcome and replay receipt | Outside the first display pilot. Cannot be generated by model text or a JSON flag |
A revision-observation receipt fixes which work and content revision the display belongs to. Read timestamps and ReadTimes alone prove neither the current revision nor successful execution. A work-revision mismatch or edited content makes the previous proposal stale: do not place it, record the mismatch, read the new revision and compute a separate proposal. Never substitute time, row numbers or invented0/1for canonical revisions.
The existing bridge can compute annotation proposals without a canonical version. Such output remains unanchored content advice, not a verified display for the current work item. Until the new harness validates placement revisions, leave it unavailable instead of attaching it to work. Existing Request/Work types do not carry separate content revisions or event receipts, so that binding is additional design/implementation work.
Separate Work/label/status reads are not one atomic snapshot. Preserve observation times and catalog_consistency:unqualified. A later live display must still validate read permission, scope, the current catalog and work/content revisions in the application. Shadow output grants no lasting permission.
Verification to prepare with synthetic test doubles
The table is a test design, not an execution performed while writing this document. Record policy tests using manually supplied fake predictions separately from actual locked-model classification tests. Passing fake scores is neither model accuracy nor successful native lookup.
| Synthetic condition | Expected behavior |
|---|---|
| Current activity report, allowed progress label/emoji and matching revision | Progress-report proposal only; preserve work state |
| Sentence claiming the entire assignment is finished, allowed completion display | Completion-report proposal; no proof of successful execution or a done work state |
| Real information question mentioning completion/cancellation | Question display only; no lifecycle change |
| Some checks finished but the same assignment is still being worked on | Review progress/completion distinction; do not infer whole-task completion |
| Quotation, negation, hypothesis, contradiction or any other intent | Do not force one of the three displays |
| Missing/inactive/group/duplicate label or foreign scope | Exclude invalid references or reject the request |
| Missing/noncanonical work revision, changed revision or edited content | Invent no version; do not attach the display |
| Retransmission of the same content revision/command or an existing display | No duplicate display; distinguish replay from existing-annotation no-op |
| Reader error or Work missing from the scope | Expose no source text/credentials; do not classify/place that Work |
| Inconsistent numeric probabilities, guarded input or oversized text | Abstain/reject |
Observations should retain model hash, intent, confidence, interpretable abstention reason, opaque case/family reference, current label-allowlist fingerprint, revision-binding result, existing-display condition and classification/read/total processing costs. Keep actual text and native lookup separately private. Existing zero-allocation classifier hot-path measurements are not total reader/JSON/harness costs.
Staged sample plan:100→400→2,400
- Review the first100: first check placement/abstention boundaries with public synthetic doubles. When actual data is authorized, keep development-work source messages private and, where possible, have reviewers annotate without seeing predictions. A completion claim and actual completed-work evidence occupy separate fields. These100are development/review cases, not a new final benchmark.
- 400different work/context cases: include different assignments, phases, styles, Korean/English/mixed language, questions, quotations, negation, partial completion and current activity. Preserve observed failures as separate public synthetic reproductions without exporting actual source text. Count work/incident/conversation-context families alongside rows.
- Expand to2,400total rows: increase missing family coverage. Repetitions, renamed variants, translations and revision sequences from the same work are not independent samples. Keep related variants together across train/validation/calibration/final-test boundaries.2,400rows does not mean2,400families.
Before fitting again, freeze allowed source-data use, annotation rubric, family grouping, partitions, model candidates, calibration and the fixed gate. Do not reuse the observed100/400or exposed V1/V2 test as the next final selection benchmark. Reserve fresh families for a separately reviewed final evaluation. At every stage report incorrect completion-report displays, question errors, abstentions, duplicate/stale displays and complete cost by language/intent/family. Retain failures instead of removing them or lowering the gate to pass.
Private data and publication claims
Actual source text, event receipts, native IDs, work links and credentials are private by default. Publish only authorized summaries and aggregates. A public example needs separately authorized content or a newly authored synthetic reproduction. Do not automatically publish source-text hashes or opaque references either.
Count policy-double verification, synthetic model evaluation, actual read-only lookup, real UI displays and work-state transitions separately. This draft establishes the three-display scope and a concrete next verification plan only. It does not establish new fitting, live-reader access, installed displays, service writes, completed100/400/2,400-case review or improved operational accuracy.
Publication/full-file verification. OS RSS, Go heap and GPU metrics have different scopes and must not be added.