Status (2026-08-03): historical experiment decision. The experiment shaped the current hybrid design, but its proposed module topology and staging are no longer the current operating contract. See pulse_consolidation.md for current architecture and pulse_v2_proof_carrying_architecture.md for the retained measurements and later decisions.

Date: 2026-07-29

Primary objective: Reliability

Secondary objective: Cost efficiency

Scope: Workflow Pulse Gate, evidence normalization, deterministic signals, semantic review, automatic repair boundaries, notifications, and Goal Advisor experiments

Background discussion: Pulse v2 architecture and review exchange

Pulse will not be rewritten wholesale and Gate will not be globally removed.

The approved direction is a hybrid reliability architecture:

• deterministic runtime facts are normalized in code;
• a Sentinel signal layer identifies indisputable conditions;
• Gate remains available for ambiguity, semantic drift, assurance expiry, and sampled review of apparently healthy runs;
• existing reviewer skills remain the core semantic review capability;
• operational truth stops expanding inside builder/improve.html;
• consequential changes continue to use the existing Pulse human-decision flow;
• Goal Advisor experiments move into a dedicated Lab lifecycle, which may be built now;
• broader Sentinel, Autopilot, and event-ledger work proceeds only when production measurement justifies it.

The next commitment is an experiment, not a control-plane replacement.

Pulse exists so a small team can operate 100+ autonomous workflows by exception.

It should reliably answer:

1. What did the workflow claim happened?
2. What current evidence supports or contradicts that claim?
3. What materially changed?
4. What requires deterministic handling, semantic review, or a user decision?
5. What was changed, and has the change been verified?

Reliability takes precedence over token reduction. Cost optimization is pursued only where it does not materially reduce assurance.

When reliability and cost conflict, choose the more reliable path.

Cost remains important because excessive Pulse cost prevents the system from scaling, but cost savings do not justify removing semantic review without evidence.

Sentinel handles deterministic facts and may eventually own narrowly proven routes.

Gate remains responsible for:

• ambiguous or conflicting evidence;
• semantic correctness;
• healthy-but-wrong detection;
• external or environmental drift that lacks a deterministic verifier;
• expired assurance;
• risk-weighted sampled audits;
• unfamiliar signal types.

There is no plan to globally delete Gate.

Users choose notification behavior during Pulse setup.

Run outcome and Pulse maintenance are configured separately:

• Run outcome: every run, transitions only, or digest.
• Pulse activity: normally transitions/decisions only.

Recommended defaults:

• externally side-effecting workflows: run outcome after every run;
• high-frequency read-only workflows: transitions or digest;
• Pulse maintenance: transitions only;
• dashboard: always updated.

The existing run_summary and pulse_summary preference model remains the configuration surface.

Proof means evidence that an important workflow effect or user-visible claim is true. It does not mean exposing or validating hidden chain-of-thought.

Examples:

Use these states:

• success_verified
• success_unverified
• failed

success_unverified means the action may have happened, but Pulse cannot prove it.

For a material or high-risk claim, success_unverified:

• prevents a healthy Pulse verdict;
• triggers semantic review;
• may notify the user;
• never automatically retries an external side effect;
• remains open until proof, contradiction, user resolution, or expiry.

Use failed when evidence disproves the claim or a required fail-closed contract explicitly defines absence of proof as failure.

Pulse continues to use structured human-input requests for consequential changes.

Automatic work is initially limited to bookkeeping and narrowly registered, reversible repairs. Plan, strategy, destination, recipient, model, schedule, database meaning, and business-semantic changes require exact approval.

Lab is approved as a separate, proposal-first strategy experiment lifecycle.

It does not wait for Sentinel measurement, but it must not become a second operational Fixer.

Do not build or migrate to a universal Pulse event ledger during Stage A.

Use existing SQLite state for the measurement experiment. Lab may have a dedicated experiment table and history.

A unified event ledger remains an option after Stage A if state fragmentation continues to justify its implementation and migration cost.

The current reviewer skills are product assets:

• design-plan
• improve-evaluation
• improve-report
• improve-database
• improve-learnings
• improve-knowledge
• pulse-bug-review
• cost/timing and LLM operations guidance

The architecture distinguishes:

• Control-plane prose: scheduling, module enumeration, concurrency, authorization, transaction state, recovery, and finalization. Move exact invariants into code where practical.
• Reviewer expertise: how to investigate, compare evidence, diagnose, and recommend. Preserve and evaluate this semantic content.

The objective is not fewer prompts. It is prompts doing judgment instead of acting as an unreliable transaction coordinator.

Pulse should infer an initial risk class from workflow capabilities and ask the user only when uncertain.

• sends external messages;
• publishes content;
• trades, pays, purchases, or moves money;
• changes external or production data;
• uses authentication or credentials;
• affects recipients, accounts, or destinations;
• generates business decisions people act on without review.

Any workflow that communicates externally or mutates an external system is high-risk by default.

• produces reports, recommendations, or evaluations;
• updates internal stores;
• prepares externally used content that a person reviews first;
• influences later workflows without directly causing an external effect.

• read-only research;
• drafts;
• local transformations;
• tests and simulations without external effects.

Risk class controls:

• assurance expiry;
• sampling frequency;
• required boundary coverage;
• notification recommendation;
• permitted automatic actions;
• verification strength.

Do not author claims for every step.

Begin with critical assurance boundaries:

• workflow success criteria;
• external effects and their account/destination;
• current-run and current-group binding;
• durable writes and the runtime consumers expected to read them;
• user-visible report and evaluation claims;
• safety-critical stop, retry, and fail-closed behavior.

Claims may be inherited from:

• step type;
• tool type;
• validation schema;
• success criteria;
• reviewer-discovered assurance gaps.

Claims that encode business meaning require user or builder approval.

Coverage must be explicit:

required critical boundaries: 7
proved: 5
legitimately not exercised: 1
unknown/not instrumented: 1

unknown, not_instrumented, stale, or contradicted critical boundaries prevent routine semantic-review skipping.

A workflow with no claim coverage is coverage_unknown, not healthy.

Do not require the entire tool ecosystem to adopt new contracts immediately.

Introduce receipt extraction around a small set of important boundaries:

• publishing;
• messaging;
• external writes;
• database writes/read-back;
• report/eval run binding;
• backup and publish completion.

Adapters may normalize existing tool arguments and results. Native typed receipts can be introduced later when the tool is owned and the benefit is proven.

Sentinel's routing policy is deterministic and initially runs only in shadow mode. Shadow mode does not require hiding deterministic facts from Gate.

Existing code-owned facts may remain live Gate inputs. This includes open_concerns, plan_change_backlog, module review history, and the validated loop-closure findings. For loop closure:

• Gate receives the findings and coverage state as read-only evidence;
• no finding mandates a module or overrides the three-module cap;
• no finding authorizes mutation;
• an empty result is clean only when coverage is verified;
• the server snapshots the same facts with Gate's worklist for measurement.

This measures whether a deterministic routing decision could replace Gate. It does not claim to be a counterfactual test of how Gate would behave if the underlying facts had been hidden. New fact families without equivalent operational validation remain shadow-only.

Candidate deterministic signals:

• run or step failed;
• required output missing;
• evaluation score missing or zero;
• current-run/group mismatch;
• unreviewed plan change;
• recurring open concern;
• expired module/reviewer baseline;
• material cost/timing change;
• backup/publish failure;
• required boundary proof missing, stale, or contradicted.

Sentinel does not diagnose root cause or declare semantic health.

Gate remains the only authoritative scheduler during Stage A.

Gate consumes normalized facts and continues selecting the existing reviewer skills.

During Stage A, shadow signals:

• record what they would have selected;
• never launch or suppress a reviewer;
• never mutate module state;
• never compete with Gate.

Live fact feeds may influence Gate's reasoning, just as existing concerns and plan-change backlog already do. What remains shadow-only is the deterministic decision that maps those facts to reviewer selection or suppression.

If a deterministic route is later approved:

• ownership transfers only for that signal family and risk class;
• exactly one scheduler is authoritative;
• Gate remains the fallback for unknown signal types, but not an ambiguous second owner of an already transferred route;
• a critical miss revokes deterministic ownership of that route until it is redesigned and re-evaluated.

Do not use a synthetic numeric confidence score in Stage A.

Use a deterministic predicate:

routine_doctor_skip_eligible =
  no_open_critical_or_high_signal
  AND no_watched_change_detected
  AND all_required_critical_boundaries_instrumented
  AND all_required_receipts_current
  AND no_contradictory_evidence
  AND semantic_assurance_not_expired
  AND not_selected_for_control_sample

Every clause returns:

• pass
• fail
• unknown
• evidence/reason

unknown makes the run ineligible for skipping Doctor review. It does not by itself mark the workflow failed.

Initial dimensions:

• workflow, plan, prompt, validation, eval, report, and schedule versions;
• resolved model/provider and relevant runtime/tool versions;
• account, destination, permission, and credential state;
• source schema and material input-distribution indicators;
• external contract/page-shape fingerprints where available;
• output, timing, cost, and receipt shape.

Risk-specific windows and tolerances are measured in Stage A rather than guessed in advance.

Use categorical state:

• eligible_for_routine_skip
• semantic_review_due
• coverage_unknown
• evidence_contradicted

Each category carries its exact determining clauses.

Initially automatic:

• update projections;
• deduplicate signals;
• schedule a recheck;
• record freshness from verified runtime evidence;
• render status;
• preserve or advance deterministic cursors.

Eligible only after its action type is individually implemented and proven:

• correct current-run/group binding;
• restore a required field without changing meaning;
• repair a known report query against an unchanged contract;
• roll back a previous automatic action.

Requirements:

• expected target hash/version;
• bounded parameters or patch;
• rollback;
• verification policy;
• no business-semantic change;
• no recipient, destination, account, model, or schedule change.

User approval required:

• prompts that change external behavior;
• model/provider;
• schedule;
• destination/recipient/account;
• database semantics or migration;
• business thresholds;
• external side-effect behavior.

Lab experiment plus user approval required:

• strategy replacement;
• goal or success-criterion change;
• material plan restructuring;
• externally visible experiment.

Every action type declares:

• minimum verification method;
• permitted stronger methods;
• whether it may be applied before verification;
• maximum unverified lifetime;
• rollback requirement;
• escalation when verification is unavailable.

Policy may strengthen the requirement based on risk. Doctor may request a stronger method but cannot weaken the action-policy minimum.

1. Static/schema/contract assertion.
2. Deterministic fixture or subgraph replay.
3. Read-only postcondition query against the real system.
4. Sandbox or synthetic-account execution.
5. Shadow decision with side effects suppressed.
6. Limited canary.
7. Next legitimate production evidence boundary.

Full workflow replay is not assumed.

An action may become applied_unverified only when its registered policy permits next-boundary verification.

It records:

• action and finding IDs;
• target/version;
• application time;
• expected evidence boundary;
• expiry;
• rollback state;
• user-visible impact;
• escalation owner.

Expired items become verification_overdue, stay visible in Pulse, and block overlapping actions on the same target.

Lab is a strategy experiment system, not a maintenance reviewer.

draft
→ proposed
→ awaiting_user
→ approved
→ running
→ awaiting_evidence
→ adopted | rejected | retired | blocked

Each experiment stores:

• workflow;
• goal criterion;
• current strategy and baseline;
• proposed alternative;
• hypothesis;
• expected effect;
• measurement method;
• evidence checkpoint;
• risks;
• approval;
• relevant plan/config versions;
• result and disposition.

• one active strategy experiment per workflow;
• Goal Advisor and critic remain read-only;
• the existing Pulse human-input system obtains approval;
• existing builder tools apply the exact approved change;
• no automatic strategy changes;
• approval expires when the proposal target/version materially changes;
• browser workflows may use future legitimate runs as evidence;
• Lab writes a dedicated SQLite experiment table/history;
• Lab does not require the global Pulse event-ledger migration.

Stage A is the only approved Sentinel implementation stage.

Create one canonical Go registry for current Pulse module metadata.

Generate or validate:

• module order and IDs;
• current versus legacy aliases;
• reviewer artifact path validation;
• tool schemas/enums;
• result validation;
• scheduler labels;
• UI metadata;
• guidance/documentation invariants where feasible.

Parity tests must fail when a current module/tool appears in one required registry surface and not another.

This work is justified independently by demonstrated production defects and does not wait for the experiment.

Effective immediately:

• do not add new authoritative state or recovery semantics to builder/improve.html;
• use SQLite/runtime state as authoritative for new behavior;
• treat HTML as an explanatory view;
• keep the current hidden recovery ledger until its durable replacement exists;
• do not expand its responsibilities.

Normalize facts already available today:

EvidenceBundle
  workflow/run identity
  workflow/plan/config versions
  step terminal states
  run metadata
  validation outcomes
  evaluation verdicts
  concern recurrence
  plan-change backlog
  cost/timing summaries
  report/eval current-run binding
  backup/publish state
  human-input state
  reviewer/module history
  available boundary receipts

The bundle is read-only input to Gate and shadow Sentinel.

For every pass, record:

• EvidenceBundle identity/version;
• Gate decisions and reasons;
• deterministic signals present;
• modules/reviewers selected;
• reviewer findings and severity;
• actions and verification outcomes;
• Gate, reviewer, Fixer, finalizer, and recovery cost/timing.

Begin with a small set:

• run/step failed;
• missing required output;
• missing current eval verdict;
• unreviewed plan change;
• recurring concern;
• backup/publish failure;
• current-run mismatch;
• material cost/timing change.

Do not begin with every proposed signal family.

The experiment must review some runs that Sentinel would classify as requiring no Doctor review. Otherwise it cannot measure false negatives.

Start with three workflows:

1. one high-risk browser/publishing workflow;
2. one report/data/evaluation workflow;
3. one mostly deterministic workflow.

• review every run after a material change;
• additionally review one apparently shadow-clean run per pilot workflow per day;
• continue sequentially until the evidence threshold, experiment budget, or a verified critical miss stops the experiment;
• target approximately 60 clean sampled reviews as an initial planning number, not an automatic promotion threshold.

If zero misses occur in n independent samples, the rough 95% upper bound on the true miss rate is approximately 3/n. Sixty zero-miss samples therefore support only a roughly 5% upper bound, not proof of zero risk.

Track samples by:

• workflow type;
• risk class;
• recent change versus observational stability;
• first baseline versus previously reviewed state;
• external side-effect versus read-only behavior.

Use the existing read-only reviewer skills as Doctor reviewers.

Human-adjudicate a smaller subset to calibrate whether agent findings are real.

A shadow-policy miss is:

• a verified actionable correctness, evidence-integrity, safety, or material goal-measurement finding;
• supported by evidence that existed when the shadow decision was made;
• not represented or routed by a shadow signal.

Style preferences, unsupported suspicions, facts that occurred after the decision, and findings below the predeclared materiality threshold do not count.

Examples:

• wrong recipient/account/destination;
• unsafe external action;
• false success on a materially failed workflow;
• stale evidence causing an incorrect material conclusion;
• financial/authentication failure;
• corruption or loss of production data.

One verified critical miss:

• stops deterministic ownership promotion for that route;
• keeps or returns it to hybrid Gate/Doctor handling;
• requires signal/coverage redesign;
• resets evidence collection for that route.

It does not cancel unrelated deterministic routes.

• percentage of Gate selections explainable by deterministic facts;
• Gate-only actionable-finding rate and severity;
• deterministic-route actionable-finding rate;
• shadow-clean sampled miss rate;
• critical miss count;
• changed versus observationally stable finding yield;
• coverage state by risk class;
• false positives and false negatives after adjudication.

• actionable findings per reviewer;
• actionable findings per token/cost;
• reviewer-skill yield independent of trigger;
• duplicate or unsupported finding rate;
• time from signal to verified resolution.

current_pulse_cost =
  gate
  + reviewers
  + fixer
  + finalizer
  + recovery/retry

Report per pass, workflow/day, and organization/day where data permits.

stage_a_cost =
  engineering
  + telemetry/storage
  + shadow evaluation
  + control-sample Doctor reviews
  + human adjudication

Also record:

• production defects found;
• incidents prevented or shortened;
• Pulse-caused failures;
• operator investigation time;
• time to detection;
• time to verified resolution.

The Stage B decision compares reliability, assurance value, cost, and migration risk—not token savings alone.

Before running the experiment, declare:

• observation/run budget;
• Doctor review budget;
• minimum samples per important risk stratum;
• maximum acceptable critical/high miss rate;
• materiality definitions;
• adjudication procedure;
• stop/extend rule;
• inconclusive outcome.

Possible outcomes:

Approved only for signal families and risk classes that meet the predeclared reliability evidence.

Sentinel normalizes facts and prioritizes work, while Gate remains the semantic scheduler.

This is the expected default unless narrow replacement is strongly supported.

Keep Gate scheduling. Ship only the improvements justified independently:

• canonical registry;
• normalized facts;
• structured state;
• reporting/projection discipline;
• measurement and reviewer evaluations.

If the budget expires without adequate assurance, do not promote Sentinel. Extend the experiment or choose Outcome 2/3.

Stage C exists only after an explicit Stage B decision.

• select one workflow, signal family, and risk class;
• exactly one system owns new decisions for that route;
• legacy Gate/module state owns earlier occurrences;
• late events route by original run/correlation identity;
• do not dual-write authoritative decisions.

• refuse cutover with an in-flight fixing row;
• leave changed_unverified work under its legacy owner until terminal;
• snapshot relevant module state, audit, concerns, reviewer artifacts, and recovery metadata;
• block overlapping legacy and signal actions.

• preserve module audit and reviewer artifacts immutably;
• expose history through read-through adapters;
• seed signal projections with explicit legacy references;
• preserve concern fingerprints and recurrence counts.

• disable the route at a recorded cursor;
• return future decisions to Gate;
• never delete signal history;
• keep already-applied actions under their verification policy until terminal.

During Stage A:

• existing pulse_module_state remains authoritative;
• existing pulse_module_audit remains the fixer history;
• pulse_review_log remains reviewer history;
• run_concerns remains durable concern state;
• pulse_final_command_state remains finalizer state;
• structured finding evidence and compact review receipts remain forensic evidence;
• current HTML recovery state is preserved but not expanded.

No live-state migration occurs during measurement.

1. Canonical module/tool registry and invariant tests.
2. No-new-HTML-truth rule.
3. Gate/reviewer/fixer/finalizer instrumentation.
4. Minimal EvidenceBundle.
5. Small shadow signal set.
6. Three-workflow control sample.
7. Stage B decision.

1. Experiment schema and SQLite history.
2. One-active-experiment invariant.
3. Goal Advisor proposal and critic output mapped into the experiment record.
4. Existing human-input approval.
5. Exact approved-change application through current builder tools.
6. Measurement checkpoint and adopted/rejected/retired disposition.
7. Pulse UI representation.

Track B does not wait for Stage A, but the two tracks must not share mutation ownership or introduce a second Fixer.

• every current module is accepted by reviewer persistence;
• scheduler/tool/UI registries match;
• legacy aliases are explicit and read-only where appropriate;
• parity tests reproduce the defect class found during the July refactor.

• shadow routing decisions never launch or suppress live work;
• validated live fact feeds may inform Gate but never mandate routing;
• every comparison is tied to the exact EvidenceBundle and Gate decision;
• sampled clean runs receive independent Doctor review;
• misses are adjudicated;
• cost and finding yield are reportable;
• no operational truth is added to HTML.

• no strategy change occurs without exact approval;
• one active experiment per workflow;
• baseline and measurement are explicit;
• approval expires on material target drift;
• experiment history survives restarts;
• Goal Advisor remains read-only;
• Lab cannot overwrite operational Fixer state.

• one authoritative route owner;
• in-flight legacy work drains before cutover;
• history and concern recurrence remain legible;
• rollback preserves actions and evidence;
• one critical miss revokes route ownership.

• global Gate removal;
• universal claims for every step;
• typed receipts for every tool;
• universal workflow replay;
• broad automatic plan/config changes;
• automatic strategy adoption;
• global Pulse event-ledger migration;
• deterministic HTML/published-page rendering rewrite;
• retirement of legacy module orchestration.

These remain hypotheses or later improvements, not approved work.

The architecture is settled enough to begin the two approved tracks, but Stage A planning still needs operational numbers:

• current number of Pulse-enabled workflows;
• runs per workflow/day;
• current Gate/reviewer/Fixer/finalizer cost;
• Stage A Doctor-review budget;
• experiment duration/run budget;
• acceptable high-severity miss bound;
• materiality/adjudication owner;
• initial three pilot workflows.

These values configure the experiment. They do not reopen the architectural decisions above.

Proceed with:

1. the canonical registry;
2. Stage A instrumentation and shadow measurement;
3. the independent, proposal-only Lab MVP.

Do not proceed yet with:

• Gate replacement;
• broad proof/receipt rollout;
• general Autopilot;
• unified event-ledger migration;
• live-state cutover.

The final operating principle is:

Reliability comes first. Deterministic code should own facts and exact invariants; Gate and the existing reviewer skills should continue to own semantic judgment until production evidence proves that a narrower route can be safely automated.