backup_history_consolidation - manishiitg/agentworks GitHub Wiki
Status: Phase 1 + 2 + 3 implemented (2026-06-21); Phase 4 (per-commit historyβrestore linkage + auto-snapshot) deferred β needs a data model that does not exist yet (changelog entries are not tagged with backup commit refs, and there are no builder hooks for "risky bulk op").
2026-09-30: the ZIP workspace export/import endpoints and the
local_zipstrategy were removed (see the decisions log). References to them below describe the pre-removal design.
Implemented so far:
-
Phase 1 β Versions removed. Deleted
WorkflowVersionsPopup.tsx, the toolbar Versions button + state, the 4 version API methods, theWorkflowVersionMetatype, the 4 backend handlers + routes (workflow.goversion block +server.goroutes), and thelocal_versionsbackup strategy. Shared test helpers that lived inworkflow_versions_test.gowere preserved in newworkspace_mock_test.go. -
Phase 2 β Restore from Backup + git-default tiering.
handleRunWorkflowBackupnow acceptsaction:"restore"with an optionaltarget_ref, backed by a new agent-drivenbuildWorkflowRestoreAgentPrompt(git checkout into the workspace, no remote rewrite). Backup popup has a Restore button + confirmation (restores latest; commit picker deferred to Phase 4). Git is marked the default primary strategy and the configure prompt defaults to a zero-config local git repo, nudging a GitHub remote for off-box durability. -
Phase 3 β History view. The "Pulse" tab is renamed History and is now a
two-lens surface: Timeline (the existing
improve.htmlnarrative viaLogViewer) and Plan edits (new). Plan edits is backed by a newGET /api/workflow/plan-changelogendpoint (workflow_changelog.go) that mergesplanning/changelog/*.jsoninto a newest-first feed of tool Β· reason Β· per-field oldβnew diffs, rendered byPlanChangelogFeed.tsxinsideHistoryView.tsx.
We accumulated three overlapping features for "don't lose / understand my workflow":
- Backup β AI-driven, multi-destination, off-box durability. Now the primary strategy.
-
Versions β local numbered snapshots (
versions/v{N}/), manual rollback points. - Changelog / History β the narrative of what changed and why.
These were built at different times and now confuse each other. Backup is clearly primary. Versions no longer earns its own top-level slot. History is the most durably valuable artifact but is under-framed. This doc defines the target: two first-class citizens β Backup (durability + restore) and History (what & why) β and the removal of Versions as a standalone feature.
- Config in
workflow.json.backup; operational status inbackup/status.json. - Strategies (
workflow_backup.go:82):git/GitHub,object_store(R2/S3/B2),huggingface,local_zip,local_versions(β this is the versions feature, already modeled as a backup strategy type). - Agent-driven (
handleRunWorkflowBackup,workflow_backup.go:343). States:not_configured,configured_not_verified,running,healthy,stale,partial,failed. - Tracked for staleness (
workflow_backup.go:161,172):workflow.json,planning/*.json,reports/report_plan.json,variables/variables.json, +knowledgebase/andlearnings/folders. - UI: dedicated Backup popup + toolbar button with status dot (added 2026-06-19).
- Endpoints
workflow.go:4524/4674/4804/4915(publish/list/revert/delete). - On disk:
versions/v{N}/version_meta.json+ snapshotted files. - Covers a strict subset of backup: config files +
learnings/, but notknowledgebase/. - Manual-publish only β no auto-snapshot before AI edits, revert, or harden/improve.
-
Effectively unused: e.g. the
linkedinworkflow has exactly one version, published once in April 2026. Nobody remembers to click Publish. - Why it's redundant: a git backup destination already produces a durable, off-box, automatic commit timeline β a strictly better version history than local snapshots.
Two distinct logs, both kept:
-
builder/improve.htmlβ the high-level human-facing narrative (applied/proposed changes, review findings, monitor notes, user rules). Already surfaced in the UI as the "Pulse" tab (WorkflowCanvas.tsx:307βLogViewer.tsx, viagetBuilderDoc('improve'), endpointauto_improvement_endpoints.go:145). Archives viagetBuilderDocArchives. -
planning/changelog/*.jsonβ automatic, per-plan-mod-tool-call audit trail with mandatoryreasonand per-field oldβnew diffs (planning_agent.go:964). No UI today.
Backup β "Can I get it back?" durability + restore. Multi-destination, AI-driven.
History β "What changed & why?" narrative + plan-edit audit. Read-only timeline.
(Versions: deleted β its job is absorbed by git-backup history + Backup's restore.)
Two toolbar citizens instead of three. The standalone Versions button and popup are removed.
- Frontend:
WorkflowVersionsPopup.tsx, its toolbar button + state inWorkflowToolbar.tsx, thelistVersions/publishVersion/revertToVersion/deleteVersionAPI methods. - Backend: the four version handlers (
workflow.go:4507β4994) and their routes. - Drop
local_versionsfrom the supported-strategies list (workflow_backup.go:82).
Killing versions removes the only one-click rollback. Restore must move to Backup, consistent with the agent-driven model:
- New action in the Backup popup: "Restore from backup".
- Source = a configured durable destination (git commit, or object-store snapshot).
- For git: list recent commits on the backup branch; selecting one runs a builder "restore" task that checks out that commit's tracked files back into the workspace.
- Implemented the same way as
handleRunWorkflowBackupβ an agent task with arestoreaction and a target ref, writing progress tobackup/status.json.
-
Do not auto-delete existing
versions/v{N}/folders β harmless on disk. - Stop writing new ones; hide all UI.
- One-time: a small notice in the Backup popup if
versions/exists ("Local snapshots are deprecated β set up a git backup for durable history"). No automatic conversion.
Killing versions makes restore depend on a configured durable destination. A workflow with no backup configured has no rollback. Mitigations:
- Backup popup nudges setup when
not_configured. - Keep
local_zipexport as the offline safety net (already in the Backup popup). - Optional follow-up: a lightweight auto-commit to a local git repo under the workspace as a zero-config default destination (gives restore without remote setup). Deferred.
Most of this exists as Pulse. The work is reframing + adding plan-level detail + a restore hook, not building from zero.
A History surface (extend the existing Pulse tab, or a sibling tab/popup) with:
-
Timeline (default) β render
builder/improve.html(reuseLogViewer). This is the authored "what & why" narrative. Newest-on-top, with archive months fromgetBuilderDocArchives. -
Plan edits (new) β parse
planning/changelog/*.jsoninto a granular feed: timestamp Β· tool Β·reasonΒ· affected step ids Β· per-field oldβnew diffs. Needs a new read endpoint (or reuse the workspace file API) to list + read changelog files. - Restore hook β where an entry corresponds to a backup commit, offer "Restore to this point" β triggers the Backup restore flow above. (Linking entries to commits is phase 2; phase 1 ships the read-only timeline.)
Keep the user-facing name "Pulse" (already established) or rename to "History" β to be decided at implementation. Either way it is the single "what changed & why" window.
Backup gets an opinionated, tiered default instead of a flat list of equal strategies. Everyone uses git, so git is the baseline; heavier content escalates to bulk storage. Two axes: git is the timeline (local default β GitHub upgrade); object_store and HuggingFace are interchangeable bulk stores for content that shouldn't live in git.
| Tier | Strategy | Holds | Rationale |
|---|---|---|---|
| Normal default |
git (local) |
workflow.json, planning, knowledgebase, learnings, scripts | text + small JSON, diffable; this is the rollback timeline, zero config |
| Primary upgrade | GitHub (git + remote) | same as above, off-box | durable, off-box copy of the timeline; the recommended upgrade |
| Tertiary (bulk storage) |
object_store (R2/S3/B2) + huggingface
|
run folders, media, large artifacts, datasets | both are storage backends; HF Hub also offers storage/LFS. Agent picks per content type |
The git strategy is currently labeled "Git / GitHub" (workflow_backup.go:85) and
conflates two things:
-
Local git (
.gitin the workspace, auto-commit) β zero config, instant rollback, but on-box (not a real off-box backup; it's a better Versions). - GitHub (git + remote) β off-box durability, needs auth (token/SSH).
Default = local-git auto-commit. Every workflow gets a real commit timeline + restore the moment it's created, no setup. The Backup popup then nudges "add a GitHub remote for off-box durability." This is also the clean answer to the Decision 1 risk: with local-git as the default, killing Versions never leaves a workflow without rollback.
The builder agent escalates to object_store only when the workflow actually produces
large/media artifacts (driven by the existing BestFor metadata). Text-only workflows
stay git-only. No user choice required for the common case.
- Split the strategy concept:
git(local, zero-config default) vsgit_remote/GitHub as the durable upgrade β or keep onegitstrategy with an optionalremotefield and treat "no remote" as the local default. - On workflow creation, initialize a local git repo + first commit as the implicit default destination.
-
Phase 1 β Remove Versions. β
Done. Deleted UI + endpoints +
local_versionsstrategy.local_zipexport remains the offline fallback. -
Phase 2 β Restore from Backup. β
Done (restore-latest). Agent
restoreaction + Restore button/confirm in the Backup popup. Git-commit picker deferred to Phase 4. -
Phase 3 β History view. β
Done. "Pulse" β "History" with Timeline + Plan edits
sub-tabs; new
/api/workflow/plan-changelogendpoint +PlanChangelogFeed. - Phase 4 β Link history β restore. Deferred. Associate changelog/decision entries with backup commits; "Restore to this point" from the timeline; auto-create a backup commit + decision entry before risky bulk operations. Blocked on: the backup agent does not yet stamp changelog entries with the commit it captured them in, and there is no hook that fires before harden/replan/bulk edits. Needs that linkage data first.
- Phase 1 and Phase 2 should ship together β confirm we're not releasing the deletion before restore exists. (Strong recommendation: yes, together.)
- Rename "Pulse" β "History", or keep "Pulse"?
-
Zero-config local-git defaultβ decided (Decision 3): yes, local-git auto-commit is the default primary; GitHub/object_store are upgrades. - Do we surface
planning/changelogto users at all, or keep it agent-only and only elevateimprove.html? (Granular diffs may be noise for non-builders.) - One
gitstrategy with optionalremote, or splitgit(local) vsgit_remote(GitHub)? Affects how the agent reasons about durability state.
- No manual JSON editor for backup config (stays AI-driven β prior decision).
- Not building a general git client UI β only a commit picker scoped to the backup branch.
- Not auto-deleting existing
versions/data.