Experiment trajectory: 8 October 2026

Back to the experiment index

Recovery study preparation

The user requested explained notebooks for the reserved-image setup and the next model-preparation step. The preparation notes below precede the reported shared-stage completion at the end of this page. No recovery result is available. The earlier unlearning findings remain those on the 7 October page.

Notebook 16 creates CPU-only task-3 lists: 400 initial-training images and 100 reserved images per class, with nested recovery budgets of 1, 5, and 20 images per class. Validation stays separate. The notebook exports the lists and their fingerprint, without moving images or loading a model. Existing all-images checkpoints cannot satisfy this reserved-data design retrospectively. Synthetic-file checks verified counts, repeatability, overlap rejection, and refusal to overwrite an incompatible split. No real dataset was downloaded during these checks.

Notebook 17 prepares fresh study models, not recovery adaptation. It defaults to CPU CHECK mode with training disabled. CPU PREPARE saves the protocol and image manifest to R2. Explicit GPU TRAIN runs one seed and one stage per session.

The selected setup uses task 3 as the target and task 2 as the unseen control. Both branches first learn task 0 from one shared checkpoint per seed. This gives identical learned chunk embeddings before branching; they stay frozen thereafter. The unlearned branch follows L0, L3, L9, L5, L17, L1, L7, L14, U3. The never-learned branch follows L0, L9, L5, L17, L1, L7, L14. Task 2 is excluded from both. Moving L0 before L3 is a deliberate diagnostic change, not an exact reproduction of the paper or a continuation of the previous pair.

Task 3 uses the 4,000-image initial-training list. Retained tasks use 5,000 images each. Settings are Hyperfan-in, ResNet50, learning beta 0.1, learning and unlearning LR 0.0001, five epochs per lesson, batch size 64, evaluation batch size 256, unlearning gamma 0.01, ten noise samples, and 100 unlearning updates. U3 uses actual generated target weights, including scales and offsets. The notebook preserves shared, never-learned final, before-unlearning, and after-unlearning checkpoints. It saves epoch reports and request-boundary progress to R2 and releases a successful Colab GPU session only after verification.

The gate keeps the proposal's retained mean above 40% and summed absolute spill below 3 points. It also checks target accuracy before unlearning at least 25% and afterward at most 12%. These are practical entry criteria. The previous pair's 10.8-point summed spill fails this gate; its 1.54-point mean retained loss is a different measure. A failed new gate is saved without an automatic sweep or recovery run.

Four small CPU tests passed for branch isolation, shared-state reuse, frozen-state preservation, request-boundary resume, configuration mismatch refusal, spill arithmetic, and partition mismatch rejection. These tests do not establish study-model accuracy or GPU runtime. The notebook code cells compile. Cloud credentials, dataset download, R2 transfers, and A100 training were not executed in this review.

Next decision: run CPU image preparation and protocol checks first. When training is explicitly resumed, prepare and review seed 0 before committing further GPU sessions. Recovery adaptation and component tests still require a later runner. Main-repository records are updated now; wiki publication stays on its Wednesday/Saturday schedule.

Manifest persistence after a disconnected session

The user reported that the notebook-16 session had disconnected and asked where its manifest was saved. The original notebook saved local JSON files and exported a ZIP; it did not upload to R2. A read-only bucket check found no object at uncle/recovery_study/recovery_task3_reserved_v1/manifest.json. Whether the user downloaded the ZIP or whether the disconnected runtime retains local files is unknown.

Notebook 16 now uploads the manifest and preparation metadata to R2 by default, with content verification, before offering the ZIP export. It uploads no images or checkpoints. Notebook 17 now reads the R2 manifest in CPU CHECK and PREPARE modes as well as TRAIN. LOCAL remains an explicit alternative. A missing remote manifest stops with instructions to run notebook 16's CPU preparation and R2 save cell. The same split seed and class partition regenerate the same selection for the same extracted dataset layout.

Both notebooks' code cells compile. Mocked-storage execution checked notebook 16's JSON uploads, notebook 17's CPU CHECK and PREPARE paths, and the missing-object message. No new manifest was uploaded to the user's bucket by this work. No dataset or model training was performed.

Seed 0 shared stage completed

The user supplied notebook 17 output reporting Completed: shared Seed: 0 and task-0 validation accuracy of 51.4%. The notebook reported verification of these R2 objects:

• uncle/recovery_study/recovery_task3_reserved_v1/seed_0/shared/checkpoint.pt
• uncle/recovery_study/recovery_task3_reserved_v1/seed_0/shared/report.json

This is user-reported output. The remote report and checkpoint have not been independently read for this update. Actual configuration, source fingerprints, epoch measurements, GPU type, runtime, and GPU release are not established by the supplied excerpt. The intended shared stage is fresh L0 with notebook 17's fixed settings, followed by reuse of that checkpoint in both sequences. No task-3 recovery, unlearning gate, or retained-task comparison is available yet.

Next: keep STUDY_ID recovery_task3_reserved_v1 and SEED 0, select TRAIN with STAGE never_learned, and restore the shared checkpoint. Continue L9, L5, L17, L1, L7, L14 without learning tasks 3 or 2. After that sequence completes, prepare the separate unlearned sequence from the same shared checkpoint, not from the never-learned final model.

R2 cleanup and paired-study notebooks

The user authorized deletion of obsolete model checkpoints after a live bucket inventory and an exact-file review. Sixteen large model files were deleted: three unused beta-trial finals, four session-replay finals, seven earlier paired-L9 probe checkpoints, and two earlier learning-only checkpoints. Verification found 187 remaining objects totaling 9,159,283,304 bytes, down from 203 objects totaling 17,751,676,508 bytes. The deletion freed 8,592,393,204 bytes (8.59 GB). Every object outside the deletion list retained its size and ETag during verification. Reports, settings, the completed U3/L15 models, their source chain, and the recovery-study objects were preserved. No archive copy of the deleted models was verified; their exact model states are no longer in R2.

The user then requested notebooks to generalize experiment 06 while limiting GPU use, supporting interruptions, saving to R2, and showing progress bars. Notebooks 18, 19, and 20 implement CPU preparation, bounded GPU execution, and CPU review.

The default matrix has three learning orders, seeds 0, 1, and 2, forget targets 3, 5, and 14, and incoming task 15. Each target appears at learning positions 1, 4, and 8 across the orders. Nine independently trained source histories support nine shared learning-only controls and 27 unlearn-then-learn branches. The 27 comparisons therefore require 45 executable jobs including source histories. Every source lesson uses the same selected diagnostic configuration, including beta 0.1. This is a fresh diagnostic study, not a recreation of the previous mixed-setting checkpoint chain or the paper's request sequences. It uses all training images and does not replace the reserved-image recovery protocol.

Learning resume saves are made at epoch boundaries; unlearning saves occur every 20 steps by default. The state includes Adam, the original request-start protection snapshot, task buffers, request counters, and random generators. R2 checkpoints and reports are uploaded and content-verified before a progress pointer is published. Two alternating resume slots bound unfinished-job storage. After verified completion, cleanup removes only that job's temporary slots, while final models and the default immediate post-unlearning models remain. The full matrix's final model storage is approximately 39 GB, with roughly 4.3 GB more for two large active resume slots. These are estimates based on observed model size. GPU and transfer time have not been measured for this workflow.

The GPU notebook defaults to training disabled and one work unit per session, with a 120-minute soft budget and 15-minute save reserve. A request boundary or network retry can exceed the budget. Resume rejects a mismatched code, configuration, or recorded runtime signature. CPU review downloads reports rather than models, checks paired controls, reports each seed, and averages paired effects with equal weight across settings that have all planned seeds complete. Invalid pairs and unfinished work remain visible. Shared controls and tasks are not treated as independent repetitions.

Small CPU tests checked uninterrupted versus resumed model equality, preservation of the original learning behavior, old-task training-data exclusion in branches, upload-failure recovery, fixed-matrix reuse, session limits, pair rejection, and aggregation. Mocked R2 execution checked the CPU preparation and review exports. Existing paired-diagnostic and storage tests also passed. These are software checks, not GPU experiments or new accuracy results. No generalization training or recovery adaptation was started by this implementation work. Wiki publication remains on its scheduled days.

Reduced model retention for the paired study

The user later reported notebook 19 releasing its A100 runtime during setup after connecting to R2, without a visible error. The supplied output does not identify the underlying exception or establish that training began. The setup handler released the runtime before Colab's normal exception display. Notebook 19 now prints and flushes the traceback before releasing the runtime, including for errors during training. A mocked runtime check verifies that the setup traceback appears before release. The saved study plan and training implementation revision are unchanged. The next step is to retry with the updated notebook and inspect the printed exception.

The user questioned the initial 38.74 GB storage estimate and requested removal of unnecessary models. The paired-study default now keeps nine source checkpoints, unfinished-job resume state, and every report, setting, hash, and figure. Completed branch final models are uploaded and content-verified before a durable completion receipt is published; cleanup then removes the branch model. CPU review uses the verified receipt and report instead of requiring a model that was intentionally discarded. Post-unlearning models are not retained by default. Retaining either type of branch model remains an explicit option chosen before preparation.

Permanent model storage is now approximately 4.84 GB for the full matrix's nine sources, plus small evidence files. Two large active resume slots can add roughly 4.3 GB temporarily. The new study had no objects in R2 at the live cleanup check, so no prepared remote protocol required migration. Eight small CPU runner tests and two mocked notebook-flow tests passed, including source preservation, averaging after branch-model deletion, skipping completed work without retraining, and refusing to delete a model changed after verification. No new GPU experiment was started.

The remaining 16 historical model checkpoints were checked as additional R2 cleanup candidates, totaling 8,603,663,551 bytes (8.60 GB). This expanded list includes the earlier U3/L15 models, source chain, Hyperfan pilots, and older sequence checkpoints. All 16 were downloaded to a local archive and verified against their recorded SHA-256 hashes. Automatic approval review initially rejected deletion because this exact expanded list had not been explicitly approved after the earlier preservation instructions. The user then explicitly approved the reviewed list. All 16 archive copies were rechecked before conditional deletion, and remote verification confirmed that only those 16 objects were removed. R2 now contains 171 objects totaling 555,619,753 bytes (0.56 GB). Every other object's size and ETag remained unchanged, including all reports and the active recovery-study files. The local archive remains available under outputs/r2-review/historical-model-archive/, with its manifest and deletion log under outputs/r2-review/second-cleanup-2026-10-08/. Older notebooks that need these historical models require restoration from that archive.

Interrupted R2 checkpoint upload

The user supplied an InvalidPart traceback from notebook 19 while publishing controls/order_01/seed_0/learn_15/resume_1.pt for paired_generalization_v2. Training had reached a checkpoint boundary, but R2 rejected multipart-upload completion. The exact cause is unknown. The progress pointer is published only after checkpoint and report verification, so this failure did not advance that pointer. Resume uses the previous verified boundary and can repeat work after it.

Notebook 19 now uses sequential 64 MiB upload parts with the classic boto3 transfer client and at most three full-upload attempts for InvalidPart. Other errors still stop immediately. This is a notebook transport change; the fixed training implementation and saved study configuration are unchanged. Four mocked notebook checks passed, including retry success, retry exhaustion, immediate access-denied failure, CPU preparation/review, and traceback display before runtime release. No live large-checkpoint transfer or GPU run was performed for this fix. The next step is to reopen the updated notebook 19 on A100, keep the same study ID and settings, and run all cells with only one active writer.