Runbooks Unity Runners After Transfer - Ambiguous-Interactive/unity-helpers GitHub Wiki

Unity Runners After Repository Transfer Runbook

This runbook explains how to restore self-hosted Unity runner access after a repository is transferred between GitHub organizations (or when a freshly provisioned runner does not pick up queued Unity jobs). Keep execution notes local. Do not paste secrets, screenshots of organization settings, or other private account metadata into this file or any tracked follow-up.

It is referenced by the runner-preflight job in .github/workflows/unity-tests.yml, .github/workflows/unity-benchmarks.yml, and .github/workflows/runner-bootstrap.yml, and by scripts/unity/ensure-editor.ps1 and .github/actions/print-self-hosted-runner-diagnostics.

Symptom

  • A queued Unity workflow run (for example Unity Tests or Unity Benchmarks) stays queued indefinitely.
  • The GitHub Actions UI shows the job waiting for a runner. There is no error, no warning, and the run never starts.
  • The organization's self-hosted runners report Online and Idle in the GitHub UI, with labels that exactly match the workflow's runs-on request (self-hosted, Windows, RAM-64GB).
  • The watchdog defined in .github/workflows/stuck-job-watchdog.yml does not recover the run because no idle runner is visible to the repository, so the watchdog's label-matching rule never fires.

Root cause

After a repository transfer between GitHub organizations, the destination organization's runner groups do not automatically include the transferred repository in their repository-access list. When a runner group is configured as "Selected repositories", any repository that is not explicitly listed cannot dispatch jobs to that group's runners. The dispatcher does not log an error in this state; the job simply stays queued.

This is a configuration-state issue, not the intermittent dispatcher bug tracked upstream as GitHub Community Discussion #186811. The dispatcher bug applies when an idle matching runner is visible to the repository through the GitHub API but never receives the job. If the API does not list the runner at all for the repository, this runbook applies instead.

Diagnose with the GitHub CLI

Run the following from any workstation with gh auth login already completed. Replace <org> with the destination organization that owns the runners.

List the organization's runner groups, including each group's visibility setting:

gh api orgs/<org>/actions/runner-groups \
  -q '.runner_groups[] | {id, name, visibility, allows_public_repositories}'

For a runner group whose visibility is selected, list the repositories that currently have access:

gh api orgs/<org>/actions/runner-groups/<group-id>/repositories \
  -q '.repositories[] | {id, name, full_name}'

If Ambiguous-Interactive/unity-helpers does not appear in that list, the dispatcher has no path to the group's runners from this repository, which matches the symptom above.

Cross-check by listing runners that the repository itself can see:

gh api repos/Ambiguous-Interactive/unity-helpers/actions/runners \
  -q '.runners[] | {id, name, status, busy, labels: [.labels[].name]}'

When this list is empty or omits the expected runner names while the organization-level inventory shows them online, the access list is the cause.

Resolution

Choose one of the following inside the destination organization. Either restores dispatch; pick the one that matches the organization's security model.

Add the transferred repository to the selected list:

  1. Organization Settings.
  2. Actions.
  3. Runner groups.
  4. Default (or the relevant group).
  5. Repository access.
  6. Add wallstop/unity-helpers to the list.
  7. Save.

Change the group's visibility to all repositories:

  1. Organization Settings.
  2. Actions.
  3. Runner groups.
  4. Default (or the relevant group).
  5. Repository access.
  6. Set visibility to all repositories.
  7. Save.

The second resolution avoids future per-transfer maintenance but exposes the runners to every repository in the organization. Use it only when that exposure is acceptable for the runner group's security posture.

After applying the chosen resolution, re-run the queued workflow from the Actions tab. The runner-preflight job in each licensed Unity test, benchmark, or bootstrap workflow validates runner access from ubuntu-latest before any matrix entry attempts to dispatch onto self-hosted; a green preflight confirms the fix.

Preflight diagnostic in this repository

Unity workflows run a runner-preflight job on ubuntu-latest before the self-hosted matrix. The preflight uses the organization reader GitHub App and requests only organization self-hosted-runner read permission. It asks GitHub for runner groups visible to Ambiguous-Interactive/unity-helpers, then considers only runners inside those groups when matching the exact runs-on labels.

The preflight fails closed. Missing reader credentials, an App authentication or API failure, no runner group visible to this repository, malformed or truncated inventory, or no accessible online runner with every required label all make the check red. It never emits a green soft pass. Busy online runners remain eligible because GitHub can queue the licensed job until one becomes idle.

The Unity CI Success aggregate job runs with always() and is the required branch-protection check. It rejects a failed or cancelled preflight and rejects an unexpected skipped licensed job, so a runner outage cannot produce a green Unity check merely because dependent jobs were skipped.

The reader App must be installed for the organization and expose the organization secrets BUILD_LOCK_READER_APP_ID and BUILD_LOCK_READER_APP_PRIVATE_KEY to this repository. Its organization permission is Self-hosted runners: read. No PAT or repository-level environment is required.

The watchdog uses the client-id input of actions/create-github-app-token pinned at bcd2ba49218906704ab6c1aa796996da409d3eb1, while retaining BUILD_LOCK_READER_APP_ID and its existing private key. That revision forwards the identity unchanged into the JWT issuer; GitHub accepts either the application ID or client ID. The retained secret is an application ID, not a newly provisioned client ID. The release workflow uses the same supported input with AUTO_COMMIT_APP_ID. No secret replacement or additional permission is required for this input migration. Missing or expired reader credentials still follow the inventory fallback and unreadable-inventory failure path.

If the preflight passes but the matrix job still stays queued, the cause is more likely the dispatcher bug (see GitHub Community Discussion #186811) than the access list. Use the recovery workflows in this repository: .github/workflows/unstick-run.yml for manual recovery of a single run, and .github/workflows/stuck-job-watchdog.yml for the automated 5-minute scan.

The watchdog's second recovery mode

The same watchdog also recovers a different symptom: a completed run reported cancelled whose jobs all succeeded step by step. GitHub occasionally marks a job cancelled even though every one of its steps, Complete job included, reported success; dependent jobs then skip on the unmet needs: and Unity CI Success reports red with no diff to explain it. A plain re-run of the same run id goes green.

The watchdog detects that signature (a job with conclusion: cancelled whose step list is non-empty and whose every step succeeded) and re-runs the run through POST /repos/{owner}/{repo}/actions/runs/{id}/rerun. A deliberate cancel never matches, because its in-flight step is itself cancelled rather than successful.

Two bounds keep it from looping. The run's run_attempt must be at most MAX_RERUN_ATTEMPT, so a re-run that lands in the same state escalates to a human, and re-runs are capped at MAX_RERUNS_PER_DAY per run id in a state file on the watchdog-state branch. Both appear in the workflow's env: block.

This re-run uses the workflow's own GITHUB_TOKEN, whose job-level actions: write grant already covers it. It does not use the build-lock reader App, which is read-only by design; do not widen that App to enable this.

Machine-name labels for runner bootstrap

The manual .github/workflows/runner-bootstrap.yml workflow performs host maintenance on one specific Windows runner. Each runner must therefore have a custom label that exactly matches its runner name:

  • DAD-MACHINE
  • ELI-MACHINE

The bootstrap job requests self-hosted, Windows, RAM-64GB, and the selected machine-name label. GitHub schedules a self-hosted job only on runners that have every requested label, so this prevents an ELI bootstrap from silently running on DAD, or the reverse. The workflow still keeps a hard runner-identity check as a final guard against label drift.

If a bootstrap dispatch stays queued after selecting a runner, verify the runner is online and that the matching machine-name label is present in Settings -> Actions -> Runners. Do not work around the queue by removing the machine-name label from the workflow; that reintroduces wrong-runner maintenance.

Install a new Windows runner agent

The workflow bootstrap can maintain only an agent that is already online. For a new machine, scripts/unity/install-windows-actions-runner.ps1 separates the work that requires elevation from the package download that should run as the ordinary runner owner. Obtain the current Windows x64 runner version, its published SHA-256, and a short-lived registration token from Settings -> Actions -> Runners -> New self-hosted runner. Never commit or transcribe the token into logs.

First, open Windows PowerShell 5.1 as administrator and prepare the service directory. Replace the account and machine name examples with the actual runner owner and one of the repository's expected runner names:

powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\unity\install-windows-actions-runner.ps1 `
  -Phase AdminPrepare -InstallRoot C:\actions-runner -RunnerUser runner-owner

Then sign in as runner-owner and install the checksum-verified runner package without elevation:

powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\unity\install-windows-actions-runner.ps1 `
  -Phase UserInstall -InstallRoot C:\actions-runner `
  -RunnerVersion <version-from-github> -ArchiveSha256 <sha256-from-github>

Finally, return to an administrator prompt and register the agent as a Windows service. Pass the token only at invocation time; the script deliberately does not print it. The runner-name label is added automatically alongside RAM-64GB, which makes the machine eligible for both Unity jobs and targeted maintenance:

powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\unity\install-windows-actions-runner.ps1 `
  -Phase AdminConfigure -InstallRoot C:\actions-runner `
  -RegistrationUrl https://github.com/Ambiguous-Interactive `
  -RegistrationToken <short-lived-token> -RunnerName DAD-MACHINE

Once the agent reports online, dispatch Runner Bootstrap (Windows) in maintenance mode. That installs the host prerequisites and every Unity editor from .github/unity-versions.json, including all supported build-target components. Host preparation runs once, then each editor runs in its own serialized six-hour job. A slow editor therefore cannot consume the timeout for later versions, and GitHub can rerun only the failed version leg.

Run maintenance directly on a Windows runner

When you are already on the runner host, you do not need to run YAML. From a checkout of this repository, run the same maintenance backend directly:

powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\unity\maintain-windows-runner.ps1

The script reads .github\unity-versions.json when -UnityVersions is omitted and writes diagnostics under .artifacts\runner-bootstrap. Its Unity install-root precedence is explicit -InstallRoot, UNITY_EDITOR_INSTALL_ROOT, the Actions runner tool-cache environment, the runner service's .env tool-cache setting, the Actions default <runner>\_work\_tool\u6-v3, then Unity\Editors on the checkout's drive. A runner whose .env sets RUNNER_TOOL_CACHE=<runner>\_tool therefore uses <runner>\_tool\u6-v3, exactly like its workflow jobs. This keeps large editor payloads off C: when the runner lives on another drive and makes interactive maintenance reuse the same editors as workflow jobs. Its default Full profile installs and verifies Windows IL2CPP, Android (including SDK/NDK/OpenJDK), WebGL, iOS, Linux Mono/IL2CPP, and macOS Mono build support; the workflow uses the same profile. Maintenance installs the standalone Unity CLI when absent and otherwise runs its current self-update command before provisioning. All CLI operations are non-interactive, use built-in download retries, stream progress, and have bounded wall-clock and no-output timeouts. UH_UNITY_CLI_UPDATE_TIMEOUT_SECONDS, UH_UNITY_CLI_INSTALL_STALL_SECONDS, UH_UNITY_CLI_UPDATE_STALL_SECONDS, and UH_UNITY_CLI_UPDATE_RETRY_ATTEMPTS override the install/update defaults for diagnosis.

Unity CLI 1.0.0-beta.9 has a confirmed upstream Windows compatibility defect where an older installation database can fail with table installs has no column named writer_kind. Maintenance recognizes only that exact signature, rolls back to 1.0.0-beta.8, retries the interrupted command, and writes a 12-hour compatibility marker beside the CLI executable so subsequent editor legs do not immediately reinstall the broken build. The hold expires automatically, allowing a fixed newer beta to be adopted without operator cleanup. If rollback fails, maintenance refuses to uninstall, quarantine, or clear payloads because the failure is CLI metadataโ€”not editor corruption.

For an audit that never installs or repairs anything:

powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\unity\maintain-windows-runner.ps1 -DetectOnly

UH_RUNNER_DISABLE_AUTO_BOOTSTRAP=1 also forces detect-only mode for both maintain-windows-runner.ps1 and bootstrap-windows-runner.ps1, matching the workflow override.

PowerShell 7 prerequisite on self-hosted runners

Self-hosted Windows Unity runners require PowerShell 7 (pwsh) in addition to Git Bash. Every Unity workflow consumes the print-self-hosted-runner-diagnostics composite action (.github/actions/print-self-hosted-runner-diagnostics/action.yml) before its own steps, and that action plus the Unity run/provision steps run with shell: pwsh. PowerShell 7 is not the Windows-built-in PowerShell 5.1 (powershell); it is a separate install that provides the pwsh executable.

Symptom

  • A self-hosted Unity job fails almost immediately with ##[error]pwsh: command not found.
  • The failure originates from the first shell: pwsh step the agent reaches.
  • Git Bash and the runner agent are otherwise healthy.

The diagnostics composite action fails fast with a clear, actionable error annotation (pwsh missing on self-hosted runner) when pwsh is absent, so this state no longer surfaces only as the cryptic pwsh: command not found. The preflight step that emits that error runs under Windows PowerShell 5.1, which is always present, so it executes even when PowerShell 7 is missing.

Install PowerShell 7

On a machine with winget:

winget install --id Microsoft.PowerShell --source winget

For machines without winget, download and run the latest MSI installer from the official releases page: https://github.com/PowerShell/PowerShell/releases.

Verify

Open a new shell (so the updated PATH is picked up) and confirm:

pwsh -v
Get-Command pwsh

pwsh -v should print the installed PowerShell 7 version, and Get-Command pwsh should resolve to the installed executable's path.

Restart the runner agent

After installing PowerShell 7, restart the self-hosted runner service/agent (or refresh the machine's PATH and restart the runner) so the agent process sees pwsh on its PATH. The runner agent inherits its environment at start time; until it is restarted it keeps reporting pwsh: command not found even though a fresh interactive shell can find pwsh. Re-run the queued Unity workflow once the agent is back online.

The persistent Unity project root

Each Unity leg generates a throwaway Unity project whose Library holds the import database and the compiled assemblies. That project lives outside the checkout, at:

<RUNNER_WORKSPACE>\unity-workspace\projects\<unity-version>-<mode>[-<scope>]
<RUNNER_WORKSPACE>\unity-workspace\cache\<unity-version>

The Unity test workflow runs EditMode and PlayMode serially with -ShareEditorProject, so both use the existing <unity-version>-editmode[-<scope>] project. The single-threaded scope keeps its different scripting defines separate from the default editor project. Standalone uses <unity-version>-standalone[-<scope>] because its player build settings differ. Each mode still writes its own logs and results.xml under .artifacts/unity/. If EditMode fails, PlayMode uses its separate <unity-version>-playmode[-<scope>] project so the second result remains independent of assets left by the failed run.

EditMode runs compiler-isolation fixtures in a fresh editor process before the broader suite. These cases carry CompilationIsolation and must all pass without skips. Their results and log remain under the mode's compilation/ artifact directory. The source API, backup recovery, import-verification, source/data apply, and recovery-record/scene fixtures each run in a separate fresh editor process with MigrationIsolation; their artifacts remain under migration/<fixture-name>/. The broader process excludes both categories, so each case runs once. All processes share the installed editor, project and acquired license context. A failed isolated phase prevents later phases and fails the mode. Verification requires passing cases from every named isolated fixture. Earlier ruleset-import tests can leave pending compilation and EditorApplication.isUpdating that cannot drain during the same batch run; each migration fixture needs an idle editor to test its real production guard without weakening it. Recovery-record scene controls also need a clean initial scene; ordinary broad-suite editor tests can leave an untitled scene dirty. The isolated process preserves this precondition without discarding authoring changes or skipping those controls. Verification requires the mode's root results.xml independently of its isolated fixture XML. A passing nested result cannot replace a missing broad-suite result. Failure dumps include root and nested editor logs.

It has to. actions/checkout runs git clean -ffdx at the top of every job and -x means gitignored, so anything under the repository's .artifacts/ tree is deleted before the job starts, including the Library the previous leg had just built on that same disk. Keeping the project one directory up puts it out of git clean's reach, so a leg reuses its own local copy instead of downloading one.

A nonzero editor exit and an accepted result artifact are separate observations. The runner reports the actual exit code, accepted NUnit counts and root result, and captured editor completion text. Known native status codes retain their descriptions; ordinary exits remain unresolved exit/artifact discrepancies. A crash marker does not establish shutdown timing. Inconclusive cases remain inconclusive, and missing, malformed, empty or failed results still fail their existing gates.

A missing configure marker also attempts to save configure.process-diagnostics.json beside the configure log. It records the child process identity, UTC launch and observation boundaries, raw and returned exits, and watchdog termination provenance. On Windows, it queries Application Error and Windows Error Reporting events once, from two seconds before launch through the earlier of capture time or fifteen seconds after the process observation ended. It examines at most 64 candidates and retains at most eight matches, with explicit truncation flags. Events require the child's executable path and process ID, or a report ID joined to an event with that identity. Unrelated events and full event messages are excluded. The report distinguishes unavailable evidence and no observed matches; a missing or delayed event does not rule out a crash. The existing Package Manager cancellation retry also attempts to preserve its first attempt in configure.first-attempt.process-diagnostics.json.

Configure and in-editor test invocations also attempt to save unity.execution-diagnostics.json beside its log, including an invocation that exits zero or later supplies passing NUnit XML. The receipt uses the same bounded Windows event matcher and records process identity, raw and returned exits, watchdog provenance, and UTC capture time. Its native-log summary counts Native Crash Reporting headers and Got a UNKNOWN while executing native code lines separately, retaining at most eight line numbers for each. These are two observations of a report and must not be added to estimate an event count. Complete counters and a truncation flag accompany the bounded locations; missing or unreadable logs have explicit states. The watchdog records whether it saved the current log; failed or unobserved persistence leaves native-log evidence unavailable, so an older readable file cannot certify a new attempt. Raw log lines and full Windows event messages are excluded from the receipt, and credential redaction still runs before writing it. Before a Package Manager retry overwrites the log and receipt, both first-attempt files are preserved with .first-attempt in their names.

On Windows, these invocations also receive an isolated -crash-report-folder and a unique -upmLogFile. Both write outside uploaded artifacts. The receipt's Process.NativeEvidence links the raw Package Manager log, its redacted text snapshot, and the private crash directory to that process observation. Each attempt has a new capture ID, so a retry cannot reuse the preceding attempt's Package Manager log or crash files. Late Package Manager output cannot append credentials to the published snapshot. This uses Unity's documented crash-folder override and Package Manager log option.

Raw crash files remain in the runner account's temporary directory under unity-helpers-native, outside uploaded artifacts. The inventory records up to 64 files within two directory levels, their observed lengths and times, and SHA-256 hashes for files no larger than 256 MiB. Readers permit active writers. Hashes cover at most the initially observed length, report the number of bytes hashed, and flag length changes; a live-file hash is a bounded observation. Only .log and .txt files are copied to artifacts, with credentials redacted and a 262,144-character limit per file. File-count limits, text truncation, unavailable hashes, failed setup, and missing files are explicit. Review the recorded raw directory on the named runner when a dump is needed for native stack and symbol analysis; remove it after preserving the required evidence. Capture observes files available at that time, so a delayed crash writer may finish afterward. An empty folder does not establish that the process was safe.

These receipts add evidence; they do not establish a native report's cause or change the existing result gates. A receipt write failure warns without replacing the original process outcome.

Use those observations to investigate the original process on the named runner before selecting a repair. A successful later launch does not explain an earlier failure, and the diagnostic report does not replace the fresh configure marker or required player results.

Opt-in native debugger capture

For a recurring Unity 6000.6 access violation, dispatch Unity Tests with unity-version: 6000.6.0f1, test-mode: editmode, and native-diagnostics: true. The workflow retains its static version matrix: the version input does not restrict that matrix. Regular jobs select EditMode; the separate SINGLE_THREADED job still runs all three modes. After the regular EditMode step succeeds, the licensed runner executes one existing migration test directly and then under Microsoft's CDB debugger in the same prepared project. The selected case, editor hash, machine, process IDs, UTC boundaries, actual process exits, target exit events, XML outcomes, and watchdog actions are recorded separately. These are diagnostic observations; they cannot satisfy acceptance or explain a different earlier failure. Normal tests, license return, lock release, and cleanup checks still run.

The driver downloads only twelve x64 debugger files from a versioned Microsoft bundle using bounded HTTP ranges. Each file must match its committed size and SHA-256 digest, and CDB must have a valid Microsoft signature. It does not register a system debugger or install tools globally. CDB captures second-chance access violations with exception details, registers, stack, module list, and a private minidump. It then continues the exception as unhandled. A separate breakpoint records requests to exit; the target's exit event remains distinct from the debugger's exit code. That breakpoint also runs read-only !gflag to observe the target's effective global flags. Control programs must preserve normal exit zero and access-violation exit 0xC0000005 before interpreting a new debugger configuration.

Runner startup diagnostics and the capture receipt read the 64-bit local-machine GlobalFlag settings for Unity.exe under Image File Execution Options and for the system under Session Manager. Each read reports observed, absent, unavailable, or invalid. Only observed numeric values report whether the loader-snaps bit is set; absent or unreadable settings do not establish the target's effective flags. These observations do not change the registry or enable debugging. Loader tracing in debugger output is a lead to investigate, not proof of a Unity fault's cause.

Raw dumps and logs remain under the runner account's local application data in unity-helpers-native-debugger/<capture-id>, whose exact path is in the receipt. Only the receipt and redacted console, editor, and Package Manager tails enter the EditMode artifacts under native-diagnostics/. Console retention is limited to 524,288 characters; each file tail reads at most 524,288 bytes of its initially observed file. Dump hashes are attempted only up to 256 MiB. Full debugger console output is not retained beyond its tail. A missing dump or exit event is explicit. Retrieve available private files on the named runner for further analysis, then remove them after preserving the required evidence. The command can also be run locally with -DebuggerDirectory pointing to the same verified files and -TestFilter selecting a single existing case. Each artifact path must be new.

Tool files and per-process capture directories occupy separate paths. Missing, incomplete, or unexpected NUnit XML is recorded as diagnostic data after log tails and dump metadata are saved; it does not interrupt fault-evidence collection or count as test acceptance.

Operator notes:

  • Confirm reuse. Every run logs Library: warm (reused) or Library: cold (first run on this runner) in the Ephemeral Unity project group. A leg that reports cold on every run means the root is not surviving between jobs; check whether something is wiping RUNNER_WORKSPACE.
  • Disk is the trade. This buys back cache transfer time by keeping the projects on local disk. run-ci-tests.ps1 prunes least-recently-used sibling projects when free space drops below its floor (-ProjectRootMinimumFreeGb, default 60 GB) and never prunes the leg that is about to run. The runner diagnostics block prints Workspace volume: <drive> N GB free of M GB every run.
  • Force a cold rebuild for one leg by deleting its directory under unity-workspace\projects; the next run regenerates it.
  • A leg that moves between the two runners is handled automatically: Clear-StaleUnityCompilationCache notices the repo-root marker changed and clears Library\Bee, ScriptAssemblies, PlayerScriptAssemblies, and Il2cppBuildCache.

Git compression tools for Actions cache

Self-hosted Windows Unity runners also need Git for Windows' Unix tools available to GitHub Actions cache steps. actions/cache restores and saves archives through tar and gzip; when the runner PATH exposes Git Bash but omits C:\Program Files\Git\usr\bin, cache post steps can warn with gzip: command not found and fail to save a cache entry. The Unity Library is no longer among them (see the section above), but the prepend stays because it is cheap and any future cache step on these runners needs it.

The print-self-hosted-runner-diagnostics composite action prepends Git's usr\bin directory to $GITHUB_PATH when it finds both gzip.exe and tar.exe, and emits a warning when that directory is absent. To verify locally on the runner:

Get-Command gzip.exe
Get-Command tar.exe

If either command is missing, install Git for Windows or add C:\Program Files\Git\usr\bin to the runner service PATH, then restart the runner agent.

Never use plain shell: bash on self-hosted Windows runners

On a self-hosted Windows runner, shell: bash can resolve to the WSL stub at C:\Windows\System32\bash.exe, which tries to launch a WSL distro that is usually not installed and fails with "Windows Subsystem for Linux has no installed distributions." The diagnostics composite warns when the runner PATH resolves bash to that stub (Git Bash must precede System32 in PATH). Unity workflow steps therefore use shell: pwsh (or shell: powershell for steps that must run before PowerShell 7 is installed) rather than shell: bash.

Windows host prerequisites (0xC0000135 / STATUS_DLL_NOT_FOUND)

If Unity.exe fails at startup with -1073741515 / 0xC0000135 (STATUS_DLL_NOT_FOUND), the host is missing an OS-level dependency Unity imports, most commonly the Microsoft Visual C++ Redistributables (both the 2010 SP1 and the 2015-2022 x64 generations). This is an OS-level fix; ensure-editor.ps1's Unity-reinstall retry loop cannot repair it (the missing DLL is on the OS, not in the Unity install). ensure-editor.ps1 detects this case and short-circuits with a clear error rather than retrying futilely.

The manual workflow_dispatch workflow .github/workflows/runner-bootstrap.yml is the supported remote remediation path for this state. Dispatch it with the affected machine-name label (DAD-MACHINE or ELI-MACHINE) and leave detect-only disabled to run host maintenance. The workflow calls scripts/unity/maintain-windows-runner.ps1, which first runs scripts/unity/bootstrap-windows-runner.ps1 for OS prerequisites and then verifies every Unity editor listed in .github/unity-versions.json through ensure-editor.ps1. If you are logged into the runner host directly, run scripts\unity\maintain-windows-runner.ps1 from the checkout instead.

For emergency manual repair, apply the same host prerequisites that the bootstrap backend manages:

  1. Install the Microsoft Visual C++ 2010 SP1 x64 Redistributable (provides MSVCP100.dll / MSVCR100.dll).
  2. Install the Microsoft Visual C++ 2015-2022 x64 Redistributable (provides VCRUNTIME140.dll, VCRUNTIME140_1.dll, MSVCP140.dll).
  3. Enable Windows long paths (git config --system core.longpaths true and the LongPathsEnabled registry value).
  4. Add Windows Defender exclusions for the Unity install root and the runner work directory to avoid scan-induced timeouts.
  5. Install PowerShell 7 (see above).

Re-run the queued Unity workflow once the bootstrap run completes successfully or the host is prepared manually. If the bootstrap dispatch stays queued, first verify that the target runner is online and carries the matching machine-name label; do not remove that label from runner-bootstrap.yml to force scheduling.

Required secrets

The Unity workflows expect the following repository (or organization) secrets. They are NOT provisioned by this batch; a maintainer must add them before the first self-hosted run:

  • UNITY_SERIAL, UNITY_EMAIL, UNITY_PASSWORD: classic serial Unity activation (all three required together).
  • BUILD_LOCK_APP_ID, BUILD_LOCK_APP_PRIVATE_KEY: dedicated GitHub App credentials for the wallstop-organization-builds organization build lock (Ambiguous-Interactive/ambiguous-organization-build-lock); both are required together and should be provisioned as organization secrets with access to this repository.
  • BUILD_LOCK_READER_APP_ID, BUILD_LOCK_READER_APP_PRIVATE_KEY: read-only GitHub App credentials used by the hosted runner preflight; both are required together and should be provisioned as organization secrets with access to this repository.
  • UNITY_ACCELERATOR_ENDPOINT: optional; enables the Unity Accelerator cache namespace when set.

Provision the required Unity and build-lock credentials as organization secrets selected for this repository. The licensed workflows intentionally do not bind jobs to a per-repository environment, so trusted pull requests from branches in this repository validate automatically without an environment approval. Pull requests from forks remain ineligible for licensed jobs and do not receive these secrets.

Focused native acceptance

Manually dispatch Unity Tests with acceptance set to sentinel, intmap, or all. The default is none. The selected normal test modes still run; acceptance adds work under the same organization license lease. unity-version can select a supported editor to bound the run. Artifacts named unity-<version>-acceptance contain separate result directories and a summary. Each requested test, verification, credential redaction and upload must succeed. These optional runs are separate from the benchmark dispatch and release export workflows.

For fresh IL2CPP conversion evidence, set fresh-conversion=true on a manual Unity Tests dispatch. The default is false, preserving normal warm CI. This clears only Bee, ScriptAssemblies, PlayerScriptAssemblies, and Il2cppBuildCache in each selected Standalone project before configuration, inside the existing organization license lease. Regular, SINGLE_THREADED, and WProto-only Standalone builds receive the request; EditMode and PlayMode keep their caches. Package and import caches remain available. The script records the explicit fresh-compilation reason in the runner job log; acceptance still requires the actual converter invocation, source bindings, and passing native results. Direct script callers can pass -ForceFreshCompilationCache only with Standalone IL2CPP mode.

The runner creates fresh projects and removes them after results are copied outside the project. If an editor remains active, cleanup retains the project and fails with its path for diagnosis. There are no automatic measurement retries. An inconclusive result is evidence to inspect, not a reason to repeat runs until a favorable number appears.

run-ci-tests.ps1 -TestFilter 'Namespace.Fixture.Method' forwards Unity's test-name filter in EditMode, PlayMode and standalone builds. It preserves the assembly and category filters. The script no longer resolves an editor itself: set UNITY_EDITOR_PATH to a CI-managed editor first (in CI the central ensure-unity-editor gate publishes it; locally run the repository's ensure-editor.ps1 helper with -CiManagedOnly -RequireHealthyExisting). Without it the script fails closed. A filtered run requires at least one passing test: zero matches, all skipped, and entirely inconclusive results cannot establish acceptance. The NUnit XML and logs remain available when this check fails, including an IntMap measurement rejected for unstable timing.

The explicit ValidationWorkspaceInteractionTests.NativePanelCallbacksRetainDraftAndPersistSettings fixture exercises native field-change and button-submit callbacks. It requires batchmode and a fresh, expendable project whose directory name begins sentinel-interaction-. Before starting Unity, set WALLSTOP_SENTINEL_INTERACTION_PROJECT to that exact absolute project root, set WALLSTOP_SENTINEL_INTERACTION_TOKEN to a new GUID in 32-character N format, and write the same token to .sentinel-interaction-disposable in the project root. Select the full test name in namespace WallstopStudios.UnityHelpers.Tests.Editor.Validation, using its validation test assembly and a fresh -ProjectPath. The fixture checks every marker before changing settings, Undo or native objects.

An exclusive CI checkout with no other active editor may supply the package through the runner's normal file: dependency. A second package copy is unnecessary in that case: this fixture writes only its disposable project's settings and creates objects it destroys afterward. A local shared checkout requires an isolated source copy. The existing Library provenance markers do not authorize this test in a developer project, and batchmode alone is insufficient.

Retain the NUnit XML and editor log, then discard the marked project. The fixture restores its recorded settings and Undo group as cleanup, but it does not promise to preserve arbitrary preexisting Undo/redo history or global subscribers. A pass proves callback delivery, draft retention and saved settings; it does not prove pixels, mouse hit testing, focus, graph dragging or domain-reload recovery.

Licensed benchmarks

Unity Benchmarks accepts manual dispatch. The hosted weekly scheduler dispatches the same workflow on protected main; both workflows must be enabled in repository settings. The benchmark job runs the Performance and Random assemblies in PlayMode on the latest supported editor. It acquires the organization lock, returns the license, classifies cleanup, and releases the lock before reporting and artifact uploads. Hosted fallback and an always-reporting aggregate reject incomplete cleanup or missing execution.

The benchmark lane streams each test start and completion. Its no-output guard allows 35 minutes of silence because individual performance cases allow up to 30 minutes. The overall Unity pass remains bounded at 110 minutes, with cleanup time reserved before the step limit. Compatibility tests retain their shorter default silence guard.

Superseded runs cancel within their ref group. Unity Tests uses separate groups for distinct pushed commits, preserving the exact validation required by automatic release publication. The central enrollment audit requires literal cancellation settings for the licensed queues. A benchmark branch dispatch records diagnostic results without replacing the canonical baseline; only complete successful measurements on main can refresh the published report.

The metric extractor retains dataset and operation labels for repeated tables. It accepts Markdown headers followed by HTML data rows, as emitted by the sorting benchmarks, and keeps each measured cell distinct. Duplicate identities still prevent publication.

Package exports

Release Publish creates the .unitypackage on an ubuntu-latest runner without opening Unity or using a license. scripts/unity/create-unitypackage.js stages the validated npm payload, renames Samples~ to Samples, preserves every tracked asset GUID, and writes the Unity archive plus its SHA256 sidecar. Generated package-root and renamed-sample folder GUIDs are stable hashes of their paths instead of fresh values from each temporary Unity project.

The Unity Tests smoke job remains on the self-hosted, Windows, RAM-64GB fleet. It imports and compiles the same staged Assets payload with release optimization before using Unity's native export API. This retains sample compilation and native import/export coverage. Release Publish requires the latest Unity Tests run attempt at the selected source commit to have completed this native smoke job, including successful acquire, export, return, classification, lock release, and confirmed-cleanup steps in order. Missing, skipped, stale, or failed evidence prevents tag mutation and publication. The smoke job acquires, activates, returns, classifies, and releases in the same physical runner context, with a final cleanup gate.

The release gate script comes from the workflow commit. Its evidence query remains bound to the selected source SHA, allowing an older validated source to use the current release controls.

The shell exporter remains available for local Docker comparison and delegates staging to the same Node implementation. Compare the portable and native archives by logical path, asset bytes, and metadata; Unity may rewrite text assets while importing them, so archive-byte equality is not a valid compatibility check.

For a release-export canary, dispatch Release Publish with export_only: true, the package's current version, and the candidate commit as source_ref. It validates metadata and produces both package artifacts without waiting for a Unity runner, then skips tag mutation and publication. The selected source SHA is retained in the job outputs so the canary can be compared with the smoke run on the same revision.

โš ๏ธ **GitHub.com Fallback** โš ๏ธ