Testing - prnvkhndlwl/Kinetrace GitHub Wiki
make_test_video.py generates synthetic videos with known ground truth and
validates exports against it:
python make_test_video.py demo.mp4 --frames 600 --size 1920x1080 --dots 4
python make_test_video.py demo.mp4 --check exported_tracks.csv
For a first look, make the demo clip with
python make_test_video.py test600.mp4 --seed 0 (inside .venv), open it,
place the four dots (press N, click a dot; once per dot), press Track, and
watch.
The verification suites live in tests/ (run them with the environment's
Python: .venv\Scripts\python.exe on Windows, .venv/bin/python on Linux and
macOS — tests/verify_<name>.py; each prints PASSED and exits non-zero on
failure). tests/run_suites.py --cpu|--gpu runs a whole group and generates
the synthetic test videos it needs first; --affected runs only the suites
for the files you changed (the table is tests/suite_map.py), and --stamp
prints which suites passed on exactly the current commit, for a pull request.
Every pull request also runs the suites that need no GPU on Ubuntu and
Windows on GitHub (the PR check), which must pass before it is merged.
Ball markers have
verify_balls.py and the automatic re-track verify_retrack.py (both GPU);
verify_sweep_fixes.py pins the fixes of the 2026-09-22 release sweep —
camera timing and exports, undo and keyframes, the multi-camera, canvas and
timeline behaviour (offscreen, no GPU, part of --cpu). The error log has
verify_crashlog.py (no GPU): an error in a menu entry, a key handler, a
background thread and a Python thread is logged with its context, the notice is
rate-limited, Help → Error Report shows and copies it, the log rotates, a native
crash and Qt's fatal message are recorded, an unwritable folder changes nothing,
and the real main() opens and closes a logged session. Updating has
verify_update.py (no GPU, no internet — a local server stands in for
GitHub): the version check and its errors in words, a ZIP install replacing
only changed files and never the user's folders, refusing unsafe or foreign
archives, a git checkout moved exactly to the release tag, the one-line
launchers, the release workflow reading the version, and Help → About /
Check for Updates driven by real clicks (also: a download link off GitHub is
refused, Restart now really starts the launcher). Model downloads have
verify_downloads.py (no GPU, no internet — a local server): every model in
use is the pinned one, progress in bytes, a wrong checksum deleted and never
used, a cut-off download resuming, Cancel between chunks, a stalled server
ending in a sentence within the timeout, a code archive with an unsafe path or
another version refused, a checkpoint that would run code refused, and the
Track dialog naming the right model with MB and time left and a working
Cancel. The Moving spot point model has verify_spots.py (no GPU): synthetic
clips with exact ground truth (a flapping spot over water with glints, a dark
bird against the sky, a bat among rocks as dark as it) followed by the right
cue to a fraction of a pixel, its stops (vanished, two crossing, leaving the
picture), the test's 20-click rule, the test recommending the right cue with 0
corrections, and through the app: the Track ▾ choice saved with the project,
a run that stops where the spot vanishes, the test dialog, the hints, the
manual's section, a semi-automatic step and an every-camera run. The project folder has
verify_projectfile.py (exact round trips from the CSVs and from the cache,
a save writing only what changed, a hand-edited CSV winning over the cache, a
save cut short by a crash undone on the next open, going back one save, files
written by hand or on another operating system, speed budgets, and a project
made by someone else that tries to delete or write files outside its folder
through its .history / exports / project id — refused) and saving /
unsaved-work recovery has verify_recovery.py (through the app, offscreen:
also an older single-file project turned into a folder, Save As guards, Export
Project as One File and the exports kept up to date on save); files from and for other programs (DeepLabCut, SLEAP, DLTdv / Argus, Anipose, OpenCV, MATLAB, Blender) and the command-line converter have verify_interop.py (hand-written files as each program writes them, exact round trips, calibrations checked to a thousandth of a pixel). Portability has
verify_portable.py (no GPU): the one device rule and the hardware report,
SAM 3D Body greyed out without CUDA, --check, the installer's choices, the
launchers' and CI's shared install marker and bootstrap URLs, and AllTracker
tracking on the CPU. The GitHub workflow Install check (started by hand from
the Actions tab) installs from scratch on Ubuntu 22.04 / 24.04, macOS 14 and
Windows Server 2022 through the launchers' own --check mode — once with a
Python on the machine and once with every Python hidden, so the launcher has
to fetch its private copy — then runs the CPU suites and
verify_decode_identity.py: small clips that stress what differs between FFmpeg
builds (B-frames behind an edit list, a late start, a dropped frame), generated
with imageio-ffmpeg and decoded with that OS's OpenCV, must show the same
picture at each frame number as the Windows fingerprints committed in
tests/data/decode_identity. The frame check itself has verify_fingerprint.py
(no Qt: a shifted start found as −1 / +1, a 1-px-per-frame high-speed clip, a
still clip's "cannot tell" with its bound, a capture whose seeks land a frame
late read exactly, a real dropped-frame file) and verify_fingerprint_gui.py
(through the app: the evidence window after Open Project, exact frames typed
into the frame box, the still-video fallback). Two audit tools are
not suites: tests\audit_sweep.py drives every enabled menu entry, hotkey and
context-menu entry in every state offscreen and must end with 0 exceptions;
tests/audit_gui_real.py drives the app with real mouse clicks and key
presses on the real desktop, screenshots every window, dialog, wizard
page and menu into tests\out\gui_real\, and checks that nothing raises,
menus show their tooltips, hotkeys work wherever the focus is, the view never
moves on its own, a pause lands within a second and nothing is clipped at
1366×768 (--no-gpu skips the segment and tracking states).
The 3D layer has two of its own: verify_3d.py (cameras, triangulation,
undistortion, sub-frame sync, visual hull — synthetic ground truth, no GPU)
and verify_3d_gui.py (the 3D menu end to end, offscreen). The body layer has
verify_body.py (joint angles against a synthetic walker built from known
angles, storage, exports, the backend registry, the drawing — no GPU, no
weights) and verify_body_gui.py (the real pose model through the app, from
the run dialog to the exported video).