Segments Silhouettes and Skeletons - prnvkhndlwl/Kinetrace GitHub Wiki
Point trackers hold textured features (an eye, a snout, a marker) but lose thin undulating tails and dark textureless bodies — measured on real 4K footage. The app therefore tracks the animal as a whole too:
- Animals are layers. The right panel's LAYERS tree lists each animal (an individual animal or object) with its points underneath, then Scene for the points that belong to no animal (wand ends, reference markers on a rock, calibration points). ＋ Animal makes an animal with no silhouette — the silhouette is optional — and ＋ Point a point in the selected animal. A point of an animal is named "<animal> <part>" everywhere (exports, 3D, the other cameras), e.g. squirrel snout; the tree shows just snout. Renaming an animal renames its points in every camera. Names that differ only in capitals are one name: head beside Head becomes head (2) (cameras join points by name). A new point (N + click) joins the selected animal, else the animal whose silhouette is under the click, else the selected point's animal, else Scene. Drag points onto another animal or onto Scene to move them (or right-click → Move to ▸; a silhouette landmark needs its animal and does not go to Scene). Track tracks exactly what is selected: an animal row = its silhouette only (and the landmarks derived from it); Ctrl+click its points to track them with it. Delete asks whether an animal's points move to Scene or go with it.
- One click defines the animal. SAM 3 (or SAM 2.1) segments it on that frame and, during tracking, on every frame — following it through occlusion with a memory of what it looks like. S outlines the animal selected in LAYERS (the first click makes one when there is none; with several animals, select one first). The crop that the point tracker works in follows the silhouette, and extra hidden support points sampled inside the silhouette anchor the joint tracking to the body.
- Silhouette landmarks are geometry, not appearance: the body midline (anchored at your head landmark), the tail tip (end of the midline), points at 25/50/75 % of body length, the centroid, and extremities (the farthest protrusions on each side — feet, wing or patagium tips). They are ordinary points in the export. Left and right are named as seen from above (a dorsal view): filmed from below, the animal's left and right swap, so rename those landmarks (or swap their rules). The same swap happens on the frames where an animal filmed from above rolls onto its back; the program cannot tell a roll from a turn, so check such stretches by eye. Exports made before 2026-09-22 from dorsal footage had them the wrong way round.
- When the body touches itself (the tail over the flank, a foot on an upturned tail, a fish's closed C-curl) the outline closes a loop and the midline can cut through the contact: tail tip, midline points and feet are then wrong for as long as it lasts. The program detects it (an enclosed gap in the outline AND a midline clearly shorter than the last one without such a gap) and marks those landmarks 30 % confident from there until the outline encloses no gap at all (red on the timeline, J stops there). Positions are not changed, the centroid is not affected, and a foot simply resting on the body (gap, but the same length) is not marked. A run that starts in contact is marked until the contact ends.
- Skeletons give every landmark a fixed name so exports line up across
videos and, later, across cameras. Built-in templates cover lizards, generic
quadrupeds, gliding mammals, flying lizards, undulating bodies and birds/bats;
Custom skeleton… saves yours as JSON in
skeletons/. A skeleton belongs to one animal: Skeleton ▾ puts it on the animal selected in LAYERS (asks when several or none are; makes an animal when there is none), so two animals can share a template ("squirrel snout", "chipmunk snout"). Select two points of one animal → right-click → Connect them with a bone; a point's menu → Use it as <animal>'s head sets the midline's anchor; the animal's menu → Save its points and bones as a skeleton template… makes it reusable from Skeleton ▾. Deleting a point removes its bones. Right-click any point → Data source to switch it between appearance tracking and a silhouette rule (it asks before erasing the point's existing track; Ctrl+Z brings it back). - Points can be kept on the animal. With the animal's Keep its points on its silhouette ticked (right-click the animal; off by default), a tracked point the model puts just outside the silhouette (within about 3 % of the silhouette's diagonal, at least 8 px) is nudged back onto it — that is outline jitter. A landmark that goes further has left the animal: the run stops at that frame, its track ends on the frame before, and the app takes you there. Pulling it back onto the nearest silhouette pixel would put it on some other spot of the body and carry on as if nothing had happened. Click it where it really is and press Track. Exempt one point of the animal with right-click → May leave its silhouette (free point); a mark that lives elsewhere (a ground marker) belongs in Scene, whose points are never held. The outline it holds to is the one tracked in the same run (the animal's row selected too) or, for points selected alone, the silhouette saved by an earlier run — free on frames that have none. Selecting a point never starts the silhouette program on its own.
- Corrections are clicks. Pause, go to the frame, press S, click (or Shift+click), press Track: the correction applies from that frame on. A landmark that leaves the silhouette stops the run (above); one that sits off the body at low confidence turns red on the timeline; and with Auto-pause on the run also stops when a point stays unreliable, or when the animal itself is lost or leaves the frame.
- Segment ▾ picks the segmentation model right on the button — the same way
Track ▾ picks the point model — and each entry says whether its weights are
ready, need downloading (with the size), or are gated behind a token. SAM 3 is
the default when its weights are in
models\sam3(best masks, ~13 fps at 4K); SAM 2.1 base+ is the fast fallback (~27 fps). Robustness was chosen over speed by design. Settings (Ctrl+,) holds the same choice plus the Hugging Face token and the mask opacity; the last menu entry opens it. Every model also runs without a GPU (on the CPU, or a Mac's Metal GPU), several times slower — there SAM 2.1 base+ is the practical choice, and Help → System Check… says what this computer runs.
A step-by-step guide for new users is in docs/MANUAL.md (also in the app: F1).