Decision making - KasperSkytte/slurmjobpacker GitHub Wiki
Free space counts only as far as typical jobs can use it. For a node with fc free CPUs
and fm MB free memory:
phi = Σ over the demand mix of weight × min(fc, fm / mem_per_cpu)
The demand mix ([policy] demand) is a weighted list of typical memory-per-CPU requests.
Fit your own with python3 -m sjp.fit --since 2026-01-01. Placing a job costs the phi
it destroys.
sjpd precomputes a table: jobs bucketed by memory per CPU, CPUs and time limit
([policy] buckets); each partition scored by its best node with room. Allowed: the
cheapest partition plus any within tolerance (0.25 CPU per job CPU). No room anywhere:
every partition big enough. Slurm then tries them in PriorityTier order.
Idle cluster → jobs go where their ratio fits. Busy cluster → costs converge, so everything big enough is allowed. Time limit is a bucket key but doesn't affect scores yet.
Treated as planned jobs: whole nodes or the listed cores (× threads), memory at the node's memory per CPU.
- Active: counted as allocated (less what the reservation's own jobs use), everywhere.
- Upcoming: counted against jobs whose time limit would reach them (all of them for jobs without a limit), in the room check, pins and flex moves.
sjp.place():
- skips interactive, GPU, reservation,
--constraint, multi-node and--mem=0jobs; - looks the job up and drops partitions too small for it; for a job asking for nodes
(
--nodelist,--exclude), considers only those nodes, and partitions holding them (each holding every--nodelistnode), or with[policy] user_nodes = "ignore"drops the request; - if no node in them has the CPUs and memory free now: returns
false, "no room", or withalways_placeuses the partitions the job's shape fits best on an empty cluster (markedroom=no); - sets partitions, and the node for a pin;
- writes
AdminComment, returnstrue. A job it skips getssjp:skipped=<reason>.
Table older than [cadence] policy_max_age (300 s) or node data older than
[pin] max_age (10 s): false, "stale".
For a job that can start now, sjp sets --nodelist:
- only single-node jobs with an explicit memory request; not
--nodelist(the user chose, unlessuser_nodes = "ignore"),--gres,--exclusive, held, future--begin, or several tasks without-N 1; not when the user is at the QOS CPU cap; - not an array at submission: one node list would hold all its tasks on one node. The array is placed by one task's shape, and each task waiting is then a job of its own;
- within the highest-
PriorityTierpartition with room; - among nodes within
min_gain(1 CPU of phi) of the best, the closest memory per CPU wins; - no pin if only one node fits, or all are about equal (
min_gain,min_ratio_gain).
Pins not started within release_after (60 s) are removed and partitions restored.
Pending jobs tied to one node (pins or --nodelist) count as taken there, so a burst of
submissions isn't pinned to the same free space.
[pin] time_aware = true adds to the node choice:
time_cost = max(0, log(job time limit / time until the node's jobs end))
Empty nodes count as busy for 1 minute, so jobs join busy nodes and long jobs gather.
The plugin also counts the end times of its own pins for 30 s, since sjpd reads the
queue only every 8 s.
With [pin] wait_for_room, a job no node has room for is pinned to the node expected to
have room for it first: its free space, less the jobs already waiting there, plus what its
running jobs release by their time limits. Ties: higher PriorityTier, then name. Marked
eta=. Not released after release_after; the recheck moves it, and it is released once
the job is past its [starvation] budget (then widened as usual).
Jobs that start, end or are cancelled change what fits where, so a choice made at
submission goes stale. Every minute ([recheck] interval) sjp chooses again for each pending
job it placed (None, Resources, Priority or waiting on its pinned node) and moves it
if the choice changed and a node there has room, dropping any wait pin. A job that still
cannot start is re-pinned to the node now expected to have room first (wait_for_room).
Highest priority first, only as many as fit, so a job whose priority you raise is moved
first. Jobs the plugin skipped for want of room (skipped=no room, or stale) are taken
over the same way once room opens: your rules placed them only as a fallback. Not jobs
skipped for other reasons, nor jobs on the user's own --nodelist. Each waiting array task
is rechecked, moved and pinned on its own (Slurm then gives it a job record of its own);
a waiting job stays on its node unless room is expected sooner elsewhere.
Jobs sjp placed that pend for Resources or Priority beyond their budget (1 h at
≥ 6000 MB/CPU, 4 h otherwise, from eligibility) are widened once to every partition big
enough ([starvation] enabled); each array task on its own. Not GPU, pinned or
multi-node jobs, nor jobs your own rules placed.