Plans and policy - FullmetalBober/Indexterity GitHub Wiki
What a cluster owner can configure, what each plan entitles them to, and why the defaults are where they are. The engine that acts on all of it is described in Architecture; this page is the surface a customer touches.
| knob | effect | default |
|---|---|---|
readOnly |
compute everything, never write | on |
workloadAnalysis |
the create/merge/update engine; turn it OFF to silence the create side | on |
instantCreate |
build critical missing indexes without approval | off |
observeWindowDays |
baseline bake time for a hidden index; scaled per index (below) | 30 |
maxCollectionSizeBytes |
size ceiling for building new indexes | โ |
autoApplyScore |
the one auto-approval control: empty = you approve everything, 0 = everything auto-approves, 1โ100 = a confidence floor. 70 is the suggested setting. Advisories never auto-approve. The settings page says how many of your current findings a given number would approve โ read that rather than guessing, because the scores are not evenly spread and two adjacent numbers can differ by a lot |
empty |
changeWindowStartHour / EndHour
|
elective changes run only in this UTC window; safety rollbacks never wait | engine-chosen |
One so far, and it lives apart from the table above for a reason worth stating: a per-cluster policy row is created after its cluster exists, so it cannot decide anything about connecting one.
| knob | effect | default |
|---|---|---|
requireLeastPrivilege |
refuse to store credentials that can create users or roles, at connect and at rotate | off |
Settings โ Organization โ Credential policy, owner-only to change and visible to every member โ somebody refused a connect has to be able to read the rule that refused them.
Two shapes, and the second is not a narrower version of the first.
- The credentials can create users or roles. Refused with a 422 whose message points at the provisioning path, which is one button along on the same screen: the admin string is used once to create a scoped user and is never stored. The connect form withdraws Use these credentials as-is entirely rather than greying it out, because a disabled control is still an offer and sends the reader hunting for what unlocks it instead of pressing the button beside it that works.
- The deployment has authentication disabled. Also refused, and provisioning is not offered โ a server that asks for no credentials grants every privilege to anyone who can reach it, no grant narrows that, and there would be nothing to authenticate a scoped user as. The remedy is on the server.
A rotation whose diagnosis could not be taken is not refused. The string pinged, so the credentials work; what failed is our ability to describe them, and refusing on that would mean a cluster with a flaky diagnosis cannot have its password changed โ the one operation an incident needs most. The posture is recorded as unknown instead, and the connection card says so.
Switching it on cannot un-store an admin string that is already sealed, and it deliberately does not try. Clusters connected under the old rule keep collecting โ an organization that had to choose between its policy and its analysis would never switch the policy on โ and each one's connection card marks it out of policy with a rotation as the fix. A cluster whose posture was never recorded gets its own sentence rather than the violation's, because rotating records the posture and may well record that it was fine all along.
Turning it on or off writes an ORG_POLICY_CHANGED row to the security trail
with both sides of the change, and only when the value actually moved: a form
saved twice is one decision.
observeWindowDays is the baseline; each pending drop gets its own, decided
once at hide time and recorded with its reason โ which the dashboard shows on
the drop date, so a 7-day window beside a 60-day one reads as measured rather
than as arbitrary. The window is set by whichever question is still open, and
the two run on different clocks โ will anything want this again at the cadence
of the workload, did hiding it hurt at the rate the index is queried:
| the index | window | |
|---|---|---|
| periodic usage (monthly report, weekly batch) | 2ร the largest activity gap, โค 90d | might be wanted next cycle |
| queried about daily, and still when hidden | 7d | a regression arrives in days, not weeks |
| in place โฅ 2ร the policy and used in that time | 1.5ร the policy, โค 90d | may have a cadence longer than we have watched |
| zero usage across โฅ 2ร the policy | half the policy, โฅ 7d | the history already was the observation |
| appeared on our watch, never used since | about as long as it has existed, โฅ 7d | its whole life is on record |
| anything else | the policy |
An owner can end an observation early โ "Drop sooner" on a hidden row โ which shortens the window to the time already served. It never lengthens one, and it never skips a gate: the drop still waits for the change window and still passes the regression check, so what is skipped is the waiting and nothing else.
Every row above turns on activity, and activity is a difference. MongoDB's
$indexStats counter is cumulative โ it climbs while the index is queried and
holds perfectly still while it is not โ so one reading only says the index was
used at some point, never when. Indexterity reads usage as the change between
two consecutive readings, which is why an index whose counter has been frozen
for a month counts as a month of silence rather than as a month of use. A
counter that restarts under us โ a node restart, a stepdown โ is counted in full
rather than as a difference, so a restarted member never reads as a burst of
activity or as negative usage.
Order matters in the first two rows. A quarterly job that runs densely for a week is periodic, not busy, so the cadence rule is checked first. And "still" is narrow: an index that was busy and went quiet a week ago gets no fast verdict from being hidden, because nothing is querying it to notice โ that one is back to a cadence question, and waiting is the only answer.
The last two handle a hand-made ad-hoc index: created, used once, forgotten. Its whole life is on record, so it leaves in about a week instead of a month. Age only counts when the index appeared after we started watching โ snapshots begin at onboarding, so an index in the first one may be five years old.
Decided in D17.
Every recommendation carries 0โ100, and the scale is calibrated so 100 is reachable and means "as sure as this engine gets".
- Drops โ the argument is worth 55 (redundant), 50 (never used) or 35 (was periodic, went quiet); a month of unbroken history adds up to 25; reclaimable space up to 20.
- Creates โ 40 for a live collection scan, up to 35 for frequency, up to 25 for collection size.
- Each past regression on the same index subtracts 40, so one is nearly disqualifying and two are โ while it still counts. A regression carries full weight for as long as the cooldown it caused, then fades to nothing over the same span again, so an index is not disqualified for the life of the cluster by one failed experiment against a workload that has since been rewritten. The count itself is kept either way, and the parked panel still shows it.
The score gates entry only. Every safety stage runs regardless of it.
Every five minutes is the probe's schedule, whichever clock drives it.
FAST_PASS_INTERVAL_MINUTES can stretch it, and on a free serverless database
fifteen is the number, for the reason in
Running Indexterity โ Configuration, under Choosing the interval on free serverless tiers.
Two things, in order.
First the server as a whole. CPU is not available โ mongod reports none outside
FTDC โ but serverStatus reports what the query engine is doing: collection
scans, documents and index keys walked, sorts run with no index to order by, and
operations queued behind the global lock. The signal is documents walked per
index key, which is volume-independent, so it reads the same on a small
cluster and a large one. A loaded CPU could be a backup or a noisy neighbour;
thousands of documents per key is a missing index and nothing else. This catches
a scan storm spread thinly across many collections that no single collection's
latency would show.
Then per collection: a missing index shows up as average read latency climbing
while the collection keeps serving traffic. When reads get sharply slower than
their own baseline, the workload pass runs immediately instead of waiting for
the hourly one. Only the busiest 20 collections are probed, and the readings are
never written to latency_samples โ that table's hourly cadence is what the
activity gate and the change-window inference reason over.
serverStatus is the one privilege that reads beyond index metadata, so it is
optional: a cluster without it onboards clean and simply loses the first half of
the probe. Connecting a cluster says exactly what it
exposes.
Left unset, the engine buckets the cluster's own traffic into the four 6h slots of the UTC day and takes the quietest, re-deriving after every collect. It declines to guess on a flat day or thin history, and an explicit setting always wins.
| clusters | seats | index suggestions | unattended changes | history | |
|---|---|---|---|---|---|
| FREE | 1 | 3 | yes | โ | 90 days |
| PRO | 5 | 15 | yes | yes | 183 days |
| SCALE | unlimited | unlimited | yes | yes | 365 days |
| SELF_HOSTED | 1 | unlimited | yes | yes | 365 days |
A plan is per organization. Every number above is that org's, so a customer with a free side project and a paid production team holds two orgs on two plans and is billed for one of them. How many organizations you make is therefore not metered โ capping it would cap how much you can buy โ and the free tier is held by the cluster limit, applied inside each org one at a time. What has been spent of that limit is drawn beside the connect form, before anything is typed into it: a quota nobody sees until it refuses them is a 402 in the middle of someone's work.
Free gives away the analysis and sells the automation. Every plan sees every
recommendation, with the reasoning, and can approve any of them by hand. What a
paid plan adds is not having to: autoApplyScore (approve by score) and
instantCreate (build a critical missing index immediately). The safety
pipeline โ hide, observe, regression-gate, roll back โ is what makes unattended
changes safe to run, and it is the part that took the work.
The rules live in one table in apps/api/src/billing/plans.ts; nothing else
decides them. Limits are enforced by the api, not drawn in the dashboard, and a
refusal comes back as 402 rather than 403 โ the caller is an owner, so
"forbidden" would send them looking for a permissions problem they do not have.
Seats count members plus outstanding invites, so an org cannot invite past
its plan and leave the refusal for whoever clicks the link; better-auth's own
membershipLimit, which counts members only, is deliberately left unset so that
there is one limit with one name. A downgrade never deletes anything: an org
over its new limit keeps what it has and simply cannot add more, and an
auto-approve score saved on a paid plan stops being obeyed without being erased
โ it comes back on upgrading.
Each org's history is kept for its own plan's window, and the daily sweep at 03:00 UTC deletes whatever is older (#549). The same window is applied at every read of the time-series tables: the engine's reads as much as the dashboard's, since a longer series is precisely what lets the engine call an index unused at all. That read filter also covers the day a row can sit past its window between two sweeps.
It used to be split. Deletion ran one cutoff for the whole deployment (the longest window any plan could claim), and each plan's window was applied only on the way out. That kept an upgrade's history ready: a free org moving to a paid plan saw its extra months at once. The price was storing history no reader was entitled to. On a deployment of FREE and PRO orgs that meant everything past 183 days of a 365-day sweep, and on a small database that is what reaches the storage limit first.
So, stated plainly:
- An upgrade starts from what the old plan kept. A free org moving to PRO has its 90 days, and the rest of the new window fills as time passes.
-
A downgrade deletes history at the next sweep, and moving back up does not
return it.
set-plan.jsprints both windows when it moves an org down.
Settled recommendations age out on the same clock as the history they were made from.
The mechanism is Architecture ยง11.3.
It is the BUSL Additional Use Grant expressed as entitlements โ one production cluster, everything else on โ and it is what the chart ships. The licence caps production clusters and says nothing about features, seats or history, so neither does this. Shipping self-hosters the hosted free tier would restrict them further than the licence they are complying with, on hardware they pay for themselves.
DEFAULT_ORG_PLAN, validated against the plan list at boot, stamped onto the row
by the organization-create hook. The chart sets it to SELF_HOSTED; a deployment
that says nothing gets FREE, because a process that has not been told where it
runs should offer the least.
organizations.plan deliberately has no database default. It had one โ
FREE โ and for a release that default was the decision rather than a fallback:
the create hook wrote no plan, so every self-hosted install ran on the hosted free
tier (three seats, no unattended apply, ninety days) while the chart asked for
SELF_HOSTED and the variable reached the container and stopped
(#132). A column that
cannot fall back is what makes a path that forgets fail at the insert instead of
silently choosing the most restrictive plan.
Upgrading an install that has been running a while? Its organizations predate
that fix and are sitting on FREE. List them:
node apps/api/dist/set-plan.js --backfill SELF_HOSTED
Nothing changes until you add --apply. It moves only organizations that are
on FREE โ the plan the database default used to impose โ and whose plan was
never chosen deliberately: no set-plan.js run, no webhook, no billing
subscription attached. So it can raise a plan and never lower one, and the same
command is safe to hand a hosted operator: a paying customer's org is skipped by
construction.
The FREE guard is there because "nobody ever chose this" is not quite what a
null timestamp means. Anything that writes the column without also writing
plan_updated_at โ an integration test, a hand-run UPDATE โ looks identical to
an untouched organization, and without that guard a SCALE org would have been
silently downgraded by a command called backfill.
Plans are set with node apps/api/dist/set-plan.js <org> <PLAN> [note] โ enough
to charge by invoice today, from anywhere, with no processor account. Whoever
eventually takes the money only decides which plan an org is on; a webhook
would write the same column that CLI does, and the entitlements above would not
change.