Connecting a cluster - FullmetalBober/Indexterity GitHub Wiki
SQL Server? Sections below are MongoDB. For SQL Server, the login needs
VIEW SERVER STATE(usage counters) andVIEW DATABASE STATEon every user database to analyze, plus database-levelALTERto apply; Query Store should be ON per database for the latency gates. Bothmssql://โฆand ADOServer=โฆ;User Id=โฆ;Password=โฆstrings paste straight into the same connect form โ the engine is detected from the string, and the form now names both dialects up front and tells you which one it read yours as before you press anything. Details, including what hide means there and the version floor/ceiling: Architecture ยง9.3.
Indexterity never reads your documents. Give it a dedicated user with an index-only role โ start with analyze-only, upgrade to live-manage when you flip the cluster out of read-only mode.
MongoDB 6.0 to 9.x. 4.4 and 5.0 are past end-of-life and have no
$queryStats, so they are refused rather than supported half-well. A server
below the floor is refused at connect time with that explanation, and every
write re-checks the version immediately before running, so a cluster downgraded
or repointed later cannot be half-changed.
A major series newer than anything tested is refused too โ this engine drops and
builds indexes on a live database, and a major release is where command
behaviour moves. Set ALLOW_UNTESTED_DATABASE_VERSION=true to run ahead of
the tested range โ one flag, whichever engine the cluster is.
Enough for collection/index stats, latency and $queryStats workload analysis:
use admin
db.createRole({
role: "indexterityAnalyze",
privileges: [
{ resource: { db: "", collection: "" },
actions: ["listCollections", "listIndexes", "indexStats", "collStats"] },
{ resource: { cluster: true }, actions: ["listDatabases"] },
// Workload analysis via $queryStats. Read the two notes below โ the grant
// alone is not enough to make the store useful.
{ resource: { cluster: true }, actions: ["queryStatsRead"] },
// Optional: the five-minute health probe. See "What serverStatus exposes".
{ resource: { cluster: true }, actions: ["serverStatus"] },
],
roles: [],
})
db.createUser({
user: "indexterity",
pwd: "<generate a strong password>",
roles: ["indexterityAnalyze"],
})$queryStats does not record anything by default before 9.0.
internalQueryStatsRateLimit is 0 on a stock 6.0โ8.x server, and at 0 the
store stays empty no matter how privileged the user is. Set it to -1 (record
every shape) or to a per-second cap such as 100.
On 9.0 it records a 1% sample by default. internalQueryStatsSampleRate
ships at 0.01, and above 0 it decides instead of the rate limit. Measured on
9.0.2, 400 runs of one query counted 41 times at 0.1 and 208 at 0.5. The
store fills and the numbers look plausible, but they are about a hundredth of
the workload. Indexterity reads the counts as they are, so index suggestions
come out weaker than they should, and a query that runs rarely may never appear.
Set it to 1 to count every execution.
Indexterity checks both when you connect, if the credentials can read server parameters, and says so when either is holding the store back.
Before MongoDB 8.0 the store reports execution counts only โ no
docsExamined, keysExamined or hasSortStage, which is the difference
between knowing a query ran and knowing it scanned. On 6.0 and 7.0 the profiler
is therefore the only source that can suggest an index, and Indexterity falls
back to it automatically. 9.0 keeps the same three under metrics.queryExec and
metrics.queryPlanner, and Indexterity reads them from wherever the server's
release puts them.
Profiler fallback (any version): additionally grant find on each analyzed
database's system.profile collection and enable the profiler
(db.setProfilingLevel(2), or level 1 with a slow-ms threshold). The profiler
is the richer source below 8.0 โ it records the plan summary, the documents
walked, whether a blocking in-memory sort ran, and the client's app name.
The profiler also powers the two drop safeguards on any version: it is the only
record of an index named by hint(), and of an operation that failed while
an index was hidden. The second needs the profiler to keep fast operations, and
a slow-ms threshold does not: a hint at a hidden index fails in 0 ms.
You do not have to run the profiler for them. With enableProfiler in the role
(section 2), Indexterity turns it on itself for a database while a drop there is
in flight โ failed operations and hints only, your slow-query log kept as it was,
your settings given back afterwards โ and waits a day of that before hiding, so a
failure after the hide has something to be compared against. A database whose
profiler already runs at level 2 needs neither. Without the action, every hide
says failed operations not watched: โฆ and why, and the drop says the check was
skipped.
Only failures a hidden index can cause count against a drop: a query that names
the index in hint(), a query run past its maxTimeMS, a sort over its memory
limit. Your application's own errors (a duplicate key, a document your validator
refuses, a client that hangs up) are listed in the audit line as not counted.
The day only reaches as far back as system.profile still holds. It is a
capped collection, 1 MB per database unless you made it bigger, and a busy
database turns that over in under an hour. The cluster's overview says so above
the recommendations, once a watched database's ring holds less than a day, and
the HIDE line says it again:
only 54 minutes of history before the hide can be read โฆ so only a failed hint at the index can roll it back. A hinted query still protects the index, but a query
pushed past its maxTimeMS cannot be told from one that was already timing out.
To give the check its day, make the ring bigger. Do it while no drop is in flight
on that database: Indexterity reads a profiler turned off under its own watch as
yours, and leaves it off.
use app // each database you want watched
db.getProfilingStatus() // note the level and filter to put back
db.setProfilingLevel(0)
db.system.profile.drop()
db.createCollection("system.profile", { capped: true, size: 64 * 1024 * 1024 })
// then set the level back as it was, if it was not 0Size it from the HIDE line. 54 minutes in 1 MB is about 27 MB a day, so 64 MB is two days with room to spare. Only operations the profiler keeps go in it, and the disk it takes is the whole cost.
serverStatus is optional and sits outside the index-only story, so it is worth
being explicit: it is a server-wide command, and it reports more than
indexes. Indexterity reads the query-engine counters from it (collection scans,
documents and index keys walked, unindexed sorts, operations queued behind the
global lock) to notice a scan storm spread across many collections. The same
document also carries connection counts, network byte totals, replication state
and storage-engine internals, and granting the action means the user can read
all of it.
It still cannot read a single one of your documents โ that needs find, which
this role never grants. Skip the privilege if you would rather not share the
rest: the cluster onboards clean without it and only loses the health probe.
Adds the three index write operations โ still nothing that touches documents โ
and, optionally, enableProfiler, which lets Indexterity watch a hidden index's
queries for failures (above):
use admin
db.grantPrivilegesToRole("indexterityAnalyze", [
{ resource: { db: "", collection: "" },
actions: ["createIndex", "dropIndex", "collMod", // collMod = hide/unhide
"enableProfiler"] }, // optional: the failure watch
])Sections 1 and 2 are the manual path. If the string you paste on the connect
form can create users, the form offers to do it for you instead: it creates the
indexterityEngine role โ the privileges above and nothing else, so still no
find on your collections โ and an indexterity user holding it. The string you
pasted is used once and never stored; only the new user's string is kept
(envelope-encrypted), and it is shown to you once.
The offer needs three actions on admin:
| Action | What it is for |
|---|---|
createRole |
defining the indexterityEngine role |
createUser |
creating the indexterity user |
grantRole |
attaching that role to that user |
userAdminAnyDatabase carries all three, and so does root. The preflight
lists them alongside the engine's own privileges, and when one is missing it
says which โ grant it and press Check access again. Connecting without them
is not a failure, it is the other path: the string you pasted is what gets
stored (encrypted) and dialed on every collect.
Atlas has none of the three. Atlas manages database users through its own UI and API, and no connection string it issues can create one โ follow sections 1 and 2 there and connect with the user you made.
Revoke the provisioned user whenever you like: db.dropUser("indexterity") in
admin. Disconnecting the cluster hands you that exact command, because by then
the control plane no longer holds credentials that could run it.
The scoped user is called indexterity on every engine โ not a random
idx_<hex> per connection. Two things follow.
A cluster stops collecting abandoned users. Provisioning creates the user with admin credentials that are then thrown away, so nothing in Indexterity can ever drop one. With a random name, connect โ disconnect โ connect left three logins behind for somebody to find later and be unable to attribute.
The same cluster cannot be connected twice. The second provision asks the
server to create a user it already has, and the server refuses. That refusal is
the duplicate check โ no rule invented over connection strings, which can spell
one server a dozen ways (mongodb+srv versus a seed list, an alias versus an IP,
with or without a database, options in any order). The name is the identity.
You will see it as a 422 naming both readings, because the server cannot tell them apart: either the cluster is already connected here โ open it instead of adding it again โ or it was disconnected and the user outlived it, in which case the message carries the exact statements that remove it.
| Engine | What removes it |
|---|---|
| MongoDB | db.getSiblingDB("admin").dropUser("indexterity") |
| SQL Server |
DROP USER in each database, then DROP LOGIN indexterity
|
| PostgreSQL |
DROP OWNED BY "indexterity" in each database, then DROP ROLE "indexterity"
|
The two-step forms are not belt-and-braces, they are required. SQL Server refuses
to drop a login while any database user still maps to it, and provisioning
created one in every database. PostgreSQL refuses to drop a role its grants still
point at โ the CONNECT on each database and USAGE on each schema are enough
to hold it down:
DROP ROLE "indexterity";
ERROR: role "indexterity" cannot be dropped because some objects depend on it
DETAIL: 1 object in database postgres
DROP OWNED BY clears them, and it clears them per database: run it in one
and the DROP ROLE still fails, naming the next. Both measured on PostgreSQL
18.4.
The role is created once, from an admin string that is never kept, so a release that can do more with one more action cannot add it to a role that already exists. What happens instead:
-
Nothing breaks. A privilege a release adds is always optional: the one feature it serves stays off โ and says why, in the activity trail โ until it is granted. A test holds every engine to that.
-
You are told once, and only if you lack it. The hourly collect asks the cluster what the credentials hold, over the connection it already has open. Credentials that hold the privilege already are never told. Admin strings qualify, and so do broad built-in roles: Atlas's
atlasAdmincarriesenableProfilerthroughdbAdminAnyDatabase. Credentials found to lack it get three things:- a notice on the cluster's pages naming the privilege and what it turns on;
- an explanation in the cluster's settings of how to grant it;
- one mail to the owners.
The check repeats every collect while the privilege is missing, so a grant run by hand clears the notice within the hour.
-
One statement grants it, on a role Indexterity created โ for 0.29.0's
enableProfiler:db.getSiblingDB("admin").grantPrivilegesToRole("indexterityEngine", [{"resource":{"db":"","collection":""},"actions":["enableProfiler"]}])
On a role you made yourself, add the action to it as in sections 1 and 2.
-
Or upgrade it from the dashboard. Settings โ Connection takes an admin string for the same cluster, uses it once to bring the role to today's definition, checks the result with the stored credentials, and forgets the string โ provisioning's terms. The admin needs
userAdminAnyDatabase, orviewRole,grantRoleandrevokeRoleon every database includingconfig: the role readsconfig.collections, so the same three on{db: "", collection: ""}are refused (measured on 7.0). -
Not granting it is an answer too. "Mark as reviewed" removes the notice and changes nothing on the cluster. Opening the credentials card after granting by hand clears it at once, rather than at the next collect, because that card asks the cluster what the credentials actually hold.
The same offer, with SQL Server's own vocabulary. An admin string that can
create logins gets you an indexterity login, granted exactly what the connect
form probes and nothing else:
| Grant | Scope | What it is for |
|---|---|---|
VIEW SERVER STATE |
server |
sys.dm_db_index_usage_stats โ the usage counters |
VIEW DATABASE STATE |
each database | sizes, row counts, Query Store |
ALTER |
each schema that owns tables | the index DDL: disable, rebuild, create, drop |
ALTER is granted per schema rather than per database on purpose: database-wide
ALTER would also let the login create, alter and drop tables anywhere in that
database, and SQL Server has no index-only permission to grant instead (there is
no ALTER ANY INDEX at any scope). No SELECT anywhere โ the login cannot
read a row of your data, and the server is what enforces that.
Making the offer needs three things of the string you paste, and the preflight
names whichever is missing: ALTER ANY LOGIN (create the login), ALTER ANY USER in each database (create its user), and CONTROL SERVER (grant it
VIEW SERVER STATE โ a login that lacks this creates the login and then fails
at the grant, so it is checked up front). sysadmin carries all three.
A database created after provisioning is not covered, because its grants did not exist when the login was made. Nothing is silent about it: the connect form's checks turn red and name that database, and re-running the provisioning with an admin string fixes it. Same for a brand-new schema in an existing database.
Query Store is not a permission, it is a per-database setting, and it is the only
workload signal on SQL Server that survives a restart โ read/write latency per
table (which is what the regression gates read) and hinted-index detection both
come from it. The connect form lists it beside the permissions for that reason, and
when it is off it now hands you the statements rather than a sentence with an
ellipsis in it: one ALTER DATABASE per database that is missing it, with a copy
button. Run them as an owner of the server โ the login Indexterity uses holds
ALTER on schemas, not on the database, so it cannot enable this itself.
The statements carry MAX_STORAGE_SIZE_MB = 1000 as well as the enable, and that
is measured rather than tasteful. On a new database:
| SQL Server 2017 | SQL Server 2022 | |
|---|---|---|
| Query Store | OFF |
READ_WRITE (on by default) |
QUERY_CAPTURE_MODE |
ALL |
AUTO |
MAX_STORAGE_SIZE_MB |
100 | 1000 |
STALE_QUERY_THRESHOLD_DAYS |
30 | 30 |
SIZE_BASED_CLEANUP_MODE |
AUTO |
AUTO |
A bare enable on the 2016/2017 generation therefore turns on ALL capture into a
100 MB budget. When that fills, Query Store flips to READ_ONLY and stops
capturing โ quietly, which is the failure worth avoiding. 1000 MB is simply what
2019 and later ship as the default, so naming it normalises the older generation
and changes nothing on a newer server.
QUERY_CAPTURE_MODE is deliberately left alone. AUTO skips trivial one-off
statements, which is the right trade on a production server: the queries an index
decision rests on run repeatedly, so AUTO captures them, and ALL would buy
coverage of statements nobody is optimising at a cost you pay all day.
What it costs while it is on. Runtime stats go to an in-memory buffer flushed
every 15 minutes (flush_interval_seconds 900), aggregated into one-hour intervals
(interval_length_minutes 60), kept for 30 days, inside the storage budget above
with size-based cleanup at ~90% of it. Enabling is an online ALTER DATABASE: no
restart, no exclusive lock, no detach. Overhead is a small CPU and memory cost on
query compilation and execution paths, which is why the capture mode above matters
more than the switch itself.
Two things it will not do. It cannot be enabled on master or tempdb โ which is
why those never appear in the database list. And a read-only replica captures
nothing, so on an availability group the latency signal comes from the primary;
Query Store for secondary replicas is a 2022+ opt-in that Indexterity does not turn
on for you.
Sections 1 to 3 are about what to grant. This is the same list read backwards: what a string you already connected holds and Indexterity never uses.
The connection card's badge says how privileged the stored credentials are โ
scoped user, scoped credentials, admin credentials. That is true and it is
not actionable: it never says what more, so the one thing a reader wants to do
about it has no target. Under it, Check what these credentials hold dials the
cluster and splits the answer into three:
- provided โ what the credentials actually hold, right now
- required โ the CORE / APPLY / WORKLOAD checks the engine needs on this cluster, for the databases it observes
- redundant โ held, never used, each with the statement that removes it
The redundant statements run the opposite direction from everything else on the
connect form: those close a gap (GRANT โฆ), these take a grant away
(ALTER ROLE โฆ NOCREATEROLE, REVOKE โฆ, db.revokeRolesFromUser).
Two things worth knowing about the read. It dials the cluster, so it happens when you open the panel rather than on every page view, and it spends the same per-account dial budget as testing a string (ยง5). And the answer is stamped with when it was taken โ nothing else in the product re-checks an existing cluster, so a figure without a timestamp would be indistinguishable from one measured at connect time on credentials since rotated.
A cluster running on a user Indexterity provisioned reports an empty redundant group that says it is empty, by construction โ the role in ยง1 and ยง2 is the required set. "Nothing surplus" is the reassuring answer and blank space does not deliver it.
Measured against real servers rather than reasoned from documentation, and the list agrees with the role tables above by construction: anything the scoped role holds is required, and this is what is left.
MongoDB
| Held | Why it is surplus | What removes it |
|---|---|---|
root |
everything on the deployment, documents included | db.revokeRolesFromUser |
userAdminAnyDatabase |
creating and altering users and roles โ how any grant is made | db.revokeRolesFromUser |
readWriteAnyDatabase |
reading and writing every document | db.revokeRolesFromUser |
dbOwner |
reading, writing and administering everything in that database | db.revokeRolesFromUser |
dbAdminAnyDatabase |
administering every database, dropDatabase included |
db.revokeRolesFromUser |
insert / update / remove
|
nothing in the pipeline writes a document | revoke the role above, or db.updateRole on a custom one |
The revoke is issued against the user's own database, not admin.
revokeRolesFromUser resolves the user in the database it is run from, so a user
created in app answers Could not find user "owner" for db "admin" (measured
on 8.0) โ and a customer's application user usually lives beside its data.
PostgreSQL
| Held | Why it is surplus | What removes it |
|---|---|---|
SUPERUSER |
everything on the server; the engine uses none of it | ALTER ROLE โฆ NOSUPERUSER |
CREATEROLE |
needed once to provision; never by a stored connection | ALTER ROLE โฆ NOCREATEROLE |
CREATEDB |
nothing in the analysis or apply path creates a database | ALTER ROLE โฆ NOCREATEDB |
INSERT / UPDATE / DELETE on tables it does not own |
nothing in the pipeline writes a row |
REVOKE โฆ ON ALL TABLES IN SCHEMA, plus ALTER DEFAULT PRIVILEGES
|
SUPERUSER is listed separately from the other two even though it implies them,
because NOCREATEDB on a superuser changes nothing โ only NOSUPERUSER removes
anything.
Read access is deliberately not on this list, and PostgreSQL is the only
engine where that is true. Applying here requires table ownership โ there is
no grantable index privilege, and pg 17's MAINTAIN is not one either โ and an
owner can always SELECT. So a cluster you intend to apply on holds read access
as a consequence of a requirement, and calling it surplus would be telling you to
break your own APPLY row. Writes are different: nothing in the pipeline writes a
row, so a write grant on a table the role does not own is surplus outright.
The revoke statements are grouped per database under a psql \connect, because
no SQL statement on this engine can cross a database boundary. The
ALTER DEFAULT PRIVILEGES line beside each REVOKE is the half people forget โ
without it the next table created in that schema arrives with the grant again.
SQL Server
| Held | Why it is surplus | What removes it |
|---|---|---|
sysadmin |
everything on the instance, and every permission below it implicitly | ALTER SERVER ROLE [sysadmin] DROP MEMBER |
CONTROL SERVER |
every permission on the instance, granting them onward included | REVOKE CONTROL SERVER FROM |
db_owner |
reading and rewriting every table in that database | USE โฆ; ALTER ROLE [db_owner] DROP MEMBER |
A sysadmin is reported as one finding, not three, and that is measured rather
than tidy. HAS_PERMS_BY_NAME reports CONTROL SERVER as held for a sysadmin
after it has been revoked, so that row's statement would do nothing; and a
sysadmin maps to dbo in every database, so the db_owner row would emit
ALTER ROLE db_owner DROP MEMBER [dbo], which SQL Server refuses outright โ
Msg 15405, Cannot use the special principal 'dbo' (verified on 2022). The
membership is the cause of all three, and dropping it is the only change that
does anything.
The db_owner statement names the database user, not the login: a login's
user is a different name in every database it has one in, so a statement built
from SUSER_SNAME() would fail on at least one of them.
Connect to the mongos. Stats aggregate across shards, and each collection's
shard key is read from config.collections so any index it prefixes is
protected. For that protection, also grant read on config:
db.grantRolesToUser("indexterity", [{ role: "read", db: "config" }])Without config read, the collection is treated as unsharded.
mongodb://indexterity:<password>@host1,host2/?replicaSet=rs0&authSource=admin
mongodb:// and mongodb+srv:// are the MongoDB schemes; SQL Server takes
mssql://, sqlserver:// or an ADO Server=โฆ;User Id=โฆ;Password=โฆ list. The
connect form lists whichever of those the build you are on supports, so the
answer to "does this take my database" is on the screen rather than in these
docs. The string is envelope-encrypted at rest in the control plane; rotate the
password anytime and re-connect the cluster.
Nothing asks you to pick an engine, and that is deliberate: the schemes are
disjoint, so the string says which engine it is and the api decides from the
string it was actually given. The form shows what it read yours as โ a guess off
the scheme while you type, replaced by the api's own verdict the moment you check
access โ and offers a choice in exactly one case, where no engine recognises the
string at all. A picker in front of the strings detection does recognise would be
a way to be wrong: choose MongoDB, paste Server=โฆ, and the choice is honoured
and a perfectly good string is refused.
Testing and connecting both make the control plane dial the hosts you named, and that is capped at 10 attempts a minute per account โ a security control on what we will connect to, not a throttle on you. Past it the form says so, and says how many seconds are left; the window is rolling, so it clears on its own without anyone unlocking anything. Worth knowing when you are iterating on a string that will not connect.
Three things you can change afterwards without losing anything, all under the cluster's Settings tab: the connection string (rotated in place โ the new one is dialed and verified before it replaces the old), the name, and which databases are observed (ยง5.1). The name has to be unique inside your organization, because it is what the sidebar lists and what every alert about the cluster is subject-lined with. Neither costs you history; disconnecting does โ it deletes every measurement, recommendation, ROI figure and audit row for that cluster, so it is not the way to fix a typo.
A cluster is often not one database. One production database beside three staging copies, a per-tenant clone per customer, and last week's restore is an ordinary shape, and walking all of it is work nobody asked for โ plus recommendations for indexes on data that is about to be dropped.
So the connect form asks. Check access first: the answer lists every user database the credentials can see, each ticked, and unticking one leaves it alone. The cluster's Settings tab has the same list afterwards, read live from the cluster, so a database created last week is offerable today.
What counts as a user database, which is the same question on all three engines and now has the same answer: the databases the engine owns and uses are excluded by name, and nothing else is.
| Engine | Never listed |
|---|---|
| MongoDB |
admin, local, config
|
| PostgreSQL | none by name โ templates and anything with connections disabled cannot be dialled at all, so they never appear |
| SQL Server |
master, tempdb, model, msdb (and anything not ONLINE) |
PostgreSQL's postgres is the awkward one and gets a narrower rule: it is not a
system database โ initdb creates it so that a client has somewhere to connect
before anything else exists โ but on almost every install it holds nothing. So it
is listed only when it holds a table of yours. An install that really does
keep application tables in postgres sees it and can observe it; everyone else
sees one application database and is not asked to choose between it and an empty
one.
If your cluster has exactly one database, nothing is asked and it is observed. The choice appears at two.
Everything is observed unless you say otherwise. That is the default at
connect time and it is what every cluster connected before this existed does โ a
scripted POST /clusters that has never heard of the field keeps working
unchanged.
A narrowed selection does not grow by itself. Add a database to the cluster next month and it is not observed until somebody ticks it. That is the cost of naming databases rather than excluding them, and it is the honest trade: the alternative silently starts reading every database anyone adds. The Settings screen counts what it is leaving out for exactly this reason, so the answer to "why are there no recommendations for the new service" is on the screen. A cluster observing all of them keeps observing all of them, additions included โ which is why "all" is stored as "all" and not as a list of today's names.
It sharpens the privilege check. Indexterity's per-database requirements pass when every database in scope is covered by the role. Before you could narrow the scope, a role granted on one database of twelve read as a missing privilege, and the only way to connect that cluster was to widen the role over the eleven nobody wanted read. Now the form offers to re-check against the databases you ticked, and the same role reads as a grant. The re-check appears only when something is actually missing โ a role that covers everything says the same thing about one database as about twelve.
It does not narrow what a provisioned user may reach, and that is a decision
rather than an oversight. Let Indexterity create the login (ยง3) and it is granted
across every user database that exists at that moment โ on SQL Server
VIEW DATABASE STATE plus ALTER on each schema holding tables, per database,
because that is the only scope those permissions have. What it is never granted,
anywhere, is SELECT: the login cannot read a row in any database, observed or
not, and the server enforces that (Msg 229 on a plain SELECT, verified on 2022).
The alternative was tried and reverted. Granting only where the selection pointed made the selection un-editable in practice: provisioning runs once, from an admin string that is never stored, so a login created for two databases of twelve could never be given the third โ ticking it was a dead end with no way out inside the product. Keeping the two apart means "which databases do we look at" stays a setting you can change at any time, and "which databases could this login look at" was settled once, when you consented to it.
One gap remains, and it is refused rather than silent. A database created on
the cluster after the login was has no user for it, because nothing re-runs the
grants. SQL Server will still offer that database on the settings screen โ every
login sees every row of sys.databases, since VIEW ANY DATABASE is granted to
public โ and every read of it fails with Msg 916 ("The server principal โฆ is not
able to access the database โฆ under the current security context"). So the save
probes each newly ticked database with the credentials actually stored and names
the ones they cannot read. Two ways forward, both keeping every measurement: grant
that login access on the cluster yourself, or rotate to a connection string that
already has it.
The same shape reaches every engine, from your own connection strings. A credential that can list a cluster's databases and read only some of them is what a scoped user normally is, whoever created it:
-
MongoDB โ a user holding the cluster-wide
listDatabasesaction with per-database grants that do not cover everything. Every read of an uncovered database answersnot authorized on <db>. -
PostgreSQL โ a role without
CONNECTon a database, which is what revokingPUBLIC's default grant leaves behind. The refusal ispermission denied for database "<db>", and it arrives when we first dial that database. - SQL Server โ Msg 916, as above.
All three are treated identically: the settings screen names the databases the stored credentials cannot read and refuses the save rather than starting to "observe" something we will never see.
If access is lost after the tick โ a user dropped, a database restored over, a grant revoked โ the collect skips that one database and keeps walking the rest of the cluster. It used to abort the entire pass on the first database it could not reach, so one inaccessible database cost every measurement on the cluster.
(MongoDB's provisioned role is cluster-wide for the same reason, and gives up less by being so: it grants index metadata actions and explicitly withholds reading a document, so a database outside the selection is one we can list indexes on and still cannot read a row from.) What happens to what was already collected. Nothing is deleted. Measurements for a database you untick are kept โ the next collect simply stops walking it โ and open proposals for it are discarded, because a proposal is a suggestion to change something we have stopped watching. Changes already in flight there are not discarded: an index the engine has hidden pending a drop stays tracked through its observe window and is restored on disconnect like any other. And approving a proposal in a database that is no longer observed is refused rather than quietly executed.
A database that goes away. Drop or rename a selected database and nothing breaks: each collect walks what is both selected and present, so the missing name simply stops matching, and if it comes back it is picked up again with no re-ticking. The Settings screen points out a selected name the cluster no longer reports, since that is the one place you would ever see it.
A cluster observing nothing is refused. Untick the last database and the form says so. It would be indistinguishable from a broken cluster on every panel afterwards; disconnect it instead.
Everything above assumes Indexterity can open a socket to your database. A cluster inside a VPC with no public endpoint, or on-prem behind a firewall, is not reachable from a control plane sitting somewhere else โ and no connection string fixes that.
Paste the WireGuard config that network already uses. Settings โ VPN
tunnels takes a whole wg0.conf, the file your VPN admin exports. Indexterity
terminates the peering itself, in userspace: no agent to install, no container to
run, no privileges to grant on your side. Then pick that tunnel on the connect
form and the cluster connects like any other.
What moves those packets is wireguard-go with gvisor's network stack โ the same pairing Tailscale's userspace mode uses โ in a service that runs beside the control plane and nowhere else. That is a detail you never touch, and it is here only so nobody has to guess: your peer is talking to a standard WireGuard implementation.
If Settings โ VPN tunnels says the feature is turned off, this deployment has no tunnel service running and no peering can be brought up. On a self-hosted install that is one setting away; on a hosted one it is worth asking about rather than working around.
[Interface] โ our side of the peering
PrivateKey = ... โ encrypted before it is stored, never shown again
Address = 10.9.0.2/32
DNS = 10.9.0.1 โ who resolves names INSIDE the tunnel
[Peer] โ your gateway
PublicKey = ...
Endpoint = vpn.example.com:51820
AllowedIPs = 10.0.0.0/8 โ what this tunnel is allowed to reach
Pick the tunnel when you connect, not afterwards. The connect form checks what your credentials can do before storing anything, and that check dials your database โ so it has to go through the tunnel too. There is a How to reach it picker above the certificate boxes; it only appears once you have registered a tunnel.
AllowedIPs is a limit, not a hint. It is what your peer agreed to carry,
so it is what Indexterity will dial and nothing else. A connection string
pointing outside it is refused with that reason rather than timing out, because
a timeout is indistinguishable from a database that is merely down.
Give the tunnel a DNS line if you address hosts by name. A private replica
set's member names mean something only on your side of it. Without a resolver
in the config, Indexterity cannot look them up โ the tunnels screen says so, and
the failure otherwise arrives as "unreachable", which reads like a firewall
problem and sends you looking in the wrong place.
When something stops collection, the cluster says so. A cluster nobody can reach used to show one thing: last collected 7 days ago. That is true and it is not enough โ a paused schedule and a quiet cluster look identical to a cluster whose firewall changed, so the badge that needed you read like the two that did not. The cluster's heading now carries a red badge and a banner underneath it naming what happened, how long it has been happening, the failure in the driver's own words, and what to do about it. The wording distinguishes the cases that matter: we could not reach it, versus we declined to dial it because the stored string would connect in plaintext, versus your VPN gateway is not answering while the database itself may be fine. Your owners still get one email a day about it; this is what is there when they open the page a week later.
A tunnel that is down is a condition of the tunnel. The tunnels screen shows each one as Up, Connecting, Down, or Idle โ Idle simply meaning nothing has needed it since the server started, which is the normal state of a healthy new one. A cluster behind a down tunnel is unreachable, not broken, and both screens say so rather than blaming the database.
Test it rather than believing it. Registering a tunnel proves the file
parses, and nothing more. A mistyped PublicKey, a gateway that moved, a UDP
port a firewall drops and a peering your VPN admin revoked are all perfectly
valid files, and all four fail identically at the first collect โ as a timeout
you would reasonably read as a problem with your database. Each row has a
Test: it negotiates a handshake with your gateway there and then and tells
you what came back. It forces a fresh handshake rather than reporting the
current state, because a tunnel can sit on a session negotiated an hour ago
against a gateway that has been switched off since โ WireGuard has nothing to
say about that until traffic needs a new key. Silence is the one answer with no
cause to report: an endpoint that is not there, a dropped UDP port and a
PublicKey your gateway does not recognise look identical from our side, so the
message names the three rather than guessing between them.
Editing: rename freely, replace the config whole. Edit on a row renames
a tunnel โ the name is only how the connect form refers to it โ and takes a
whole new wg0.conf when a key is rotated or a gateway moves. There is no
partial edit of a stored config, and that is not an omission: the PrivateKey
in it never comes back from the api, so there is nothing anyone could show you
to amend. Leave the config box empty and only the name changes.
Saving a new config drops the live peering rather than swapping it underneath. It comes back up on the next collect, or the moment you press Test, and clusters behind it are unreachable for that gap rather than dialling with a key that has just been rotated away โ which is the failure that would be silent.
An IPv6-only peering works. A wg0.conf whose [Interface] Address is a v6
address and whose AllowedIPs is a v6 prefix used to be refused with a message
saying the network stack carried IPv4 only. That limit is gone, and it has been
tested end to end against a real WireGuard peer rather than assumed โ the tunnel
comes up and a database inside it answers on its v6 address. The gateway's own
Endpoint is a separate matter and may be either family.
Every one of these acts is on the security trail, including pressing Test. Registering a tunnel, replacing its config and removing it each write a row naming who did it, the gateway involved and โ for a replacement โ what the peering reached before and after. A test writes one too, with what came back: a gateway that has been asked six times and answered none of them is worth being able to see. The config itself is never recorded: it holds a private key, and the trail is read by people who are not meant to be able to bring the tunnel up.
What it does not change. TLS is still required: the tunnel carries
ciphertext and adds a second layer rather than replacing the first, so a cluster
reached through it still needs a certificate Indexterity can validate. Cloud
metadata and the other never-a-database ranges stay refused whatever route
exists to them โ a peer whose AllowedIPs covers 169.254.169.254 is a
misconfiguration. And your credentials still leave your network: the connection
string is stored by Indexterity and opened by Indexterity. A tunnel changes
which addresses are reachable, not where the secret lives. Only an agent that
runs the queries inside your network would change that, and that is
#272.
If you run Indexterity yourself and would rather your own network stack carry
the tunnel, you can skip all of the above: run a WireGuard client as a sidecar
in the api pod and set ALLOW_PRIVATE_CLUSTER_TARGETS=true. The route it
creates is one the api dials through, with no tunnel registered in the product
at all. The recipe, with values, is in the chart's
README.
That flag is global, which is why it is a self-hosted answer and not the hosted one: on a shared install it would let every tenant aim a connection string at the operator's own private network. A registered tunnel is scoped to the clusters that use it instead.
$indexStats and $collStats latencyStats are per member; both sum all of
them, so an index used only on a secondary counts as used, and a cluster's write
latency is read from the node that actually takes writes. Both are aggregations,
so the driver routes them by read preference โ a connection string carrying
readPreference=secondaryPreferred had every latency reading landing on a
secondary, whose write counters are permanently zero because oplog application
is not a client write op.
Which members, exactly: every one hello names, and that is two arrays, not
one. Electable members are listed under hosts, but a priority: 0 member is
listed under passives โ and priority 0 is the normal setting for a secondary
in another region, so that it cannot win an election across a WAN. Priority
governs elections and nothing else, so those members serve secondaryPreferred
reads like any other secondary and both arrays are collected from.
Hidden members are not, and this is a deliberate limit rather than an
oversight. They appear in neither array, so reaching them needs
replSetGetStatus or replSetGetConfig โ cluster privileges the engine role
does not ask for. Drivers never route reads to a hidden member either, so what
is missed is direct-connected traffic: BI tools, backups, delayed members. If an
index exists solely for a hidden analytics node, hint it or keep it out of the
pipeline.
What Atlas hands you carries two settings outside its own text: the scheme
defaults tls to true, and authSource arrives in a DNS TXT record the driver
reads at connect.
Per-member connections have to rewrite the string as plain mongodb:// โ an SRV
seed cannot be pointed at one host, since its targets live in DNS and the scheme
forbids a port โ so both are carried across explicitly from the live client
rather than re-parsed from the original text. Without that the member
connections are plaintext, authenticate against the database in the path, and
fail; a plain multi-host string was never affected, because its options are in
the string.