Connecting a cluster - FullmetalBober/Indexterity GitHub Wiki

Connecting a cluster: the scoped user Indexterity needs

SQL Server? Sections below are MongoDB. For SQL Server, the login needs VIEW SERVER STATE (usage counters) and VIEW DATABASE STATE on every user database to analyze, plus database-level ALTER to apply; Query Store should be ON per database for the latency gates. Both mssql://โ€ฆ and ADO Server=โ€ฆ;User Id=โ€ฆ;Password=โ€ฆ strings paste straight into the same connect form โ€” the engine is detected from the string, and the form now names both dialects up front and tells you which one it read yours as before you press anything. Details, including what hide means there and the version floor/ceiling: Architecture ยง9.3.

Indexterity never reads your documents. Give it a dedicated user with an index-only role โ€” start with analyze-only, upgrade to live-manage when you flip the cluster out of read-only mode.

0. Supported versions

MongoDB 6.0 to 9.x. 4.4 and 5.0 are past end-of-life and have no $queryStats, so they are refused rather than supported half-well. A server below the floor is refused at connect time with that explanation, and every write re-checks the version immediately before running, so a cluster downgraded or repointed later cannot be half-changed.

A major series newer than anything tested is refused too โ€” this engine drops and builds indexes on a live database, and a major release is where command behaviour moves. Set ALLOW_UNTESTED_DATABASE_VERSION=true to run ahead of the tested range โ€” one flag, whichever engine the cluster is.

1. Analyze-only (read-only mode)

Enough for collection/index stats, latency and $queryStats workload analysis:

use admin
db.createRole({
  role: "indexterityAnalyze",
  privileges: [
    { resource: { db: "", collection: "" },
      actions: ["listCollections", "listIndexes", "indexStats", "collStats"] },
    { resource: { cluster: true }, actions: ["listDatabases"] },
    // Workload analysis via $queryStats. Read the two notes below โ€” the grant
    // alone is not enough to make the store useful.
    { resource: { cluster: true }, actions: ["queryStatsRead"] },
    // Optional: the five-minute health probe. See "What serverStatus exposes".
    { resource: { cluster: true }, actions: ["serverStatus"] },
  ],
  roles: [],
})
db.createUser({
  user: "indexterity",
  pwd: "<generate a strong password>",
  roles: ["indexterityAnalyze"],
})

$queryStats does not record anything by default before 9.0. internalQueryStatsRateLimit is 0 on a stock 6.0โ€“8.x server, and at 0 the store stays empty no matter how privileged the user is. Set it to -1 (record every shape) or to a per-second cap such as 100.

On 9.0 it records a 1% sample by default. internalQueryStatsSampleRate ships at 0.01, and above 0 it decides instead of the rate limit. Measured on 9.0.2, 400 runs of one query counted 41 times at 0.1 and 208 at 0.5. The store fills and the numbers look plausible, but they are about a hundredth of the workload. Indexterity reads the counts as they are, so index suggestions come out weaker than they should, and a query that runs rarely may never appear. Set it to 1 to count every execution.

Indexterity checks both when you connect, if the credentials can read server parameters, and says so when either is holding the store back.

Before MongoDB 8.0 the store reports execution counts only โ€” no docsExamined, keysExamined or hasSortStage, which is the difference between knowing a query ran and knowing it scanned. On 6.0 and 7.0 the profiler is therefore the only source that can suggest an index, and Indexterity falls back to it automatically. 9.0 keeps the same three under metrics.queryExec and metrics.queryPlanner, and Indexterity reads them from wherever the server's release puts them.

Profiler fallback (any version): additionally grant find on each analyzed database's system.profile collection and enable the profiler (db.setProfilingLevel(2), or level 1 with a slow-ms threshold). The profiler is the richer source below 8.0 โ€” it records the plan summary, the documents walked, whether a blocking in-memory sort ran, and the client's app name.

The profiler also powers the two drop safeguards on any version: it is the only record of an index named by hint(), and of an operation that failed while an index was hidden. The second needs the profiler to keep fast operations, and a slow-ms threshold does not: a hint at a hidden index fails in 0 ms.

You do not have to run the profiler for them. With enableProfiler in the role (section 2), Indexterity turns it on itself for a database while a drop there is in flight โ€” failed operations and hints only, your slow-query log kept as it was, your settings given back afterwards โ€” and waits a day of that before hiding, so a failure after the hide has something to be compared against. A database whose profiler already runs at level 2 needs neither. Without the action, every hide says failed operations not watched: โ€ฆ and why, and the drop says the check was skipped.

Only failures a hidden index can cause count against a drop: a query that names the index in hint(), a query run past its maxTimeMS, a sort over its memory limit. Your application's own errors (a duplicate key, a document your validator refuses, a client that hangs up) are listed in the audit line as not counted.

Giving the failure check its day

The day only reaches as far back as system.profile still holds. It is a capped collection, 1 MB per database unless you made it bigger, and a busy database turns that over in under an hour. The cluster's overview says so above the recommendations, once a watched database's ring holds less than a day, and the HIDE line says it again: only 54 minutes of history before the hide can be read โ€ฆ so only a failed hint at the index can roll it back. A hinted query still protects the index, but a query pushed past its maxTimeMS cannot be told from one that was already timing out. To give the check its day, make the ring bigger. Do it while no drop is in flight on that database: Indexterity reads a profiler turned off under its own watch as yours, and leaves it off.

use app                                  // each database you want watched
db.getProfilingStatus()                  // note the level and filter to put back
db.setProfilingLevel(0)
db.system.profile.drop()
db.createCollection("system.profile", { capped: true, size: 64 * 1024 * 1024 })
// then set the level back as it was, if it was not 0

Size it from the HIDE line. 54 minutes in 1 MB is about 27 MB a day, so 64 MB is two days with room to spare. Only operations the profiler keeps go in it, and the disk it takes is the whole cost.

What serverStatus exposes

serverStatus is optional and sits outside the index-only story, so it is worth being explicit: it is a server-wide command, and it reports more than indexes. Indexterity reads the query-engine counters from it (collection scans, documents and index keys walked, unindexed sorts, operations queued behind the global lock) to notice a scan storm spread across many collections. The same document also carries connection counts, network byte totals, replication state and storage-engine internals, and granting the action means the user can read all of it.

It still cannot read a single one of your documents โ€” that needs find, which this role never grants. Skip the privilege if you would rather not share the rest: the cluster onboards clean without it and only loses the health probe.

2. Live-manage (after "Go live")

Adds the three index write operations โ€” still nothing that touches documents โ€” and, optionally, enableProfiler, which lets Indexterity watch a hidden index's queries for failures (above):

use admin
db.grantPrivilegesToRole("indexterityAnalyze", [
  { resource: { db: "", collection: "" },
    actions: ["createIndex", "dropIndex", "collMod", // collMod = hide/unhide
              "enableProfiler"] },                    // optional: the failure watch
])

3. Or let Indexterity create the user for you

Sections 1 and 2 are the manual path. If the string you paste on the connect form can create users, the form offers to do it for you instead: it creates the indexterityEngine role โ€” the privileges above and nothing else, so still no find on your collections โ€” and an indexterity user holding it. The string you pasted is used once and never stored; only the new user's string is kept (envelope-encrypted), and it is shown to you once.

The offer needs three actions on admin:

Action What it is for
createRole defining the indexterityEngine role
createUser creating the indexterity user
grantRole attaching that role to that user

userAdminAnyDatabase carries all three, and so does root. The preflight lists them alongside the engine's own privileges, and when one is missing it says which โ€” grant it and press Check access again. Connecting without them is not a failure, it is the other path: the string you pasted is what gets stored (encrypted) and dialed on every collect.

Atlas has none of the three. Atlas manages database users through its own UI and API, and no connection string it issues can create one โ€” follow sections 1 and 2 there and connect with the user you made.

Revoke the provisioned user whenever you like: db.dropUser("indexterity") in admin. Disconnecting the cluster hands you that exact command, because by then the control plane no longer holds credentials that could run it.

The name is fixed, and that is the point

The scoped user is called indexterity on every engine โ€” not a random idx_<hex> per connection. Two things follow.

A cluster stops collecting abandoned users. Provisioning creates the user with admin credentials that are then thrown away, so nothing in Indexterity can ever drop one. With a random name, connect โ†’ disconnect โ†’ connect left three logins behind for somebody to find later and be unable to attribute.

The same cluster cannot be connected twice. The second provision asks the server to create a user it already has, and the server refuses. That refusal is the duplicate check โ€” no rule invented over connection strings, which can spell one server a dozen ways (mongodb+srv versus a seed list, an alias versus an IP, with or without a database, options in any order). The name is the identity.

You will see it as a 422 naming both readings, because the server cannot tell them apart: either the cluster is already connected here โ€” open it instead of adding it again โ€” or it was disconnected and the user outlived it, in which case the message carries the exact statements that remove it.

Removing the scoped user

Engine What removes it
MongoDB db.getSiblingDB("admin").dropUser("indexterity")
SQL Server DROP USER in each database, then DROP LOGIN indexterity
PostgreSQL DROP OWNED BY "indexterity" in each database, then DROP ROLE "indexterity"

The two-step forms are not belt-and-braces, they are required. SQL Server refuses to drop a login while any database user still maps to it, and provisioning created one in every database. PostgreSQL refuses to drop a role its grants still point at โ€” the CONNECT on each database and USAGE on each schema are enough to hold it down:

DROP ROLE "indexterity";
ERROR:  role "indexterity" cannot be dropped because some objects depend on it
DETAIL:  1 object in database postgres

DROP OWNED BY clears them, and it clears them per database: run it in one and the DROP ROLE still fails, naming the next. Both measured on PostgreSQL 18.4.

When a release can use one more privilege

The role is created once, from an admin string that is never kept, so a release that can do more with one more action cannot add it to a role that already exists. What happens instead:

  • Nothing breaks. A privilege a release adds is always optional: the one feature it serves stays off โ€” and says why, in the activity trail โ€” until it is granted. A test holds every engine to that.

  • You are told once, and only if you lack it. The hourly collect asks the cluster what the credentials hold, over the connection it already has open. Credentials that hold the privilege already are never told. Admin strings qualify, and so do broad built-in roles: Atlas's atlasAdmin carries enableProfiler through dbAdminAnyDatabase. Credentials found to lack it get three things:

    • a notice on the cluster's pages naming the privilege and what it turns on;
    • an explanation in the cluster's settings of how to grant it;
    • one mail to the owners.

    The check repeats every collect while the privilege is missing, so a grant run by hand clears the notice within the hour.

  • One statement grants it, on a role Indexterity created โ€” for 0.29.0's enableProfiler:

    db.getSiblingDB("admin").grantPrivilegesToRole("indexterityEngine",
      [{"resource":{"db":"","collection":""},"actions":["enableProfiler"]}])

    On a role you made yourself, add the action to it as in sections 1 and 2.

  • Or upgrade it from the dashboard. Settings โ†’ Connection takes an admin string for the same cluster, uses it once to bring the role to today's definition, checks the result with the stored credentials, and forgets the string โ€” provisioning's terms. The admin needs userAdminAnyDatabase, or viewRole, grantRole and revokeRole on every database including config: the role reads config.collections, so the same three on {db: "", collection: ""} are refused (measured on 7.0).

  • Not granting it is an answer too. "Mark as reviewed" removes the notice and changes nothing on the cluster. Opening the credentials card after granting by hand clears it at once, rather than at the next collect, because that card asks the cluster what the credentials actually hold.

SQL Server

The same offer, with SQL Server's own vocabulary. An admin string that can create logins gets you an indexterity login, granted exactly what the connect form probes and nothing else:

Grant Scope What it is for
VIEW SERVER STATE server sys.dm_db_index_usage_stats โ€” the usage counters
VIEW DATABASE STATE each database sizes, row counts, Query Store
ALTER each schema that owns tables the index DDL: disable, rebuild, create, drop

ALTER is granted per schema rather than per database on purpose: database-wide ALTER would also let the login create, alter and drop tables anywhere in that database, and SQL Server has no index-only permission to grant instead (there is no ALTER ANY INDEX at any scope). No SELECT anywhere โ€” the login cannot read a row of your data, and the server is what enforces that.

Making the offer needs three things of the string you paste, and the preflight names whichever is missing: ALTER ANY LOGIN (create the login), ALTER ANY USER in each database (create its user), and CONTROL SERVER (grant it VIEW SERVER STATE โ€” a login that lacks this creates the login and then fails at the grant, so it is checked up front). sysadmin carries all three.

A database created after provisioning is not covered, because its grants did not exist when the login was made. Nothing is silent about it: the connect form's checks turn red and name that database, and re-running the provisioning with an admin string fixes it. Same for a brand-new schema in an existing database.

Query Store, and what turning it on costs

Query Store is not a permission, it is a per-database setting, and it is the only workload signal on SQL Server that survives a restart โ€” read/write latency per table (which is what the regression gates read) and hinted-index detection both come from it. The connect form lists it beside the permissions for that reason, and when it is off it now hands you the statements rather than a sentence with an ellipsis in it: one ALTER DATABASE per database that is missing it, with a copy button. Run them as an owner of the server โ€” the login Indexterity uses holds ALTER on schemas, not on the database, so it cannot enable this itself.

The statements carry MAX_STORAGE_SIZE_MB = 1000 as well as the enable, and that is measured rather than tasteful. On a new database:

SQL Server 2017 SQL Server 2022
Query Store OFF READ_WRITE (on by default)
QUERY_CAPTURE_MODE ALL AUTO
MAX_STORAGE_SIZE_MB 100 1000
STALE_QUERY_THRESHOLD_DAYS 30 30
SIZE_BASED_CLEANUP_MODE AUTO AUTO

A bare enable on the 2016/2017 generation therefore turns on ALL capture into a 100 MB budget. When that fills, Query Store flips to READ_ONLY and stops capturing โ€” quietly, which is the failure worth avoiding. 1000 MB is simply what 2019 and later ship as the default, so naming it normalises the older generation and changes nothing on a newer server.

QUERY_CAPTURE_MODE is deliberately left alone. AUTO skips trivial one-off statements, which is the right trade on a production server: the queries an index decision rests on run repeatedly, so AUTO captures them, and ALL would buy coverage of statements nobody is optimising at a cost you pay all day.

What it costs while it is on. Runtime stats go to an in-memory buffer flushed every 15 minutes (flush_interval_seconds 900), aggregated into one-hour intervals (interval_length_minutes 60), kept for 30 days, inside the storage budget above with size-based cleanup at ~90% of it. Enabling is an online ALTER DATABASE: no restart, no exclusive lock, no detach. Overhead is a small CPU and memory cost on query compilation and execution paths, which is why the capture mode above matters more than the switch itself.

Two things it will not do. It cannot be enabled on master or tempdb โ€” which is why those never appear in the database list. And a read-only replica captures nothing, so on an availability group the latency signal comes from the primary; Query Store for secondary replicas is a 2022+ opt-in that Indexterity does not turn on for you.

3.1 Which of a stored string's privileges are needed, and which are surplus

Sections 1 to 3 are about what to grant. This is the same list read backwards: what a string you already connected holds and Indexterity never uses.

The connection card's badge says how privileged the stored credentials are โ€” scoped user, scoped credentials, admin credentials. That is true and it is not actionable: it never says what more, so the one thing a reader wants to do about it has no target. Under it, Check what these credentials hold dials the cluster and splits the answer into three:

  • provided โ€” what the credentials actually hold, right now
  • required โ€” the CORE / APPLY / WORKLOAD checks the engine needs on this cluster, for the databases it observes
  • redundant โ€” held, never used, each with the statement that removes it

The redundant statements run the opposite direction from everything else on the connect form: those close a gap (GRANT โ€ฆ), these take a grant away (ALTER ROLE โ€ฆ NOCREATEROLE, REVOKE โ€ฆ, db.revokeRolesFromUser).

Two things worth knowing about the read. It dials the cluster, so it happens when you open the panel rather than on every page view, and it spends the same per-account dial budget as testing a string (ยง5). And the answer is stamped with when it was taken โ€” nothing else in the product re-checks an existing cluster, so a figure without a timestamp would be indistinguishable from one measured at connect time on credentials since rotated.

A cluster running on a user Indexterity provisioned reports an empty redundant group that says it is empty, by construction โ€” the role in ยง1 and ยง2 is the required set. "Nothing surplus" is the reassuring answer and blank space does not deliver it.

What counts as surplus, per engine

Measured against real servers rather than reasoned from documentation, and the list agrees with the role tables above by construction: anything the scoped role holds is required, and this is what is left.

MongoDB

Held Why it is surplus What removes it
root everything on the deployment, documents included db.revokeRolesFromUser
userAdminAnyDatabase creating and altering users and roles โ€” how any grant is made db.revokeRolesFromUser
readWriteAnyDatabase reading and writing every document db.revokeRolesFromUser
dbOwner reading, writing and administering everything in that database db.revokeRolesFromUser
dbAdminAnyDatabase administering every database, dropDatabase included db.revokeRolesFromUser
insert / update / remove nothing in the pipeline writes a document revoke the role above, or db.updateRole on a custom one

The revoke is issued against the user's own database, not admin. revokeRolesFromUser resolves the user in the database it is run from, so a user created in app answers Could not find user "owner" for db "admin" (measured on 8.0) โ€” and a customer's application user usually lives beside its data.

PostgreSQL

Held Why it is surplus What removes it
SUPERUSER everything on the server; the engine uses none of it ALTER ROLE โ€ฆ NOSUPERUSER
CREATEROLE needed once to provision; never by a stored connection ALTER ROLE โ€ฆ NOCREATEROLE
CREATEDB nothing in the analysis or apply path creates a database ALTER ROLE โ€ฆ NOCREATEDB
INSERT / UPDATE / DELETE on tables it does not own nothing in the pipeline writes a row REVOKE โ€ฆ ON ALL TABLES IN SCHEMA, plus ALTER DEFAULT PRIVILEGES

SUPERUSER is listed separately from the other two even though it implies them, because NOCREATEDB on a superuser changes nothing โ€” only NOSUPERUSER removes anything.

Read access is deliberately not on this list, and PostgreSQL is the only engine where that is true. Applying here requires table ownership โ€” there is no grantable index privilege, and pg 17's MAINTAIN is not one either โ€” and an owner can always SELECT. So a cluster you intend to apply on holds read access as a consequence of a requirement, and calling it surplus would be telling you to break your own APPLY row. Writes are different: nothing in the pipeline writes a row, so a write grant on a table the role does not own is surplus outright.

The revoke statements are grouped per database under a psql \connect, because no SQL statement on this engine can cross a database boundary. The ALTER DEFAULT PRIVILEGES line beside each REVOKE is the half people forget โ€” without it the next table created in that schema arrives with the grant again.

SQL Server

Held Why it is surplus What removes it
sysadmin everything on the instance, and every permission below it implicitly ALTER SERVER ROLE [sysadmin] DROP MEMBER
CONTROL SERVER every permission on the instance, granting them onward included REVOKE CONTROL SERVER FROM
db_owner reading and rewriting every table in that database USE โ€ฆ; ALTER ROLE [db_owner] DROP MEMBER

A sysadmin is reported as one finding, not three, and that is measured rather than tidy. HAS_PERMS_BY_NAME reports CONTROL SERVER as held for a sysadmin after it has been revoked, so that row's statement would do nothing; and a sysadmin maps to dbo in every database, so the db_owner row would emit ALTER ROLE db_owner DROP MEMBER [dbo], which SQL Server refuses outright โ€” Msg 15405, Cannot use the special principal 'dbo' (verified on 2022). The membership is the cause of all three, and dropping it is the only change that does anything.

The db_owner statement names the database user, not the login: a login's user is a different name in every database it has one in, so a statement built from SUSER_SNAME() would fail on at least one of them.

4. Sharded clusters

Connect to the mongos. Stats aggregate across shards, and each collection's shard key is read from config.collections so any index it prefixes is protected. For that protection, also grant read on config:

db.grantRolesToUser("indexterity", [{ role: "read", db: "config" }])

Without config read, the collection is treated as unsharded.

5. Connection string

mongodb://indexterity:<password>@host1,host2/?replicaSet=rs0&authSource=admin

mongodb:// and mongodb+srv:// are the MongoDB schemes; SQL Server takes mssql://, sqlserver:// or an ADO Server=โ€ฆ;User Id=โ€ฆ;Password=โ€ฆ list. The connect form lists whichever of those the build you are on supports, so the answer to "does this take my database" is on the screen rather than in these docs. The string is envelope-encrypted at rest in the control plane; rotate the password anytime and re-connect the cluster.

Nothing asks you to pick an engine, and that is deliberate: the schemes are disjoint, so the string says which engine it is and the api decides from the string it was actually given. The form shows what it read yours as โ€” a guess off the scheme while you type, replaced by the api's own verdict the moment you check access โ€” and offers a choice in exactly one case, where no engine recognises the string at all. A picker in front of the strings detection does recognise would be a way to be wrong: choose MongoDB, paste Server=โ€ฆ, and the choice is honoured and a perfectly good string is refused.

Testing and connecting both make the control plane dial the hosts you named, and that is capped at 10 attempts a minute per account โ€” a security control on what we will connect to, not a throttle on you. Past it the form says so, and says how many seconds are left; the window is rolling, so it clears on its own without anyone unlocking anything. Worth knowing when you are iterating on a string that will not connect.

Three things you can change afterwards without losing anything, all under the cluster's Settings tab: the connection string (rotated in place โ€” the new one is dialed and verified before it replaces the old), the name, and which databases are observed (ยง5.1). The name has to be unique inside your organization, because it is what the sidebar lists and what every alert about the cluster is subject-lined with. Neither costs you history; disconnecting does โ€” it deletes every measurement, recommendation, ROI figure and audit row for that cluster, so it is not the way to fix a typo.

5.1 Which databases are observed

A cluster is often not one database. One production database beside three staging copies, a per-tenant clone per customer, and last week's restore is an ordinary shape, and walking all of it is work nobody asked for โ€” plus recommendations for indexes on data that is about to be dropped.

So the connect form asks. Check access first: the answer lists every user database the credentials can see, each ticked, and unticking one leaves it alone. The cluster's Settings tab has the same list afterwards, read live from the cluster, so a database created last week is offerable today.

What counts as a user database, which is the same question on all three engines and now has the same answer: the databases the engine owns and uses are excluded by name, and nothing else is.

Engine Never listed
MongoDB admin, local, config
PostgreSQL none by name โ€” templates and anything with connections disabled cannot be dialled at all, so they never appear
SQL Server master, tempdb, model, msdb (and anything not ONLINE)

PostgreSQL's postgres is the awkward one and gets a narrower rule: it is not a system database โ€” initdb creates it so that a client has somewhere to connect before anything else exists โ€” but on almost every install it holds nothing. So it is listed only when it holds a table of yours. An install that really does keep application tables in postgres sees it and can observe it; everyone else sees one application database and is not asked to choose between it and an empty one.

If your cluster has exactly one database, nothing is asked and it is observed. The choice appears at two.

Everything is observed unless you say otherwise. That is the default at connect time and it is what every cluster connected before this existed does โ€” a scripted POST /clusters that has never heard of the field keeps working unchanged.

A narrowed selection does not grow by itself. Add a database to the cluster next month and it is not observed until somebody ticks it. That is the cost of naming databases rather than excluding them, and it is the honest trade: the alternative silently starts reading every database anyone adds. The Settings screen counts what it is leaving out for exactly this reason, so the answer to "why are there no recommendations for the new service" is on the screen. A cluster observing all of them keeps observing all of them, additions included โ€” which is why "all" is stored as "all" and not as a list of today's names.

It sharpens the privilege check. Indexterity's per-database requirements pass when every database in scope is covered by the role. Before you could narrow the scope, a role granted on one database of twelve read as a missing privilege, and the only way to connect that cluster was to widen the role over the eleven nobody wanted read. Now the form offers to re-check against the databases you ticked, and the same role reads as a grant. The re-check appears only when something is actually missing โ€” a role that covers everything says the same thing about one database as about twelve.

It does not narrow what a provisioned user may reach, and that is a decision rather than an oversight. Let Indexterity create the login (ยง3) and it is granted across every user database that exists at that moment โ€” on SQL Server VIEW DATABASE STATE plus ALTER on each schema holding tables, per database, because that is the only scope those permissions have. What it is never granted, anywhere, is SELECT: the login cannot read a row in any database, observed or not, and the server enforces that (Msg 229 on a plain SELECT, verified on 2022).

The alternative was tried and reverted. Granting only where the selection pointed made the selection un-editable in practice: provisioning runs once, from an admin string that is never stored, so a login created for two databases of twelve could never be given the third โ€” ticking it was a dead end with no way out inside the product. Keeping the two apart means "which databases do we look at" stays a setting you can change at any time, and "which databases could this login look at" was settled once, when you consented to it.

One gap remains, and it is refused rather than silent. A database created on the cluster after the login was has no user for it, because nothing re-runs the grants. SQL Server will still offer that database on the settings screen โ€” every login sees every row of sys.databases, since VIEW ANY DATABASE is granted to public โ€” and every read of it fails with Msg 916 ("The server principal โ€ฆ is not able to access the database โ€ฆ under the current security context"). So the save probes each newly ticked database with the credentials actually stored and names the ones they cannot read. Two ways forward, both keeping every measurement: grant that login access on the cluster yourself, or rotate to a connection string that already has it.

The same shape reaches every engine, from your own connection strings. A credential that can list a cluster's databases and read only some of them is what a scoped user normally is, whoever created it:

  • MongoDB โ€” a user holding the cluster-wide listDatabases action with per-database grants that do not cover everything. Every read of an uncovered database answers not authorized on <db>.
  • PostgreSQL โ€” a role without CONNECT on a database, which is what revoking PUBLIC's default grant leaves behind. The refusal is permission denied for database "<db>", and it arrives when we first dial that database.
  • SQL Server โ€” Msg 916, as above.

All three are treated identically: the settings screen names the databases the stored credentials cannot read and refuses the save rather than starting to "observe" something we will never see.

If access is lost after the tick โ€” a user dropped, a database restored over, a grant revoked โ€” the collect skips that one database and keeps walking the rest of the cluster. It used to abort the entire pass on the first database it could not reach, so one inaccessible database cost every measurement on the cluster.

(MongoDB's provisioned role is cluster-wide for the same reason, and gives up less by being so: it grants index metadata actions and explicitly withholds reading a document, so a database outside the selection is one we can list indexes on and still cannot read a row from.) What happens to what was already collected. Nothing is deleted. Measurements for a database you untick are kept โ€” the next collect simply stops walking it โ€” and open proposals for it are discarded, because a proposal is a suggestion to change something we have stopped watching. Changes already in flight there are not discarded: an index the engine has hidden pending a drop stays tracked through its observe window and is restored on disconnect like any other. And approving a proposal in a database that is no longer observed is refused rather than quietly executed.

A database that goes away. Drop or rename a selected database and nothing breaks: each collect walks what is both selected and present, so the missing name simply stops matching, and if it comes back it is picked up again with no re-ticking. The Settings screen points out a selected name the cluster no longer reports, since that is the one place you would ever see it.

A cluster observing nothing is refused. Untick the last database and the form says so. It would be indistinguishable from a broken cluster on every panel afterwards; disconnect it instead.

5.2 Reaching a cluster with no public endpoint

Everything above assumes Indexterity can open a socket to your database. A cluster inside a VPC with no public endpoint, or on-prem behind a firewall, is not reachable from a control plane sitting somewhere else โ€” and no connection string fixes that.

Paste the WireGuard config that network already uses. Settings โ†’ VPN tunnels takes a whole wg0.conf, the file your VPN admin exports. Indexterity terminates the peering itself, in userspace: no agent to install, no container to run, no privileges to grant on your side. Then pick that tunnel on the connect form and the cluster connects like any other.

What moves those packets is wireguard-go with gvisor's network stack โ€” the same pairing Tailscale's userspace mode uses โ€” in a service that runs beside the control plane and nowhere else. That is a detail you never touch, and it is here only so nobody has to guess: your peer is talking to a standard WireGuard implementation.

If Settings โ†’ VPN tunnels says the feature is turned off, this deployment has no tunnel service running and no peering can be brought up. On a self-hosted install that is one setting away; on a hosted one it is worth asking about rather than working around.

[Interface]                    โ† our side of the peering
PrivateKey = ...               โ† encrypted before it is stored, never shown again
Address    = 10.9.0.2/32
DNS        = 10.9.0.1          โ† who resolves names INSIDE the tunnel

[Peer]                         โ† your gateway
PublicKey  = ...
Endpoint   = vpn.example.com:51820
AllowedIPs = 10.0.0.0/8        โ† what this tunnel is allowed to reach

Pick the tunnel when you connect, not afterwards. The connect form checks what your credentials can do before storing anything, and that check dials your database โ€” so it has to go through the tunnel too. There is a How to reach it picker above the certificate boxes; it only appears once you have registered a tunnel.

AllowedIPs is a limit, not a hint. It is what your peer agreed to carry, so it is what Indexterity will dial and nothing else. A connection string pointing outside it is refused with that reason rather than timing out, because a timeout is indistinguishable from a database that is merely down.

Give the tunnel a DNS line if you address hosts by name. A private replica set's member names mean something only on your side of it. Without a resolver in the config, Indexterity cannot look them up โ€” the tunnels screen says so, and the failure otherwise arrives as "unreachable", which reads like a firewall problem and sends you looking in the wrong place.

When something stops collection, the cluster says so. A cluster nobody can reach used to show one thing: last collected 7 days ago. That is true and it is not enough โ€” a paused schedule and a quiet cluster look identical to a cluster whose firewall changed, so the badge that needed you read like the two that did not. The cluster's heading now carries a red badge and a banner underneath it naming what happened, how long it has been happening, the failure in the driver's own words, and what to do about it. The wording distinguishes the cases that matter: we could not reach it, versus we declined to dial it because the stored string would connect in plaintext, versus your VPN gateway is not answering while the database itself may be fine. Your owners still get one email a day about it; this is what is there when they open the page a week later.

A tunnel that is down is a condition of the tunnel. The tunnels screen shows each one as Up, Connecting, Down, or Idle โ€” Idle simply meaning nothing has needed it since the server started, which is the normal state of a healthy new one. A cluster behind a down tunnel is unreachable, not broken, and both screens say so rather than blaming the database.

Test it rather than believing it. Registering a tunnel proves the file parses, and nothing more. A mistyped PublicKey, a gateway that moved, a UDP port a firewall drops and a peering your VPN admin revoked are all perfectly valid files, and all four fail identically at the first collect โ€” as a timeout you would reasonably read as a problem with your database. Each row has a Test: it negotiates a handshake with your gateway there and then and tells you what came back. It forces a fresh handshake rather than reporting the current state, because a tunnel can sit on a session negotiated an hour ago against a gateway that has been switched off since โ€” WireGuard has nothing to say about that until traffic needs a new key. Silence is the one answer with no cause to report: an endpoint that is not there, a dropped UDP port and a PublicKey your gateway does not recognise look identical from our side, so the message names the three rather than guessing between them.

Editing: rename freely, replace the config whole. Edit on a row renames a tunnel โ€” the name is only how the connect form refers to it โ€” and takes a whole new wg0.conf when a key is rotated or a gateway moves. There is no partial edit of a stored config, and that is not an omission: the PrivateKey in it never comes back from the api, so there is nothing anyone could show you to amend. Leave the config box empty and only the name changes.

Saving a new config drops the live peering rather than swapping it underneath. It comes back up on the next collect, or the moment you press Test, and clusters behind it are unreachable for that gap rather than dialling with a key that has just been rotated away โ€” which is the failure that would be silent.

An IPv6-only peering works. A wg0.conf whose [Interface] Address is a v6 address and whose AllowedIPs is a v6 prefix used to be refused with a message saying the network stack carried IPv4 only. That limit is gone, and it has been tested end to end against a real WireGuard peer rather than assumed โ€” the tunnel comes up and a database inside it answers on its v6 address. The gateway's own Endpoint is a separate matter and may be either family.

Every one of these acts is on the security trail, including pressing Test. Registering a tunnel, replacing its config and removing it each write a row naming who did it, the gateway involved and โ€” for a replacement โ€” what the peering reached before and after. A test writes one too, with what came back: a gateway that has been asked six times and answered none of them is worth being able to see. The config itself is never recorded: it holds a private key, and the trail is read by people who are not meant to be able to bring the tunnel up.

What it does not change. TLS is still required: the tunnel carries ciphertext and adds a second layer rather than replacing the first, so a cluster reached through it still needs a certificate Indexterity can validate. Cloud metadata and the other never-a-database ranges stay refused whatever route exists to them โ€” a peer whose AllowedIPs covers 169.254.169.254 is a misconfiguration. And your credentials still leave your network: the connection string is stored by Indexterity and opened by Indexterity. A tunnel changes which addresses are reachable, not where the secret lives. Only an agent that runs the queries inside your network would change that, and that is #272.

Self-hosted: routing at the node instead

If you run Indexterity yourself and would rather your own network stack carry the tunnel, you can skip all of the above: run a WireGuard client as a sidecar in the api pod and set ALLOW_PRIVATE_CLUSTER_TARGETS=true. The route it creates is one the api dials through, with no tunnel registered in the product at all. The recipe, with values, is in the chart's README.

That flag is global, which is why it is a self-hosted answer and not the hosted one: on a shared install it would let every tenant aim a connection string at the operator's own private network. A registered tunnel is scoped to the clusters that use it instead.

6. Replica sets โ€” which members get read

$indexStats and $collStats latencyStats are per member; both sum all of them, so an index used only on a secondary counts as used, and a cluster's write latency is read from the node that actually takes writes. Both are aggregations, so the driver routes them by read preference โ€” a connection string carrying readPreference=secondaryPreferred had every latency reading landing on a secondary, whose write counters are permanently zero because oplog application is not a client write op.

Which members, exactly: every one hello names, and that is two arrays, not one. Electable members are listed under hosts, but a priority: 0 member is listed under passives โ€” and priority 0 is the normal setting for a secondary in another region, so that it cannot win an election across a WAN. Priority governs elections and nothing else, so those members serve secondaryPreferred reads like any other secondary and both arrays are collected from.

Hidden members are not, and this is a deliberate limit rather than an oversight. They appear in neither array, so reaching them needs replSetGetStatus or replSetGetConfig โ€” cluster privileges the engine role does not ask for. Drivers never route reads to a hidden member either, so what is missed is direct-connected traffic: BI tools, backups, delayed members. If an index exists solely for a hidden analytics node, hint it or keep it out of the pipeline.

7. mongodb+srv:// strings

What Atlas hands you carries two settings outside its own text: the scheme defaults tls to true, and authSource arrives in a DNS TXT record the driver reads at connect.

Per-member connections have to rewrite the string as plain mongodb:// โ€” an SRV seed cannot be pointed at one host, since its targets live in DNS and the scheme forbids a port โ€” so both are carried across explicitly from the live client rather than re-parsed from the original text. Without that the member connections are plaintext, authenticate against the database in the path, and fail; a plain multi-host string was never affected, because its options are in the string.

โš ๏ธ **GitHub.com Fallback** โš ๏ธ