ADR 005 Cosmos Full Text Search Ruled Out - digitalspace/eagle-demi GitHub Wiki

ADR-005: Cosmos DB full-text search ruled out

Status

Rejected 2026-07-31. Azure AI Search serves search instead — see Search-and-Retrieval.

Traps 1-5 below are properties of the Cosmos SDK, not of this decision, and they apply to any ranked query.

Context

Cosmos DB for NoSQL added full-text search. DEMI already stores every chunk's content in the chunks container, so serving search from the same database would have meant no second service, no second bill and no sync lag.

Base FTS works and is fast: 0.3–0.9 s, ~25 RU per ranked query, roughly $9/month. By far the cheapest option costed.

Decision

Rejected, because fuzzy matching is a hard requirement and Cosmos cannot serve it.

The backfill never ran. chunks_fts held 0 rows and is deleted, along with COSMOS_FTS_FUZZY, searchText, queryRanked/drainRanked and buildSnippet.

Why: fuzzy is a silent no-op

demi-cosmos-dev carries EnableNoSQLFullTextSearchPreviewFeatures, the enrolment Microsoft's docs require. It made no difference. The syntax parses, the term matches, and distance does nothing.

query	fuzzy	hits
`quokkafluxion`	true	1 — exact still matches
`quokafluxion` (1 deletion)	true	0
`riparia` / `riparians`	true	1 — that is the stemmer, not fuzzy

The decisive case is a nonsense token one edit from an indexed one, which no stemmer can bridge.

Eliminated by measurement, in order: not enrolment; not the app flag; not parameter binding (rewritten as an inline SQL literal, verified in wwwroot, identical); not query shape; and not a stale index — the container was deleted and recreated from scratch after enrolment and behaved identically. What remains is server-side: region gating (canadacentral) or the preview not honouring distance yet.

Why the original spike said fuzzy worked

The first spike reported that fuzzy on real typos (rivver, propnent, climat) returned all 20 hits. 20 was the TOP cap, and three different typos all returning exactly the cap is the signature of trap 4 below — a ranked query whose full-text predicate matched nothing returns the whole container, ranked but unfiltered. That reads as "fuzzy works brilliantly" and is in fact "the predicate did nothing".

A fuzzy probe must use a rare or nonsense term and assert an exact expected count, never a capped one.

Six silent-failure traps

None of these raise an error. All return zero rows, and none are documented by Microsoft.

fetchNext() returns empty early pages for ranked queries. Measured: fetchAll gave 5 hits at 24.72 RU; one fetchNext gave 0 hits at 4.83 RU with hasMoreResults: true; draining 3 pages gave 5 hits. src/db/cosmos-nosql.js:query() runs exactly one fetchNext() whenever maxItemCount is set, so every repository method built on it returns nothing for a ranked query. This produced three wrong diagnoses during the spike.
Index lag after a bulk load. Ranked queries return 0 until x-ms-documentdb-collection-index-transformation-progress (a container read with populateQuotaInfo) reaches 100. FULLTEXTCONTAINS masks it by scanning; ORDER BY RANK cannot. Poll it before any cutover.
distance: 3 returns 0 rows and does not error. The documented maximum is 2.
ORDER BY RANK without a WHERE returns everything, ranked but unfiltered.
A ranked query matching zero rows against a populated container blocks the Node event loop. FULLTEXTCONTAINSALL with ORDER BY RANK that matches nothing spins synchronously inside the SDK's client-side merge: the whole app stops answering, /api/config included, for minutes, with no error, no crash, no container restart and nothing in any log. It then recovers on its own, which makes it look like a network fault.

Verified at index progress 100, and it reproduces with fuzzy=false, so it is GA surface rather than preview. No client-side timeout can defend against it — AbortSignal.timeout() and elapsed-time checks between pages are both timers, and a blocked event loop runs no timers. Two fixes built on that premise were deployed and failed. The only defence is to not issue the query: run a COUNT with the identical predicate and no ORDER BY RANK first, and return empty on zero.
Fuzzy {term, distance} is a no-op, as above.

Durable Cosmos facts from the spike

These hold regardless of the search backend.

Container creation is control-plane. The app's managed identity gets 403 ... cannot be authorized by AAD token in data plane. Containers come from ARM/Bicep only.
A full-text policy is immutable. Enabling FTS means a new container plus a copy. Vector policies are narrower than they read: adding one to an existing but empty container succeeded in place. What cannot change is an existing embedding's dimensions or metric.
CONTAINS matches mid-word. Query env returned "Grenville to Kincolith Road". STARTSWITH matches only the start of a field, so neither one gives token-prefix behaviour on its own; that needs STARTSWITH(f,q) OR CONTAINS(f,' '+q).
Ranked queries cannot page by continuation token, so chunk search is TOP-N with no pagination.
Metadata search on live containers with no FTS policy (pure STARTSWITH/CONTAINS) is cheap, measured 2026-07-31: 393 projects at 3–6.5 RU / 61–70 ms; 60,578 documents STARTSWITH 9.23 RU / 75 ms, CONTAINS 67.7 RU / 81 ms, multi-field OR with ACL 39.3 RU / 62 ms, project-scoped 15.1 RU / 34 ms.
Chunks are durable in Cosmos regardless of backend. src/controllers/nosql/document.js writes the full content into the chunks container. The search backend is a query layer over it, which is what made replacing it a configuration change rather than a data migration.