What to monitor on a vector database
Most vector-database dashboards show CPU, memory, request rate, and p99 latency. All four are real metrics. None of them tells you the thing you most need to know, which is whether the search is still returning good results.
A vector database can degrade in quality while every infrastructure signal stays flat. That is the central monitoring problem here and it’s why this needs its own dashboard rather than a copy of your service template.
Tier 1: the signals that predict
These lead incidents. Alert on them.
Resident memory as a fraction of the limit. The single most valuable signal. Vector workloads fail memory-first, and the failure is a cliff rather than a slope — everything is fine until the index no longer fits, then it’s swapping or OOM-killed. Alert well before the limit, because you need lead time to shard or resize, not a page at 98%.
Headroom for a rebuild. A derived signal worth computing explicitly: current usage plus your rebuild strategy’s peak multiplier, against the limit. This is the number that tells you a routine reindex will take the service down, and it’s better to learn that from a dashboard than from triggering one. See sizing for where the multiplier comes from.
Vector count and growth rate. Trivial to collect, routinely missing. You want the count, the daily delta, and a projection against your memory ceiling. This converts capacity planning from a periodic panic into a line on a graph with a date on it.
Index build duration. Track every build. It creeps up with corpus size, and the trend tells you when your maintenance window stops being long enough — usually months before it actually does.
Segment or shard count. Engines that write segments and compact them in the background can accumulate segments faster than they merge. A rising count means query fan-out is rising too, which shows up as latency later. This is a genuine leading indicator.
Tier 2: the signals that confirm
These tell you something is wrong now. Dashboard them; alert selectively.
Query latency percentiles — p50, p95, p99, separately. Never the mean; ANN latency distributions have a long tail and the mean hides it. The shape is diagnostic: p50 rising with a stable p99 usually means the whole index got slower (memory pressure, more segments). A stable p50 with a rising p99 usually means a subset of queries got harder — often filtered ones. See when your vector database gets slow.
Queries per second, split by type. Filtered versus unfiltered, and by collection. Aggregate QPS hides a tenant who started hammering one collection with expensive filters.
Result set sizes. Specifically, the rate of empty or short results. A rise means a filter is over-selective, a backfill has a gap, or something upstream is sending malformed queries. This is one of the few cheap signals that catches quality problems, and almost nobody collects it.
Write and delete throughput. Deletes especially — see the tombstone section below.
Error rate by class. Timeouts, OOMs, and rejected requests mean different things and should not be a single counter.
Tier 3: the quality probe
This is the one that doesn’t come from your engine’s metrics endpoint, and it’s the reason the rest of the dashboard isn’t enough.
Run your fixed query set on a schedule and record recall against known-good results.
Take the committed query set from backing up a vector database — representative queries with expected document IDs. Run it hourly against production. For each query, compute the fraction of expected IDs present in the returned top-k. Record the aggregate.
Now you have a quality time series. Things it catches that nothing else does:
- An index that rebuilt with wrong parameters and is returning plausible but worse neighbours.
- A partial backfill that left a slice of the corpus unsearchable.
- A quantization or dimensionality change that cost more recall than expected.
- Metadata filters silently excluding valid documents.
- An embedding model swap that nobody flagged as a breaking change.
Every one of those leaves CPU, memory, and latency completely normal.
Two cautions. Alert on a sustained drop, not a single sample — approximate search is nondeterministic and individual runs vary. And regenerate the expected results after any intentional model or corpus change, or the probe will report a successful migration as an outage.
Tombstones and delete accumulation
Worth its own section because it’s the most common slow-degradation pattern.
Most vector engines don’t remove a deleted vector from the graph immediately. They mark it, keep it in the structure, and skip it at query time — then reclaim the space during compaction or a rebuild.
That means a high-churn collection accumulates dead entries the search still has to traverse. Query latency rises gradually, memory doesn’t fall after deletes, and recall can drop because the traversal spends its budget on tombstones instead of live neighbours.
Track deleted count as a fraction of total. When it crosses a threshold — engine-dependent, and worth establishing on your own workload — compaction or a rebuild is due. Without this metric the symptom presents as “the database just got slower over the last two months” with no obvious cause.
The dashboard
One screen. Roughly:
| Panel | Signal | Why |
|---|---|---|
| Capacity | Resident memory vs limit; projected rebuild peak | The cliff |
| Growth | Vector count, daily delta, projection to ceiling | Lead time |
| Quality | Recall probe, hourly | The one nothing else covers |
| Latency | p50 / p95 / p99, filtered vs unfiltered | Shape is diagnostic |
| Health | Segment count, deleted fraction, build duration | Slow degradation |
| Traffic | QPS by collection, empty-result rate, errors by class | Blame assignment |
Alerting
Page on: memory approaching the limit; sustained recall drop; error rate above baseline; p99 beyond your latency budget for a sustained window.
Ticket, don’t page: segment count high, deleted fraction over threshold, build duration trending up, projected capacity date inside the next quarter. These are all weeks of warning, and paging on them trains people to ignore the dashboard.
Never alert on: raw CPU. On a vector workload it’s neither leading nor diagnostic — it’s high during builds, which is correct, and unremarkable during the memory exhaustion that will actually take you down.
Getting started
If you have none of this, add two things this week: memory as a fraction of the limit, and the hourly recall probe. Those two catch most of what will hurt you. The rest can follow.