Cold start retrieval: warm and cold queries are different benchmarks
A database on object storage answers a query on open data from RAM in about a millisecond, and the first query to data it doesn’t hold in tens of milliseconds. With many tenants that sit idle between sessions, both numbers matter. Measure them separately, and look at what each query had to read.
Three temperatures, reported on every answer
SkimmerDB keeps every namespace in object storage and uses servers’ RAM and disk as a cache. Each search, count, vector and SQL answer says which of these it was and how many object-store reads it made (temperature, store_gets, store_bytes).
| temperature | What the server had | Object-store reads |
|---|---|---|
warm | The namespace is open on the server: RAM and local NVMe only | 0 |
local | Opened from the server’s disk, which kept its files | 0, unless a file has to be re-checked |
cold | Opened from object storage: nothing of it on this server | One or more, reported per answer |
Cold and warm, measured
From our benchmark runs of October 5–6, 2026. The real-S3 rows are c7gd.2xlarge servers in us-east-1 with a 400 MB cache each, 8 clients and a Zipf mix of 50% text, 20% vector and 30% small-tenant queries. The other rows use a local object store with 20 ms added to every request to model S3.
| Workload | Cold | Warm |
|---|---|---|
| Real S3, 1 server, Zipf mix (p50 / p90 / p99) | 1.0 / 33.0 / 130.5 ms | 0.8 / 25.0 / 64.3 ms |
| Real S3, 3 servers, same mix (p50 / p90 / p99) | 2.0 / 40.3 / 413.9 ms | 1.4 / 26.4 / 71.6 ms |
| Small namespace, store +20 ms a request | 25.6 ms | 0.14 ms |
| Big namespace (50,000 docs), store +20 ms | 34.8 ms | 2.09 ms |
| One of 10,000 namespaces, evicted from RAM and disk (p50 / p99) | 24.0 / 31.3 ms | 0.0 / 0.1 ms |
| Vector query, 1.34M × 768-d (p50 / p90) | 63 / 70 ms | 0.80 ms p50 |
In the real-S3 mix, “cold” is a phase that starts with empty disks, so its p50 is already mostly warm queries: the cost of cold data shows in the p90 and p99. That’s the first lesson for your own test: a median over a mixed run hides the cold queries.
What makes a cold query slower
- Round trips, not bytes. Each sequential trip to S3 costs 20 to 50 ms. A small namespace opens in one read; a cold vector query reads the clusters it probes in two parallel rounds.
- Namespace size. Our small namespace made 0.74 reads a cold query; a 50,000-document one made 11.5.
- Query shape. A selective filter or a regex with a literal reads less. A regex with no literal reads every document.
- Servers. With 3 servers each sees fewer repeats of the same data, so the cold tail was longer (p99 414 ms vs 131 ms).
What a cold query doesn’t do
- Download the namespace. It fetches the pieces the query needs, by range.
- Lose writes. A write is acknowledged only once it’s in object storage; a fresh server on an empty disk found 50,040 of 50,040 acknowledged documents.
- Cost compute while idle. An idle namespace keeps 806 bytes of state plus its documents in object storage, and nothing runs for it.
What cold reads cost
Cold queries are billed as a query plus their object-store reads, at $0.50 per million reads on pro. At the read counts we measured, a million queries’ reads would cost:
| Workload | Reads a query | Reads per million queries |
|---|---|---|
| Small namespace | 0.74 | $0.37 |
| Big namespace (50,000 docs) | 11.5 | $5.75 |
| Real S3 Zipf mix, cold phase, 1 server | 1.44 | $0.72 |
| Real S3 Zipf mix, warm phase, 1 server | 0.43 | $0.21 |
Plus $0.50 per million queries, storage and writes, and a $10 monthly minimum that usage counts against. The free plan includes 1 million cold reads a month. Pricing has the rest.
Warm a namespace before the agent needs it
When you know an agent is about to work on one customer’s namespace (a support chat opens, a coding session starts), warm it with the searches it is likely to run. The session’s first queries then answer from RAM. Log the same fields from every answer next to your own timing.
B=https://api.skimmerdb.com/v1 H=(-H "Authorization: Bearer $SKIMMERDB_KEY" -H "Content-Type: application/json") # Before an agent starts a session on a customer's namespace: open it and fetch # what its likely searches need (up to 20). Counts as one query. curl -fsS "$B/namespaces/kb-birch/warm" "${H[@]}" \ -d '{"queries":["refund","ERR-404"]}' | jq -c '{temperature, store_gets, store_bytes, took_ms}' # temperature is what it was BEFORE warming: warm, local or cold # Every answer says what it had to read; log these next to your own timing curl -fsS "$B/namespaces/kb-birch/search" "${H[@]}" \ -d '{"q":"refund"}' | jq -c '{temperature, store_gets, store_bytes, took_ms}'
How to test it on your own data
- Write the workload down first: documents and sizes, vector dimensions, query mix, filter selectivity, concurrency, region, and where the client runs.
- Make the cache state on purpose. A new client process doesn’t make the server cold, and a restart that keeps the disk is
local, notcold. Use a namespace nobody has queried, and checktemperatureon the answer. - Check answers before timing them. Compare ids for exact search and recall against ground truth for vectors. Count partial or timed-out answers instead of dropping them.
- Report distributions: p50, p90 and p99 with the sample count, for cold and warm separately. Never a best run.
- Price the whole month: how often each tenant goes cold, times the reads a cold query makes, plus queries, storage and writes.
Questions
What is a cold start in a vector database?
The first query to data the server doesn’t hold yet. On a database built on object storage, it has to fetch the index pieces it needs from S3 before it can answer, which costs one or more round trips of 20 to 50 ms each. Later queries on the same data are warm and read only RAM and local disk.
How slow is a cold query on SkimmerDB?
In our runs: about 25 ms for a small namespace and 35 ms for a 50,000-document one with 20 ms added to each object-store request, 24 ms p50 and 31 ms p99 for one of 10,000 namespaces evicted from memory and disk, and 63 ms p50 for a vector query over 1.34M 768-d vectors. These are our workloads and hardware; measure yours.
Does a cold vector query download the whole index?
No. Without a filter or text, it reads only the clusters it probes, in two parallel rounds of object-store reads. A text search opens the namespace’s document lists for the terms it needs.
How do I avoid cold starts for an agent?
Call POST /v1/namespaces/{name}/warm with the searches the agent is likely to run before its session starts. It opens the namespace and fetches what those searches need, so the first queries are warm. It counts as one query.
What does a cold query cost?
A query, plus the object-store reads it makes, at $0.50 per million reads on pro (the free plan includes 1 million a month). Every answer reports store_gets, so you can see it per query and in GET /v1/usage per day.
Does an idle namespace cost anything?
Only its stored bytes, at $0.10 per GB-month on pro. Nothing runs for it until the next query, which is why that query is cold.
Try it with your own documents
Free up to 1 GB stored and 100k queries a month. No card to start.