SkimmerDB vs DuckDB and MotherDuck
DuckDB is the analytical engine you embed in your process. SkimmerDB is a hosted database that apps and agents search over an API, one namespace per tenant, with SQL on the same data. They meet at Parquet and DuckLake, so you don’t have to pick one.
Side by side
Different tools for different jobs, compared where they overlap.
| DuckDB / MotherDuck | SkimmerDB | |
|---|---|---|
| What it is | An analytical SQL engine you run in your process (MIT); MotherDuck hosts it | A hosted database over HTTP and MCP: search first, SQL on the same data |
| Where data lives | Your files (Parquet, CSV, DuckLake) or a .duckdb file | Object storage we run, or read from your S3 / R2 / B2 bucket and DuckLake |
| Exact and regex search | LIKE and regexp_matches scan every row | A trigram index answers substrings and RE2 regex, newest first, with exact counts |
| Full-text index | fts extension: BM25, not updated when the table changes unless made incremental or rebuilt | Every write is searchable within tens of ms; no index to rebuild |
| Vectors | vss extension: HNSW in RAM; on-disk persistence experimental, deletes not reflected until compacted | IVF + RaBitQ on object storage, hybrid with text and filters, recall@10 0.97 by default |
| Many tenants | A file or database each, or one table and a filter | A namespace per tenant and keys that reach only theirs; idle ones cost their bytes |
| Agents | You write the tool | A remote MCP server with OAuth, scoped to the user’s namespaces |
| SQL breadth | Very wide: the reference for analytical SQL | Joins, subqueries, windows, CTEs (recursive too), JSON; narrower than DuckDB |
Choose DuckDB when
- The analysis runs where you are. A notebook, a script, a pipeline step, over files you already have.
- The SQL is the job. Wide analytical SQL, extensions, and the fastest joins and million-group GROUP BYs we’ve measured against.
- One analyst, one copy. No tenants to keep apart and no API to serve.
Choose SkimmerDB when
- An app or agent searches it. Substrings, regex, filters and vectors answered from indexes, over REST or MCP.
- Every customer gets their own. A namespace and a key per tenant, made on the first write; idle ones cost their bytes.
- Nobody runs a server. Writes are searchable in tens of milliseconds and stored in object storage, with no index to rebuild.
Use them together
Keep the lake where it is. SkimmerDB syncs a DuckLake table into a namespace (only the rows that changed), answers the searches, and hands answers back as Parquet in your bucket.
# keep a DuckLake table searchable: only what changed is written curl -s https://api.skimmerdb.com/v1/namespaces/tickets/lake/sync \ -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d '{ "catalog": "postgres://lake_reader:...@lake-db.example.com:5432/lake?sslmode=require", "table": "main.tickets", "text": ["subject", "body"], "s3": {"endpoint": "https://<account-id>.r2.cloudflarestorage.com", "region": "auto", "access_key_id": "...", "secret_access_key": "..."}}' # then the search DuckDB has no index for curl -s https://api.skimmerdb.com/v1/namespaces/tickets/search \ -H "Authorization: Bearer $KEY" -d '{"q": "ERR_[0-9]{4}", "regex": true, "filter": {"status": "open"}}'
-- and back to DuckDB: an export delivered to your bucket as Parquet
-- POST /v1/sql/exports {"sql": "SELECT * FROM tickets WHERE ...", "format": "parquet",
-- "destination": {"path": "s3://analytics/tickets.parquet", "s3": {...}}}
SELECT status, count(*) FROM 's3://analytics/tickets.parquet' GROUP BY status;Measured, both ways
Our numbers, on our machines; DuckDB on the same machine each time. Faster on single-table scans, even on distinct counts, slower on some joins and on GROUP BY with millions of groups.
| Workload | Result |
|---|---|
| ClickBench, 43 queries, 10M rows, hot, 8 threads (geomean) | SkimmerDB’s column engine 0.39x DuckDB 1.5.5’s time; won 35, tied 8, lost 0 |
| ClickBench, 100M rows, 16 threads | 0.47x DuckDB’s time; lost one query (Q9, 232 vs 205 ms) |
| Join + GROUP BY, 2M orders x 4M items | 47 ms vs DuckDB 35–47 ms |
| COUNT(DISTINCT sku) by order, 4M items, 1.7M groups | 32 ms vs DuckDB 33 ms |
| GROUP BY id over 2M orders (2M groups) | 22 ms vs DuckDB 11 ms: DuckDB 2x faster |
| GROUP BY order_id, sku over 4M items (3.9M groups) | 37 ms vs DuckDB 20 ms |
ClickBench: SkimmerDB’s column engine (the one its SQL runs on, measured directly, not through the HTTP API) against DuckDB 1.5.5 on EC2 m7gd.4xlarge, every engine started fresh for each query, 3 runs interleaved, best of runs 2–3, October 2026. Joins and groups: 2M orders and 4M items on a 14-core laptop, SkimmerDB’s SQL engine and DuckDB’s CLI on the same files, best of 5–7, October 2026; the many-group queries are wrapped in an outer count and sum, so neither side prints millions of rows. The laptop was loaded, so treat the gaps as approximate. Through the HTTP API, ClickBench’s 10M-row geomean was 9.3 ms (agent analytics).
What DuckDB’s own docs say about search
Read October 22, 2026.
| Topic | What the docs say |
|---|---|
| DuckDB full-text search | Indexes are made with PRAGMA create_fts_index and ranked with BM25. “By default, the FTS index will not update automatically when the input table changes”: create it with incremental = 1 or make it again. |
| DuckDB vector search | HNSW indexes on FLOAT arrays. On-disk persistence sits behind hnsw_enable_experimental_persistence, which the docs recommend against in production; “the index itself is not buffer managed and must be able to fit into RAM”; deletes are marked until the index is compacted. |
| MotherDuck | “The serverless cloud data warehouse built on DuckDB”, whose architecture “gives every user or AI agent an isolated compute instance”; it connects to DuckLake. |
Questions
Is SkimmerDB a DuckDB replacement?
No. Use DuckDB for analytics in your own process or notebook, over files you already have. Use SkimmerDB when an app or agent needs to search that data over an API, many tenants each need their own copy and key, or nobody wants to run a server. Many teams will use both: SkimmerDB exports Parquet that DuckDB reads as is, and keeps DuckLake tables searchable.
Does SkimmerDB run on DuckDB?
No. Its SQL runs on its own column engine and its search on a trigram index, both written for object storage. It reads DuckLake catalogs and Parquet, and writes Parquet, so data moves between them without conversion.
Is SkimmerDB faster than DuckDB?
On ClickBench’s single-table scans and aggregations, its column engine measured faster than DuckDB 1.5.5 on the same machine, and COUNT(DISTINCT) over 1.7M groups ran even. On some joins and on GROUP BY with millions of groups DuckDB is faster, up to about 2x. Those are our measurements on our machines; run your own queries before deciding on speed.
What about Amazon buying DuckLabs?
Amazon announced on August 26, 2026 that it agreed to acquire DuckLabs, the company behind DuckDB, while DuckDB itself stays MIT-licensed under the DuckDB Foundation. SkimmerDB reads data on AWS S3 and on S3-compatible stores such as Cloudflare R2, Backblaze B2 and MinIO alike, and exports everything as Parquet.
Can I search a DuckLake table?
Yes. POST /v1/namespaces/{name}/lake/sync reads the catalog (Postgres) and the Parquet files where they are, on S3, R2, B2 or MinIO, with keys used for that sync only, and writes only the rows that changed since the last sync.
Search what DuckDB can’t index
Free up to 1 GB stored and 100k queries a month. No card to start.