Search your DuckLake, where it is
DuckDB answers your analytics. SkimmerDB adds what a lake has no index for: substring and regex search, filters and vectors for apps and agents, one namespace per table or tenant. Each sync writes only the rows that changed.
One call to keep a table searchable
Point a namespace at a table. The first sync writes every live row; later ones write the inserts, updates and deletes since the last snapshot.
curl -s https://api.skimmerdb.com/v1/namespaces/tickets/lake/sync \
-H "Authorization: Bearer $KEY" -H "Content-Type: application/json" -d '{
"catalog": "postgres://lake_reader:...@lake-db.example.com:5432/lake?sslmode=require",
"table": "main.tickets",
"text": ["subject", "body"],
"s3": {"region": "us-east-1", "access_key_id": "...", "secret_access_key": "..."}}'
# {"snapshot": 42, "previous": 40, "full": false, "upserted": 120, "deleted": 3, ...}What a sync does
| Reads the catalog where it is | DuckLake 1.0 in Postgres, at the latest snapshot or one you name. The host must be public; the connection goes to the address checked. |
| Reads Parquet by byte range | The footer, then the columns needed, from AWS S3 or an S3-compatible store (Cloudflare R2, Backblaze B2, Wasabi, MinIO) with your keys. |
| Writes only what changed | Rows whose version began after the last synced snapshot are upserted; rows live then and gone now are deleted. Compactions and flushes don’t count as changes. |
| Follows the lake’s rules | Inlined rows and deletes, delete files, updates (same row id), renamed and dropped columns, time travel. Checked against DuckDB’s own answer at every snapshot of our test lake. |
Then search it
Every row is a document: its id the key column, its text the columns you chose, its metadata every other column. Search, filter and query it with SQL over REST, from Python or TypeScript, or from an agent over MCP.
# substrings and RE2 regex from an index, filtered by the table's own columns curl -s $B/namespaces/tickets/search -H "Authorization: Bearer $KEY" \ -d '{"q": "timeout after [0-9]+ ms", "regex": true, "filter": {"status": "open"}}' # SQL on the same rows, text predicates included curl -s $B/namespaces/tickets/sql -H "Authorization: Bearer $KEY" \ -d '{"sql": "SELECT priority, count(*) FROM docs WHERE contains(text, '\''refund'\'') GROUP BY 1"}'
Questions
Does SkimmerDB copy my lake?
It keeps the rows of the table you sync as documents in a namespace, so search and SQL don’t touch your files at query time. Your Parquet files and catalog are only read, and only during a sync.
Are my credentials stored?
No. The catalog URL and the bucket keys come with each sync request and are dropped when it ends. Give it a read-only Postgres role and keys that can only read the lake’s bucket.
My lake is on Cloudflare R2 or Backblaze B2. Does that work?
Yes. Add the service’s endpoint to the s3 object, such as https://<account-id>.r2.cloudflarestorage.com with region auto for R2, or https://s3.<region>.backblazeb2.com for B2. Requests are path-style and signed with AWS Signature Version 4.
How often should I sync?
After the lake’s commits: from the job that writes the lake, or on a schedule. A sync with nothing new returns at once, and two syncs of one namespace never interleave (the second gets a 409).
What is not supported yet?
SQLite and DuckDB-file catalogs, nested column types and blobs (listed in the answer’s skipped), and stored connections that sync on their own.
Sync your first table
Free up to 1 GB stored and 100k queries a month. No card to start.