Benchmarks

Don't take our word for it. Take the harness.

Vendor benchmarks are marketing until the harness is public. Ours is: same corpus, same client, same instance class, config files committed. Where a competitor wins, the table says so.

Run configuration
Harnessperegrine/bench v0.9.4
CorpusMS MARCO + synthetic
Dimensions1536, float32
Clientc3-standard-8, same zone
Duration30 min steady state
Date2026-07-12

Query latency by corpus size

Strongly consistent reads, cache-resident namespace, top-K 10 with a 20% selective metadata filter.

Systemp50p95p99
{{ r.name }} {{ r.p50 }} {{ r.p95 }} {{ r.p99 }}
{{ b.name }}
{{ b.value }}

{{ sizeNote }}

Write commit latency

4 KB documents, single namespace, 200 writes/sec offered load. Measured to durable ack, not to queryable.

Systemp50p90p99
{{ w.name }} {{ w.p50 }} {{ w.p90 }} {{ w.p99 }}

Under concurrent load our adaptive group-commit window adds 5–15 ms; we quote 25 ms typical and sub-50 ms under load rather than a best case.

Cold start

First query against a namespace idle for one hour, nothing resident. No prewarm API called on any system.

System1 M10 M100 M
{{ c.name }} {{ c.a }} {{ c.b }} {{ c.c }}

Cold latency is the number that decides whether your p99 is survivable. It is also the number vendors publish least often.

Recall

Latency is meaningless without recall.

Recall@10 against exact brute-force search on the same 10 M corpus, at each system's documented high-recall configuration — then again with a 1% selective filter, where centroid indexes usually fall apart.

System Recall@10 Filtered 1% QPS / node Index build
{{ r.name }} {{ r.recall }} {{ r.filtered }} {{ r.qps }} {{ r.build }}
Reproduce

Run it against your own account.

The harness drives every system through its official client with credentials you supply. Results write to a CSV with the raw histograms, not just the percentiles we chose to show.

View harness on GitHub →
bash 
# clone, configure credentials for each system under test
git clone github.com/peregrine/bench && cd bench
cp targets.example.toml targets.toml

# load the 10M corpus into every configured target
./bench load --corpus msmarco-10m --dims 1536

# run the full suite; ~2h wall clock, writes results/*.csv
./bench run --suite full --duration 30m --warmup 5m

# regenerate the tables on this page from your own results
./bench report --format html > report.html
Caveats

Where these numbers are weakest.

If we only listed our wins you would be right to discount the whole page.

We are the vendor running the test

Our engineers tuned our system daily and the others from documentation. Treat competitor numbers as a floor on their performance, not a ceiling.

Single-region, same-zone client

Every system was measured from a client in its own zone. If your application runs in AWS, add 10–30 ms of cross-cloud round trip to our figures and the advantage narrows sharply.

Feature maturity is not measured here

Sparse vectors, hybrid tuning knobs and full-text scoring options are more mature in the incumbents. Latency tables do not capture that, and it may matter more to you.

Failover pauses writes

A hard leader crash makes one namespace unwritable for up to the ~5 s lease TTL. That is the honest cost of serialized strong consistency, and it is in the SLA.

Think we measured it wrong?

Open an issue on the harness. We publish corrections against our own numbers and date every revision.

Start free Read the methodology