TL;DR: Memvid reduces retrieval errors by 50-66% compared to leading vector databases while delivering 4x better accuracy-per-millisecond than alternatives.
Headline Results
92.7%
Top-1 AccuracyMemvid finds the correct document on the first try 92.7% of the time, 8.5 to 14.5 points higher than any competitor.
50-66%
Error ReductionMemvid reduces retrieval errors by 50-66% compared to Chroma, Weaviate, LanceDB, and Qdrant.
4.2x
Accuracy per msMemvid delivers 4.2x more accuracy per millisecond than Chroma, the best quality-to-latency ratio.
0.95
MRR ScoreMean Reciprocal Rank of 0.95. The correct result consistently appears in the top 1-2 hits.
Accuracy Comparison
Top-1 Accuracy (Higher is Better)
The most important metric: does the system return the correct document first?| System | Accuracy@1 | vs Memvid |
|---|---|---|
| Memvid | 92.72% | baseline |
| LanceDB | 84.24% | -8.5 pts |
| Qdrant | 84.24% | -8.5 pts |
| Weaviate | 80.68% | -12.0 pts |
| Chroma | 78.24% | -14.5 pts |
Error Reduction
Another way to look at accuracy: how often does the system get it wrong?| System | Error Rate | Memvid Reduces Errors By |
|---|---|---|
| Chroma | 21.76% | 66% fewer errors |
| Weaviate | 19.32% | 62% fewer errors |
| LanceDB | 15.76% | 54% fewer errors |
| Qdrant | 15.76% | 54% fewer errors |
| Memvid | 7.28% | baseline |
Memvid returns the wrong answer 3x less often than Chroma.
Accuracy at Different k Values
| System | @1 | @3 | @5 | @10 |
|---|---|---|---|---|
| Memvid | 92.72% | 96.96% | 97.56% | 98.12% |
| LanceDB | 84.24% | 92.72% | 94.60% | 96.24% |
| Qdrant | 84.24% | 92.72% | 94.60% | 96.24% |
| Weaviate | 80.68% | 88.52% | 90.16% | 91.52% |
| Chroma | 78.24% | 85.80% | 87.28% | 88.68% |
Ranking Quality
MRR (Mean Reciprocal Rank)
MRR measures how high the correct result appears in the ranking. A score of 1.0 means perfect top-1 placement every time.| System | MRR | Interpretation |
|---|---|---|
| Memvid | 0.949 | Correct result typically #1 |
| LanceDB | 0.888 | Correct result typically #1-2 |
| Qdrant | 0.888 | Correct result typically #1-2 |
| Weaviate | 0.849 | Correct result typically #2 |
| Chroma | 0.823 | Correct result typically #2 |
NDCG@10 (Normalized Discounted Cumulative Gain)
NDCG measures overall ranking quality across the top 10 results.| System | NDCG@10 |
|---|---|
| Memvid | 0.967 |
| LanceDB | 0.929 |
| Qdrant | 0.929 |
| Weaviate | 0.886 |
| Chroma | 0.858 |
Latency Performance
Query Latency (Lower is Better)
| System | p50 | p95 | p99 | QPS |
|---|---|---|---|---|
| Weaviate | 5.3ms | 7.1ms | 7.9ms | 180 |
| Memvid | 16.0ms | 17.4ms | 19.7ms | 61 |
| LanceDB | 16.0ms | 17.5ms | 19.3ms | 61 |
| Qdrant | 28.0ms | 30.4ms | 31.4ms | 36 |
| Chroma | 55.6ms | 61.0ms | 65.2ms | 18 |
Cold Start Time
| System | Cold Start |
|---|---|
| Memvid | 0.5ms |
| Chroma | 66.3ms |
| Qdrant | 71.8ms |
| LanceDB | 72.4ms |
| Weaviate | 147.7ms |
Memvid cold-starts 130-300x faster than alternatives. This matters for serverless deployments and edge computing where startup time is critical.
The Efficiency Frontier
Accuracy per Millisecond
We compute accuracy divided by p95 latency to measure quality-per-latency:| System | Accuracy@1 | p95 Latency | Accuracy/ms |
|---|---|---|---|
| Memvid | 92.72% | 17.4ms | 0.053 |
| Weaviate | 80.68% | 7.1ms | 0.114 |
| LanceDB | 84.24% | 17.5ms | 0.048 |
| Qdrant | 84.24% | 30.4ms | 0.028 |
| Chroma | 78.24% | 61.0ms | 0.013 |
Memvid delivers 4.2x more accuracy per millisecond than Chroma and leads all systems except Weaviate (which sacrifices 12 points of accuracy for speed).If accuracy matters, Memvid is the clear winner. If you can tolerate 12% worse results, Weaviate is faster.
The Frontier Chart
Storage Efficiency
Single-File Advantage
Memvid is unique among memory systems because everything lives inside a single portable file; (data, embeddings, indices, and metadata) There are no sidecar files, external indexes, database directories, or hidden dependencies. One file contains the entire memory: easy to move, copy, version, share, or embed into an agent.| System | Storage | Compression | Bytes/Doc |
|---|---|---|---|
| LanceDB | 213 MB | 0.71x | 5,428 |
| Qdrant | 212 MB | 0.72x | 5,396 |
| Memvid | 508 MB | 0.30x | 12,911 |
| Weaviate | 1,009 MB | 0.15x | 25,646 |
| Chroma | 1,025 MB | 0.15x | 26,068 |
Memvid’s higher bytes-per-document value reflects its richer internal structure: embedded indices, a write-ahead log, a time index, and the metadata required for hybrid semantic + keyword + time-travel search. Instead of scattering these components across multiple files or services, Memvid packages the entire memory system into a single
.mv2 file, delivering portability and simplicity that traditional systems can’t match.Methodology
Dataset
- Wikipedia: 39,324 documents
- Queries: 2,500 natural language queries with known correct answers
Systems Tested
- Memvid v2 (hybrid search mode)
- Chroma 0.4.x (default HNSW)
- LanceDB (default IVF-PQ)
- Qdrant (default HNSW)
- Weaviate (default HNSW)
Metrics
- Accuracy@k: Fraction of queries where the correct document appears in top-k results
- MRR: Mean Reciprocal Rank (1/position of first correct result)
- NDCG@10: Normalized Discounted Cumulative Gain at 10
- Latency: Query time in milliseconds (p50, p95, p99)
- QPS: Queries per second throughput
Environment
- Apple M-series Mac
- All systems using default configurations
- Same embedding model across all systems
- Each query executed 1x (no caching)
Key Takeaways
Best Accuracy
Memvid achieves 92.7% top-1 accuracy, 8.5 to 14.5 points higher than any competitor.
Lowest Error Rate
Memvid reduces errors by 50-66% compared to leading vector databases.
Best Ranking
Highest MRR (0.95) and NDCG@10 (0.97). The correct result consistently appears first.
Fastest Cold Start
0.5ms cold start, 130-300x faster than alternatives for serverless deployments.
Run the Benchmarks Yourself
The benchmark suite is open source. Run it on your own hardware:results/results.json.