> ## Documentation Index
> Fetch the complete documentation index at: https://docs.memvid.com/llms.txt
> Use this file to discover all available pages before exploring further.
# The Memvid Approach
> A portable memory architecture with flexible embedding options
**Memvid is not another vector database.** It's a portable, single-file memory system with flexible embedding options. Use local models for privacy or connect to external providers like OpenAI, NVIDIA, and more.
***
## The Problem with Traditional RAG
Most memory systems today follow the same pattern:
```mermaid theme={null}
flowchart LR
A[Your Data] --> B[Embedding API]
B --> C[Vector Database]
C --> D[Similarity Search]
D --> E[Results]
```
This approach has serious limitations:
| Problem | Impact |
| ----------------------- | -------------------------------------------- |
| **API dependency** | Can't work everyhwere, costs money per query |
| **Embedding drift** | Model updates break your index |
| **No exact matching** | "Error 404" doesn't find "error 404" |
| **Black box relevance** | Hard to debug why results are wrong |
| **Cold start** | Need to embed everything before first search |
***
## The Memvid Innovation
Memvid takes a completely different approach:
```mermaid theme={null}
flowchart LR
A[Your Data] --> B[Frames]
B --> C[Multiple Indices]
C --> D[Smart Retrieval]
D --> E[Results]
C -.- F[BM25 Lexical]
C -.- G[Vector Embeddings]
C -.- H[SimHash Dedup]
C -.- I[Time Index]
```
**Flexible embedding options.** Memvid combines multiple search strategies:
* **BM25 lexical search** - Battle-tested, fast, explainable
* **Vector embeddings** - Local models (Nomic, BGE, GTE) or external APIs (OpenAI, NVIDIA)
* **SimHash deduplication** - Find near-duplicates instantly
* **Time-aware retrieval** - When something was added matters
* **Hybrid search** - Combines lexical + semantic for best results
***
## How Frames Change Everything
Traditional systems store "chunks" - arbitrary text splits. Memvid stores **Frames** - structured units of memory:
```mermaid theme={null}
flowchart LR
subgraph Frame[Smart Frame]
A[content]
B[uri]
C[timestamp]
D[content_hash]
E[simhash]
F[metadata]
G[entities]
H[connections]
end
```
| Field | Example |
| -------------- | ----------------------------------------- |
| `content` | "The quarterly revenue exceeded \$10M..." |
| `uri` | reports/q4-2024.pdf |
| `timestamp` | 2024-12-15T10:30:00Z |
| `content_hash` | blake3(...) |
| `simhash` | 0x8f3a2b1c... |
Each frame knows:
* **What it contains** (content + hash)
* **Where it came from** (URI + metadata)
* **When it was created** (timestamp)
* **What it's similar to** (SimHash)
* **What entities it mentions** (Logic Mesh)
* **How it connects to other frames** (relationships)
Frames are the foundation of Memvid's multi-index approach, enabling both lexical and semantic search.
***
## Multiple Indices, Flexible Providers
Every `.mv2` file contains multiple search indices:
| Index | Purpose | How It Works |
| ------------------ | ------------------------ | --------------------------------------- |
| **Lexical (BM25)** | Keyword search | TF-IDF scoring, exact matches |
| **Vector** | Semantic similarity | Local or external embedding models |
| **SimHash** | Near-duplicate detection | 64-bit locality-sensitive hash |
| **Time** | Temporal queries | B-tree on timestamps |
| **Logic Mesh** | Entity-relationship | Triple store (subject-predicate-object) |
**Embedding flexibility** - Choose what works for your use case:
* **Local models** (Nomic, BGE-small, BGE-base, GTE-large) - Fast, private, works offline
* **External APIs** (OpenAI, NVIDIA) - Higher quality, no local compute needed
***
## Choosing Your Embedding Provider
Memvid supports multiple embedding providers. Choose based on your needs:
| Provider | Models | Best For |
| ------------------- | ------------------------------------- | ---------------------------------- |
| **Local (Default)** | bge-small, bge-base, nomic, gte-large | Privacy, offline use, no API costs |
| **OpenAI** | openai, openai-small, openai-ada | Highest quality, multilingual |
| **NVIDIA** | nvidia | Enterprise, high throughput |
```bash theme={null}
# Use local embeddings (default - works offline)
memvid create notes.mv2
memvid put notes.mv2 --input docs/ --embedding
# Use OpenAI embeddings (requires OPENAI_API_KEY)
memvid put notes.mv2 --input docs/ --embedding -m openai
# Use specific local model
memvid put notes.mv2 --input docs/ --embedding -m bge-base
```
***
## Logic Mesh: Relationships Without ML
Traditional systems need expensive NER models for entity extraction. Memvid's Logic Mesh uses:
* **Rule-based extraction** - Fast, free, no API
* **Pattern matching** - Dates, emails, numbers
* **Co-occurrence** - Entities mentioned together
* **Temporal reasoning** - When facts changed
```bash theme={null}
# Enable Logic Mesh during ingestion
memvid put memory.mv2 --input docs/ --logic-mesh
# Query relationships
memvid follow traverse memory.mv2 --start "John" --link "works_at"
# Result: John → works_at → Acme Corp (since 2024-01-15)
```
***
## SimHash: Smart Deduplication
Instead of comparing embeddings, Memvid uses **SimHash** - a locality-sensitive hash that detects near-duplicates:
```
Document A: "The quick brown fox jumps over the lazy dog"
Document B: "The quick brown fox leaps over the lazy dog"
SimHash A: 0x8f3a2b1c4d5e6f70
SimHash B: 0x8f3a2b1c4d5e6f71
Hamming distance: 1 bit → Near duplicate detected!
```
Benefits:
* **Instant** - O(1) comparison
* **No API calls** - Computed locally
* \*\*Works anywhere \*\*- No internet needed
* **Deterministic** - Same input = same hash
***
## Real-World Performance
Memvid's local-first approach delivers fast performance:
| Operation | Traditional Vector DB | Memvid (Local Embeddings) |
| ----------------------- | --------------------- | ------------------------- |
| Ingest 1,000 docs | 5-10 minutes | **30 seconds** |
| First search | After embedding | **Instant** |
| Offline search | ❌ | ✅ |
| Cost per query | \$0.0001+ | **\$0** |
| Single file portability | ❌ | ✅ |
***
## Getting Started
Get started with Memvid in minutes:
```bash theme={null}
# Install
npm install -g memvid-cli
# Create memory
memvid create my-memory.mv2
# Add your documents (uses local embeddings by default)
memvid put my-memory.mv2 --input ./documents/
# Search with hybrid lexical + semantic
memvid find my-memory.mv2 --query "quarterly report"
# Ask questions
memvid ask my-memory.mv2 --question "What were the Q4 results?"
```
That's it. Local embeddings work out of the box, no API keys required.
***
## Switching Embedding Providers
Need higher quality embeddings? Switch to an external provider:
```bash theme={null}
# Rebuild vector index with OpenAI embeddings
memvid doctor my-memory.mv2 --rebuild-vec-index -m openai
# Search with semantic mode
memvid find my-memory.mv2 --query "financial performance" --mode sem
# Or use hybrid (lexical + semantic)
memvid find my-memory.mv2 --query "financial performance" --mode hybrid
```
Your existing data stays intact. Only the vector index is rebuilt.
***
## Next Steps
Get up and running fast
Deep dive into frames
Entity-relationship graphs
SimHash and content hashing