> ## Documentation Index
> Fetch the complete documentation index at: https://docs.memvid.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Local Models with Ollama

> Run AI-powered Q&A locally without sending data to external APIs

Memvid supports local LLM inference through [Ollama](https://ollama.com), allowing you to run AI-powered Q\&A without sending your data to external APIs. This is ideal for:

* **Privacy-sensitive data** - Keep everything on your machine
* **Offline usage** - No internet connection required after setup
* **Cost savings** - No API fees for inference
* **Low latency** - No network round-trips

***

## Quick Setup

### 1. Install Ollama

<Tabs>
  <Tab title="macOS">
    ```bash theme={null}
    brew install ollama
    ```
  </Tab>

  <Tab title="Linux">
    ```bash theme={null}
    curl -fsSL https://ollama.com/install.sh | sh
    ```
  </Tab>

  <Tab title="Windows">
    Download from [ollama.com/download](https://ollama.com/download)
  </Tab>
</Tabs>

### 2. Start Ollama Server

```bash theme={null}
# Start in foreground (see logs)
ollama serve

# Or run as background service (macOS)
brew services start ollama
```

### 3. Pull a Model

```bash theme={null}
# Recommended: Qwen2.5 1.5B (best quality/size ratio)
ollama pull qwen2.5:1.5b
```

### 4. Use with Memvid

```bash theme={null}
memvid ask knowledge.mv2 \
  --question "What is the main topic?" \
  --use-model "ollama:qwen2.5:1.5b"
```

***

## Recommended Models

| Model          | Size    | Speed  | Quality   | Best For                   |
| -------------- | ------- | ------ | --------- | -------------------------- |
| `qwen2.5:0.5b` | \~400MB | Fast   | Good      | Quick queries, limited RAM |
| `qwen2.5:1.5b` | \~1GB   | Fast   | Great     | **Recommended default**    |
| `qwen2.5:3b`   | \~2GB   | Medium | Excellent | Complex questions          |
| `phi3:mini`    | \~2GB   | Medium | Great     | Reasoning tasks            |
| `gemma2:2b`    | \~1.6GB | Medium | Great     | General purpose            |
| `llama3.2:1b`  | \~1.3GB | Fast   | Good      | Conversational             |
| `llama3.2:3b`  | \~2GB   | Medium | Great     | Balanced performance       |

### Pull Commands

```bash theme={null}
# Small & fast
ollama pull qwen2.5:0.5b

# Recommended (best balance)
ollama pull qwen2.5:1.5b

# Higher quality
ollama pull qwen2.5:3b
ollama pull phi3:mini
ollama pull gemma2:2b

# Meta's Llama
ollama pull llama3.2:1b
ollama pull llama3.2:3b
```

***

## CLI Usage

### Basic Q\&A

```bash theme={null}
# Ask with local model
memvid ask knowledge.mv2 \
  --question "What are the key findings?" \
  --use-model "ollama:qwen2.5:1.5b"

# With JSON output
memvid ask knowledge.mv2 \
  --question "Summarize the main points" \
  --use-model "ollama:qwen2.5:1.5b" \
  --json
```

### Advanced Options

```bash theme={null}
# More context for complex questions
memvid ask knowledge.mv2 \
  --question "Explain the architecture in detail" \
  --use-model "ollama:qwen2.5:3b" \
  --top-k 15 \
  --snippet-chars 800

# Filter by scope
memvid ask knowledge.mv2 \
  --question "What API endpoints exist?" \
  --use-model "ollama:qwen2.5:1.5b" \
  --scope "mv2://api/"

# Time-travel query
memvid ask knowledge.mv2 \
  --question "What was the status?" \
  --use-model "ollama:qwen2.5:1.5b" \
  --as-of-frame 100
```

***

## Python SDK Usage

```python theme={null}
from memvid_sdk import use

mem = use('basic', 'knowledge.mv2')

# Ask with local Ollama model
response = mem.ask(
    "What are the main conclusions?",
    model="ollama:qwen2.5:1.5b",
    k=10
)

print(response['answer'])
```

### With Different Models

```python theme={null}
# Quick answer with small model
quick_response = mem.ask(
    "What is this document about?",
    model="ollama:qwen2.5:0.5b"
)

# Detailed analysis with larger model
detailed_response = mem.ask(
    "Provide a comprehensive analysis of the findings",
    model="ollama:qwen2.5:3b",
    k=15
)
```

***

## Node.js SDK Usage

```javascript theme={null}
import { use } from '@memvid/sdk';

const mem = await use('basic', 'knowledge.mv2');

// Ask with local Ollama model
const response = await mem.ask(
  'What are the key takeaways?',
  { model: 'ollama:qwen2.5:1.5b', k: 10 }
);

console.log(response.answer);
```

***

## Model Selection Guide

### By Use Case

| Use Case        | Recommended Model | Why              |
| --------------- | ----------------- | ---------------- |
| Quick lookups   | `qwen2.5:0.5b`    | Fastest response |
| General Q\&A    | `qwen2.5:1.5b`    | Best balance     |
| Technical docs  | `qwen2.5:3b`      | Better reasoning |
| Code analysis   | `phi3:mini`       | Strong at code   |
| Research papers | `qwen2.5:3b`      | Complex content  |

### By Hardware

| RAM Available | Recommended Model           |
| ------------- | --------------------------- |
| 4GB           | `qwen2.5:0.5b`              |
| 8GB           | `qwen2.5:1.5b`              |
| 16GB+         | `qwen2.5:3b` or `phi3:mini` |

***

## Ollama Management

### List Downloaded Models

```bash theme={null}
ollama list
```

### Remove a Model

```bash theme={null}
ollama rm qwen2.5:0.5b
```

### Update a Model

```bash theme={null}
ollama pull qwen2.5:1.5b
```

### Check Ollama Status

```bash theme={null}
# Check if server is running
curl http://localhost:11434/api/tags
```

### Run as Background Service

<Tabs>
  <Tab title="macOS">
    ```bash theme={null}
    # Start service
    brew services start ollama

    # Stop service
    brew services stop ollama

    # Check status
    brew services list | grep ollama
    ```
  </Tab>

  <Tab title="Linux (systemd)">
    ```bash theme={null}
    # Start service
    sudo systemctl start ollama

    # Enable on boot
    sudo systemctl enable ollama

    # Check status
    sudo systemctl status ollama
    ```
  </Tab>
</Tabs>

***

## Troubleshooting

### Ollama Not Running

```
Error: Failed to contact LLM provider
```

**Solution:**

```bash theme={null}
# Start Ollama server
ollama serve

# Or as background service (macOS)
brew services start ollama
```

### Model Not Found

```
Error: model 'qwen2.5:1.5b' not found
```

**Solution:**

```bash theme={null}
# Pull the model first
ollama pull qwen2.5:1.5b
```

### Slow Response Times

**Solutions:**

* Use a smaller model: `ollama:qwen2.5:0.5b`
* Reduce context: `--top-k 5 --snippet-chars 300`
* Close other memory-intensive applications
* Ensure you have enough RAM for the model

### Out of Memory

**Solutions:**

* Use a smaller model
* Close other applications
* Increase swap space (not recommended for performance)

***

## Comparison: Local vs Cloud Models

| Aspect      | Local (Ollama)        | Cloud (OpenAI, Claude, Gemini) |
| ----------- | --------------------- | ------------------------------ |
| **Privacy** | Data stays local      | Data sent to API               |
| **Cost**    | Free after setup      | Per-token pricing              |
| **Speed**   | Depends on hardware   | Usually faster                 |
| **Quality** | Good to great         | Excellent                      |
| **Offline** | Yes                   | No                             |
| **Setup**   | Requires installation | Just API key                   |

### When to Use Local Models

* Sensitive/confidential data
* Offline environments
* Cost-sensitive applications
* Privacy requirements

### When to Use Cloud Models

* Best possible answer quality
* Limited local compute
* Quick prototyping
* Complex reasoning tasks

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Search & Ask CLI" icon="terminal" href="/cli/search-and-ask">
    Full CLI reference for search and Q\&A commands
  </Card>

  <Card title="Python SDK" icon="python" href="/python-sdk/overview">
    Use local models in Python applications
  </Card>
</CardGroup>