eidolon-search

# Eidolon Search

FTS5-based memory search for AI Agents. Index markdown files once, search with 90%+ token savings.

## Quick Start

### 1. Index memory files (once)

```bash
python3 scripts/build-index.py <memory_dir> <db_path>
```

Example:
```bash
python3 scripts/build-index.py ./memory ./memory.db
```

This creates a SQLite database with FTS5 full-text index of all `.md` files in the directory (recursive).

### 2. Search

```bash
python3 scripts/search.py <query> [limit] [db_path]
```

Example:
```bash
python3 scripts/search.py "Physical AI roadmap" 5
python3 scripts/search.py "project timeline" 10 ./memory.db
```

Default limit: 10. Default db_path: `./memory.db`

Output: matching snippets with file paths and relevance scores.

### 3. Re-index when files change

Run `build-index.py` again. It rebuilds the index from scratch (fast, <1 second for typical workspaces).

## When to Use

- **Memory search**: Find specific information across many daily notes or memory files
- **Token-limited contexts**: When reading all files would exceed context limits
- **Repeated searches**: Index once, search many times
- **Large workspaces**: 10+ markdown files with cumulative size >50KB

## When NOT to Use

- Single small file (<5KB): just read it directly
- Need semantic/meaning-based search: FTS5 is keyword-based only
- See [references/QUALITY.md](references/QUALITY.md) for known limitations

## Search Tips for Agents

FTS5 is keyword-based. Improve results by:

- Use specific terms: "Jetson Orin" not "hardware plans"
- Use OR for synonyms: "car OR vehicle OR automobile"
- Use quotes for phrases: `"Physical AI"`
- Try multiple queries if first attempt returns nothing
- Check file paths in results to read full context when needed

## Benchmarks

- Token savings: 90%+ (measured 93-98.9%)
- Speed: 15x faster (measured 10-20x)
- Details: [references/PERFORMANCE.md](references/PERFORMANCE.md)

Run benchmarks yourself:
```bash
python3 scripts/benchmark-recall.py    # Recall@5, Reca...