Query Markup Documents
On-device search for everything
you need to remember
By Tobi Lutke — github.com/tobi/qmd
16k+ stars · TypeScript · MIT License · Created Dec 2025
The simple version anyone can understand
You have hundreds of notes, meeting transcripts, docs scattered everywhere.
QMD is like hiring a librarian who:
Like Ctrl+F on steroids. Finds exact words.
"Find every doc that says budget Q4"
Understands what you mean, not just what you type.
"Find docs about spending plans for year-end"
Combines both + an AI judge picks the best results.
# Keywords $ qmd search "budget Q4" # Meaning $ qmd vsearch "spending plans" # Both + AI reranking $ qmd query "what did we spend?"
The big idea: State-of-the-art hybrid search that used to require Elasticsearch + OpenAI + a server farm, now runs on your MacBook with zero cloud dependencies.
QMD speaks the Model Context Protocol (MCP) — the universal language for AI tools.
Your data never leaves your machine. The AI just gets the relevant snippets.
Architecture, algorithms, and implementation details
QMD is a TypeScript monolith (~260KB of source) built around a single SQLite database with three access patterns:
| Layer | Technology |
|---|---|
| CLI | Node.js parseArgs + tsx |
| MCP Server | stdio + HTTP/SSE transports |
| SDK | createStore() library API |
| Storage | SQLite (better-sqlite3 / bun:sqlite) |
| Full-text | FTS5 with Porter stemming |
| Vectors | sqlite-vec (cosine distance) |
| LLM Runtime | node-llama-cpp (GGUF models) |
| Config | YAML synced to SQLite |
src/ store.ts 154KB core engine cli/ qmd.ts 115KB CLI commands formatter 13KB output formats llm.ts 50KB model layer collections 13KB YAML config mcp/ server.ts 30KB MCP protocol db.ts 3KB SQLite compat index.ts 17KB SDK exports finetune/ query expansion model
Everything lives in ~/.cache/qmd/index.sqlite:
-- Content-addressable storage CREATE TABLE content ( hash TEXT PRIMARY KEY, doc TEXT NOT NULL, created_at TEXT ); -- Document index (soft-delete) CREATE TABLE documents ( collection TEXT, path TEXT, title TEXT, hash TEXT REFERENCES content, active INTEGER DEFAULT 1, UNIQUE(collection, path) ); -- Full-text search CREATE VIRTUAL TABLE documents_fts USING fts5( filepath, title, body, tokenize='porter unicode61' );
-- Vector embeddings (sqlite-vec) CREATE VIRTUAL TABLE vectors_vec USING vec0( hash_seq TEXT PRIMARY KEY, embedding float[N] distance_metric=cosine ); -- Chunk metadata CREATE TABLE content_vectors ( hash TEXT, seq INTEGER, pos INTEGER, model TEXT, PRIMARY KEY(hash, seq) ); -- LLM response cache CREATE TABLE llm_cache ( hash TEXT PRIMARY KEY, result TEXT, created_at TEXT );
Key insight: Content-addressable storage means identical documents across collections are stored once. The hash (SHA256) is the universal join key.
A compatibility layer that works under both Bun and Node.js:
const isBun = typeof globalThis.Bun !== "undefined"; if (isBun) { // Dynamic import prevents tsc errors const mod = "bun:" + "sqlite"; _Database = (await import(mod)).Database; // macOS: swap in Homebrew SQLite // (Apple's omits LOAD_EXTENSION) if (process.platform === "darwin") { BunDatabase.setCustomSQLite( "/opt/homebrew/opt/sqlite/lib/..." ); } } else { _Database = (await import( "better-sqlite3" )).default; }
Graceful degradation: QMD works without vectors. BM25 search is always available.
All models run via node-llama-cpp and auto-download from HuggingFace on first use:
| Model | Purpose | Size | Base |
|---|---|---|---|
| EmbeddingGemma | Vector embeddings — converts text to float arrays for semantic similarity | ~300MB (Q8) | 300M params |
| Qwen3-Reranker | Re-ranks candidate results by relevance. Scores doc-query pairs. | ~640MB (Q8) | 600M params |
| qmd-query-expansion | Expands user query into keyword + semantic + hypothetical doc variants | ~1.1GB (Q4_K_M) | Qwen3-1.7B (fine-tuned) |
~2GB total RAM for all three. Models cache locally and use inactivity timeouts (5 min) to free VRAM when idle. Promise guards prevent duplicate loads.
What happens when you run qmd query "what did we decide about pricing?"
A fine-tuned 1.7B model transforms your query into typed retrieval variants:
Input: "what did we decide about pricing?" Output (3 line types): lex: pricing decision meeting notes lex: pricing strategy final vec: discussions about product pricing and strategy decisions vec: revenue model and price changes hyde: After reviewing the market data, the team decided to move forward with the tiered pricing approach starting Q2...
Strong signal shortcut: If BM25 alone returns a result with score ≥ 0.85 and a gap ≥ 0.15 from second place, expansion is skipped entirely.
The repo includes the full fine-tuning pipeline in finetune/:
| Base model | Qwen3-1.7B |
| Method | LoRA (rank 16, alpha 32) |
| Targets | All q/k/v/o/gate/up/down projections |
| Training data | ~2,290 examples |
| Epochs | 5 |
| Batch size | 16 (4 x 4 grad accum) |
| LR schedule | Cosine from 2e-4 |
| Hardware | A10G (24GB VRAM) |
Train accuracy: 97.4%
Eval accuracy: 93.8%
// User syntax -> FTS5 query buildFTS5Query("budget Q4 -sports") // Returns: "budget"* "Q4"* NOT "sports"* buildFTS5Query('"exact phrase" docs') // Returns: "exact phrase" "docs"* // Features: // - Prefix matching with * suffix // - Quoted phrase = exact match // - Negation via NOT binary op // - Porter stemming + Unicode
// BM25 returns negative values // Normalize: |bm25| / (1 + |bm25|) score = -10 → 0.91 // strong score = -2 → 0.67 // medium score = -0.5 → 0.33 // weak
Why BM25? It is the gold standard for keyword retrieval. SQLite FTS5's built-in BM25 is fast, battle-tested, and requires zero infrastructure beyond the database file.
-- Step 1: KNN from vector table SELECT hash_seq, distance FROM vectors_vec WHERE embedding MATCH ? AND k = ? -- Step 2: Join back to documents SELECT d.path, d.title, cv.pos FROM content_vectors cv JOIN documents d ON d.hash = cv.hash WHERE cv.hash || '_' || cv.seq IN (...) AND d.active = 1
Score = 1 − cosine_distance. Deduplicated by filepath (best chunk wins).
Known quirk: sqlite-vec JOINs hang indefinitely. The two-step pattern is intentional.
Merging results from BM25 and vector search into a single ranked list:
// RRF formula per document: RRF(d) = Σ for each query q: weight(q) / (k + rank(d, q)) // where k = 60 (standard constant) // weight(q) depends on query type: // original query = 2x weight // expansion variants = 1x weight
// Top ranks: trust retrieval more rank 1-5: 75% retrieval + 25% reranker rank 6-15: 65% retrieval + 35% reranker rank 16+: 40% retrieval + 60% reranker
Smart confidence: Top results likely matched keywords exactly, so retrieval signals are trusted more (75%). Lower results may benefit from the LLM reranker's semantic understanding (60%).
| Score | Meaning |
|---|---|
| 0.8 – 1.0 | Highly relevant |
| 0.5 – 0.8 | Moderately relevant |
| 0.2 – 0.5 | Somewhat relevant |
| 0.0 – 0.2 | Low relevance |
// Position-dependent blending if (rank <= 5) { final = 0.75 * retrieval + 0.25 * reranker; } else if (rank <= 15) { final = 0.65 * retrieval + 0.35 * reranker; } else { final = 0.40 * retrieval + 0.60 * reranker; } // Intuition: // Top hits: retrieval was confident // -> trust the original ranking // Lower hits: less confident // -> let the reranker decide
Use --explain flag to see the full score trace.
Documents are split at ~900 tokens per chunk with 15% overlap. The algorithm prioritizes natural semantic boundaries.
| Pattern | Score |
|---|---|
| # H1 heading | 100 |
| ## H2 heading | 90 |
| Code block boundary ``` | 80 |
| Horizontal rule --- | 60 |
| Blank line (paragraph) | 20 |
| List item - item | 5 |
| Any newline | 1 |
// Prefer splits near the target size // but accept earlier good boundaries normalizedDist = distance / window; multiplier = 1.0 - (dist**2) * 0.7; finalScore = bp.score * multiplier; // At target: 1.00x // 25% back: 0.96x (gentle) // 50% back: 0.83x (moderate) // 75% back: 0.61x (steep) // Edge: 0.30x
Code fence protection: Never splits inside a ```code block```. Fences are detected and excluded from candidate break points.
Each doc gets a 6-char hash ID (#abc123) from the first 6 characters of its SHA256 content hash. Used throughout for retrieval: qmd get #abc123
Dimension auto-detection: the vectors_vec table is created on first embed with the model's actual output dimensions.
Running three models locally requires careful memory management:
// Scoped session pattern await withLLMSession(llm, async (session) => { // session has abort signal // and duration limits const embeddings = await session.embedBatch(texts); }); // resources freed on exit // Reference counting tracks active // sessions per model class LLMSessionManager { refs: Map<model, count> canUnload(): boolean }
QMD exposes 4 tools via the Model Context Protocol, with both stdio and HTTP transports:
| Tool | Purpose |
|---|---|
| query | Multi-type search (lex/vec/hyde). The flagship tool. Accepts 1-10 typed sub-queries. |
| get | Single doc by path or docid. Supports :line ranges. Suggests similar files on 404. |
| multi_get | Batch retrieval via glob or CSV. 10KB default max per file. |
| status | Index health: doc counts, embedding coverage, collection metadata. |
Read-only markdown access via qmd://{+path} resource templates. Discovery is search-only (no directory listing).
# Stdio (Claude Code, Cursor) $ qmd mcp # HTTP (shared, long-lived) $ qmd mcp --http --port 8181 # Background daemon $ qmd mcp --http --daemon $ qmd mcp stop
On MCP initialize, the server builds a system prompt including:
# YAML config (~/.config/qmd/index.yml) collections: meeting-notes: path: ~/Documents/meetings pattern: "**/*.md" ignore: - "**/drafts/**" context: /: "Weekly team meeting notes" /2025: "Current year meetings" /2025/q1: "Q1 planning sessions" docs: path: ~/projects/api/docs pattern: "**/*.{md,mdx}"
Universal addressing across collections:
# Full virtual path qmd://meeting-notes/2025/standup.md # Context resolution (longest prefix) qmd://meeting-notes/2025/q1/jan.md → matches /2025/q1 context → "Q1 planning sessions"
# Index your files $ qmd collection add ~/notes --name notes $ qmd embed # Three search modes $ qmd search "budget Q4" # BM25 $ qmd vsearch "spending plans" # vector $ qmd query "what did we spend" # hybrid # Retrieve documents $ qmd get meeting.md:50 # from line 50 $ qmd get "#abc123" # by docid $ qmd multi-get "journals/2025*" # Maintenance $ qmd update --pull # git pull + reindex $ qmd status # index health $ qmd cleanup # vacuum + prune
| Flag | Format | Use Case |
|---|---|---|
| (default) | Colorized CLI | Humans |
| --json | Structured JSON | AI agents |
| --csv | CSV | Spreadsheets |
| --md | Markdown | Documentation |
| --xml | XML | Pipelines |
| --files | File list | Scripting |
Use QMD as an embedded search engine in your own Node.js/Bun apps:
import { createStore } from '@tobilu/qmd'; // Initialize from YAML, inline config, or DB path const store = await createStore({ dbPath: './my-index.sqlite', collections: { docs: { path: './content', pattern: '**/*.md' } } }); // Hybrid search const results = await store.search({ queries: [ { type: 'lex', query: 'API rate limits' }, { type: 'vec', query: 'throttling and quotas' } ], limit: 10, minScore: 0.3 }); // Document retrieval const doc = await store.get('api/auth.md'); const body = await store.getDocumentBody( doc.filepath, { fromLine: 1, maxLines: 50 } ); await store.close();
Config sources: Inline config, YAML file, or existing DB path. Supports lazy-loaded LLM models and write-through sync between YAML and SQLite.
| What | On-device hybrid search engine for markdown/knowledge bases |
| How | BM25 + vectors + LLM reranking, all in SQLite |
| Models | 3 GGUF models, ~2GB total, via node-llama-cpp |
| Integration | CLI, MCP server, and npm SDK |
| Secret sauce | Fine-tuned query expansion + position-aware score blending |
github.com/tobi/qmd · MIT License · npm install -g @tobilu/qmd