MyAiConfiguration/docs/superpowers/specs/2026-10-08-qmd-local-search-design.md

QMD local search design

Date: 2026-10-08 Status: approved in conversation, pending written-spec review

Goal

Replace the Python semantic retrieval stack with QMD as the only local search engine, used by Claude Code and Codex alike.

Role	Model	QMD URI
Embedding	Qwen3-Embedding-0.6B	hf:Qwen/Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf
Reranking	Qwen3-Reranker-0.6B	hf:ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF/qwen3-reranker-0.6b-q8_0.gguf
Query expansion	QMD Query Expansion 1.7B	hf:tobil/qmd-query-expansion-1.7B-gguf/qmd-query-expansion-1.7B-q4_k_m.gguf

Success criteria:

• No Python model code, virtual environment, or Python model dependency remains.
• QMD is no longer an extension in the plugin selector; the local-models question alone controls it.
• When enabled, agents use the search skill for every question whose identifiers are unknown, and the search answers in seconds because the index is warm and the models are loaded.
• When disabled or unsuitable, no skill, hook, instruction line, QMD package, or model installed by the setup remains.

Motivation from the 2026-10-08 Claude A/B benchmark: the Python search was never used in the three concept-search tasks, while the setup's fixed context made every run about 2.5 times more expensive in input tokens. A permanent MCP server would add to that cost, so the integration is a skill over a CLI wrapper.

Decisions

1. Search scope: primarily the current Git repository; additionally user-registered QMD collections (includeByDefault: true), ranked together in one query.
2. Index location: one global QMD index (~/.cache/qmd). Each repository is a collection named repo-<short hash of the absolute repository path> with includeByDefault: false. No files are written into repositories.
3. Agent access: an own semantic-search skill for both clients that calls a Node wrapper. QMD's qmd@qmd plugin and MCP tools are not used.
4. Warm-up: a SessionStart hook indexes in the background and starts an own search daemon built on the QMD SDK so all three models stay loaded. QMD's HTTP daemon (qmd mcp --http) is not used: its POST /query accepts only pre-typed lex/vec/hyde searches and never runs query expansion.
5. Daemon lifetime: the daemon exits by itself after 30 idle minutes (QMD_IDLE_MINUTES).
6. Auto mode: benchmark GPU first, then CPU; both thresholds must pass; if no device passes, remove what the setup installed and continue as No.

Components

Removed

• shared/retrieval/ (server.py, benchmark.py, requirements.txt).
• scripts/lib/semantic-retrieval.py, .sh, .ps1.
• scripts/tests/test-retrieval-server.py, scripts/tests/test-semantic-retrieval.py.
• The retrieval job in .gitea/workflows/ci.yml.
• The QMD row in adapters/plugins.tsv and QMD-specific ownership in scripts/lib/managed-extensions.py, scripts/lib/plugins.sh, and scripts/lib/plugins.ps1.
• Qwen/Python references in README.md, docs/plugins.md, docs/scripts.md, and other affected documentation.

Added under shared/qmd/

• qmd-daemon.mjs: long-lived search process using the QMD SDK (createStore({ dbPath, configPath }), store.search({ query, collections }) with expansion and reranking, store.update(), store.embed()). Listens on 127.0.0.1 only, on a port chosen at start; writes {port, token, pid} to ~/.my-ai-configuration/qmd/daemon.json readable only by the user; rejects requests without the token header. Exits after QMD_IDLE_MINUTES (default 30) without a request. A second start while one is healthy is a no-op.
• qmd-search.mjs: ensures the repository collection, refreshes it incrementally, sends the search to the daemon, starts the daemon if it is not running, and falls back to qmd query --json when the daemon cannot be used. Collections searched: the repository collection plus every collection with includeByDefault: true. Output: JSON list of {path, line, score, snippet} with repository-relative paths, best first.
• qmd-warm.mjs: started detached by the SessionStart hook. Starts the daemon with the stored device if it is not running and asks it to refresh and embed the repository collection. A per-repository lock file prevents parallel indexing from concurrent sessions.
• qmd-benchmark.mjs: the Auto benchmark (see below).
• Thin .sh and .ps1 entry points for every script, as the repository requires.

Changed

• shared/skills/research/semantic-search/SKILL.md: rewritten for QMD; name kept so references and the A/B benchmark stay stable.
• scripts/lib/qmd.sh, scripts/lib/qmd.ps1: install, update, benchmark, and removal, called from install and update.
• Hooks: SessionStart entries for Claude and Codex are generated only when local models are enabled.
• shared/rules/general.md (and the generated global instructions): the semantic-search line becomes mandatory and is generated only when the skill is installed.
• scripts/benchmarks/ab.py: setup-no-search blocks the skill and wrapper instead of the Python CLI; --warm-index runs qmd-warm.mjs.

Repository collection

• Name: repo-<first 12 hex characters of SHA-256 of the absolute repository path>.
• Mask: **/*.{md,py,cs,ts,tsx,js,jsx,mjs,go,rs,java,ps1,sh,json,yml,yaml,xml,sql}.
• Chunking: --chunk-strategy auto (AST chunking where QMD supports it; regex chunking elsewhere, including C#).
• QMD's built-in exclusions apply (node_modules, .git, .cache, vendor, dist, build).

Configuration and state

• ~/.config/qmd/index.yml (or QMD_CONFIG_DIR): the setup writes only the models: block; other keys and collections are preserved. An unparsable file aborts without writing. The daemon also writes the ignore list (the repository's git-ignored paths) of its own repo-* collections, because QMD re-syncs SQLite from this file and its addCollection write-through drops ignore.
• ~/.my-ai-configuration/qmd.json: {version, device: "gpu" | "cpu", owned_package: bool, owned_models: [...]}.
• Device mapping when starting QMD: gpu sets QMD_LLAMA_GPU=auto (llama.cpp uses all detected GPUs); cpu sets QMD_FORCE_CPU=1.

Install and update flow

Question, identical in Bash and PowerShell:

Enable local QMD search models (Qwen3 embedding, Qwen3 reranker, QMD query expansion)? [y/N/a]

• The default is the stored selection. The selection key semantic_retrieval and the -SemanticRetrieval / --semantic-retrieval parameters keep their names for compatibility.
• An update that keeps the stored selection does not rerun the benchmark; the benchmark runs only when a is chosen explicitly.

Prerequisites: Node.js 22 or newer and npm. Missing with Yes: abort with a clear message before any change. Missing with Auto: treat as unsuitable (No) and report the reason.

Yes:

1. npm install -g @tobilu/qmd@<pinned version>; record ownership only if QMD was not already installed.
2. Write the models: block into index.yml.
3. qmd pull.
4. Device: gpu when QMD's device probe (qmd doctor) reports a usable GPU, otherwise cpu. No benchmark.
5. Install skill, hook, scripts, and the mandatory instruction line.

Auto:

1. Steps 1 to 3 as for Yes.
2. Benchmark on GPU; if no GPU exists or it fails, benchmark on CPU.
3. The first passing device continues as Yes with that device.
4. If none passes: report the measurements, delete the models the setup downloaded, uninstall QMD only if the setup installed it, restore the configuration, store the selection as false, and continue as No.

No:

• Remove skill, hook, scripts, and instruction line; stop a running daemon.
• Remove the setup-owned QMD package and setup-owned models.
• Never remove a QMD installation, model, or collection the setup did not create.

Migration on the first update after this change:

• Remove the Python installation: ~/.my-ai-configuration/semantic-retrieval/ (virtual environment, model cache, data), its daemon, and semantic-retrieval.json.
• Remove previous QMD extension records (Claude plugin qmd@qmd, Codex QMD record) from the extension state. If the answer is Yes or a passing Auto, the existing npm package is adopted by the new path instead of being uninstalled.

Dry run: print every step as DRYRUN ...; no download, benchmark, or removal.

Auto benchmark

• Fixture: a small fixed document set shipped with the scripts, indexed into a temporary named index (qmd --index ai-config-benchmark), removed afterwards.
• Models are warmed before measuring.
• Search: median latency of three qmd query runs with expansion, vector search, and reranking of 40 candidates must be at most 5 s (QMD_BENCHMARK_MAX_QUERY_SECONDS).
• Indexing: embedding throughput must be at least 20 chunks per second (QMD_BENCHMARK_MIN_CHUNKS_PER_SECOND).
• Order: GPU first, then CPU. Timeout per device: 10 minutes; a timeout fails that device.
• Result: JSON {device, querySeconds, chunksPerSecond, passed} per measured device, printed in the install summary.

Runtime

SessionStart hook (Claude and Codex, .sh and .ps1):

• Runs only inside a Git repository. Starts qmd-warm.mjs detached and returns immediately with exit code 0.
• Errors go to ~/.my-ai-configuration/logs/qmd-warm.log; the hook never blocks or fails a session.

Search (qmd-search.mjs):

1. Refresh the repository collection when files changed since the last refresh (git status and modification times), then qmd embed for it.
2. Query the daemon; start it if it is not running; fall back to qmd query --json when the daemon cannot be reached or started.
3. Print the JSON results.

Strict skill:

• Description: "MUST use before reading or grepping code whenever the exact symbol, file, or string is not already known: concept questions, unfamiliar areas, 'where/how is X done', bug reports that describe behavior. Use rg only for known identifiers, paths, or exact strings."
• Body: search first with a full behavioral question; open and verify the top hits; then use rg for callers and tests. If no relevant hit appears, run one rephrased search before switching to rg. The previous cost check (status, repository-size thresholds) is removed.

Global instruction line (only when the skill is installed): "When semantic-search is installed, run it first for any question whose identifiers are unknown; fall back to rg with a brief note only if it fails."

Error handling

• Model download or npm failure during install or update: roll back what this run created. Yes fails the install; Auto treats it as unsuitable and reports the reason.
• Benchmark hang: per-device timeout of 10 minutes.
• Hook: detached, logged, always exit code 0. qmd embed uses QMD's 30-minute session cap and resumes incrementally on the next start.
• Wrapper failure: non-zero exit with a short message on stderr; the skill continues with rg and states the limitation.
• Stale daemon.json (process gone or token rejected): the wrapper deletes it and starts a new daemon; if that fails it uses the CLI fallback.
• index.yml that cannot be parsed: abort without writing.

Testing

• Node unit tests (node --test): collection name and mask, benchmark decision (thresholds, GPU before CPU, unsuitable result), daemon idle exit with an injected clock, token rejection, wrapper fallback to the CLI, index.yml merge preserving foreign keys.
• Install guard tests in Bash and PowerShell with a mocked qmd: Yes, No, Auto pass and fail, dry run, ownership (a foreign QMD survives No), migration (Python installation removed), skill, hook, and instruction line present only when enabled.
• Integration job replacing the retrieval CI job: install QMD, pull models (cached), index a small fixture, assert the wrapper returns the expected hit, start and stop the daemon.
• ab.py: the setup-no-search arm and --warm-index use the QMD path.
• prompt-budget: the longer skill description changes the baseline; update it deliberately and report the difference.

Risks to verify during implementation

• Whether the Codex workspace-write sandbox can reach the daemon on 127.0.0.1 and read ~/.cache/qmd. If not, the Codex skill needs an approval rule for the wrapper or must run it outside the sandbox.
• Whether qmd doctor's device probe is reliable enough to choose gpu for Yes without a benchmark.
• Disk and VRAM use: about 2.4 GB of models; the daemon holds them in VRAM until the idle stop.

Out of scope

• Commands for managing user collections; users register them with qmd collection add.
• Any MCP integration of QMD.