MyAiConfiguration/docs/superpowers/specs/2026-10-08-qmd-local-search-design.md

QMD local search design

Date: 2026-10-08 Status: approved in conversation, pending written-spec review

Goal

Replace the Python semantic retrieval stack with QMD as the only local search engine, used by Claude Code and Codex alike.

Role	Model	QMD URI
Embedding	Qwen3-Embedding-0.6B	hf:Qwen/Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf
Reranking	Qwen3-Reranker-0.6B	hf:ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF/qwen3-reranker-0.6b-q8_0.gguf
Query expansion	QMD Query Expansion 1.7B	hf:tobil/qmd-query-expansion-1.7B-gguf/qmd-query-expansion-1.7B-q4_k_m.gguf

Success criteria:

• No Python model code, virtual environment, or Python model dependency remains.
• QMD is no longer an extension in the plugin selector; the local-models question alone controls it.
• When enabled, agents use the search skill for every question whose identifiers are unknown, and the search answers in seconds because the index is warm and the models are loaded.
• When disabled or unsuitable, no skill, hook, instruction line, QMD package, or model installed by the setup remains.

Motivation from the 2026-10-08 Claude A/B benchmark: the Python search was never used in the three concept-search tasks, while the setup's fixed context made every run about 2.5 times more expensive in input tokens. A permanent MCP server would add to that cost, so the integration is a skill over a CLI wrapper.

Decisions

1. Search scope: primarily the current Git repository; additionally user-registered QMD collections (includeByDefault: true), ranked together in one query.
2. Index location: one global QMD index (~/.cache/qmd). Each repository is a collection named repo-<short hash of the absolute repository path> with includeByDefault: false. No files are written into repositories.
3. Agent access: an own semantic-search skill for both clients that calls a Node wrapper. QMD's qmd@qmd plugin and MCP tools are not used.
4. Warm-up: a SessionStart hook indexes in the background and starts QMD's HTTP daemon (qmd mcp --http --daemon) so models stay loaded.
5. Daemon lifetime: a watcher stops the daemon after 30 idle minutes (QMD_IDLE_MINUTES).
6. Auto mode: benchmark GPU first, then CPU; both thresholds must pass; if no device passes, remove what the setup installed and continue as No.

Components

Removed

• shared/retrieval/ (server.py, benchmark.py, requirements.txt).
• scripts/lib/semantic-retrieval.py, .sh, .ps1.
• scripts/tests/test-retrieval-server.py, scripts/tests/test-semantic-retrieval.py.
• The retrieval job in .gitea/workflows/ci.yml.
• The QMD row in adapters/plugins.tsv and QMD-specific ownership in scripts/lib/managed-extensions.py, scripts/lib/plugins.sh, and scripts/lib/plugins.ps1.
• Qwen/Python references in README.md, docs/plugins.md, docs/scripts.md, and other affected documentation.

Added under shared/qmd/

• qmd-search.mjs: ensures the repository collection, refreshes it incrementally, queries the daemon with POST /query on localhost:8181, starts the daemon if it is not running, and falls back to qmd query --json when the daemon cannot be used. Collections searched: the repository collection plus every collection with includeByDefault: true. Output: JSON list of {path, line, score, snippet} with repository-relative paths, best first. Records the last-search timestamp for the watcher.
• qmd-warm.mjs: started detached by the SessionStart hook. Ensures the collection, runs qmd update and qmd embed for that collection, starts the daemon with the stored device, and starts the watcher. A per-repository lock file prevents parallel indexing from concurrent sessions.
• qmd-watch.mjs: checks every minute; after QMD_IDLE_MINUTES (default 30) without a search it runs qmd mcp stop and exits.
• qmd-benchmark.mjs: the Auto benchmark (see below).
• Thin .sh and .ps1 entry points for every script, as the repository requires.

Changed

• shared/skills/research/semantic-search/SKILL.md: rewritten for QMD; name kept so references and the A/B benchmark stay stable.
• scripts/lib/qmd.sh, scripts/lib/qmd.ps1: install, update, benchmark, and removal, called from install and update.
• Hooks: SessionStart entries for Claude and Codex are generated only when local models are enabled.
• shared/rules/general.md (and the generated global instructions): the semantic-search line becomes mandatory and is generated only when the skill is installed.
• scripts/benchmarks/ab.py: setup-no-search blocks the skill and wrapper instead of the Python CLI; --warm-index runs qmd-warm.mjs.

Repository collection

• Name: repo-<first 12 hex characters of SHA-256 of the absolute repository path>.
• Mask: **/*.{md,py,cs,ts,tsx,js,jsx,mjs,go,rs,java,ps1,sh,json,yml,yaml,xml,sql}.
• Chunking: --chunk-strategy auto (AST chunking where QMD supports it; regex chunking elsewhere, including C#).
• QMD's built-in exclusions apply (node_modules, .git, .cache, vendor, dist, build).

Configuration and state

• ~/.config/qmd/index.yml (or QMD_CONFIG_DIR): the setup writes only the models: block; other keys and collections are preserved. An unparsable file aborts without writing.
• ~/.my-ai-configuration/qmd.json: {version, device: "gpu" | "cpu", owned_package: bool, owned_models: [...]}.
• Device mapping when starting QMD: gpu sets QMD_LLAMA_GPU=auto (llama.cpp uses all detected GPUs); cpu sets QMD_FORCE_CPU=1.

Install and update flow

Question, identical in Bash and PowerShell:

Enable local QMD search models (Qwen3 embedding, Qwen3 reranker, QMD query expansion)? [y/N/a]

• The default is the stored selection. The selection key semantic_retrieval and the -SemanticRetrieval / --semantic-retrieval parameters keep their names for compatibility.
• An update that keeps the stored selection does not rerun the benchmark; the benchmark runs only when a is chosen explicitly.

Prerequisites: Node.js 22 or newer and npm. Missing with Yes: abort with a clear message before any change. Missing with Auto: treat as unsuitable (No) and report the reason.

Yes:

1. npm install -g @tobilu/qmd@<pinned version>; record ownership only if QMD was not already installed.
2. Write the models: block into index.yml.
3. qmd pull.
4. Device: gpu when QMD's device probe (qmd doctor) reports a usable GPU, otherwise cpu. No benchmark.
5. Install skill, hook, scripts, and the mandatory instruction line.

Auto:

1. Steps 1 to 3 as for Yes.
2. Benchmark on GPU; if no GPU exists or it fails, benchmark on CPU.
3. The first passing device continues as Yes with that device.
4. If none passes: report the measurements, delete the models the setup downloaded, uninstall QMD only if the setup installed it, restore the configuration, store the selection as false, and continue as No.

No:

• Remove skill, hook, scripts, instruction line, and watcher; stop a running daemon.
• Remove the setup-owned QMD package and setup-owned models.
• Never remove a QMD installation, model, or collection the setup did not create.

Migration on the first update after this change:

• Remove the Python installation: ~/.my-ai-configuration/semantic-retrieval/ (virtual environment, model cache, data), its daemon, and semantic-retrieval.json.
• Remove previous QMD extension records (Claude plugin qmd@qmd, Codex QMD record) from the extension state. If the answer is Yes or a passing Auto, the existing npm package is adopted by the new path instead of being uninstalled.

Dry run: print every step as DRYRUN ...; no download, benchmark, or removal.

Auto benchmark

• Fixture: a small fixed document set shipped with the scripts, indexed into a temporary named index (qmd --index ai-config-benchmark), removed afterwards.
• Models are warmed before measuring.
• Search: median latency of three qmd query runs with expansion, vector search, and reranking of 40 candidates must be at most 5 s (QMD_BENCHMARK_MAX_QUERY_SECONDS).
• Indexing: embedding throughput must be at least 20 chunks per second (QMD_BENCHMARK_MIN_CHUNKS_PER_SECOND).
• Order: GPU first, then CPU. Timeout per device: 10 minutes; a timeout fails that device.
• Result: JSON {device, querySeconds, chunksPerSecond, passed} per measured device, printed in the install summary.

Runtime

SessionStart hook (Claude and Codex, .sh and .ps1):

• Runs only inside a Git repository. Starts qmd-warm.mjs detached and returns immediately with exit code 0.
• Errors go to ~/.my-ai-configuration/logs/qmd-warm.log; the hook never blocks or fails a session.

Search (qmd-search.mjs):

1. Refresh the repository collection when files changed since the last refresh (git status and modification times), then qmd embed for it.
2. Query the daemon; start it if it is not running; fall back to qmd query --json when the daemon cannot be reached or started.
3. Print the JSON results and record the search timestamp.

Strict skill:

• Description: "MUST use before reading or grepping code whenever the exact symbol, file, or string is not already known: concept questions, unfamiliar areas, 'where/how is X done', bug reports that describe behavior. Use rg only for known identifiers, paths, or exact strings."
• Body: search first with a full behavioral question; open and verify the top hits; then use rg for callers and tests. If no relevant hit appears, run one rephrased search before switching to rg. The previous cost check (status, repository-size thresholds) is removed.

Global instruction line (only when the skill is installed): "When semantic-search is installed, run it first for any question whose identifiers are unknown; fall back to rg with a brief note only if it fails."

Error handling

• Model download or npm failure during install or update: roll back what this run created. Yes fails the install; Auto treats it as unsuitable and reports the reason.
• Benchmark hang: per-device timeout of 10 minutes.
• Hook: detached, logged, always exit code 0. qmd embed uses QMD's 30-minute session cap and resumes incrementally on the next start.
• Wrapper failure: non-zero exit with a short message on stderr; the skill continues with rg and states the limitation.
• Port 8181 owned by another process: detected through GET /health and the response signature; the wrapper uses the CLI fallback.
• index.yml that cannot be parsed: abort without writing.

Testing

• Node unit tests (node --test): collection name and mask, benchmark decision (thresholds, GPU before CPU, unsuitable result), watcher timeout with an injected clock, wrapper fallback to the CLI, index.yml merge preserving foreign keys.
• Install guard tests in Bash and PowerShell with a mocked qmd: Yes, No, Auto pass and fail, dry run, ownership (a foreign QMD survives No), migration (Python installation removed), skill, hook, and instruction line present only when enabled.
• Integration job replacing the retrieval CI job: install QMD, pull models (cached), index a small fixture, assert the wrapper returns the expected hit, start and stop the daemon.
• ab.py: the setup-no-search arm and --warm-index use the QMD path.
• prompt-budget: the longer skill description changes the baseline; update it deliberately and report the difference.

Risks to verify during implementation

• Whether the Codex workspace-write sandbox can reach localhost:8181 and read ~/.cache/qmd. If not, the Codex skill needs an approval rule for the wrapper or must run it outside the sandbox.
• Whether qmd doctor's device probe is reliable enough to choose gpu for Yes without a benchmark.
• Disk and VRAM use: about 2.4 GB of models; the daemon holds them in VRAM until the idle stop.

Out of scope

• Commands for managing user collections; users register them with qmd collection add.
• Any MCP integration of QMD.