docs(specs): design QMD as the local search engine
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Julian lechnerandClaude Opus 5.5 committed
1 parent
5b098fef70
commit
6dd7cad9f5
1 file changed
+168
@@ -0,0 +1,168 @@
|
||||
# QMD local search design
|
||||
|
||||
Date: 2026-10-08
|
||||
Status: approved in conversation, pending written-spec review
|
||||
|
||||
## Goal
|
||||
|
||||
Replace the Python semantic retrieval stack with QMD as the only local search engine, used by Claude Code and Codex alike.
|
||||
|
||||
| Role | Model | QMD URI |
|
||||
|---|---|---|
|
||||
| Embedding | Qwen3-Embedding-0.6B | `hf:Qwen/Qwen3-Embedding-0.6B-GGUF/Qwen3-Embedding-0.6B-Q8_0.gguf` |
|
||||
| Reranking | Qwen3-Reranker-0.6B | `hf:ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF/qwen3-reranker-0.6b-q8_0.gguf` |
|
||||
| Query expansion | QMD Query Expansion 1.7B | `hf:tobil/qmd-query-expansion-1.7B-gguf/qmd-query-expansion-1.7B-q4_k_m.gguf` |
|
||||
|
||||
Success criteria:
|
||||
|
||||
- No Python model code, virtual environment, or Python model dependency remains.
|
||||
- QMD is no longer an extension in the plugin selector; the local-models question alone controls it.
|
||||
- When enabled, agents use the search skill for every question whose identifiers are unknown, and the search answers in seconds because the index is warm and the models are loaded.
|
||||
- When disabled or unsuitable, no skill, hook, instruction line, QMD package, or model installed by the setup remains.
|
||||
|
||||
Motivation from the 2026-10-08 Claude A/B benchmark: the Python search was never used in the three concept-search tasks, while the setup's fixed context made every run about 2.5 times more expensive in input tokens. A permanent MCP server would add to that cost, so the integration is a skill over a CLI wrapper.
|
||||
|
||||
## Decisions
|
||||
|
||||
1. Search scope: primarily the current Git repository; additionally user-registered QMD collections (`includeByDefault: true`), ranked together in one query.
|
||||
2. Index location: one global QMD index (`~/.cache/qmd`). Each repository is a collection named `repo-<short hash of the absolute repository path>` with `includeByDefault: false`. No files are written into repositories.
|
||||
3. Agent access: an own `semantic-search` skill for both clients that calls a Node wrapper. QMD's `qmd@qmd` plugin and MCP tools are not used.
|
||||
4. Warm-up: a SessionStart hook indexes in the background and starts QMD's HTTP daemon (`qmd mcp --http --daemon`) so models stay loaded.
|
||||
5. Daemon lifetime: a watcher stops the daemon after 30 idle minutes (`QMD_IDLE_MINUTES`).
|
||||
6. Auto mode: benchmark GPU first, then CPU; both thresholds must pass; if no device passes, remove what the setup installed and continue as No.
|
||||
|
||||
## Components
|
||||
|
||||
### Removed
|
||||
|
||||
- `shared/retrieval/` (`server.py`, `benchmark.py`, `requirements.txt`).
|
||||
- `scripts/lib/semantic-retrieval.py`, `.sh`, `.ps1`.
|
||||
- `scripts/tests/test-retrieval-server.py`, `scripts/tests/test-semantic-retrieval.py`.
|
||||
- The `retrieval` job in `.gitea/workflows/ci.yml`.
|
||||
- The `QMD` row in `adapters/plugins.tsv` and QMD-specific ownership in `scripts/lib/managed-extensions.py`, `scripts/lib/plugins.sh`, and `scripts/lib/plugins.ps1`.
|
||||
- Qwen/Python references in `README.md`, `docs/plugins.md`, `docs/scripts.md`, and other affected documentation.
|
||||
|
||||
### Added under `shared/qmd/`
|
||||
|
||||
- `qmd-search.mjs`: ensures the repository collection, refreshes it incrementally, queries the daemon with `POST /query` on `localhost:8181`, starts the daemon if it is not running, and falls back to `qmd query --json` when the daemon cannot be used. Collections searched: the repository collection plus every collection with `includeByDefault: true`. Output: JSON list of `{path, line, score, snippet}` with repository-relative paths, best first. Records the last-search timestamp for the watcher.
|
||||
- `qmd-warm.mjs`: started detached by the SessionStart hook. Ensures the collection, runs `qmd update` and `qmd embed` for that collection, starts the daemon with the stored device, and starts the watcher. A per-repository lock file prevents parallel indexing from concurrent sessions.
|
||||
- `qmd-watch.mjs`: checks every minute; after `QMD_IDLE_MINUTES` (default 30) without a search it runs `qmd mcp stop` and exits.
|
||||
- `qmd-benchmark.mjs`: the Auto benchmark (see below).
|
||||
- Thin `.sh` and `.ps1` entry points for every script, as the repository requires.
|
||||
|
||||
### Changed
|
||||
|
||||
- `shared/skills/research/semantic-search/SKILL.md`: rewritten for QMD; name kept so references and the A/B benchmark stay stable.
|
||||
- `scripts/lib/qmd.sh`, `scripts/lib/qmd.ps1`: install, update, benchmark, and removal, called from `install` and `update`.
|
||||
- Hooks: SessionStart entries for Claude and Codex are generated only when local models are enabled.
|
||||
- `shared/rules/general.md` (and the generated global instructions): the semantic-search line becomes mandatory and is generated only when the skill is installed.
|
||||
- `scripts/benchmarks/ab.py`: `setup-no-search` blocks the skill and wrapper instead of the Python CLI; `--warm-index` runs `qmd-warm.mjs`.
|
||||
|
||||
### Repository collection
|
||||
|
||||
- Name: `repo-<first 12 hex characters of SHA-256 of the absolute repository path>`.
|
||||
- Mask: `**/*.{md,py,cs,ts,tsx,js,jsx,mjs,go,rs,java,ps1,sh,json,yml,yaml,xml,sql}`.
|
||||
- Chunking: `--chunk-strategy auto` (AST chunking where QMD supports it; regex chunking elsewhere, including C#).
|
||||
- QMD's built-in exclusions apply (`node_modules`, `.git`, `.cache`, `vendor`, `dist`, `build`).
|
||||
|
||||
### Configuration and state
|
||||
|
||||
- `~/.config/qmd/index.yml` (or `QMD_CONFIG_DIR`): the setup writes only the `models:` block; other keys and collections are preserved. An unparsable file aborts without writing.
|
||||
- `~/.my-ai-configuration/qmd.json`: `{version, device: "gpu" | "cpu", owned_package: bool, owned_models: [...]}`.
|
||||
- Device mapping when starting QMD: `gpu` sets `QMD_LLAMA_GPU=auto` (llama.cpp uses all detected GPUs); `cpu` sets `QMD_FORCE_CPU=1`.
|
||||
|
||||
## Install and update flow
|
||||
|
||||
Question, identical in Bash and PowerShell:
|
||||
|
||||
`Enable local QMD search models (Qwen3 embedding, Qwen3 reranker, QMD query expansion)? [y/N/a]`
|
||||
|
||||
- The default is the stored selection. The selection key `semantic_retrieval` and the `-SemanticRetrieval` / `--semantic-retrieval` parameters keep their names for compatibility.
|
||||
- An update that keeps the stored selection does not rerun the benchmark; the benchmark runs only when `a` is chosen explicitly.
|
||||
|
||||
Prerequisites: Node.js 22 or newer and `npm`. Missing with Yes: abort with a clear message before any change. Missing with Auto: treat as unsuitable (No) and report the reason.
|
||||
|
||||
Yes:
|
||||
|
||||
1. `npm install -g @tobilu/qmd@<pinned version>`; record ownership only if QMD was not already installed.
|
||||
2. Write the `models:` block into `index.yml`.
|
||||
3. `qmd pull`.
|
||||
4. Device: `gpu` when QMD's device probe (`qmd doctor`) reports a usable GPU, otherwise `cpu`. No benchmark.
|
||||
5. Install skill, hook, scripts, and the mandatory instruction line.
|
||||
|
||||
Auto:
|
||||
|
||||
1. Steps 1 to 3 as for Yes.
|
||||
2. Benchmark on GPU; if no GPU exists or it fails, benchmark on CPU.
|
||||
3. The first passing device continues as Yes with that device.
|
||||
4. If none passes: report the measurements, delete the models the setup downloaded, uninstall QMD only if the setup installed it, restore the configuration, store the selection as `false`, and continue as No.
|
||||
|
||||
No:
|
||||
|
||||
- Remove skill, hook, scripts, instruction line, and watcher; stop a running daemon.
|
||||
- Remove the setup-owned QMD package and setup-owned models.
|
||||
- Never remove a QMD installation, model, or collection the setup did not create.
|
||||
|
||||
Migration on the first update after this change:
|
||||
|
||||
- Remove the Python installation: `~/.my-ai-configuration/semantic-retrieval/` (virtual environment, model cache, data), its daemon, and `semantic-retrieval.json`.
|
||||
- Remove previous QMD extension records (Claude plugin `qmd@qmd`, Codex QMD record) from the extension state. If the answer is Yes or a passing Auto, the existing npm package is adopted by the new path instead of being uninstalled.
|
||||
|
||||
Dry run: print every step as `DRYRUN ...`; no download, benchmark, or removal.
|
||||
|
||||
## Auto benchmark
|
||||
|
||||
- Fixture: a small fixed document set shipped with the scripts, indexed into a temporary named index (`qmd --index ai-config-benchmark`), removed afterwards.
|
||||
- Models are warmed before measuring.
|
||||
- Search: median latency of three `qmd query` runs with expansion, vector search, and reranking of 40 candidates must be at most 5 s (`QMD_BENCHMARK_MAX_QUERY_SECONDS`).
|
||||
- Indexing: embedding throughput must be at least 20 chunks per second (`QMD_BENCHMARK_MIN_CHUNKS_PER_SECOND`).
|
||||
- Order: GPU first, then CPU. Timeout per device: 10 minutes; a timeout fails that device.
|
||||
- Result: JSON `{device, querySeconds, chunksPerSecond, passed}` per measured device, printed in the install summary.
|
||||
|
||||
## Runtime
|
||||
|
||||
SessionStart hook (Claude and Codex, `.sh` and `.ps1`):
|
||||
|
||||
- Runs only inside a Git repository. Starts `qmd-warm.mjs` detached and returns immediately with exit code 0.
|
||||
- Errors go to `~/.my-ai-configuration/logs/qmd-warm.log`; the hook never blocks or fails a session.
|
||||
|
||||
Search (`qmd-search.mjs`):
|
||||
|
||||
1. Refresh the repository collection when files changed since the last refresh (`git status` and modification times), then `qmd embed` for it.
|
||||
2. Query the daemon; start it if it is not running; fall back to `qmd query --json` when the daemon cannot be reached or started.
|
||||
3. Print the JSON results and record the search timestamp.
|
||||
|
||||
Strict skill:
|
||||
|
||||
- Description: "MUST use before reading or grepping code whenever the exact symbol, file, or string is not already known: concept questions, unfamiliar areas, 'where/how is X done', bug reports that describe behavior. Use rg only for known identifiers, paths, or exact strings."
|
||||
- Body: search first with a full behavioral question; open and verify the top hits; then use `rg` for callers and tests. If no relevant hit appears, run one rephrased search before switching to `rg`. The previous cost check (`status`, repository-size thresholds) is removed.
|
||||
|
||||
Global instruction line (only when the skill is installed): "When `semantic-search` is installed, run it first for any question whose identifiers are unknown; fall back to `rg` with a brief note only if it fails."
|
||||
|
||||
## Error handling
|
||||
|
||||
- Model download or `npm` failure during install or update: roll back what this run created. Yes fails the install; Auto treats it as unsuitable and reports the reason.
|
||||
- Benchmark hang: per-device timeout of 10 minutes.
|
||||
- Hook: detached, logged, always exit code 0. `qmd embed` uses QMD's 30-minute session cap and resumes incrementally on the next start.
|
||||
- Wrapper failure: non-zero exit with a short message on stderr; the skill continues with `rg` and states the limitation.
|
||||
- Port 8181 owned by another process: detected through `GET /health` and the response signature; the wrapper uses the CLI fallback.
|
||||
- `index.yml` that cannot be parsed: abort without writing.
|
||||
|
||||
## Testing
|
||||
|
||||
- Node unit tests (`node --test`): collection name and mask, benchmark decision (thresholds, GPU before CPU, unsuitable result), watcher timeout with an injected clock, wrapper fallback to the CLI, `index.yml` merge preserving foreign keys.
|
||||
- Install guard tests in Bash and PowerShell with a mocked `qmd`: Yes, No, Auto pass and fail, dry run, ownership (a foreign QMD survives No), migration (Python installation removed), skill, hook, and instruction line present only when enabled.
|
||||
- Integration job replacing the `retrieval` CI job: install QMD, pull models (cached), index a small fixture, assert the wrapper returns the expected hit, start and stop the daemon.
|
||||
- `ab.py`: the `setup-no-search` arm and `--warm-index` use the QMD path.
|
||||
- `prompt-budget`: the longer skill description changes the baseline; update it deliberately and report the difference.
|
||||
|
||||
## Risks to verify during implementation
|
||||
|
||||
- Whether the Codex `workspace-write` sandbox can reach `localhost:8181` and read `~/.cache/qmd`. If not, the Codex skill needs an approval rule for the wrapper or must run it outside the sandbox.
|
||||
- Whether `qmd doctor`'s device probe is reliable enough to choose `gpu` for Yes without a benchmark.
|
||||
- Disk and VRAM use: about 2.4 GB of models; the daemon holds them in VRAM until the idle stop.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Commands for managing user collections; users register them with `qmd collection add`.
|
||||
- Any MCP integration of QMD.
|
||||
Reference in new issue
Block a user