Compare commits

..
76 Commits
Author SHA1 Message Date
Julian lechner 2192b4613d fix(ci): refresh agent metadata budget baselines
CI / CI configuration security checks (push) Successful in 3s
CI / Capabilities and shell quality (push) Successful in 16s
CI / Python tests and compilation (push) Successful in 23s
CI / QMD unit tests (push) Successful in 12s
CI / Build and inspect generated packages (push) Successful in 17s
CI / PowerShell tests on Ubuntu (push) Successful in 1m6s
CI / Installed client configuration smoke tests (push) Successful in 1m20s
CI / Bash integration and installation guards (push) Successful in 1m21s
2026-10-08 16:09:11 +02:00
Julian lechner 5e80d30936 fix(config): harden doctor checks and enable Codex full access
CI / CI configuration security checks (push) Successful in 3s
CI / Capabilities and shell quality (push) Successful in 18s
CI / Python tests and compilation (push) Successful in 25s
CI / QMD unit tests (push) Successful in 11s
CI / Build and inspect generated packages (push) Successful in 17s
CI / PowerShell tests on Ubuntu (push) Successful in 1m11s
CI / Installed client configuration smoke tests (push) Successful in 1m10s
CI / Bash integration and installation guards (push) Failing after 1m13s
2026-10-08 15:58:47 +02:00
Julian lechner 8aae370bdb fix(agents): align research tools and clarify role boundaries 2026-10-08 15:43:30 +02:00
Julian lechner 8288ea51f0 docs: refresh configuration guides and remove obsolete plans 2026-10-08 15:37:03 +02:00
Julian lechner 17d01d58f5 fix(ci): install YAML dependency for QMD unit tests 2026-10-08 15:30:23 +02:00
Julian lechner 7f441d388d fix(qmd): resolve hoisted dependencies from package context
CI / CI configuration security checks (push) Successful in 3s
CI / Capabilities and shell quality (push) Successful in 15s
CI / Python tests and compilation (push) Successful in 23s
CI / QMD unit tests (push) Failing after 11s
CI / Build and inspect generated packages (push) Successful in 17s
CI / PowerShell tests on Ubuntu (push) Successful in 1m2s
CI / Installed client configuration smoke tests (push) Successful in 1m7s
CI / Bash integration and installation guards (push) Successful in 1m17s
2026-10-08 15:25:33 +02:00
Julian lechner 88f7c9984b docs(instructions): explain conditional custom instruction loading
CI / CI configuration security checks (push) Successful in 3s
CI / Capabilities and shell quality (push) Successful in 18s
CI / Python tests and compilation (push) Successful in 23s
CI / QMD unit tests (push) Failing after 9s
CI / Build and inspect generated packages (push) Successful in 15s
CI / PowerShell tests on Ubuntu (push) Successful in 1m11s
CI / Installed client configuration smoke tests (push) Successful in 52s
CI / Bash integration and installation guards (push) Successful in 1m14s
2026-10-08 15:16:30 +02:00
Julian lechner b1170604eb fix(instructions): refresh startup rule when custom instructions change 2026-10-08 15:16:19 +02:00
Julian lechner 83091d247a fix(instructions): add startup rule only for existing custom instructions 2026-10-08 15:16:07 +02:00
Julian lechner 6a981bd7f2 fix(instructions): check optional personal file before reading 2026-10-08 15:02:05 +02:00
Julian lechner 73cb30c32b ci(qmd): keep model execution out of ci 2026-10-08 14:59:32 +02:00
Julian lechner 2fe612c8e1 fix(tests): expect codex session start hook in packages 2026-10-08 14:59:19 +02:00
Julian lechner 32283c428a docs(plugins): explain automatic triggers and opt-outs 2026-10-08 14:49:23 +02:00
Julian lechner 99d30163ce feat(skills): enforce automatic plugin workflow routing 2026-10-08 14:49:10 +02:00
Julian lechner 8e9b28ffb0 refactor(rules): shorten mandatory research guidance 2026-10-08 14:48:55 +02:00
Julian lechner 1cdb6b1881 refactor(image): retain focused SEO and performance guidance 2026-10-08 14:38:49 +02:00
Julian lechner 65c99ac76d feat(codex): show remaining usage and fast mode in status bar 2026-10-08 14:38:33 +02:00
Julian lechner 5c66f79a76 fix(hooks): initialize flashbang when its DLL cache is unavailable 2026-10-08 14:38:18 +02:00
Julian lechner 3586245408 feat(workflows): enforce research checkpoints across development phases 2026-10-08 14:38:05 +02:00
Julian lechner fe7a4d7a25 feat(research): require Context7 evidence for external behavior 2026-10-08 14:37:47 +02:00
Julian lechner fa43cb13ac docs(skills): explain mandatory research workflow
CI / CI configuration security checks (push) Successful in 3s
CI / Capabilities and shell quality (push) Successful in 17s
CI / Python tests and compilation (push) Successful in 24s
CI / Build and inspect generated packages (push) Successful in 42s
CI / Bash integration and installation guards (push) Failing after 20s
CI / PowerShell tests on Ubuntu (push) Failing after 1m27s
CI / Installed client configuration smoke tests (push) Successful in 40s
CI / QMD local search integration (push) Failing after 13m6s
2026-10-08 13:55:28 +02:00
Julian lechner 21e3352456 docs(rules): enforce research gates across workflows 2026-10-08 13:55:27 +02:00
Julian lechner d929c6df26 docs(skills): require primary evidence for every technical lookup 2026-10-08 13:55:27 +02:00
Julian lechner f497a23aaa docs(skills): apply semantic routing before every repository lookup 2026-10-08 13:55:26 +02:00
Julian lechner e12caa0d9f docs(skills): require research checks for every repository lookup 2026-10-08 13:55:26 +02:00
Julian lechner a1427f18a3 feat(skills): add mandatory research gate 2026-10-08 13:55:25 +02:00
Julian lechner 766f08af26 fix(codex): use elevated Windows sandbox 2026-10-08 13:55:24 +02:00
Julian lechnerandClaude Sonnet 5.5 c041521668 feat(codex): enable api_key_model_discovery feature
CI / CI configuration security checks (push) Successful in 3s
CI / Capabilities and shell quality (push) Successful in 15s
CI / Python tests and compilation (push) Successful in 22s
CI / Build and inspect generated packages (push) Successful in 27s
CI / PowerShell tests on Ubuntu (push) Failing after 1m7s
CI / Bash integration and installation guards (push) Failing after 24s
CI / Installed client configuration smoke tests (push) Successful in 1m9s
CI / QMD local search integration (push) Failing after 12m46s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 13:11:58 +02:00
Julian lechnerandClaude Opus 5.5 3f51bc908e docs(skills): make technical research mandatory for unverified external behavior
CI / PowerShell tests on Ubuntu (push) Failing after 3s
CI / CI configuration security checks (push) Successful in 4s
CI / Capabilities and shell quality (push) Successful in 22s
CI / Python tests and compilation (push) Successful in 23s
CI / Build and inspect generated packages (push) Successful in 19s
CI / Bash integration and installation guards (push) Failing after 22s
CI / Installed client configuration smoke tests (push) Successful in 1m16s
CI / QMD local search integration (push) Failing after 12m46s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:56:29 +02:00
Julian lechnerandClaude Opus 5.5 36851c34e2 docs(skills): hand semantic search hits to repository research
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:56:28 +02:00
Julian lechnerandClaude Opus 5.5 ede0c37d24 docs(skills): make repository research mandatory and route unknown identifiers to semantic search
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:56:28 +02:00
Julian lechnerandClaude Opus 5.5 b8ebe452bf chore(gitignore): ignore .NET build output of benchmark fixtures
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:54:08 +02:00
Julian lechnerandClaude Opus 5.5 1e93e1dad4 chore(gitignore): ignore Python bytecode caches
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:54:07 +02:00
Julian lechnerandClaude Opus 5.5 c0fa0961c3 docs(qmd): document the git ignore list in repository collections
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:48:09 +02:00
Julian lechnerandClaude Opus 5.5 06c4848336 fix(qmd): handle a lost daemon start race and stale daemon tokens
startDaemon accepts the daemon of another launch when its own child
exits 0. A 403 from the daemon marks daemon.json as stale: the wrapper
deletes it and starts a new daemon once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:47:41 +02:00
Julian lechnerandClaude Opus 5.5 3012e1666e fix(qmd): let only one concurrent launch start the daemon
An exclusive daemon.lock (stale after 2 minutes) is held from before the
health check until the daemon listens; a launch that loses exits 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:47:03 +02:00
Julian lechnerandClaude Opus 5.5 691ad6a69f fix(qmd): skip git-ignored paths in repository collections
The daemon passes `git ls-files --others --ignored --directory` as
escaped ignore globs to addCollection. QMD's YAML write-through drops
`ignore` and index.yml wins on every re-sync, so the list is also
written into index.yml and re-synced whenever it changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:45:27 +02:00
Julian lechnerandClaude Opus 5.5 abface2293 fix(qmd): write index.yml atomically
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:42:40 +02:00
Julian lechnerandClaude Opus 5.5 ced6d8f119 fix(qmd): keep models loaded until the daemon's own idle exit
QMD 2.8.3 createStore unloads the models after 5 idle minutes, which
made the next search reload them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:42:05 +02:00
Julian lechnerandClaude Opus 5.5 1618cd7cd0 docs(qmd): note the CLI fallback under the Codex sandbox
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:41:49 +02:00
Julian lechnerandClaude Opus 5.5 233152ff54 fix(qmd): honor XDG_CACHE_HOME for the setup-owned model files
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:41:42 +02:00
Julian lechnerandClaude Opus 5.5 7ebd61f35b fix(qmd): make removal wait for the daemon and survive partial failures
Removal reads the daemon PID before --stop and waits up to 10 s for it
to exit. A model file that cannot be deleted stays recorded in the state
for a retry instead of aborting removal, and a failing models block
restore only warns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:41:19 +02:00
Julian lechnerandClaude Opus 5.5 0c203a3905 fix(qmd): stop the legacy Python daemon before removing its directory
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:40:37 +02:00
Julian lechnerandClaude Opus 5.5 f1d7894ba0 fix(qmd): treat Auto install and GPU probe failures as unsuitable
Auto now rolls back and reports "not suitable" when Node.js is too old,
npm or the model download fails. A failing GPU probe falls back to the
CPU in both Yes and Auto.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:40:12 +02:00
Julian lechnerandClaude Opus 5.5 60f739ceb9 fix(qmd): repair integration test, pull models first, reorder CI steps
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:19:18 +02:00
Julian lechnerandClaude Opus 5.5 5f21e4fb7c ci(qmd): replace the Qwen retrieval job with a QMD integration job
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:17:49 +02:00
Julian lechnerandClaude Opus 5.5 c31af49b70 docs(qmd): make semantic search mandatory and document QMD local search
Prompt budget stays within +10% of baseline (instruction 1485 -> 1331,
skill metadata 3163 -> 3017, rule catalog claude 28818 -> 30517); baseline
files unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:15:15 +02:00
Julian lechnerandClaude Opus 5.5 5bb0bdf62f refactor(retrieval): remove the Python Qwen stack and the QMD extension entry
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:12:22 +02:00
Julian lechnerandClaude Opus 5.5 505e6ae084 feat(install): ask for QMD local search models and run the QMD installer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:10:19 +02:00
Julian lechnerandClaude Opus 5.5 953401a431 fix(qmd): reject unreliable npm listings and restore non-mapping models verbatim
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:55:47 +02:00
Julian lechnerandClaude Opus 5.5 7f89ae70bd fix(qmd): preserve user models block, adopt-and-remove ledger package, guard symlinks
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:53:22 +02:00
Julian lechnerandClaude Opus 5.5 59796d99d7 feat(qmd): install, benchmark, migrate, and remove QMD local search
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:49:22 +02:00
Julian lechnerandClaude Opus 5.5 29e712e751 fix(qmd): measure embed throughput after warm-up and fail on embed errors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:47:48 +02:00
Julian lechnerandClaude Opus 5.5 d8a5ad7492 feat(qmd): add GPU and CPU benchmark for Auto selection
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:46:05 +02:00
Julian lechnerandClaude Opus 5.5 f483ea7fe7 fix(qmd): stop hook group growth on re-sync and harden the warm-up lock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:44:18 +02:00
Julian lechnerandClaude Opus 5.5 f3c88a089e feat(qmd): warm the index and daemon from a SessionStart hook
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:37:47 +02:00
Julian lechnerandClaude Opus 5.5 08f015de92 fix(qmd): start the search daemon only when none runs and fail fast on child exit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:29:05 +02:00
Julian lechnerandClaude Opus 5.5 a40ff270d8 feat(qmd): add search wrapper with daemon start and CLI fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:25:28 +02:00
Julian lechnerandClaude Opus 5.5 f4dd193ea1 fix(qmd): harden daemon refresh, fingerprint, and request handling
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:23:55 +02:00
Julian lechnerandClaude Opus 5.5 cba9a7a2f4 feat(qmd): add token-protected search daemon with idle exit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:16:52 +02:00
Julian lechnerandClaude Opus 5.5 1ad1880704 feat(qmd): add shared Node helpers and index.yml model block editing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:15:08 +02:00
Julian lechnerandClaude Opus 5.5 c7c7c57621 docs(plans): simplify the warm-up step
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:10:18 +02:00
Julian lechnerandClaude Opus 5.5 76de4f8dab docs(plans): plan QMD local search implementation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:10:04 +02:00
Julian lechnerandClaude Opus 5.5 6850ef5f2a docs(specs): run QMD search through an SDK daemon with expansion
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:03:03 +02:00
Julian lechnerandClaude Opus 5.5 6dd7cad9f5 docs(specs): design QMD as the local search engine
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:58:42 +02:00
Julian lechnerandClaude Opus 5.5 5b098fef70 test(benchmarks): phrase the transient retry prompt as a task
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:40:22 +02:00
Julian lechnerandClaude Opus 5.5 0779e5c7db fix(benchmarks): resume Codex sessions without the rejected --sandbox flag
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:40:21 +02:00
Julian lechnerandClaude Opus 5.5 af1e248243 fix(benchmarks): pass prompts through stdin so Windows rework turns keep newlines
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:40:21 +02:00
Julian lechnerandClaude Opus 5.5 e6b516f828 feat(benchmarks): summarize A/B results per task category
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:03 +02:00
Julian lechnerandClaude Opus 5.5 449ffa3631 test(benchmarks): tag tasks as small change, larger implementation, or search
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:02 +02:00
Julian lechnerandClaude Opus 5.5 56f9981cc9 docs(rules): merge repeated completion checks in definition of done
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:02 +02:00
Julian lechnerandClaude Opus 5.5 0af6b13a16 docs(rules): leave commented-out code to the definition of done
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:02 +02:00
Julian lechnerandClaude Opus 5.5 bd4d94e451 docs(rules): keep Java composition guidance in modern-java only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:01 +02:00
Julian lechnerandClaude Opus 5.5 f4f38b38c9 docs(rules): give each C# rule statement one authoritative file
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:01 +02:00
Julian lechnerandClaude Opus 5.5 30257e72cd ci: fail PowerShell job when the runtime cannot be installed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:00 +02:00
Julian lechnerandClaude Opus 5.5 f254a1ca40 fix(context-report): accept "< 20" token counts from /context
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:00 +02:00
146 changed files with 3228 additions and 2690 deletions

No files matched your search

+23 -29
View File
@@ -64,23 +64,23 @@ jobs:
timeout-minutes: 20
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Check PowerShell runtime
- name: Install PowerShell runtime
shell: bash
run: |
if command -v pwsh >/dev/null 2>&1; then
echo 'POWERSHELL_AVAILABLE=true' >> "$GITHUB_ENV"
exit 0
fi
if sudo apt-get update && sudo apt-get install --yes --no-install-recommends powershell; then
echo 'POWERSHELL_AVAILABLE=true' >> "$GITHUB_ENV"
else
echo 'POWERSHELL_AVAILABLE=false' >> "$GITHUB_ENV"
echo '::warning::PowerShell is unavailable; PowerShell checks skipped for this runner.'
set -euo pipefail
if ! command -v pwsh >/dev/null 2>&1; then
# Microsoft's signed apt repository; the job fails when PowerShell cannot be installed.
. /etc/os-release
wget -q -O "$RUNNER_TEMP/packages-microsoft-prod.deb" "https://packages.microsoft.com/config/$ID/$VERSION_ID/packages-microsoft-prod.deb"
sudo dpkg -i "$RUNNER_TEMP/packages-microsoft-prod.deb"
sudo apt-get update
sudo apt-get install --yes --no-install-recommends powershell
fi
pwsh -NoLogo -NoProfile -NonInteractive -Command '$PSVersionTable.PSVersion.ToString()'
- name: Run PowerShell tests
shell: bash
run: |
if [ "${POWERSHELL_AVAILABLE:-false}" != true ]; then exit 0; fi
set -euo pipefail
pwsh -NoLogo -NoProfile -NonInteractive -File ./scripts/commands/check-capabilities.ps1
pwsh -NoLogo -NoProfile -NonInteractive -File ./scripts/commands/build.ps1 -Summary
for file in scripts/tests/test-*.ps1; do
@@ -172,29 +172,23 @@ jobs:
AI_CONFIG_REQUIRE_CLIENTS: '1'
run: timeout 900 python scripts/tests/test-client-smoke.py -v
retrieval:
name: Qwen retrieval integration
qmd:
name: QMD unit tests
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 10
needs: [python]
env:
QMD_PACKAGE_DIR: ${{ runner.temp }}/qmd-tests
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
- uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v22.16.0
with:
python-version: '3.12'
cache: pip
cache-dependency-path: shared/retrieval/requirements.txt
- name: Install retrieval dependencies
run: python -m pip install -r shared/retrieval/requirements.txt
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .cache/huggingface
key: qwen-retrieval-${{ runner.os }}-${{ hashFiles('shared/retrieval/requirements.txt') }}
- name: Run retrieval tests with real models
env:
QWEN_TEST_MODEL_CACHE: ${{ github.workspace }}/.cache/huggingface
HF_HUB_DISABLE_XET: '1'
run: python scripts/tests/test-retrieval-server.py
node-version: '24'
# CI runs model-free tests only; real-model integration requires a suitable local machine.
- name: Install QMD test dependency
run: npm install --prefix "$QMD_PACKAGE_DIR" --ignore-scripts --no-package-lock --no-audit --no-fund yaml@2.9.0
- name: Run QMD unit tests
run: node --test scripts/tests/test-qmd-lib.mjs scripts/tests/test-qmd-daemon.mjs scripts/tests/test-qmd-search.mjs scripts/tests/test-qmd-warm.mjs scripts/tests/test-qmd-benchmark.mjs
security:
name: CI configuration security checks
+4
View File
@@ -15,3 +15,7 @@ credentials.json
secrets.json
!.env.example
.idea
__pycache__/
*.pyc
bin/
obj/
+19 -17
View File
@@ -30,7 +30,7 @@ Bash:
## Quickstart
With the [requirements](#requirements) installed, clone the repository, build it, then install it.
Stop if any command fails. Installation asks for the shell, client, Flashbang, status line, Qwen retrieval, and extension choices.
Stop if any command fails. Installation asks for the shell, client, Flashbang, status line, local QMD search models, and extension choices.
PowerShell:
@@ -96,11 +96,10 @@ Install and update also rebuild internally; the explicit build above lets you ch
- Separate install, update, and uninstall scripts: install refuses to run against an already-installed destination,
update refuses to run against one that is not installed yet, and uninstall refuses to run against one that was
never installed.
- User-level extension installation for Ponytail, i-have-adhd, Superpowers, Context7, Caveman, QMD, MCPorter,
- User-level extension installation for Ponytail, i-have-adhd, Superpowers, Context7, Caveman, MCPorter,
Humanizer, Impeccable, and Anthropic Frontend Design, with an interactive extension toggle during both install and update. Only extensions
installed and recorded by this setup can be removed through deselection.
- Optional local semantic retrieval for both clients using Qwen3-Embedding-0.6B embeddings and Qwen3-Reranker-0.6B
reranking, disabled by default and removable through the install/update choice.
- Optional local search for both clients through QMD with Qwen3-Embedding-0.6B, Qwen3-Reranker-0.6B, and QMD Query Expansion 1.7B (Yes/No/Auto; Auto benchmarks GPU, then CPU).
## Requirements
@@ -108,8 +107,7 @@ Install and update also rebuild internally; the explicit build above lets you ch
- Bash 4.3 or newer, with standard utilities such as `awk`, `sed`, and `sha256sum`.
- Python 3.11 or newer for configuration validation and task-state hooks/helpers (`python` for PowerShell scripts, `python3` for Bash scripts).
- `jq` for Claude's Bash status line; without it, that status line produces no output.
- Python `venv` and `pip` when the optional semantic retrieval layer is enabled.
- Node.js 22 or newer and `npm` when QMD or MCPorter is selected.
- Node.js 22 or newer and `npm` when local QMD search or MCPorter is selected.
Core Bash setup scripts do not require PowerShell. The credential commands use Windows Credential Manager and require
PowerShell, including when invoked through their Bash wrappers. The PowerShell scripts remain compatible with
@@ -129,6 +127,11 @@ shared definitions -> client adapters -> generated packages -> user installation
| [`scripts/`](docs/scripts.md) | Build, installation, update, diagnosis, and validation scripts. |
| `docs/` | Focused documentation about the configuration design. |
Client settings start in `adapters/codex/config/config.toml` and
`adapters/claude/config/settings.json`. Shared Markdown and metadata define behavior;
adapters supply client-specific formats and placeholders resolved during build or installation.
Generated output is not a source to edit.
The build creates:
```text
@@ -143,16 +146,13 @@ generated/
| Document | Contents |
| --- | --- |
| [Architecture](docs/architecture.md) | Shared sources, adapters, generated packages, and installation. |
| [Scripts](docs/scripts.md) | Commands, checks, tests, and internal script layout. |
| [Configuration](docs/configuration.md) | Client settings, permission defaults, status displays, and verification. |
| [Configuration](docs/configuration.md) | Client settings, permissions, status displays, local QMD search, and verification. |
| [Extending the configuration](docs/extending-configuration.md) | Add shared rules, skills, and agents. |
| [Plugins](docs/plugins.md) | Plugin sources, selection, installation, updates, and removal. |
| [Hooks](docs/hooks.md) | Registered events, session-state helpers, notifications, and optional utilities. |
| [Agents](docs/agents.md) | Agent roles and when delegation is useful. |
| [Skills](docs/skills.md) | Skill categories and conditional rule loading. |
| [Codex adapter](adapters/codex/README.md) | Codex-specific configuration files. |
| [Claude Code adapter](adapters/claude/README.md) | Claude-specific configuration files. |
| [Hook package](shared/hooks/README.md) | Overview shipped with the hook scripts. |
| [Repository instructions](AI-Instructions.md) | Rules for maintaining this repository. |
@@ -232,11 +232,11 @@ Dry runs still rebuild `generated/`, but do not modify installation destinations
Selecting Codex also installs shared skills to `.agents/skills`. An interactive extension list includes plugins and
Codex skills; unchecked extensions are skipped on installation. Normal install and update ask whether to enable
Flashbang, apply the custom status line, and enable Qwen3 semantic retrieval. Choices are saved for Quickupdate.
Flashbang, apply the custom status line, and enable local QMD search. Choices are saved for Quickupdate.
The shared skill catalog includes development skills and focused writing, image, site structure, psychology, and
conversion skills. All retained skills are included in both generated clients.
conversion skills. All retained skills are included in both generated clients; installation includes `semantic-search` only when local QMD search is enabled.
The status line defaults to enabled to preserve existing behavior. Disabling it removes the custom client setting and,
for Claude, the setup-owned status line script. Disabling retrieval on update removes its setup-owned runtime and index.
for Claude, the setup-owned status line script. Disabling local QMD search removes its setup-owned package, models, and repository collections once no client still uses it. See [Local QMD search](docs/configuration.md#local-qmd-search) for storage and removal details.
Dry runs preview changes without updating extensions or retrieval. See [Plugins](docs/plugins.md) for client-specific behavior.
## Update Script
@@ -263,7 +263,7 @@ setup; pre-existing or manually installed extensions are retained. Quickupdate r
## Uninstall Script
Remove everything this setup previously installed for the selected client(s): managed files tracked in each
destination's manifest, extensions and skills recorded in the extension ledger, an enabled semantic retrieval setup,
destination's manifest, extensions and skills recorded in the extension ledger, an enabled local QMD search setup,
and the saved Quickupdate selection (once no destination remains installed for any client).
PowerShell:
@@ -289,8 +289,8 @@ touched. Codex skill backups are kept under `.agents/.ai-config-skill-backups/`.
deleted once it is fully empty; backups already on disk are always kept.
Extensions and skills recorded in the ledger are removed the same way `update` removes a deselected extension:
pre-existing or manually installed ones outside the ledger are left alone, and a skill file modified since it was
installed is kept with a warning. An enabled semantic retrieval setup is disabled and its setup-owned runtime and
index removed for the selected client(s). The saved Quickupdate selection (`~/.my-ai-configuration/selection.json`)
installed is kept with a warning. Local QMD search is disabled for the selected client(s). Its shared runtime,
owned models, and recorded repository collections are removed only when no client still uses it; the shared QMD database and foreign collections are retained. The saved Quickupdate selection (`~/.my-ai-configuration/selection.json`)
is only removed once uninstalling leaves no client installed at all, so uninstalling just one client out of a
`Both` installation keeps the saved choices for the client that remains.
@@ -301,7 +301,9 @@ keep outside this setup's ledger, or files this setup never installed.
Codex and Claude Code receive the same shared rules, skills, agents, hooks, and status data. Codex loads applicable rules from generated files, while Claude Code exposes technology- and situation-specific rules as generated skills.
Codex defaults to workspace-write sandboxing and automatic review for eligible escalation requests. Claude uses
Codex defaults to full access under the account that launches it, with unrestricted command network access.
Launch Codex as the repository owner; Codex does not switch to that owner automatically. Full access removes
Codex's filesystem sandbox boundaries. Eligible approval requests still use automatic review. Claude uses
automatic permissions and enables its sandbox where the platform supports it; explicit deny rules still apply where it
does not. Both clients include the same logical agent roles and compact status displays. See
[Configuration](docs/configuration.md) for settings, permissions, and limitations.
-3
View File
@@ -1,3 +0,0 @@
# Claude Code adapter
Claude Code uses global `CLAUDE.md`, `settings.json`, skills, and agents under the user profile. This adapter supplies client metadata; semantic content remains in `shared/`.
+2 -2
View File
@@ -1,6 +1,6 @@
role tools
architect Read,Grep,Glob,WebFetch,WebSearch
researcher Read,Grep,Glob,WebFetch,WebSearch
architect Read,Grep,Glob,WebFetch,WebSearch,__SHELL__
researcher Read,Grep,Glob,WebFetch,WebSearch,__SHELL__
reviewer Read,Grep,Glob,__SHELL__
verifier Read,Grep,Glob,__SHELL__
implementer Read,Grep,Glob,Edit,Write,__SHELL__
1 role tools
2 architect Read,Grep,Glob,WebFetch,WebSearch Read,Grep,Glob,WebFetch,WebSearch,__SHELL__
3 researcher Read,Grep,Glob,WebFetch,WebSearch Read,Grep,Glob,WebFetch,WebSearch,__SHELL__
4 reviewer Read,Grep,Glob,__SHELL__
5 verifier Read,Grep,Glob,__SHELL__
6 implementer Read,Grep,Glob,Edit,Write,__SHELL__
+4 -1
View File
@@ -16,7 +16,10 @@
},
"hooks": {
"PreCompact": [{"hooks": [{"type": "command", "command": "__HOOK_COMMAND__ \"__AI_CONFIG_ROOT__/hooks/scripts/__COMPACT_SCRIPT__\"", "timeout": 4}]}],
"SessionStart": [{"hooks": [{"type": "command", "command": "__HOOK_COMMAND__ \"__AI_CONFIG_ROOT__/hooks/scripts/__SESSION_POINTER_SCRIPT__\"", "timeout": 4}]}],
"SessionStart": [
{"hooks": [{"type": "command", "command": "__HOOK_COMMAND__ \"__AI_CONFIG_ROOT__/hooks/scripts/__SESSION_POINTER_SCRIPT__\"", "timeout": 4}]},
{"hooks": [{"type": "command", "command": "__HOOK_COMMAND__ \"__AI_CONFIG_ROOT__/hooks/scripts/__QMD_WARM_SCRIPT__\"", "timeout": 4}]}
],
"Stop": [
{
"hooks": [
-3
View File
@@ -1,3 +0,0 @@
# Codex adapter
Codex uses global `AGENTS.md`, `config.toml`, skills, and agent configuration under the user profile. This adapter supplies client metadata; semantic content remains in `shared/`.
+10 -9
View File
@@ -1,16 +1,9 @@
approval_policy = "on-request"
approvals_reviewer = "auto_review"
sandbox_mode = "workspace-write"
[sandbox_workspace_write]
network_access = true
writable_roots = ['__AI_CONFIG_ROOT__/../.my-ai-configuration/semantic-retrieval/data']
[windows]
sandbox = "unelevated"
sandbox_mode = "danger-full-access"
[tui]
status_line = ["model", "reasoning", "approval-mode", "project-name", "git-branch", "context-window-size", "context-used", "used-tokens"]
status_line = ["model", "reasoning", "fast-mode", "approval-mode", "five-hour-limit", "weekly-limit", "project-name", "git-branch", "context-window-size", "context-used", "used-tokens"]
[agents]
enabled = true
@@ -19,6 +12,7 @@ max_depth = 1
[features]
hooks = true
api_key_model_discovery = true
[[hooks.Stop]]
[[hooks.Stop.hooks]]
@@ -26,3 +20,10 @@ type = "command"
command = '__HOOK_COMMAND__ "__AI_CONFIG_ROOT__/hooks/__HOOK_SCRIPT__"'
command_windows = '__POWERSHELL_HOOK_COMMAND__ "__AI_CONFIG_ROOT__/hooks/__POWERSHELL_HOOK_SCRIPT__"'
timeout = 8
[[hooks.SessionStart]]
[[hooks.SessionStart.hooks]]
type = "command"
command = '__HOOK_COMMAND__ "__AI_CONFIG_ROOT__/hooks/scripts/__QMD_WARM_SCRIPT__"'
command_windows = '__POWERSHELL_HOOK_COMMAND__ "__AI_CONFIG_ROOT__/hooks/scripts/__QMD_WARM_SCRIPT__"'
timeout = 4
-1
View File
@@ -4,7 +4,6 @@ i-have-adhd ayghri/i-have-adhd i-have-adhd@i-have-adhd ayghri/i-have-adhd i-have
Superpowers anthropics/claude-plugins-official superpowers@claude-plugins-official openai-curated-remote superpowers@openai-curated-remote plugin - -
Context7 upstash/context7 context7@context7-marketplace upstash/context7 context7@context7-marketplace plugin - - context7 https://mcp.context7.com/mcp
Caveman JuliusBrussee/caveman caveman@caveman thinkhome-org/caveman-codex caveman@thinkhome-caveman plugin - -
QMD tobi/qmd qmd@qmd - - qmd @tobilu/qmd qmd
MCPorter - mcporter - mcporter cli mcporter -
Humanizer blader/humanizer humanizer@humanizer - - skill blader/humanizer humanizer
Impeccable pbakaus/impeccable impeccable@impeccable pbakaus/impeccable impeccable skill pbakaus/impeccable impeccable
1 name claude_marketplace claude_plugin codex_marketplace codex_plugin codex_method codex_source codex_skill mcporter_name mcporter_url
4 Superpowers anthropics/claude-plugins-official superpowers@claude-plugins-official openai-curated-remote superpowers@openai-curated-remote plugin - -
5 Context7 upstash/context7 context7@context7-marketplace upstash/context7 context7@context7-marketplace plugin - - context7 https://mcp.context7.com/mcp
6 Caveman JuliusBrussee/caveman caveman@caveman thinkhome-org/caveman-codex caveman@thinkhome-caveman plugin - -
QMD tobi/qmd qmd@qmd - - qmd @tobilu/qmd qmd
7 MCPorter - mcporter - mcporter cli mcporter -
8 Humanizer blader/humanizer humanizer@humanizer - - skill blader/humanizer humanizer
9 Impeccable pbakaus/impeccable impeccable@impeccable pbakaus/impeccable impeccable skill pbakaus/impeccable impeccable
+2 -2
View File
@@ -24,8 +24,8 @@
"codex": 8510
},
"agent_metadata_tokens": {
"claude": 126,
"codex": 52
"claude": 204,
"codex": 124
}
}
}
+2 -2
View File
@@ -9,5 +9,5 @@ auto_loaded_rule_tokens claude 678
auto_loaded_rule_tokens codex 678
rule_catalog_tokens claude 28818
rule_catalog_tokens codex 8510
agent_metadata_tokens claude 126
agent_metadata_tokens codex 52
agent_metadata_tokens claude 204
agent_metadata_tokens codex 124
1 metric client baseline_tokens
9 auto_loaded_rule_tokens codex 678
10 rule_catalog_tokens claude 28818
11 rule_catalog_tokens codex 8510
12 agent_metadata_tokens claude 126 204
13 agent_metadata_tokens codex 52 124
+9 -3
View File
@@ -2,14 +2,20 @@
The five shared agents are logical roles, not mandatory steps.
- `implementer` executes the smallest complete engineering change and performs self-review.
- `researcher` investigates code, patterns, evidence, alternatives, and unknowns without modifying implementation code by default.
- `architect` evaluates technical boundaries, dependencies, interfaces, constraints, and design options.
- `implementer` implements scoped features and fixes with proportionate tests and self-review.
- `researcher` answers questions about repository behavior, external documentation, and unknowns with source evidence, without modifying implementation code by default.
- `architect` compares design options and recommends technical boundaries and interfaces before implementation; implementation changes require an explicit assignment.
- `verifier` supplies execution evidence from relevant tests, builds, checks, generated output, and behavior reproduction. It inspects implementation only to choose checks and does not review the diff for general correctness.
- `reviewer` starts with the actual diff and changed symbols, then examines necessary callers, dependencies, tests, and contracts for analytical correctness. It does not run tests, builds, or runtime checks; it reports missing execution evidence.
Use only the roles that are likely to improve the outcome. Choose `researcher`, `reviewer`, and `verifier` based on the task's uncertainty, complexity, risk, blast radius, and verification value; none runs automatically. The agent may make evidence-based, reversible technical decisions. Ask the user when technical choices remain materially uncertain or when business behavior, user-visible behavior, UI, UX, APIs, database models, configuration formats, compatibility behavior, or other high-impact decisions are ambiguous or open to interpretation.
## Research tools
Claude's [capability manifest](../adapters/claude/capabilities.tsv) gives `architect` and `researcher` file inspection, web research, and the package's shell tool (`PowerShell` or `Bash`). Shell access supports mandatory semantic search and Context7 research through MCPorter. Their role instructions scope shell use to inspection and research; this is an instruction-level boundary, not a read-only shell sandbox.
The `reviewer` may inspect Git diffs and run required research commands through its shell tool. Tests, builds, and behavior reproduction belong to the `verifier` or parent. Missing execution evidence remains an explicit review gap. Codex receives the same role instructions; the Claude tool manifest is client-specific.
## Coordination
The main agent owns task state, decisions, and progress. Subagents report to their parent. The parent forwards only relevant conclusions, references, risks, and constraints; direct peer communication requires a concrete dependency and parent coordination. Subagents cannot create further subagents.
-5
View File
@@ -1,5 +0,0 @@
# Architecture
Shared Markdown and metadata define meaning once. Codex and Claude adapters contain only client-specific paths, filenames, and serialization details. The build script renders those inputs into `generated/`. Installation is a separate, explicit step that copies generated files into global user-level locations with backups.
Generated output is ignored because it is reproducible build output. The source files are the reviewable record.
+108 -3
View File
@@ -1,6 +1,19 @@
# Configuration
Codex receives a generated global `AGENTS.md`, agent TOML files, skills, rule files, and a `config.toml` adapter. It sets workspace sandboxing with network access for shell commands (so MCPorter calls such as Context7 work without approval), automatic approval review, `agents.enabled = true`, `agents.max_concurrent_threads_per_session = 5`, and `agents.max_depth = 1`. Claude Code receives a generated global `CLAUDE.md`, agent Markdown files, skills, and `settings.json` with `auto` permissions, explicit `Agent` and `mcporter call context7.*` shell permissions, `mcp.context7.com` as the only allowed sandbox domain where the Claude sandbox is enabled, a five-operation concurrency default, at most five concurrent subagents (`CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS`), and a subagent spawn depth of one (`CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH`), so subagents cannot start further subagents. Installation resolves user paths at runtime; the repository contains no machine-specific paths.
Source templates are [`adapters/codex/config/config.toml`](../adapters/codex/config/config.toml)
and [`adapters/claude/config/settings.json`](../adapters/claude/config/settings.json).
Installation targets `~/.codex` and `~/.claude`; Codex skills go to `~/.agents/skills`.
Choose the PowerShell or Bash package independently of the client.
Codex receives a generated global `AGENTS.md`, agent TOML files, skills, rule files, and a `config.toml` adapter.
It sets full access for commands, automatic approval review, `agents.enabled = true`,
`agents.max_concurrent_threads_per_session = 5`, and `agents.max_depth = 1`.
Claude Code receives a generated global `CLAUDE.md`, agent Markdown files, skills, and `settings.json` with
`auto` permissions and explicit `Agent` and `mcporter call context7.*` shell permissions. Where enabled, its sandbox
allows the `mcp.context7.com` domain. The adapter sets a five-operation concurrency default, at most five concurrent
subagents (`CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS`), and a spawn depth of one (`CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH`).
Installation resolves user paths at runtime. Active session policy can further restrict these defaults.
Both clients embed `shared/rules/general.md` and the rule-loading conditions from `shared/global-instructions.md`. Rules for languages and frameworks live in their own directories, with an `index.md` entry point and focused Markdown rules under `references/`. Security has its own directory with a baseline entry point and focused rules for authentication, web, APIs, data, files, network, cryptography, and supply chains. Git, Microsoft, and cross-cutting rules remain directly under `shared/rules/`. Codex receives the full directory tree; Claude generates a skill for each rule and copies supporting references alongside the index skill.
@@ -12,11 +25,30 @@ Client settings remain thin adapters. Shared semantics remain in `shared/`. User
## Permission defaults
Codex sets `approval_policy = "on-request"`, `approvals_reviewer = "auto_review"`, and `sandbox_mode = "workspace-write"`. Eligible approval requests go through automatic review. Claude sets `permissions.defaultMode = "auto"` and shell-specific deny rules. Its sandbox is enabled on supported macOS, Linux, and WSL2 environments and disabled where unavailable, including native Windows. These are generated defaults, not a guarantee that a running session, account policy, or explicit launch override uses the same mode.
Codex sets `sandbox_mode = "danger-full-access"`, `approval_policy = "on-request"`, and
`approvals_reviewer = "auto_review"`. All Codex commands run without Codex filesystem or network sandbox
restrictions, under the account that launches Codex. Launch Codex as the repository owner to use that identity;
this configuration does not impersonate another user or change filesystem ownership. OS permissions, firewalls,
and managed organization policies still apply. Eligible approval requests retain automatic review, which does
not restore a filesystem boundary. See the [official OpenAI security documentation](https://learn.chatgpt.com/docs/agent-approvals-security).
The adapter no longer selects the Windows sandbox or workspace writable roots. This avoids using the dedicated
`CodexSandboxOnline` account, whose identity differs from the repository owner and can trigger Git's ownership
check. Old `[windows]` and `[sandbox_workspace_write]` tables may survive an update as preserved local settings;
they do not activate sandboxing while `sandbox_mode` is `danger-full-access`.
Apply the generated configuration through the normal update process, then start a new Codex session under the
repository owner's account. Existing sessions keep their active permissions. Check `whoami` and
`git rev-parse --show-toplevel` there; package validation cannot prove the identity or network access of a future
session. Launch overrides and managed policies may restrict or reject the configured mode.
Claude retains `permissions.defaultMode = "auto"` and shell-specific deny rules. Its sandbox is enabled on
supported macOS, Linux, and WSL2 environments and disabled where unavailable, including native Windows.
These are generated defaults; active session or account policy may differ.
## Status displays
Codex's native `tui.status_line` contains `model`, `reasoning`, `approval-mode`, `project-name`, `git-branch`, `context-window-size`, `context-used`, and `used-tokens`, in that order. `approval-mode` reports the active command review mode. Rendering and colors are controlled by Codex; the adapter does not define custom colors per field.
Codex's native `tui.status_line` contains `model`, `reasoning`, `fast-mode`, `approval-mode`, `five-hour-limit`, `weekly-limit`, `project-name`, `git-branch`, `context-window-size`, `context-used`, and `used-tokens`, in that order. `fast-mode` shows `Fast on` or `Fast off` for supported models; it does not change the selected mode. `five-hour-limit` and `weekly-limit` show remaining usage in the primary and secondary quota windows, typically five hours and one week, and are omitted when Codex has no quota data. `approval-mode` reports the active command review mode. Rendering and colors are controlled by Codex; the adapter does not define custom colors per field. These native fields are supported by [Codex 0.161.0](https://github.com/openai/codex/blob/rust-v0.161.0/codex-rs/tui/src/chatwidget/status_surfaces.rs).
Claude runs `shared/statusline/statusline.ps1` for PowerShell or `statusline.sh` for Bash. Its two-line display is:
@@ -31,6 +63,79 @@ Model and context-window size are cyan, effort and cumulative tokens magenta, re
Claude's status line displays `permission_mode` or `permissionMode` when Claude provides it, and otherwise shows the configured `auto` review mode.
## Local QMD search
Install and update offer **Yes / No / Auto**, with No as the initial default. This option is separate
from the extension selector. The saved `semantic_retrieval` key keeps its name for compatibility.
Quickupdate reuses the saved choice and device; it does not rerun the benchmark.
- **Yes:** install `@tobilu/qmd@2.8.3` if QMD is absent, copy the shared runtime scripts, configure
and download the three models, and choose GPU when detected or CPU otherwise. An existing QMD
installation is reused, not upgraded by this step.
- **Auto:** prepare the same dependencies, then benchmark GPU when available and CPU if needed.
Both thresholds must pass: median warm search latency at most 5 seconds and embedding throughput
at least 20 chunks/second. Each device has a 10-minute timeout. No passing device disables search
and triggers setup-owned resource cleanup. Auto can download models while answering the setup prompt.
- **No:** omit the search skill and its instruction line, remove the warm-up hook registration,
and disable search for the selected clients. Dry runs do not download, benchmark, or remove QMD resources.
The models are Qwen3-Embedding-0.6B, Qwen3-Reranker-0.6B, and QMD Query Expansion 1.7B.
Exact model URIs live in [`shared/qmd/qmd-lib.mjs`](../shared/qmd/qmd-lib.mjs).
Auto thresholds can be overridden with `QMD_BENCHMARK_MAX_QUERY_SECONDS` and
`QMD_BENCHMARK_MIN_CHUNKS_PER_SECOND`. The benchmark uses a temporary database and a deterministic
100-document fixture; its measurements are not an end-to-end estimate for a user's repository.
### Search and storage
The `semantic-search` skill calls the installed wrapper from the current Git repository:
```text
node "$HOME/.my-ai-configuration/qmd/qmd-search.mjs" --query "Where are temporary upload failures retried?" --top-k 5
```
This command works in PowerShell and Bash. `--root` selects a repository explicitly. Results are JSON
objects with `path`, `line`, `score`, and `snippet`; repository hits use relative paths and other collections
use `qmd://` URIs. Known identifiers still use direct reads or `rg`; unknown identifiers use semantic retrieval first.
A token-protected daemon listens on `127.0.0.1`, refreshes changed repository collections, skips git-ignored
paths, and searches the current repository together with the user's default QMD collections. Repository
collections use `repo-` followed by a 12-character path hash. A SessionStart hook warms the daemon and index
in the background. Models stay loaded until the daemon stops after 30 idle minutes, configurable with
`QMD_IDLE_MINUTES`.
On daemon failure the wrapper tries the QMD CLI against the current repository collection only. This fallback
requires an existing index; it does not create or refresh the collection. If the wrapper fails, the skill
reports the limitation and continues with `rg`.
| Location | Contents |
| --- | --- |
| `~/.my-ai-configuration/qmd/` | Runtime scripts, daemon state, locks, collection ownership, and `warm.log`. |
| `~/.my-ai-configuration/qmd.json` | Selected device, enabled clients, and setup ownership records. |
| `$XDG_CACHE_HOME/qmd/` (default `~/.cache/qmd/`) | Shared `index.sqlite` and model cache. |
| `$QMD_CONFIG_DIR/index.yml` | Configuration when `QMD_CONFIG_DIR` is set; otherwise `$XDG_CONFIG_HOME/qmd/index.yml`, defaulting to `~/.config/qmd/index.yml`. |
With the default Codex full-access configuration, QMD uses the launching account's filesystem and network
permissions, including custom cache and configuration paths. If a session overrides this with a sandbox,
permit writes to the runtime directory, resolved cache, and QMD configuration directory. QMD writes configuration
when registering collections and persisting ignore lists.
Disabling search or uninstalling one client keeps the shared runtime while another client remains enabled.
After the last client disables it, cleanup stops the daemon, removes recorded repository collections and
setup-owned models/package, restores the previous `models:` block, and removes the runtime scripts.
It preserves foreign QMD installations, models, collections, and the shared database. Failed cleanup can leave
resources behind; inspect reported warnings rather than deleting the whole cache. Updates also migrate the
old setup-owned Python retrieval directory and QMD extension ownership records.
### Verification and troubleshooting
Check `~/.my-ai-configuration/qmd/warm.log` for background warm-up errors. First startup or changed repositories
can take longer than a warm query. An older session still using the Windows sandbox can fail Git ownership
checks or child-process pipe operations. Apply the configuration and start a new session as described in
[Permission defaults](#permission-defaults).
Model-free unit tests do not verify downloads, GPU support, sandbox access, or real search quality.
See [QMD tests](scripts.md#qmd-tests) for the separate test commands and dependency setup.
## Verification and installation
The agent chooses the smallest sufficient checks based on behavior, affected callers, risk, reversibility, and uncertainty. Inspection can suffice for a small change. Full builds, broad test suites, and independent reviews are not automatic; explicit user and project requirements remain binding.
+5 -2
View File
@@ -9,10 +9,13 @@ The adapters register these events:
| Codex and Claude | `Stop` | Run the finish notification (`flashbang`). |
| Claude | `PreCompact` | Record the compaction timestamp under the installed configuration root and point to existing workspace task state. |
| Claude | `SessionStart` | Print an absolute pointer if the workspace's `.ai-session/state.json` exists. |
| Codex and Claude | `SessionStart` | Warm the QMD index and search daemon in the background (only while local QMD search is enabled). |
The main agent alone maintains canonical task state. Hooks never write or summarize it, print its contents, or infer completed verification. They use the hook event's `cwd`, falling back to the process working directory when no workspace is supplied. Malformed event data does not fall back to another project. An absent state file is a successful no-op. Legacy state under the installed configuration root is not automatically adopted; confirm its task before migrating it. The scripts do not grant permissions, bypass approvals, or replace client permission systems.
The main agent alone maintains canonical task state. Hooks never write or summarize it, print its contents, or infer completed verification. The task-state hooks use the hook event's `cwd`, falling back to the process working directory when no workspace is supplied. Malformed event data does not fall back to another project. An absent state file is a successful no-op. Legacy state under the installed configuration root is not automatically adopted; confirm its task before migrating it. The scripts do not grant permissions, bypass approvals, or replace client permission systems.
When enabled during installation or update, Flashbang runs on `Stop`, after the main agent finishes responding. The default flash lasts about 500 milliseconds. The PowerShell command omits `-WindowStyle Hidden`, which can hide or minimize the shared terminal. The PowerShell hook creates a low-opacity click-through overlay without activating another window when its native GUI APIs are available; unsupported runtimes exit safely. The Bash hook uses Python's `tkinter`, uses `xrandr` for multi-monitor bounds when available, and falls back to `notify-send` when the overlay is unavailable. Registered hooks are best effort: errors do not fail a Claude session.
The QMD warm-up hook uses the process working directory and starts only inside a Git repository when Node.js and the installed wrapper are available. It runs detached; wrapper errors are recorded in `~/.my-ai-configuration/qmd/warm.log`. Disabling local search removes its hook registration. See [Local QMD search](configuration.md#local-qmd-search) for daemon behavior and storage.
When enabled during installation or update, Flashbang runs on `Stop`, after the main agent finishes responding. The default flash lasts about 500 milliseconds. The PowerShell command omits `-WindowStyle Hidden`, which can hide or minimize the shared terminal. The PowerShell hook creates a low-opacity click-through overlay without activating another window when its native GUI APIs are available; unsupported runtimes exit safely. Its native DLL cache is optional: a missing `LOCALAPPDATA` environment variable or an unusable cache falls back to in-process compilation instead of suppressing the notification. The Bash hook uses Python's `tkinter`, uses `xrandr` for multi-monitor bounds when available, and falls back to `notify-send` when the overlay is unavailable. Registered hooks are best effort: errors do not fail a Claude session.
`Validate-CommandSafety`/`validate-command-safety.sh` and `Invoke-PostChangeVerification`/`invoke-post-change-verification.sh` are unregistered utilities. The safety utilities check command text for selected dangerous patterns; the PowerShell variant also checks some absolute paths. Neither is a complete security boundary. The verification utilities explicitly run the repository build when invoked; they are not an automatic requirement after each change.
+39 -4
View File
@@ -2,10 +2,39 @@
The user-level plugin manifest is `adapters/plugins.tsv`. It maps each logical extension to the client-specific plugin, skill, or MCP installation method.
The current baseline includes Ponytail, i-have-adhd, Superpowers, Context7, Caveman, QMD, MCPorter, Humanizer, Impeccable, and Anthropic Frontend Design. Both scripts operate at user scope and show an extension selector unless running a dry run. Selections are collected before configuration changes. Both rebuild generated packages first.
The current baseline includes Ponytail, i-have-adhd, Superpowers, Context7, Caveman, MCPorter, Humanizer, Impeccable, and Anthropic Frontend Design. Install and update operate at user scope and show an extension selector unless running a dry run or Quickupdate. Selections are collected before configuration changes. Both rebuild generated packages before synchronizing client files and extensions. Choosing Auto for local search can prepare and benchmark QMD earlier, during option selection.
### Automatic workflow integration
The always-loaded general rules route both clients to the shared `plugin-workflow` skill at session start/resume and task or phase changes. The policy requires Superpowers at development phase boundaries, Ponytail for implementation and review, Impeccable for UI planning through verification, and i-have-adhd response structure with Caveman's concise style when available. Superpowers owns process; Ponytail constrains implementation to the smallest complete, verified change; Impeccable owns UI decisions; Frontend Design supplements uncovered needs without starting another workflow. User authorization, research gates, and proportionate checks retain precedence.
The installed i-have-adhd 0.4.1 skill declares `disable-model-invocation: true`. The shared policy therefore supplies automatic response guidance without forcing a manual-only skill invocation or relying on its optional hook. It does not diagnose the user. Style stop commands persist through task changes and resume; `normal mode` disables i-have-adhd, Caveman, and Ponytail for the session. Explicit mode levels override defaults. Missing plugins are reported once and do not trigger installation. These are instruction-level requirements, not runtime interception or a guarantee of model compliance. Existing installations receive them through the normal configuration update; builds do not modify local plugin caches or hook trust.
### Context7 routing
Selecting Context7 with MCPorter registers the server in MCPorter. Selecting Context7 without MCPorter installs its native client plugin; that explicit selection permits the native MCP route under the global instructions.
Both clients receive a mandatory Context7-first research gate through the shared general rules and `technical-research` skill. It triggers whenever answers, planning, coding, reviews, debugging, tests, configuration, or upgrades rely on external technology behavior, without requiring the user to mention Context7. The agent checks project versions, resolves the library, queries the relevant behavior, and verifies the original source. Matching evidence from the same task can be reused across phases and handoffs; repository-only facts need no external call. Missing coverage, version mismatches, and service failures require an explicit fallback to authoritative sources. [Phase checkpoints](skills.md) apply inside plugin workflows as well as the focused shared skills. These are agent instructions, not a runtime enforcement hook.
### Optional Context7 health check
For a configured MCPorter route, run these commands in PowerShell or Bash. They make live requests using a public Angular question; they do not install or change configuration. Routine builds and doctor runs do not run this check.
```text
mcporter list context7 --schema
mcporter call context7.resolve-library-id 'libraryName=Angular' 'query=How do Angular computed signals track dependencies?' --timeout 30000
```
Check that discovery lists `resolve-library-id` and `query-docs` and resolution returns the official Angular documentation. Then query the returned ID (the example assumes `/websites/angular_dev` was returned):
```text
mcporter call context7.query-docs 'libraryId=/websites/angular_dev' 'query=How do Angular computed signals track dependencies?' --timeout 30000
```
The check passes only if documentation retrieval returns relevant content with an original Angular source link. Empty results, error text, or merely a zero process exit code are insufficient. This checks connectivity and one library, not general coverage or agent compliance. For a native plugin installation, run the equivalent discovery and tool calls inside that client; do not add an MCPorter registration just for this check.
### Installation and updates
`install.ps1`/`install.sh` starts every plugin checked by default; unchecking one skips it. It refuses to run against an already-installed destination (`Already installed, use the update script.`).
`update.ps1`/`update.sh` lets you revise the saved extension selection. Checking an extension installs it if missing or updates it if owned; existing foreign installations are preserved. Deselection removes only extensions that this setup installed and recorded as owned. Pre-existing or manually installed plugins and skills are retained. Extensions removed from the manifest are also removed when recorded as owned; unrelated installations are retained. Update refuses to run if any selected destination lacks an installation manifest (`Not installed, use the install script.`).
@@ -16,11 +45,17 @@ Codex receives Caveman through its native plugin adapter. Humanizer, Impeccable,
On Windows, installation and update repair the known Caveman 1.0.0 hook launcher in setup-owned Codex installations so it resolves the plugin path under PowerShell as well as Command Prompt. Custom commands and foreign installations are preserved. Review the changed hook definitions in Codex `/hooks`; setup does not grant hook trust automatically.
[QMD](https://github.com/tobi/qmd) requires Node.js 22 or newer and `npm`. Selecting it installs `@tobilu/qmd` globally. Claude receives QMD's official `qmd@qmd` plugin; Codex can use the `qmd` CLI. Deselecting QMD removes setup-owned integrations and uninstalls the global package when the Codex-owned QMD record is removed.
Normal installation and update ask whether to enable Flashbang, apply the custom status line, and enable the optional Qwen3-Embedding-0.6B semantic retrieval layer. Retrieval uses a per-user virtual environment and index and is invoked through the installed `server.py search` CLI, disabled by default. While it is enabled, both clients receive the `semantic-search` skill, which the general rules point to for concept questions when its cost check favors it; disabling retrieval removes the skill. Codex may write only the retrieval `data/` directory from its sandbox (`sandbox_workspace_write.writable_roots`); the sandbox forbids the daemon's pipes, so each Codex search loads the models in-process and takes about half a minute. Quickupdate reuses the saved shell, client, status line, Flashbang, retrieval, and extension choices without opening menus. Disabling the status line removes its custom setting and setup-owned Claude script; disabling retrieval removes its setup-owned runtime and index. Only successful runs save selections; dry runs do not update extensions or retrieval. Install and update do not modify the Codex or Claude CLI installations.
### Setup options outside the extension selector
The `server.py search`/`rebuild` CLI transparently reuses a background daemon (spawned automatically on first use per `--data-dir` and code version, so an updated `server.py` never reuses an older daemon) that keeps the embedding and reranking models loaded in memory, so only the first query after a cold start or an idle period pays the model-loading cost; every call falls back to loading the models directly in-process if the daemon cannot be reached or started. The daemon exits on its own after 30 minutes of inactivity; `server.py stop` ends it at once. `server.py status` reports indexable, indexed, and stale files and whether the models are loaded, without loading models or starting the daemon, so agents can judge what the next search costs. Search requires a Git repository and indexes tracked files plus untracked files that Git does not ignore. Indexing covers common source, markup, style, SQL, XAML, project, and configuration file types, uses bfloat16 on GPUs that support it, and skips files larger than 512 KB and, per repository, stops after 2000 indexable files by default; set `SEMANTIC_RETRIEVAL_MAX_FILES` to raise that cap for larger monorepos. When the cap is reached, the CLI prints an explicit warning to stderr rather than silently omitting files.
Flashbang, the custom status line, and local QMD search have separate prompts during install and update.
Quickupdate reuses their saved choices together with the shell, client, and extension selection. Only successful
runs save selections; dry runs do not update extensions or local search. Disabling the status line removes its
custom setting and setup-owned Claude script.
QMD is managed by the local-search option, not as a plugin. See [Local QMD search](configuration.md#local-qmd-search)
for models, Auto thresholds, runtime behavior, sandbox paths, and ownership-aware removal.
Install and update do not modify the Codex or Claude CLI installations.
Plugin installation requires the corresponding client CLI, network access, and any authentication required by the plugin. Context7 may open an OAuth flow. Codex or plugin hooks may require a trust review in `/hooks`. The scripts never install plugins during a build or doctor run and never install them when `--dry-run` or `-DryRun` is used.
+68 -9
View File
@@ -23,7 +23,7 @@ Run scripts from the repository root. PowerShell and Bash commands have equivale
| `commands/doctor.ps1`, `commands/doctor.sh` | Check generated output and installed configuration. |
| `commands/prompt-budget.ps1`, `commands/prompt-budget.sh` | Track permanent context, skill metadata, the rule catalog, and agent metadata; warn above 10,000 permanent-context tokens, fail when any token estimate exceeds its baseline by 10%, or permanent context exceeds 15,000 tokens. |
| `commands/context-report.ps1`, `commands/context-report.sh` | Report the context installed Claude and Codex actually load, from Claude `/context`, `/skill-doctor`, and its init event, and Codex `debug prompt-input`: categories, active plugins, listed skills, installed skills missing from the listing, and skills listed without description context. `--check` fails on missing or description-less skills. No model request is made. |
| `commands/benchmark.ps1`, `commands/benchmark.sh` | Run the Python, C#/ASP.NET, and TypeScript tasks under `scripts/benchmarks/tasks/` with and without the installed setup. Both arms of a client use one pinned model and reasoning effort; a preflight records what each arm actually loads in `provenance.json` and stops when the arms are not comparable; `--arms setup setup-no-search --tasks 11-* 12-* 13-* --warm-index` compares Claude with and without semantic search on concept-search tasks in a large generated repository, and compare pass rate, rework turns, duration, tokens, tool calls, cost, and workflow checks (no planning ceremony for a small change, targeted verification for a bugfix, bounded delegation for a larger change). Starts real model sessions; use `--dry-run` to list runs. The Codex baseline needs one `codex login` with `CODEX_HOME` set to `--codex-baseline-home`. |
| `commands/benchmark.ps1`, `commands/benchmark.sh` | Compare real Claude/Codex sessions with and without the setup. See [Session benchmarks](#session-benchmarks) for isolation, costs, and the search comparison. |
| `commands/repository-context.ps1`, `commands/repository-context.sh` | Refresh the local repository-context cache. |
| `commands/session-state.ps1`, `commands/session-state.sh` | Read or update the local agent session state. |
| `commands/telemetry.ps1`, `commands/telemetry.sh` | Append a local telemetry event. |
@@ -32,20 +32,31 @@ Run scripts from the repository root. PowerShell and Bash commands have equivale
| `commands/add-credentials.ps1`, `commands/add-credentials.sh` | Store a client-scoped tool credential in the operating-system credential store. |
| `commands/remove-credentials.ps1`, `commands/remove-credentials.sh` | Delete a selected client-scoped credential from the operating-system credential store. |
Prompt KPIs estimate tokens as UTF-8 bytes divided by four, using the larger shell package per client. Skill metadata counts `SKILL.md` frontmatter outside rule skills; the rule catalog counts all generated rule entry files but excludes references, so it measures the available catalog size rather than rules loaded in a typical session; agent metadata counts frontmatter for Claude and `name`/`description` for Codex. All metrics fail CI above baseline +10%; permanent context also warns above 10,000 tokens and fails above 15,000 tokens. The baseline covers the retained skill catalog. Project-specific instructions, external plugin prompt bodies, and live MCP/tool schemas are not measured.
Prompt KPIs estimate tokens as UTF-8 bytes divided by four, using the larger shell package per client.
Skill metadata counts every `SKILL.md` frontmatter, including Claude's generated rule skills. Permanent context
combines global instructions, skill and agent metadata, and the security baseline loaded for development tasks.
The rule catalog counts Codex rule files outside `references/` and Claude's generated `rules-*/SKILL.md` entries;
it measures available guidance, not all rules loaded in a typical session. Agent metadata counts Claude frontmatter
and Codex `name`/`description` fields. All metrics fail above baseline +10%; permanent context also warns above
10,000 tokens and fails above 15,000. Project instructions, external plugin bodies, and live MCP/tool schemas
are excluded. `context-report` complements this estimate with installed-client discovery.
Custom instructions are read before generated rules, skills, and plugins. They remain under `~/.my-ai-configuration/instructions/`, so installation and update scripts never manage or reset them.
Custom instructions remain under `~/.my-ai-configuration/instructions/`; installation and update scripts never reset their contents. Only clients with a nonempty instruction file receive a startup rule in their installed `AGENTS.md` or `CLAUDE.md`. That rule requires immediate loading before any response or task work and keeps the instructions active throughout the session, including after compaction. Adding or removing instructions refreshes the rule in an existing installation; empty or missing instruction files leave no rule. Generated packages contain no personal instruction rule. Changes apply when the client next loads its global instructions.
To undo an added instruction, pass the same text to `remove-instruction` with `-Client`/`--client` and `-Instruction`/`--instruction`. It removes the last exact matching block per selected client and preserves other text. A missing match leaves the file unchanged.
Credential commands currently require PowerShell and Windows Credential Manager, including their Bash wrappers. Use `~/.my-ai-configuration/bin/with-credential.ps1` or `.sh` to run a tool with a stored key available only in that child process. The key must be a valid environment-variable name; its value is never written to configuration.
Use `remove-credentials` with `-Client`/`--client` and `-Key`/`--key` to delete only that credential. The shared `with-credential` helpers remain installed for other keys.
The A/B benchmark measures whole sessions. Qwen Auto selection uses `shared/retrieval/benchmark.py`; run it through `scripts/lib/semantic-retrieval.py benchmark --root . --home <home-directory>`. This warm-model check uses synthetic, repeated short documents. Its result does not measure model startup, first indexing, real code chunks, CLI latency, or memory use; do not use it as an end-to-end performance estimate.
The A/B benchmark measures whole sessions. Auto selection of local QMD search uses `shared/qmd/qmd-benchmark.mjs`, run through `scripts/lib/qmd.py`. It measures median search latency and embedding throughput on GPU, then CPU, and is not an end-to-end performance estimate.
## Checks and tests
Gitea Actions runs the repository checks from `.gitea/workflows/ci.yml`.
Doctor captures native-command diagnostics and checks exit codes. A warning on stderr does not abort the
PowerShell 5.1 script; failed version or configuration checks contribute to the final failure summary.
On Windows, an unavailable profile-folder lookup falls back to PowerShell's home path.
| Script | Purpose |
| --- | --- |
| `checks/validate-config.py` | Validate generated Codex TOML and Claude JSON. |
@@ -63,16 +74,64 @@ Gitea Actions runs the repository checks from `.gitea/workflows/ci.yml`.
| `tests/test-quick-update.py` | Validate saved Quickupdate choices. |
| `tests/test-remove-customizations.py` | Validate targeted instruction and credential removal without accessing the real credential store. |
| `tests/test-quick-update.ps1`, `tests/test-quick-update.sh` | Run the Quickupdate test through the relevant shell. |
| `tests/test-retrieval-server.py` | Validate retrieval indexing, ranking, and optional real-model execution. |
| `tests/test-semantic-retrieval.py` | Validate retrieval installation and removal. |
| `tests/test-qmd-lib.mjs`, `test-qmd-daemon.mjs`, `test-qmd-search.mjs`, `test-qmd-warm.mjs`, `test-qmd-benchmark.mjs` | Model-free QMD unit tests; require the YAML dependency. |
| `tests/test-qmd-integration.mjs` | Manual integration test using real QMD and downloaded models; excluded from CI. |
| `tests/test-qmd-setup.py` | Validate QMD installation, migration, and removal. |
| `tests/test-session-state.py` | Validate session-state helpers. |
| `tests/test-shell-packages.ps1`, `tests/test-shell-packages.sh` | Validate generated package contents and client settings. |
| `tests/test-runtime-status.ps1`, `tests/test-runtime-status.sh` | Validate build summaries and status-line output. |
| `tests/test-install-selections.ps1`, `tests/test-install-selections.sh` | Validate dry-run shell and client selections. |
| `tests/test-command-summaries.ps1`, `tests/test-command-summaries.sh` | Validate doctor, install, and update summary output. |
| `tests/test-telemetry.py` | Validate telemetry output. |
| `tests/test-personal-instructions.py` | Validate startup instruction pointers and preservation of personal instruction files. |
| `tests/test-user-customizations.py` | Validate user instruction overlays and credential helpers. |
## QMD tests
The CI `qmd` job installs only `yaml@2.9.0` into an isolated directory, sets `QMD_PACKAGE_DIR` to that directory,
and runs these five files. It does not install QMD or download models. With a complete global QMD installation,
the same tests resolve YAML from QMD automatically:
```text
node --test scripts/tests/test-qmd-lib.mjs scripts/tests/test-qmd-daemon.mjs scripts/tests/test-qmd-search.mjs scripts/tests/test-qmd-warm.mjs scripts/tests/test-qmd-benchmark.mjs
```
Without QMD installed, use the dependency setup from [the CI workflow](../.gitea/workflows/ci.yml).
`QMD_PACKAGE_DIR` must point to the directory containing the test dependency's `node_modules` directory.
A missing YAML dependency fails `test-qmd-lib.mjs` before its tests load.
Run the integration test separately, only with a complete QMD installation and resources for model downloads.
Unset `QMD_CONFIG_DIR` first so the test uses its temporary configuration rather than an inherited user path.
Clear a unit-test-only `QMD_PACKAGE_DIR` override as well:
```text
node --test scripts/tests/test-qmd-integration.mjs
```
The integration test creates a temporary home and repository, downloads models, searches through the daemon,
and stops it after successful assertions.
Its temporary data is not automatically removed. `QMD_TEST_CACHE` can select an existing QMD cache parent;
the test uses its `qmd/` subdirectory, so avoid pointing it at a cache containing unrelated data.
## Session benchmarks
The A/B benchmark runs Python, C#/ASP.NET, and TypeScript tasks under `scripts/benchmarks/tasks/`.
These are real model sessions and can incur usage charges; `--dry-run` lists planned runs.
Both arms use the same model and reasoning effort for each client. Preflight records loaded configuration in
`provenance.json` and normally stops when the arms are not comparable. Reports compare pass rate, rework turns,
time, tokens, tool calls, cost, and workflow checks per task category.
For Claude's semantic-search comparison, pass these arguments to the shell's benchmark command:
```text
--clients claude --arms setup setup-no-search --tasks 11-* 12-* 13-* --warm-index
```
`setup-no-search` is Claude-only. Warm-up starts QMD and indexes the fixture before timing setup-arm sessions.
The Codex baseline requires a separate login under `--codex-baseline-home`; set `CODEX_HOME` to that path for
`codex login`. Supply `--codex-model` if the installed configuration does not select a model.
Benchmark fixture READMEs describe the starting task state and are retained as test inputs.
## Internal helpers
| Script | Purpose |
@@ -88,6 +147,6 @@ Gitea Actions runs the repository checks from `.gitea/workflows/ci.yml`.
| `lib/remove-instruction.py` | Remove an exact client-specific instruction block. |
| `lib/selection-state.ps1`, `lib/selection-state.sh` | Persist choices used by Quickupdate. |
| `lib/selection-state.py` | Read and write Quickupdate state. |
| `lib/semantic-retrieval.ps1`, `lib/semantic-retrieval.sh` | Invoke semantic retrieval from the installation flow. |
| `lib/semantic-retrieval.py` | Install, index, benchmark, and remove the local retrieval service. |
| `shared/retrieval/benchmark.py` | Check warm Qwen embedding and reranking speed for Auto selection; not an end-to-end benchmark. |
| `lib/qmd.sh`, `lib/qmd.ps1` | Invoke local QMD search from the installation flow. |
| `lib/qmd.py` | Install, migrate, benchmark, and remove local QMD search. |
| [`shared/qmd/`](../shared/qmd/) (repository root) | Installed QMD scripts: search CLI, daemon, warm-up, benchmark, and shared library. |
+11 -4
View File
@@ -5,20 +5,23 @@ Skills are reusable workflows. Rules are persistent constraints and agents are s
The current structure is organized by purpose:
- `planning/`: impact analysis and work-item definition
- `research/`: technical and repository research, plus `semantic-search`, which is installed only while semantic retrieval is enabled
- `research/`: mandatory `research` gate for every search, with technical and repository specialists, plus `semantic-search`, which is installed only while local QMD search is enabled
- `reviews/`: security, performance, WPF, UI/UX, business, API, and EF Core reviews
- `implementation/`: SQL, unit-test, and integration-test writing
- `verification/`: facts and work-item verification
- `git/`: pull request reviews
- `content-strategy/`, `copy-editing/`, `copywriting/`: planning, editing, and writing text
- `image/`, `site-architecture/`: visuals and website structure
- `image/`: image assets, screenshots, SEO and performance, social-preview metadata, practical marketing workflows, design tools, and common mistakes
- `site-architecture/`: website structure
- `marketing-psychology/`, `offers/`, `product-marketing/`, `cro/`: audience understanding, value framing, and conversion
General planning, debugging, code review, and completion verification workflows come from Superpowers, so the catalog does not duplicate their triggers. The remaining skills cover focused domains Superpowers does not.
The shared `plugin-workflow` skill coordinates automatic plugin use: Superpowers for process, Ponytail for implementation and review, Impeccable for UI, and i-have-adhd with Caveman for responses. Global instructions require it at session start/resume and task or phase changes. It routes to installed skills, preserves explicit opt-outs, and handles missing or manual-only plugins without replacing upstream workflows.
Each skill defines a focused workflow with the investigation, decision points, and output expected for that task. Keep guidance specific to the skill; shared repository constraints belong in the rules.
The single catalog in `shared/skills/` includes development skills and focused skills for writing, images, content and site structure, positioning, psychology, and conversion. Builds include every retained skill for both clients. Updating a managed installation backs up and removes unchanged setup-owned skills deleted from the catalog; local changes are backed up and retained.
The single catalog in `shared/skills/` includes development skills and focused skills for writing, image assets, content and site structure, positioning, psychology, and conversion. The image skill has a short core and five focused references loaded only when relevant: optimization, OG metadata, marketing workflows, design tools, and common mistakes. AI-generation guides, model comparisons, prompting recipes, and fixed social-platform size catalogs are omitted. Builds include every retained skill for both clients. Updating a managed installation backs up and removes unchanged setup-owned skills deleted from the catalog; local changes are backed up and retained.
Claude discovers only `skills/<name>/SKILL.md`, so its generated package flattens the categorized catalog to one directory per skill name, together with the skill's `references/`; the build fails on duplicate names or a `name` that differs from its directory. Claude's package additionally has a `rules-*` skill for each focused rule file or directory in
`shared/rules/`. The rule-to-skill loading conditions live in `shared/global-instructions.md`, so full rule text loads
@@ -26,6 +29,10 @@ only when Claude invokes a matching skill instead of sitting permanently in cont
files directly. Both clients embed the general rules in their global instructions.
Language, framework, library, and security rules use directory indexes plus focused topic files. Their generated Claude skills expose each focused rule separately; the index skills also include the references for contextual lookup. See [Configuration](configuration.md).
Skills guide a task when needed; their availability does not require a full review, build, or test suite after every
Every search or lookup must use `research` and the applicable specialist, including direct reads, exact matches, familiar areas, and follow-up queries during implementation or verification. Loaded instructions can be reused; their checks apply to each search. Semantic retrieval runs first for unknown repository identifiers; known identifiers still use direct reads or `rg` within the research workflow. Disabling QMD does not disable the research requirement. These are agent instructions, not a runtime tool interceptor.
Context7 phase checkpoints are defined once in `research/technical-research/SKILL.md`. Focused planning, SQL and test-writing, and verification skills call them explicitly, as do the architect, implementer, and verifier roles. The global rule applies the same checkpoints inside Superpowers and other plugin workflows without modifying upstream plugin caches or duplicating general workflow skills. Planning verifies external assumptions before accepting a design; implementation checks them against actual versions before coding; verification requires both applicable source evidence and proportionate execution results. Evidence travels in the existing plan or handoff and is reused across phases when the behavior, version, and environment match. Queries contain only public, sanitized concepts, use bounded timeouts and refinement, and report source fallback or unresolved gaps explicitly. This enforces an instruction contract; it does not guarantee model compliance or add network calls to builds and tests.
Other skills guide a task when needed; their availability does not require a full review, build, or test suite after every
change. The shared Verification rules govern proportional checking, including when generic skill workflows call
for more work than the change needs.
+32 -24
View File
@@ -51,11 +51,15 @@ def kill_tree(process):
process.wait(timeout=30)
def run_jsonl(command, cwd, env, timeout):
"""Run a client, return (events, seconds, timed_out); output is read on a thread so the deadline holds."""
def run_jsonl(command, cwd, env, timeout, prompt):
"""Run a client, return (events, seconds, timed_out); output is read on a thread so the deadline holds.
The prompt goes through stdin: on Windows the clients start through .cmd shims that cut arguments at newlines.
"""
start = time.monotonic()
process = subprocess.Popen(command, cwd=cwd, env=env, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
process = subprocess.Popen(command, cwd=cwd, env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, text=True, encoding='utf-8', errors='replace')
threading.Thread(target=lambda: (process.stdin.write(prompt), process.stdin.close()), daemon=True).start()
lines = queue.Queue()
threading.Thread(target=lambda: ([lines.put(line) for line in process.stdout], lines.put(None)), daemon=True).start()
events, timed_out = [], False
@@ -80,8 +84,8 @@ def claude_flags(arm, args):
if arm == 'baseline':
flags += ['--setting-sources', '', '--strict-mcp-config']
if arm == 'setup-no-search':
flags += ['--disallowedTools', 'Skill(semantic-search)', 'Bash(*semantic-retrieval*)', 'PowerShell(*semantic-retrieval*)',
'--append-system-prompt', 'The semantic-search skill and the semantic retrieval CLI are unavailable in this session.']
flags += ['--disallowedTools', 'Skill(semantic-search)', 'Bash(*qmd-search*)', 'PowerShell(*qmd-search*)',
'--append-system-prompt', 'The semantic-search skill and the QMD search wrapper are unavailable in this session.']
return flags
@@ -90,12 +94,12 @@ def codex_env(arm, args):
def claude_turn(prompt, cwd, arm, args, session):
command = [shutil.which('claude'), '-p', prompt, '--output-format', 'stream-json', '--verbose',
command = [shutil.which('claude'), '-p', '--output-format', 'stream-json', '--verbose',
'--permission-mode', 'auto', '--allowedTools', *CLAUDE_TOOLS, *claude_flags(arm, args)]
if session:
command += ['--resume', session]
timeout = args.timeout
events, seconds, timed_out = run_jsonl(command, cwd, clean_env(), timeout)
events, seconds, timed_out = run_jsonl(command, cwd, clean_env(), timeout, prompt)
result = next((event for event in reversed(events) if event.get('type') == 'result'), {})
usage = result.get('usage', {})
calls = [block for event in events if event.get('type') == 'assistant'
@@ -118,11 +122,12 @@ def codex_turn(prompt, cwd, arm, args, session):
command = [shutil.which('codex'), 'exec']
if session:
command += ['resume', session]
command += ['--json', '--skip-git-repo-check', '--sandbox', 'workspace-write',
# `exec resume` rejects --sandbox; the config key works for both forms.
command += ['--json', '--skip-git-repo-check', '-c', 'sandbox_mode="workspace-write"',
'-m', args.codex_model, '-c', f'model_reasoning_effort="{args.codex_effort}"']
command += ['-C', str(cwd)] if not session else []
command.append(prompt)
events, seconds, timed_out = run_jsonl(command, cwd, codex_env(arm, args), args.timeout)
command.append('-')
events, seconds, timed_out = run_jsonl(command, cwd, codex_env(arm, args), args.timeout, prompt)
items = [event.get('item', {}) for event in events if event.get('type') == 'item.completed']
tools = [item for item in items if item.get('type') not in ('agent_message', 'reasoning', 'error')]
usage = [event.get('usage', {}) for event in events if event.get('type') == 'turn.completed']
@@ -136,7 +141,7 @@ def codex_turn(prompt, cwd, arm, args, session):
'final': next((str(item.get('text', '')) for item in reversed(items) if item.get('type') == 'agent_message'), ''),
'models': [args.codex_model],
'error': next((str(event.get('message', event.get('error', '')))[:300] for event in events if event.get('type') in ('error', 'turn.failed')
and not str(event.get('message', '')).startswith('Reconnecting')), '') if not usage else '',
and not str(event.get('message', '')).startswith('Reconnecting')), 'no turn.completed event') if not usage else '',
}
@@ -271,7 +276,7 @@ def preflight(args, clients, arms):
return provenance, problems
RETRIEVAL = Path.home() / '.my-ai-configuration/semantic-retrieval'
QMD_SCRIPTS = Path.home() / '.my-ai-configuration/qmd'
FILLER_TOPICS = [
('statement', 'download', 'Render a statement download for the export queue.'),
('trial', 'banner', 'Show the trial banner text in the marketing header.'),
@@ -298,11 +303,10 @@ def generate_filler(work, count):
def warm_index(work):
"""Build the semantic index before the session, as in a repository that was searched before."""
server = RETRIEVAL / 'server.py'
if server.is_file():
subprocess.run([sys.executable, str(server), 'rebuild', '--root', str(work), '--data-dir', str(RETRIEVAL / 'data'),
'--model-cache', str(RETRIEVAL / 'model-cache')], capture_output=True, timeout=1800)
"""Index the fixture and start the QMD daemon before the session, as in a repository that was searched before."""
script = QMD_SCRIPTS / 'qmd-warm.mjs'
if script.is_file():
subprocess.run(['node', str(script), '--root', str(work)], capture_output=True, timeout=1800)
def check(work, task):
@@ -372,7 +376,7 @@ def run_one(job, args):
'agents': sum(turn['agents'] for turn in turns), 'skills': [skill for turn in turns for skill in turn['skills']],
'commands': [command for turn in turns for command in turn['commands']],
'models': sorted({model for turn in turns for model in turn['models']}),
'searches': sum('semantic-retrieval' in command for turn in turns for command in turn['commands'])
'searches': sum('qmd-search' in command for turn in turns for command in turn['commands'])
+ sum(skill == 'semantic-search' for turn in turns for skill in turn['skills']),
'timed_out': any(turn['timed_out'] for turn in turns), 'workflow_kind': spec['workflow'].get('kind'),
'workflow_violations': workflow_violations(spec['workflow'], turns), 'check_output': '' if passed else output,
@@ -382,14 +386,18 @@ def run_one(job, args):
def summarize(results):
errors = [result for result in results if result.get('error')]
results = [result for result in results if not result.get('error')]
lines = [f'Excluded {len(errors)} runs that ended with a client or API error (for example a usage limit).', '','| client | arm | runs | pass | first try | rework turns | median s | median input tok | median output tok | median tool calls | cost USD | workflow ok |',
'|---|---|---|---|---|---|---|---|---|---|---|---|']
lines = [f'Excluded {len(errors)} runs that ended with a client or API error (for example a usage limit).',
'Rows are per task category; an overall average would hide a category where one arm is worse.', '',
'| category | client | arm | runs | pass | first try | rework turns | median s | median input tok | median output tok | median tool calls | cost USD | workflow ok |',
'|---|---|---|---|---|---|---|---|---|---|---|---|---|']
groups = {}
for result in results:
groups.setdefault((result['client'], result['arm']), []).append(result)
for (client, arm), runs in sorted(groups.items()):
spec = TASKS / result['task'] / 'task.json'
category = json.loads(spec.read_text(encoding='utf-8')).get('category', '?') if spec.is_file() else '?'
groups.setdefault((category, result['client'], result['arm']), []).append(result)
for (category, client, arm), runs in sorted(groups.items()):
median = lambda key: statistics.median(run[key] for run in runs)
lines.append(f'| {client} | {arm} | {len(runs)} | {sum(run["passed"] for run in runs) / len(runs):.0%} | '
lines.append(f'| {category} | {client} | {arm} | {len(runs)} | {sum(run["passed"] for run in runs) / len(runs):.0%} | '
f'{sum(run["first_try"] for run in runs) / len(runs):.0%} | {sum(run["rework_turns"] for run in runs)} | '
f'{median("seconds"):.0f} | {median("input_tokens"):.0f} | {median("output_tokens"):.0f} | {median("tool_calls"):.0f} | '
f'{sum(run["cost_usd"] for run in runs):.2f} | {sum(not run["workflow_violations"] for run in runs) / len(runs):.0%} |')
@@ -405,7 +413,7 @@ def main():
parser.add_argument('--clients', nargs='+', choices=('claude', 'codex'), default=['claude', 'codex'])
parser.add_argument('--arms', nargs='+', choices=('setup', 'baseline', 'setup-no-search'), default=['setup', 'baseline'],
help='setup-no-search is the setup with the semantic-search skill and CLI blocked (Claude only)')
parser.add_argument('--warm-index', action='store_true', help='build the semantic index before setup-arm sessions, outside the timing')
parser.add_argument('--warm-index', action='store_true', help='start the QMD daemon and index the fixture before setup-arm sessions, outside the timing')
parser.add_argument('--tasks', nargs='+', help='task directory names or patterns such as 11-* (default: all)')
parser.add_argument('--repetitions', type=int, default=3)
parser.add_argument('--first-repetition', type=int, default=1, help='number of the first repetition, to rerun selected repetitions')
+2 -1
View File
@@ -43,7 +43,8 @@ def skill_names(root):
def tokens(text):
text = text.strip().lstrip('~')
# Claude prints "< 20" for small nonzero counts; count it as its upper bound.
text = text.strip().lstrip('~<').strip()
if text in ('', '-'):
return 0
return round(float(text[:-1]) * 1000) if text.endswith('k') else round(float(text))
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "Change the default request timeout from 30 to 60 seconds.",
"workflow": {
"kind": "small change without planning ceremony",
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "Users report that the last item is missing on the final page returned by paginate(). Fix the bug.",
"workflow": {
"kind": "bugfix with targeted verification",
@@ -1,4 +1,5 @@
{
"category": "larger implementation",
"prompt": "Implement the three independent parsers in app/parsers (csv_parser, json_parser, ini_parser) according to their docstrings, and add unit tests for each. They share no code.",
"workflow": {
"kind": "larger change with justified delegation",
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "find_user() fails for user names that contain an apostrophe, such as O'Brien. Fix it.",
"workflow": {
"kind": "security-relevant bugfix"
@@ -1,4 +1,5 @@
{
"category": "larger implementation",
"prompt": "Add unittest-based unit tests for app/slug.py under tests/ that cover its edge cases.",
"workflow": {
"kind": "test writing"
@@ -1,4 +1,5 @@
{
"category": "larger implementation",
"prompt": "Remove the duplicated formatting logic in app/report.py without changing its output.",
"workflow": {
"kind": "behavior-preserving refactor"
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "save_settings() loses keys it does not know about when it updates the settings file. Fix it so unknown keys survive.",
"workflow": {
"kind": "data-preserving bugfix"
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "Rename the retries parameter of app.api.fetch to max_retries.",
"workflow": {
"kind": "API change with directly affected documentation"
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "Make read_upload() safe against path traversal. Legitimate names in nested folders must keep working.",
"workflow": {
"kind": "security hardening"
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "dedupe() is far too slow for 100,000 items. Make it fast while keeping its behavior and the order of first occurrences.",
"workflow": {
"kind": "performance fix"
@@ -1,4 +1,5 @@
{
"category": "search",
"prompt": "Customers whose free trial has ended can still download their monthly statements. Block that by raising PermissionError, and keep it working for paying customers and for customers whose trial is still running.",
"generate": {
"filler_modules": 250
@@ -1,5 +1,6 @@
{
"prompt": "Uploads that fail with a temporary network error are never retried. Make at most three attempts in total before giving up with the last error; other errors must not be retried.",
"category": "search",
"prompt": "Fix the upload code: uploads that fail with a temporary network error are never retried. They should make at most three attempts in total before giving up with the last error; other errors must not be retried.",
"generate": {
"filler_modules": 250
},
@@ -1,4 +1,5 @@
{
"category": "search",
"prompt": "Invoice totals are sometimes one cent too low, for example 1.005 shows as 1.00. Round half amounts up to the next cent.",
"generate": {
"filler_modules": 250
@@ -1,4 +1,5 @@
{
"category": "larger implementation",
"prompt": "POST /orders accepts orders with a zero or negative quantity and orders with an empty SKU. Reject those with HTTP 400 and a validation problem response; valid orders must still return 201.",
"check": [
"dotnet",
@@ -1,4 +1,5 @@
{
"category": "larger implementation",
"prompt": "Cancelling a report build does nothing: BuildAsync keeps running until every section is rendered. Make it honor cancellation promptly by throwing OperationCanceledException.",
"check": [
"dotnet",
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "Requesting an unknown customer with GET /customers/{id} returns HTTP 500. Return 404 instead, and keep known customers working.",
"check": [
"dotnet",
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "Back-to-back bookings, where one ends exactly when the next starts, are wrongly reported as overlapping. Fix overlaps() without breaking real overlaps.",
"check": [
"node",
@@ -1,4 +1,5 @@
{
"category": "larger implementation",
"prompt": "mergeConfig() drops nested settings from the base configuration when the override sets only some keys of a nested object. Merge nested objects deeply; arrays from the override replace base arrays, and neither input may be mutated.",
"check": [
"node",
@@ -1,4 +1,5 @@
{
"category": "small change",
"prompt": "debounce() calls the wrapped function with the arguments of the first call in a burst instead of the last one. Fix it so the trailing call uses the latest arguments.",
"check": [
"node",
+2
View File
@@ -19,5 +19,7 @@ foreach ($target in $(if ($Client -eq 'Both') { @('codex','claude') } else { @($
$path = Join-Path $directory "$target.md"
$prefix = if ((Test-Path -LiteralPath $path) -and (Get-Item -LiteralPath $path).Length -gt 0 -and -not (Get-Content -LiteralPath $path -Raw).EndsWith([Environment]::NewLine)) { [Environment]::NewLine } else { '' }
[System.IO.File]::AppendAllText($path, $prefix + $Instruction.TrimEnd() + [Environment]::NewLine, [System.Text.UTF8Encoding]::new($false))
& python (Join-Path $root 'scripts/lib/personal_instructions.py') --home $HomePath --client $target
if ($LASTEXITCODE -ne 0) { throw 'Could not refresh the personal instruction rule.' }
Write-Output "Instruction added for $target."
}
+4
View File
@@ -33,10 +33,14 @@ if [ -z "${instruction//[[:space:]]/}" ] || [ ${#instruction} -gt 65536 ]; then
case "$home_path" in /*|[A-Za-z]:[\\/]*) ;; *) printf 'Home path must be absolute.\n' >&2; exit 1 ;; esac
directory="$home_path/.my-ai-configuration/instructions"
script_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
python_command=$(command -v python || command -v python3 || true)
[ -n "$python_command" ] || { printf 'Python is required to add instructions.\n' >&2; exit 1; }
mkdir -p -- "$directory"
for target in "${targets[@]}"; do
path="$directory/$target.md"
[ ! -s "$path" ] || [ "$(tail -c 1 -- "$path" | wc -l)" -eq 1 ] || printf '\n' >> "$path"
printf '%s\n' "$instruction" >> "$path"
"$python_command" "$script_dir/../lib/personal_instructions.py" --home "$home_path" --client "$target"
printf 'Instruction added for %s.\n' "$target"
done
+2 -1
View File
@@ -209,7 +209,8 @@ foreach ($shell in @('powershell','bash')) {
$allowRulesJson = ConvertTo-Json -InputObject @('Agent', "$shellTool(mcporter call context7.*)") -Compress
$compactScript = if ($shell -eq 'powershell') { 'Record-Compact.ps1' } else { 'record-compact.sh' }
$pointerScript = if ($shell -eq 'powershell') { 'Show-SessionStatePointer.ps1' } else { 'show-session-state-pointer.sh' }
$content = $content.Replace('__HOOK_COMMAND__', $command).Replace('__POWERSHELL_HOOK_COMMAND__', $command).Replace('__HOOK_SCRIPT__', $script).Replace('__POWERSHELL_HOOK_SCRIPT__', $script).Replace('__COMPACT_SCRIPT__', $compactScript).Replace('__SESSION_POINTER_SCRIPT__', $pointerScript).Replace('__STATUSLINE_COMMAND__', $command).Replace('__STATUSLINE_SCRIPT__', $statusLineScript).Replace('__CLAUDE_SANDBOX__', $claudeSandbox).Replace('__CLAUDE_DENY_RULES__', $denyRulesJson).Replace('__CLAUDE_ALLOW_RULES__', $allowRulesJson)
$qmdWarmScript = if ($shell -eq 'powershell') { 'Start-QmdWarm.ps1' } else { 'qmd-warm.sh' }
$content = $content.Replace('__HOOK_COMMAND__', $command).Replace('__POWERSHELL_HOOK_COMMAND__', $command).Replace('__HOOK_SCRIPT__', $script).Replace('__POWERSHELL_HOOK_SCRIPT__', $script).Replace('__COMPACT_SCRIPT__', $compactScript).Replace('__SESSION_POINTER_SCRIPT__', $pointerScript).Replace('__QMD_WARM_SCRIPT__', $qmdWarmScript).Replace('__STATUSLINE_COMMAND__', $command).Replace('__STATUSLINE_SCRIPT__', $statusLineScript).Replace('__CLAUDE_SANDBOX__', $claudeSandbox).Replace('__CLAUDE_DENY_RULES__', $denyRulesJson).Replace('__CLAUDE_ALLOW_RULES__', $allowRulesJson)
Set-Content -LiteralPath $path -Value $content -Encoding UTF8
}
+3
View File
@@ -195,12 +195,14 @@ for client in codex claude; do
statusline_script=statusline.sh
compact_script=record-compact.sh
pointer_script=show-session-state-pointer.sh
qmd_warm_script=qmd-warm.sh
if [ "$shell" = powershell ]; then
command='__POWERSHELL_COMMAND__'
script=flashbang.ps1
statusline_script=statusline.ps1
compact_script=Record-Compact.ps1
pointer_script=Show-SessionStatePointer.ps1
qmd_warm_script=Start-QmdWarm.ps1
fi
path="$output/$client-$shell/$file"
content=$(<"$path")
@@ -217,6 +219,7 @@ for client in codex claude; do
content=${content//__POWERSHELL_HOOK_SCRIPT__/$script}
content=${content//__COMPACT_SCRIPT__/$compact_script}
content=${content//__SESSION_POINTER_SCRIPT__/$pointer_script}
content=${content//__QMD_WARM_SCRIPT__/$qmd_warm_script}
content=${content//__STATUSLINE_COMMAND__/$command}
content=${content//__STATUSLINE_SCRIPT__/$statusline_script}
content=${content//__CLAUDE_SANDBOX__/$sandbox}
+26 -5
View File
@@ -4,6 +4,7 @@ $ErrorActionPreference = 'Stop'
$root = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
. (Join-Path $root 'scripts/lib/manifest.ps1')
$homePath = [Environment]::GetFolderPath('UserProfile')
if ([string]::IsNullOrWhiteSpace($homePath)) { $homePath = $HOME }
$script:fail = $false
$script:checkCount = 0
function Result($state, $message) {
@@ -14,7 +15,27 @@ function Result($state, $message) {
}
}
foreach ($tool in @('codex','claude')) { if (Get-Command $tool -ErrorAction SilentlyContinue) { $version = & $tool --version 2>&1 | Select-Object -First 1; Result 'PASS' "$tool available ($version)" } else { Result 'WARN' "$tool unavailable" } }
function Invoke-DoctorCommand {
param([string]$Command, [string[]]$Arguments)
# PowerShell 5.1 turns native stderr into error records, even when the process succeeds.
$ErrorActionPreference = 'Continue'
$PSNativeCommandUseErrorActionPreference = $false
try {
$executable = Get-Command $Command -ErrorAction Stop
$output = @(& $executable @Arguments 2>&1)
return [pscustomobject]@{ Output = $output; ExitCode = $LASTEXITCODE }
} catch {
return [pscustomobject]@{ Output = @($_); ExitCode = 1 }
}
}
foreach ($tool in @('codex','claude')) {
if (Get-Command $tool -ErrorAction SilentlyContinue) {
$probe = Invoke-DoctorCommand -Command $tool -Arguments @('--version')
if ($probe.ExitCode -eq 0) { Result 'PASS' "$tool available ($($probe.Output -join ' '))" }
else { Result 'FAIL' "$tool --version failed (exit $($probe.ExitCode)): $($probe.Output -join ' ')" }
} else { Result 'WARN' "$tool unavailable" }
}
if (Get-Command jq -ErrorAction SilentlyContinue) { Result 'PASS' 'jq available for Claude Bash status line' } else { Result 'WARN' 'jq unavailable; Claude Bash status line is disabled' }
$sourceAgentCount = @(Get-ChildItem (Join-Path $root 'shared/agents') -Directory -ErrorAction SilentlyContinue).Count
@@ -50,14 +71,14 @@ if (Test-Path -LiteralPath $settingsPath) {
}
$configTomlPath = Join-Path $homePath '.codex/config.toml'
if (Test-Path -LiteralPath $configTomlPath) {
$parseResult = & python -c "import sys,tomllib,pathlib; tomllib.loads(pathlib.Path(sys.argv[1]).read_text(encoding='utf-8-sig'))" $configTomlPath 2>&1
if ($LASTEXITCODE -eq 0) { Result 'PASS' 'installed .codex/config.toml parses as TOML' } else { Result 'FAIL' "installed .codex/config.toml is invalid TOML: $parseResult" }
$parseResult = Invoke-DoctorCommand -Command 'python' -Arguments @('-c', "import sys,tomllib,pathlib; tomllib.loads(pathlib.Path(sys.argv[1]).read_text(encoding='utf-8-sig'))", $configTomlPath)
if ($parseResult.ExitCode -eq 0) { Result 'PASS' 'installed .codex/config.toml parses as TOML' } else { Result 'FAIL' "installed .codex/config.toml is invalid TOML: $($parseResult.Output -join ' ')" }
if (Get-Command codex -ErrorAction SilentlyContinue) {
$previousCodexHome = $env:CODEX_HOME
try {
$env:CODEX_HOME = Split-Path $configTomlPath -Parent
$schemaResult = & codex --strict-config --help 2>&1
if ($LASTEXITCODE -eq 0) { Result 'PASS' 'installed .codex/config.toml matches the installed Codex schema' } else { Result 'FAIL' "installed .codex/config.toml has unsupported Codex settings: $schemaResult" }
$schemaResult = Invoke-DoctorCommand -Command 'codex' -Arguments @('--strict-config', '--help')
if ($schemaResult.ExitCode -eq 0) { Result 'PASS' 'installed .codex/config.toml matches the installed Codex schema' } else { Result 'FAIL' "installed .codex/config.toml has unsupported Codex settings: $($schemaResult.Output -join ' ')" }
} finally { $env:CODEX_HOME = $previousCodexHome }
} else { Result 'WARN' 'Codex CLI unavailable; skipped installed Codex schema validation' }
}
+7 -1
View File
@@ -24,7 +24,13 @@ result() {
}
for tool in codex claude; do
if command -v "$tool" >/dev/null; then result PASS "$tool available ($($tool --version 2>&1 | head -n 1))"; else result WARN "$tool unavailable"; fi
if command -v "$tool" >/dev/null; then
if version=$("$tool" --version 2>&1); then result PASS "$tool available ($version)"
else
status=$?
result FAIL "$tool --version failed (exit $status): $version"
fi
else result WARN "$tool unavailable"; fi
done
command -v jq >/dev/null && result PASS 'jq available for Claude Bash status line' || result WARN 'jq unavailable; Claude Bash status line is disabled'
+2 -2
View File
@@ -12,7 +12,7 @@ $root = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
. (Join-Path $root 'scripts/lib/install-targets.ps1')
. (Join-Path $root 'scripts/lib/selection-state.ps1')
. (Join-Path $root 'scripts/lib/install-options.ps1')
. (Join-Path $root 'scripts/lib/semantic-retrieval.ps1')
. (Join-Path $root 'scripts/lib/qmd.ps1')
$Shell = Read-InstallShell -Shell $Shell
$shell = $Shell.ToLowerInvariant()
@@ -44,8 +44,8 @@ foreach ($item in (Get-InstallTargets -Generated $generated -HomePath $homePath
-AiConfigRoot $item.Destination.Replace('\','/') -ShellCommand $shellCommand -PowerShellCommand $powerShellCommand -ClaudeConcurrency $claudeConcurrency -FlashbangEnabled $options.flashbang -StatusLineEnabled $options.statusline -SemanticRetrievalEnabled $options.semantic_retrieval -DryRun:$DryRun -Summary:$Summary
}
Sync-Qmd -RepositoryRoot $root -HomePath $homePath -Client $Client -Enabled $options.semantic_retrieval -DryRun:$DryRun -Summary:$Summary
Sync-ConfiguredPlugins -RepositoryRoot $root -HomePath $homePath -Client $Client -DryRun:$DryRun -Summary:$Summary -Entries $plugins.Selected
Sync-SemanticRetrieval -RepositoryRoot $root -HomePath $homePath -Client $Client -Enabled $options.semantic_retrieval -DryRun:$DryRun -Summary:$Summary
if (-not $DryRun) { Save-UpdateSelection -HomePath $homePath -RepositoryRoot $root -Shell $Shell -Client $Client -Plugins $plugins -Flashbang $options.flashbang -StatusLine $options.statusline -SemanticRetrieval $options.semantic_retrieval }
if ($Summary) { Write-Output "Install: PASS | $Client, $Shell$(if ($DryRun) { ', dry-run' })" } else { Write-Output ($(if ($DryRun) { 'PASS install dry-run' } else { 'PASS install' })) }
exit 0
+2 -2
View File
@@ -21,7 +21,7 @@ root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)
. "$root/scripts/lib/install-targets.sh"
. "$root/scripts/lib/selection-state.sh"
. "$root/scripts/lib/install-options.sh"
. "$root/scripts/lib/semantic-retrieval.sh"
. "$root/scripts/lib/qmd.sh"
shell=$(read_install_shell "$shell")
client=$(read_install_client "$client")
@@ -61,8 +61,8 @@ while IFS='|' read -r source destination; do
sync_managed_destination "$source" "$destination" "$stamp" "$ai_config_root" "$shell_command" "$powershell_command" "$dry_run" "$summary" "$flashbang" "$claude_concurrency" "$statusline" "$semantic_retrieval"
done < <(get_install_targets "$generated" "$home_path" "$shell" "$client")
sync_qmd "$root" "$client" "$home_path" "$semantic_retrieval" "$dry_run" false "$summary"
sync_configured_plugins "$root" "$client" "$home_path" "$dry_run" false selected_plugins "$summary"
sync_semantic_retrieval "$root" "$client" "$home_path" "$semantic_retrieval" "$dry_run" false "$summary"
if [ "$dry_run" = false ]; then save_update_selection; fi
if [ "$summary" = true ]; then
+3 -3
View File
@@ -10,7 +10,7 @@ $root = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
. (Join-Path $root 'scripts/lib/plugins.ps1')
. (Join-Path $root 'scripts/lib/manifest.ps1')
. (Join-Path $root 'scripts/lib/install-targets.ps1')
. (Join-Path $root 'scripts/lib/semantic-retrieval.ps1')
. (Join-Path $root 'scripts/lib/qmd.ps1')
$Client = Read-InstallClient -Client $Client
$homePath = [Environment]::GetFolderPath('UserProfile')
@@ -24,7 +24,7 @@ if (-not $DryRun -and -not $Force) {
if ([Console]::IsInputRedirected -or $env:CI -eq 'true' -or $env:AI_CONFIG_NO_INTERACTIVE -eq '1') {
throw 'Refusing to uninstall without confirmation in a non-interactive session. Pass -Force to proceed.'
}
Write-Host "This removes the managed configuration, extensions, and semantic retrieval setup for $Client under $homePath."
Write-Host "This removes the managed configuration, extensions, and QMD local search setup for $Client under $homePath."
Write-Host 'Backups already on disk are kept; anything this setup never installed is left untouched.'
$confirm = Read-Host -Prompt 'Continue? [y/N]'
if ($null -eq $confirm) { $confirm = '' }
@@ -57,7 +57,7 @@ try {
}
Sync-ConfiguredPlugins -RepositoryRoot $root -HomePath $homePath -Client $Client -DryRun:$DryRun -Summary:$Summary -Entries @()
Sync-SemanticRetrieval -RepositoryRoot $root -HomePath $homePath -Client $Client -Enabled $false -DryRun:$DryRun -Summary:$Summary
Sync-Qmd -RepositoryRoot $root -HomePath $homePath -Client $Client -Enabled $false -DryRun:$DryRun -Summary:$Summary
if (-not $DryRun) {
$remaining = @($targets | Where-Object { Test-Path -LiteralPath (Get-ManagedManifestPath $_.Destination) })
+3 -3
View File
@@ -19,7 +19,7 @@ root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)
. "$root/scripts/lib/plugins.sh"
. "$root/scripts/lib/manifest.sh"
. "$root/scripts/lib/install-targets.sh"
. "$root/scripts/lib/semantic-retrieval.sh"
. "$root/scripts/lib/qmd.sh"
client=$(read_install_client "$client")
home_path=${HOME:?HOME is required}
@@ -40,7 +40,7 @@ if [ "$dry_run" = false ] && [ "$force" = false ]; then
printf 'Refusing to uninstall without confirmation in a non-interactive session. Pass --force to proceed.\n' >&2
exit 1
fi
printf 'This removes the managed configuration, extensions, and semantic retrieval setup for %s under %s.\n' "$client" "$home_path"
printf 'This removes the managed configuration, extensions, and QMD local search setup for %s under %s.\n' "$client" "$home_path"
printf 'Backups already on disk are kept; anything this setup never installed is left untouched.\n'
read -r -p "Continue? [y/N]: " confirm || confirm=""
case "$confirm" in
@@ -75,7 +75,7 @@ done
# shellcheck disable=SC2034
selected_plugins=()
sync_configured_plugins "$root" "$client" "$home_path" "$dry_run" false selected_plugins "$summary"
sync_semantic_retrieval "$root" "$client" "$home_path" false "$dry_run" false "$summary"
sync_qmd "$root" "$client" "$home_path" false "$dry_run" false "$summary"
if [ "$dry_run" = false ]; then
remaining=0
+2 -2
View File
@@ -13,7 +13,7 @@ $root = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
. (Join-Path $root 'scripts/lib/install-targets.ps1')
. (Join-Path $root 'scripts/lib/selection-state.ps1')
. (Join-Path $root 'scripts/lib/install-options.ps1')
. (Join-Path $root 'scripts/lib/semantic-retrieval.ps1')
. (Join-Path $root 'scripts/lib/qmd.ps1')
$homePath = [Environment]::GetFolderPath('UserProfile')
if ($Quick) {
@@ -54,6 +54,7 @@ foreach ($item in (Get-InstallTargets -Generated $generated -HomePath $homePath
-AiConfigRoot $item.Destination.Replace('\','/') -ShellCommand $shellCommand -PowerShellCommand $powerShellCommand -FlashbangEnabled $options.flashbang -StatusLineEnabled $options.statusline -SemanticRetrievalEnabled $options.semantic_retrieval -DryRun:$DryRun -Summary:$Summary
}
Sync-Qmd -RepositoryRoot $root -HomePath $homePath -Client $Client -Enabled $options.semantic_retrieval -DryRun:$DryRun -Update -Summary:$Summary
$previousPluginNonInteractive = $script:PluginNonInteractive
$previousCI = $env:CI
try {
@@ -64,7 +65,6 @@ try {
$script:PluginNonInteractive = $previousPluginNonInteractive
$env:CI = $previousCI
}
Sync-SemanticRetrieval -RepositoryRoot $root -HomePath $homePath -Client $Client -Enabled $options.semantic_retrieval -DryRun:$DryRun -Update -Summary:$Summary
if (-not $DryRun) { Save-UpdateSelection -HomePath $homePath -RepositoryRoot $root -Shell $Shell -Client $Client -Plugins $plugins -Flashbang $options.flashbang -StatusLine $options.statusline -SemanticRetrieval $options.semantic_retrieval }
if ($Summary) { Write-Output "Update: PASS | $Client, $Shell$(if ($DryRun) { ', dry-run' })" } else { Write-Output ($(if ($DryRun) { 'PASS update dry-run' } else { 'PASS update' })) }
exit 0
+2 -2
View File
@@ -23,7 +23,7 @@ root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)
. "$root/scripts/lib/install-targets.sh"
. "$root/scripts/lib/selection-state.sh"
. "$root/scripts/lib/install-options.sh"
. "$root/scripts/lib/semantic-retrieval.sh"
. "$root/scripts/lib/qmd.sh"
home_path=${HOME:?HOME is required}
selected_plugins=()
@@ -66,12 +66,12 @@ while IFS='|' read -r source destination; do
sync_managed_destination "$source" "$destination" "$stamp" "$ai_config_root" "$shell_command" "$powershell_command" "$dry_run" "$summary" "$flashbang" 5 "$statusline" "$semantic_retrieval"
done < <(get_install_targets "$generated" "$home_path" "$shell" "$client")
sync_qmd "$root" "$client" "$home_path" "$semantic_retrieval" "$dry_run" true "$summary"
if [ "$quick" = true ]; then
CI=true sync_configured_plugins "$root" "$client" "$home_path" "$dry_run" true selected_plugins "$summary" </dev/null
else
sync_configured_plugins "$root" "$client" "$home_path" "$dry_run" true selected_plugins "$summary"
fi
sync_semantic_retrieval "$root" "$client" "$home_path" "$semantic_retrieval" "$dry_run" true "$summary"
if [ "$dry_run" = false ]; then save_update_selection; fi
if [ "$summary" = true ]; then
+2 -2
View File
@@ -32,14 +32,14 @@ function Read-SemanticRetrievalOption {
param([bool]$Default,[string]$RepositoryRoot,[string]$HomePath)
$hint = if ($Default) { 'Y/n/a' } else { 'y/N/a' }
while ($true) {
Write-Host "Enable Qwen3 embedding and reranking semantic retrieval? [$hint] " -NoNewline
Write-Host "Enable local QMD search models (Qwen3 embedding, Qwen3 reranker, QMD query expansion)? [$hint] " -NoNewline
$answer = [Console]::ReadLine()
if ($null -eq $answer) { throw 'Input ended before semantic retrieval was confirmed.' }
switch ($answer.Trim().ToLowerInvariant()) {
'' { return $Default }
{ $_ -in @('y', 'yes') } { return $true }
{ $_ -in @('n', 'no') } { return $false }
{ $_ -in @('a', 'auto') } { return Test-SemanticRetrievalDevice -RepositoryRoot $RepositoryRoot -HomePath $HomePath }
{ $_ -in @('a', 'auto') } { return Test-QmdDevice -RepositoryRoot $RepositoryRoot -HomePath $HomePath }
}
}
}
+64 -7
View File
@@ -5,13 +5,35 @@ import re
import sys
import tomllib
from personal_instructions import CLIENT_FILES, render
def is_flashbang(hook):
return any(re.search(r'(?:[/\\\s"\x27]|^)flashbang\.(?:ps1|sh)(?:[\s"\x27]|$)', str(hook.get(key, '')), re.I)
for key in ('command', 'command_windows'))
def filter_options(path, flashbang_enabled=True, statusline_enabled=True):
def is_qmd_warm(hook):
return any(re.search(r'(?:[/\\\s"\x27]|^)(?:qmd-warm\.sh|Start-QmdWarm\.ps1)(?:[\s"\x27]|$)', str(hook.get(key, '')), re.I)
for key in ('command', 'command_windows'))
def without_hooks(groups, predicate):
remaining = []
for group in groups:
hooks = [hook for hook in group.get('hooks', []) if not predicate(hook)]
if hooks:
remaining.append(dict(group, hooks=hooks))
return remaining
def filter_instructions(text, semantic_retrieval_enabled):
if semantic_retrieval_enabled:
return text
return ''.join(line for line in text.splitlines(keepends=True) if '`semantic-search`' not in line)
def filter_options(path, flashbang_enabled=True, statusline_enabled=True, semantic_retrieval_enabled=True):
content = path.read_text(encoding='utf-8-sig')
if path.name == 'settings.json':
settings = json.loads(content)
@@ -26,6 +48,12 @@ def filter_options(path, flashbang_enabled=True, statusline_enabled=True):
settings['hooks']['Stop'] = remaining
elif 'Stop' in settings.get('hooks', {}):
del settings['hooks']['Stop']
if not semantic_retrieval_enabled:
remaining = without_hooks(settings.get('hooks', {}).get('SessionStart', []), is_qmd_warm)
if remaining:
settings['hooks']['SessionStart'] = remaining
else:
settings.get('hooks', {}).pop('SessionStart', None)
if not statusline_enabled:
settings.pop('statusLine', None)
return json.dumps(settings, indent=2, ensure_ascii=False) + '\n'
@@ -44,6 +72,17 @@ def filter_options(path, flashbang_enabled=True, statusline_enabled=True):
filtered.append(section)
result = ''.join(filtered)
result = re.sub(r'(?m)^\[\[hooks\.Stop\]\][ \t]*\r?\n\s*(?=\[|\Z)(?!\[\[hooks\.Stop\.hooks\]\])', '', result)
if not semantic_retrieval_enabled:
sections = re.split(r'(?m)(?=^\s*\[\[?[^\r\n]+\]\]?\s*$)', result)
kept = []
for section in sections:
if re.match(r'\s*\[\[hooks\.SessionStart\.hooks\]\]', section):
hook = tomllib.loads(section)['hooks']['SessionStart']['hooks'][0]
if is_qmd_warm(hook):
continue
kept.append(section)
result = ''.join(kept)
result = re.sub(r'(?m)^\[\[hooks\.SessionStart\]\][ \t]*\r?\n\s*(?=\[|\Z)(?!\[\[hooks\.SessionStart\.hooks\]\])', '', result)
if not statusline_enabled:
sections = re.split(r'(?m)(?=^\s*\[\[?[^\r\n]+\]\]?\s*$)', result)
for index, section in enumerate(sections):
@@ -69,7 +108,7 @@ def merge_json(current, managed):
for key, value in managed.items():
if key == 'hooks' and isinstance(value, dict) and isinstance(result.get(key), dict):
hooks = dict(result[key])
for event in set(value) | {'Stop'}:
for event in set(value) | {'Stop', 'SessionStart'}:
groups = value.get(event, [])
old_groups = hooks.get(event, [])
if isinstance(groups, list) and isinstance(old_groups, list):
@@ -79,7 +118,7 @@ def merge_json(current, managed):
retained.append(group)
continue
foreign = [hook for hook in group['hooks'] if not (isinstance(hook, dict) and re.search(
r'(?:flashbang|statusline|record-compact|session-state-pointer)\.(?:ps1|sh)',
r'(?:flashbang|statusline|record-compact|session-state-pointer|qmd-warm|Start-QmdWarm)\.(?:ps1|sh)',
str(hook.get('command', '')) + str(hook.get('command_windows', '')), re.I))]
if foreign:
retained.append(dict(group, hooks=foreign))
@@ -128,10 +167,21 @@ def merge_toml(current_text, managed_text, statusline_enabled=True):
managed_headers = {header for header, _ in managed_sections}
extras = []
fields_by_header = {}
for header, block in current_sections:
owned_pattern = r'(?:flashbang|statusline|record-compact|session-state-pointer|qmd-warm|Start-QmdWarm)\.(?:ps1|sh)'
for index, (header, block) in enumerate(current_sections):
plugin_table = re.match(r'^\[(?:plugins|marketplaces)(?:\.|\])', header)
array_table = header.startswith('[[')
owned_hook = any(re.search(r'(?:flashbang|statusline|record-compact|session-state-pointer)\.(?:ps1|sh)', line, re.I) for line in block)
owned_hook = any(re.search(owned_pattern, line, re.I) for line in block)
group = re.fullmatch(r'\[\[hooks\.(\w+)\]\]', header)
if group:
# A hook group header is owned when every hook table that follows it is owned (or none follows).
child = f'[[hooks.{group.group(1)}.hooks]]'
children = []
for name, lines in current_sections[index + 1:]:
if name != child:
break
children.append(any(re.search(owned_pattern, line, re.I) for line in lines))
owned_hook = all(children)
if plugin_table and header in managed_headers:
raise ValueError('Cannot merge conflicting managed plugin tables.')
if (header not in managed_headers and not array_table) or (array_table and not owned_hook):
@@ -173,17 +223,24 @@ def merge_toml(current_text, managed_text, statusline_enabled=True):
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('action', choices=('filter', 'merge-json', 'merge-toml'))
parser.add_argument('action', choices=('filter', 'filter-instructions', 'merge-json', 'merge-toml'))
parser.add_argument('--path', type=Path)
parser.add_argument('--current', type=Path)
parser.add_argument('--flashbang', choices=('true', 'false'))
parser.add_argument('--statusline', choices=('true', 'false'), default='true')
parser.add_argument('--semantic-retrieval', choices=('true', 'false'), default='true')
parser.add_argument('--managed', type=Path)
parser.add_argument('--output', type=Path)
parser.add_argument('--destination', type=Path)
args = parser.parse_args()
try:
if args.action == 'filter':
result = filter_options(args.path, args.flashbang == 'true', args.statusline == 'true')
result = filter_options(args.path, args.flashbang == 'true', args.statusline == 'true', args.semantic_retrieval == 'true')
elif args.action == 'filter-instructions':
result = filter_instructions(args.path.read_text(encoding='utf-8-sig'), args.semantic_retrieval == 'true')
if args.destination and args.path.name in CLIENT_FILES.values():
client = next(client for client, name in CLIENT_FILES.items() if name == args.path.name)
result = render(result, args.destination.parent, client)
else:
managed = args.managed.read_text(encoding='utf-8-sig') if args.managed else sys.stdin.buffer.read().decode('utf-8-sig')
current = args.current.read_text(encoding='utf-8-sig')
+2 -2
View File
@@ -41,7 +41,7 @@ read_semantic_retrieval_option() {
local default=$1 hint=y/N/a answer
[ "$default" = false ] || hint=Y/n/a
while :; do
printf 'Enable Qwen3 embedding and reranking semantic retrieval? [%s] ' "$hint" >&2
printf 'Enable local QMD search models (Qwen3 embedding, Qwen3 reranker, QMD query expansion)? [%s] ' "$hint" >&2
IFS= read -r answer || { printf 'Input ended before semantic retrieval was confirmed.\n' >&2; return 1; }
answer=${answer%$'\r'}
case "$answer" in
@@ -49,7 +49,7 @@ read_semantic_retrieval_option() {
y|Y|yes|Yes|YES) printf 'true\n'; return ;;
n|N|no|No|NO) printf 'false\n'; return ;;
a|A|auto|Auto|AUTO)
if test_semantic_retrieval_device "$root" "$home_path"; then printf 'true\n'; return; else status=$?; fi
if test_qmd_device "$root" "$home_path"; then printf 'true\n'; return; else status=$?; fi
[ "$status" -eq 1 ] && { printf 'false\n'; return; }
return "$status"
;;
-25
View File
@@ -304,9 +304,6 @@ class Manager:
if entry['codex_method'] == 'cli':
self.ensure_cli(client, entry)
return
if entry['codex_method'] == 'qmd':
self.ensure_qmd(client, entry)
return
skill = client == 'codex' and entry['codex_method'] != 'plugin'
kind = 'skill' if skill else 'plugin'
record = next((r for r in self.state['resources'] if r['client'] == client and r['name'] == entry['name']), None)
@@ -417,28 +414,6 @@ class Manager:
atomic_write(path, (json.dumps(content, indent=2) + '\n').encode('utf-8'))
print('Repaired Caveman Windows hook commands; review the changed hooks in Codex /hooks.')
def ensure_qmd(self, client, entry):
package = entry['codex_source']
if package != '@tobilu/qmd':
raise ValueError('Invalid QMD package.')
record = next((r for r in self.state['resources'] if r['client'] == 'shared' and r['name'] == entry['name']), None)
installed = package in self.global_packages()
if record is not None:
record['clients'] = sorted(set(record.get('clients', ['claude', 'codex'])) | {client})
self.save()
if record is None and installed and not self.summary:
print(f'PASS Preserving pre-existing global package: {package}')
if (record is not None or not installed) and (not installed or (self.update and package not in self.updated_packages)):
self.command(['npm', 'update' if installed else 'install', '--global', package])
self.global_packages().add(package)
self.updated_packages.add(package)
if record is None:
self.state['resources'].append(dict(client='shared', name=entry['name'], kind='qmd', package=package, clients=[client]))
self.save()
if client == 'claude':
entry = dict(entry, codex_method='plugin')
self.ensure(client, entry)
def ensure_mcporter(self, entry, clients):
server, url = entry['mcporter_name'], entry['mcporter_url']
if not re.fullmatch(r'[a-z0-9-]+', server) or not re.fullmatch(r'https://[^\s]+', url):
+7 -3
View File
@@ -227,13 +227,17 @@ function Invoke-ManagedSync {
if (-not $SemanticRetrievalEnabled -and $relative -match '(^|/)semantic-search/') { return }
$target = Resolve-ManagedPath -Destination $Destination -Relative $relative
$rawContent = Get-Content -LiteralPath $_.FullName -Raw -Encoding UTF8
$filterOptions = ((-not $FlashbangEnabled) -or (-not $StatusLineEnabled)) -and $relative -in @('settings.json', 'config.toml')
$filterOptions = ((-not $FlashbangEnabled) -or (-not $StatusLineEnabled) -or (-not $SemanticRetrievalEnabled)) -and $relative -in @('settings.json', 'config.toml')
if ($filterOptions) {
$flashbangOption = if ($FlashbangEnabled) { 'true' } else { 'false' }
$statusLineOption = if ($StatusLineEnabled) { 'true' } else { 'false' }
$rawContent = Invoke-InstallOptions -Arguments @('filter', '--path', $_.FullName, '--flashbang', $flashbangOption, '--statusline', $statusLineOption) -ErrorMessage 'Could not configure install options.'
$rawContent = Invoke-InstallOptions -Arguments @('filter', '--path', $_.FullName, '--flashbang', $flashbangOption, '--statusline', $statusLineOption, '--semantic-retrieval', $(if ($SemanticRetrievalEnabled) { 'true' } else { 'false' })) -ErrorMessage 'Could not configure install options.'
}
$needsSubstitution = $filterOptions -or $rawContent.Contains('__AI_CONFIG_ROOT__') -or $rawContent.Contains('__HOOK_COMMAND__') -or $rawContent.Contains('__POWERSHELL_HOOK_COMMAND__') -or $rawContent.Contains('__POWERSHELL_COMMAND__') -or $rawContent.Contains('__CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY__')
$filterInstructions = $relative -in @('CLAUDE.md', 'AGENTS.md') -or ((-not $SemanticRetrievalEnabled) -and $relative -eq 'rules/general.md')
if ($filterInstructions) {
$rawContent = Invoke-InstallOptions -Arguments @('filter-instructions', '--path', $_.FullName, '--destination', $Destination, '--semantic-retrieval', $(if ($SemanticRetrievalEnabled) { 'true' } else { 'false' })) -ErrorMessage 'Could not configure install options.'
}
$needsSubstitution = $filterOptions -or $filterInstructions -or $rawContent.Contains('__AI_CONFIG_ROOT__') -or $rawContent.Contains('__HOOK_COMMAND__') -or $rawContent.Contains('__POWERSHELL_HOOK_COMMAND__') -or $rawContent.Contains('__POWERSHELL_COMMAND__') -or $rawContent.Contains('__CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY__')
if ($needsSubstitution) {
$finalContent = $rawContent.Replace('__AI_CONFIG_ROOT__', $AiConfigRoot).Replace('__HOOK_COMMAND__', $ShellCommand).Replace('__POWERSHELL_HOOK_COMMAND__', $PowerShellCommand).Replace('__POWERSHELL_COMMAND__', $PowerShellCommand).Replace('__CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY__', $ClaudeConcurrency).Replace('__HOOK_SCRIPT__', 'flashbang.ps1').Replace('__POWERSHELL_HOOK_SCRIPT__', 'flashbang.ps1')
$newBytes = [System.Text.Encoding]::UTF8.GetBytes($finalContent)
+7 -2
View File
@@ -210,8 +210,13 @@ sys.stdout.buffer.write(("\n".join(lines) + "\n").encode("utf-8"))
content=$(cat -- "$source_file" && printf '\034') || return
content=${content%$'\034'}
needs_sub=false
if { [ "$flashbang_enabled" = false ] || [ "$statusline_enabled" = false ]; } && [[ "$relative" = settings.json || "$relative" = config.toml ]]; then
content=$(python3 "$(dirname -- "${BASH_SOURCE[0]}")/install-options.py" filter --path "$source_file" --flashbang "$flashbang_enabled" --statusline "$statusline_enabled" && printf '\034') || return
if { [ "$flashbang_enabled" = false ] || [ "$statusline_enabled" = false ] || [ "$semantic_retrieval_enabled" = false ]; } && [[ "$relative" = settings.json || "$relative" = config.toml ]]; then
content=$(python3 "$(dirname -- "${BASH_SOURCE[0]}")/install-options.py" filter --path "$source_file" --flashbang "$flashbang_enabled" --statusline "$statusline_enabled" --semantic-retrieval "$semantic_retrieval_enabled" && printf '\034') || return
content=${content%$'\034'}
needs_sub=true
fi
if [[ "$relative" = CLAUDE.md || "$relative" = AGENTS.md ]] || { [ "$semantic_retrieval_enabled" = false ] && [ "$relative" = rules/general.md ]; }; then
content=$(python3 "$(dirname -- "${BASH_SOURCE[0]}")/install-options.py" filter-instructions --path "$source_file" --destination "$destination" --semantic-retrieval "$semantic_retrieval_enabled" && printf '\034') || return
content=${content%$'\034'}
needs_sub=true
fi
+99
View File
@@ -0,0 +1,99 @@
"""Keep the client startup rule in sync with its personal instructions."""
import argparse
import hashlib
import os
from pathlib import Path
import stat
import tempfile
CLIENT_FILES = {'codex': 'AGENTS.md', 'claude': 'CLAUDE.md'}
START = '<!-- ai-config:personal-instructions -->'
END = '<!-- /ai-config:personal-instructions -->'
def checked_path(home, *parts):
path = home
for part in parts:
path = path / part
if path.is_symlink() or (hasattr(path, 'is_junction') and path.is_junction()):
raise ValueError(f'Personal instruction path must not be a link: {path}')
return path
def render(text, home, client):
reference = f'~/.my-ai-configuration/instructions/{client}.md'
# Migrate the two previously generated unconditional loaders.
for legacy in (
f"Read `{reference}` first when present; it is the user's direct instruction. System and developer requirements still apply.",
f"Optional personal instructions: first check whether `{reference}` is a file (`Test-Path -LiteralPath ... -PathType Leaf` in PowerShell; `test -f ...` in Bash). Read it only if it exists; otherwise continue silently. Its contents are the user's direct instructions. System and developer requirements still apply.",
):
text = text.replace(legacy + '\r\n\r\n', '').replace(legacy + '\n\n', '')
if START in text:
start = text.index(START)
end = text.find(END, start)
if end < 0:
raise ValueError('Personal instruction rule is missing its closing marker.')
end += len(END)
tail = text[end:]
for separator in ('\r\n\r\n', '\n\n'):
if tail.startswith(separator):
tail = tail[len(separator):]
break
text = text[:start] + tail
overlay = checked_path(home, '.my-ai-configuration', 'instructions', f'{client}.md')
if not overlay.is_file() or not overlay.read_text(encoding='utf-8-sig').strip():
return text
rule = (f'Read `{reference}` immediately, before any response or task work. '
'Keep its instructions active throughout the session, including after compaction. '
"These are the user's direct instructions. System and developer requirements still apply.")
return f'{START}\n{rule}\n{END}\n\n{text}'
def write_atomic(path, updated):
temporary = None
try:
with tempfile.NamedTemporaryFile(dir=path.parent, prefix='.instruction-', delete=False) as file:
temporary = Path(file.name)
file.write(updated)
os.chmod(temporary, stat.S_IMODE(path.stat().st_mode))
os.replace(temporary, path)
finally:
if temporary is not None:
temporary.unlink(missing_ok=True)
def refresh(home, client):
path = checked_path(home, f'.{client}', CLIENT_FILES[client])
if not path.is_file():
return
raw = path.read_bytes()
encoding = 'utf-8-sig' if raw.startswith(b'\xef\xbb\xbf') else 'utf-8'
updated = render(raw.decode(encoding), home, client).encode(encoding)
if updated == raw:
return
manifest = checked_path(home, f'.{client}', '.ai-config-manifest.tsv')
manifest_raw = manifest.read_bytes() if manifest.is_file() else b''
old_entry = f'{path.name}\t{hashlib.sha256(raw).hexdigest()}'.encode()
new_entry = f'{path.name}\t{hashlib.sha256(updated).hexdigest()}'.encode()
# Keep setup-owned files removable; preserve the stale hash of any preexisting user edits.
manifest_updated = b''.join(new_entry + line[len(old_entry):]
if line.rstrip(b'\r\n').lower() == old_entry.lower() else line
for line in manifest_raw.splitlines(keepends=True))
write_atomic(path, updated)
if manifest_updated != manifest_raw:
write_atomic(manifest, manifest_updated)
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--home', type=Path, required=True)
parser.add_argument('--client', choices=CLIENT_FILES, required=True)
args = parser.parse_args()
if not args.home.is_absolute():
parser.error('Home path must be absolute.')
try:
refresh(args.home, args.client)
except (OSError, ValueError) as error:
parser.exit(1, f'Personal instructions: {error}\n')
+1 -3
View File
@@ -125,9 +125,7 @@ function Select-ConfiguredPlugins {
} else { $false }
foreach ($referenceClient in $referenceClients) {
$selector = if ($referenceClient -eq 'Claude') { $entry.claude_plugin } else { $entry.codex_plugin }
if ($referenceClient -eq 'Codex' -and $entry.codex_method -eq 'qmd') {
$found = $found -or ($null -ne (Get-Command qmd -ErrorAction SilentlyContinue))
} elseif ($referenceClient -eq 'Codex' -and $entry.codex_method -ne 'plugin') {
if ($referenceClient -eq 'Codex' -and $entry.codex_method -ne 'plugin') {
$found = $found -or (Test-Path -LiteralPath (Join-Path $HomePath ('.agents/skills/' + $entry.codex_skill))) -or (Test-Path -LiteralPath (Join-Path $HomePath ('.codex/skills/' + $entry.codex_skill)))
} else {
$idField = if ($referenceClient -eq 'Claude') { 'id' } else { 'pluginId' }
+1 -3
View File
@@ -161,9 +161,7 @@ select_configured_plugins() {
fi
for reference_client in "${reference_clients[@]}"; do
if [ "$reference_client" = claude ]; then selector=$(plugin_field "$root" "$name" claude_plugin); else selector=$(plugin_field "$root" "$name" codex_plugin); fi
if [ "$reference_client" = codex ] && [ "$(plugin_field "$root" "$name" codex_method)" = qmd ]; then
command -v qmd >/dev/null 2>&1 && found=true
elif [ "$reference_client" = codex ] && [ "$(plugin_field "$root" "$name" codex_method)" != plugin ]; then
if [ "$reference_client" = codex ] && [ "$(plugin_field "$root" "$name" codex_method)" != plugin ]; then
local skill_name
skill_name=$(plugin_field "$root" "$name" codex_skill)
{ [ -d "$HOME/.agents/skills/$skill_name" ] || [ -d "$HOME/.codex/skills/$skill_name" ]; } && found=true
+18
View File
@@ -0,0 +1,18 @@
function Sync-Qmd {
param([Parameter(Mandatory=$true)][string]$RepositoryRoot,[Parameter(Mandatory=$true)][string]$HomePath,[ValidateSet('Codex','Claude','Both')][Parameter(Mandatory=$true)][string]$Client,[Parameter(Mandatory=$true)][bool]$Enabled,[switch]$DryRun,[switch]$Update,[switch]$Summary)
$arguments = @((Join-Path $PSScriptRoot 'qmd.py'), 'sync', '--root', $RepositoryRoot, '--home', $HomePath, '--client', $Client.ToLowerInvariant(), '--enabled', $Enabled.ToString().ToLowerInvariant())
if ($DryRun) { $arguments += '--dry-run' }; if ($Update) { $arguments += '--update' }
if ($Summary) { $arguments += '--summary' }
& python @arguments
if ($LASTEXITCODE -ne 0) { throw 'QMD local search reconciliation failed.' }
}
function Test-QmdDevice {
param([Parameter(Mandatory=$true)][string]$RepositoryRoot,[Parameter(Mandatory=$true)][string]$HomePath,[switch]$Summary)
$arguments = @((Join-Path $PSScriptRoot 'qmd.py'), 'benchmark', '--root', $RepositoryRoot, '--home', $HomePath)
if ($Summary) { $arguments += '--summary' }
& python @arguments | Write-Host
if ($LASTEXITCODE -eq 0) { return $true }
if ($LASTEXITCODE -eq 2) { return $false }
throw 'QMD benchmark failed.'
}
+342
View File
@@ -0,0 +1,342 @@
"""Install, benchmark, migrate, and remove the setup-owned QMD local search."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
import tempfile
import time
PACKAGE = '@tobilu/qmd'
VERSION = '2.8.3'
SCRIPTS = ('qmd-lib.mjs', 'qmd-config.mjs', 'qmd-daemon.mjs', 'qmd-search.mjs', 'qmd-warm.mjs', 'qmd-benchmark.mjs')
DEVICE_TIMEOUT = 600
WINDOWS = os.name == 'nt'
def run(arguments, env=None, timeout=1800, check=True):
result = subprocess.run(arguments, env=env, capture_output=True, text=True, encoding='utf-8', errors='replace',
timeout=timeout, shell=WINDOWS and arguments[0] in ('npm', 'qmd'))
if check and result.returncode != 0:
raise ValueError(f'{" ".join(arguments[:3])} failed: {(result.stderr or result.stdout).strip()[-300:]}')
return result.stdout
def process_alive(pid):
if WINDOWS: # os.kill(pid, 0) would send CTRL_C_EVENT on Windows
import ctypes
kernel32 = ctypes.windll.kernel32
handle = kernel32.OpenProcess(0x1000, False, pid) # PROCESS_QUERY_LIMITED_INFORMATION
if not handle:
return False
code = ctypes.c_ulong()
try:
return bool(kernel32.GetExitCodeProcess(handle, ctypes.byref(code))) and code.value == 259 # STILL_ACTIVE
finally:
kernel32.CloseHandle(handle)
try:
os.kill(pid, 0)
except ProcessLookupError:
return False
except PermissionError:
return True
return True
def decide(results):
for device in ('gpu', 'cpu'):
if any(r.get('device') == device and r.get('passed') is True for r in results):
return device
return None
def node_version_ok(text):
match = re.fullmatch(r'v(\d+)\.\d+\.\d+', text.strip())
return bool(match) and int(match.group(1)) >= 22
def digest(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
def atomic_json(path, value):
path.parent.mkdir(parents=True, exist_ok=True)
with tempfile.NamedTemporaryFile('w', encoding='utf-8', dir=path.parent, delete=False) as stream:
json.dump(value, stream, indent=2)
stream.write('\n')
os.replace(stream.name, path)
class Setup:
def __init__(self, home, root, runner=run, dry_run=False, summary=False):
self.home, self.root, self.run, self.dry_run, self.summary = Path(home), Path(root), runner, dry_run, summary
self.base = self.home / '.my-ai-configuration'
self.state_path = self.base / 'qmd.json'
self.scripts = self.base / 'qmd'
self.models = Path(os.environ.get('XDG_CACHE_HOME') or self.home / '.cache') / 'qmd' / 'models'
def say(self, text):
if not self.summary:
print(text)
def load(self):
if not self.state_path.exists():
return {'version': 1, 'clients': [], 'device': None, 'owned_package': False, 'owned_models': [], 'models_block': False, 'previous_models': None, 'files': {}}
state = json.loads(self.state_path.read_text(encoding='utf-8'))
if state.get('version') != 1 or state.get('device') not in (None, 'gpu', 'cpu'):
raise ValueError('Invalid QMD setup state.')
return state
def node(self, script, *arguments, env=None, timeout=1800):
return self.run(['node', str(self.scripts / script), *arguments], env=env, timeout=timeout)
def package_installed(self):
# npm list exits non-zero on tree problems (ELSPROBLEMS) but still prints valid JSON.
output = self.run(['npm', 'list', '--global', '--depth=0', '--json'], check=False)
try:
listing = json.loads(output)
except json.JSONDecodeError:
raise ValueError('npm list did not return JSON.')
if not isinstance(listing, dict) or not isinstance(listing.get('dependencies'), dict):
raise ValueError('npm list did not report the global packages.')
return PACKAGE in listing['dependencies']
def require_node(self):
if not node_version_ok(self.run(['node', '--version'])):
raise ValueError('QMD local search needs Node.js 22 or newer.')
def stop_legacy_daemon(self, legacy):
"""Ask the old Python daemon to exit so its files can be deleted (Windows keeps open files locked)."""
python = legacy / '.venv' / ('Scripts/python.exe' if WINDOWS else 'bin/python')
server = legacy / 'server.py'
if python.is_file() and server.is_file():
try:
self.run([str(python), str(server), 'stop', '--data-dir', str(legacy / 'data'),
'--model-cache', str(legacy / 'model-cache')], timeout=60, check=False)
except (ValueError, OSError, subprocess.SubprocessError):
pass
def migrate(self, state=None):
state = self.load() if state is None else state
legacy_state = self.base / 'semantic-retrieval.json'
legacy = self.base / 'semantic-retrieval'
if legacy_state.exists() or legacy.exists():
if legacy.is_symlink():
raise ValueError('Legacy semantic retrieval directory must not be a symbolic link.')
if legacy.exists():
self.stop_legacy_daemon(legacy)
shutil.rmtree(legacy)
legacy_state.unlink(missing_ok=True)
self.say('Removed the previous Python semantic retrieval installation.')
ledger_path = self.base / 'extensions.json'
if ledger_path.exists():
ledger = json.loads(ledger_path.read_text(encoding='utf-8'))
owned = [r for r in ledger.get('resources', []) if r.get('kind') == 'qmd']
if owned:
state['owned_package'] = True
atomic_json(self.state_path, state) # record ownership before the ledger forgets it
ledger['resources'] = [r for r in ledger['resources'] if r.get('kind') != 'qmd']
atomic_json(ledger_path, ledger)
def install_scripts(self, state):
if self.scripts.is_symlink():
raise ValueError('QMD script directory must not be a symbolic link.')
self.scripts.mkdir(parents=True, exist_ok=True)
for name in SCRIPTS:
target = self.scripts / name
if target.is_symlink():
raise ValueError(f'QMD script must not be a symbolic link: {name}')
expected = state['files'].get(name)
if target.exists() and expected and digest(target) != expected:
raise ValueError(f'Setup-owned QMD script changed: {name}')
shutil.copy2(self.root / 'shared' / 'qmd' / name, target)
state['files'][name] = digest(target)
def install(self, state):
"""Steps shared by Yes and Auto: package, scripts, models block, model download."""
self.require_node()
self.migrate(state)
if not self.package_installed():
self.run(['npm', 'install', '--global', f'{PACKAGE}@{VERSION}'])
state['owned_package'] = True
self.install_scripts(state)
atomic_json(self.state_path, state)
if not state['models_block']:
output = self.node('qmd-config.mjs', 'set-models')
try:
previous = json.loads(output.strip().splitlines()[-1])['previous']
except (IndexError, KeyError, TypeError, json.JSONDecodeError):
raise ValueError('qmd-config set-models returned no previous models.')
state['previous_models'] = previous
state['models_block'] = True
atomic_json(self.state_path, state)
before = {p.name for p in self.models.iterdir()} if self.models.is_dir() else set()
self.run(['qmd', 'pull'], timeout=3600)
after = {p.name for p in self.models.iterdir()} if self.models.is_dir() else set()
state['owned_models'] = sorted(set(state['owned_models']) | (after - before))
atomic_json(self.state_path, state)
def detect_gpu(self):
try:
found = json.loads(self.node('qmd-benchmark.mjs', '--detect', timeout=DEVICE_TIMEOUT) or '{}')
gpu = found.get('gpu') or False
except Exception as error: # any probe failure means: use the CPU
self.say(f'QMD GPU detection failed, using the CPU: {str(error)[:200]}')
return False
names = ', '.join(found.get('devices') or []) or 'unnamed device'
self.say(f"QMD devices: {f'{str(gpu).upper()}: {names}' if gpu else 'no GPU'}; CPU threads: {found.get('threads', '?')}")
return gpu
def enable(self, clients):
state = self.load()
if self.dry_run:
self.say('DRYRUN enable QMD local search: Qwen3-Embedding-0.6B, Qwen3-Reranker-0.6B, QMD query expansion 1.7B')
return
self.install(state)
if state['device'] is None:
state['device'] = 'gpu' if self.detect_gpu() else 'cpu'
state['clients'] = sorted(set(state['clients']) | set(clients))
atomic_json(self.state_path, state)
self.say(f'QMD local search enabled on {state["device"].upper()}.')
def remove_all(self, state):
daemon = self.scripts / 'daemon.json'
try:
pid = json.loads(daemon.read_text(encoding='utf-8')).get('pid') if daemon.is_file() else None
except (OSError, ValueError, AttributeError):
pid = None
try:
if self.scripts.joinpath('qmd-search.mjs').exists():
self.node('qmd-search.mjs', '--stop', timeout=60)
except (ValueError, OSError, subprocess.SubprocessError):
pass
if isinstance(pid, int) and pid > 0:
deadline = time.monotonic() + 10 # the daemon must release the model files and the package
while process_alive(pid) and time.monotonic() < deadline:
time.sleep(0.2)
collections = self.scripts / 'collections.json'
names = json.loads(collections.read_text(encoding='utf-8')) if collections.exists() else []
for name in names:
if re.fullmatch(r'repo-[0-9a-f]{12}', name):
try:
self.run(['qmd', 'collection', 'remove', name])
except (ValueError, subprocess.SubprocessError):
pass
if state['models_block'] and self.scripts.joinpath('qmd-config.mjs').exists():
previous = state.get('previous_models')
try:
if previous is not None:
self.node('qmd-config.mjs', 'restore-models', json.dumps(previous))
else:
self.node('qmd-config.mjs', 'unset-models')
except (ValueError, OSError, subprocess.SubprocessError) as error:
print(f'QMD local search: could not restore the models block in index.yml: {error}', file=sys.stderr)
kept = []
for name in state['owned_models']:
path = self.models / name
if name == Path(name).name and path.is_file() and not path.is_symlink():
try:
path.unlink()
except OSError as error:
print(f'QMD local search: could not delete model {name}; the next removal retries it: {error}', file=sys.stderr)
kept.append(name)
if state['owned_package']:
self.run(['npm', 'uninstall', '--global', PACKAGE])
if self.scripts.exists() and not self.scripts.is_symlink():
shutil.rmtree(self.scripts)
if kept:
state.update(clients=[], owned_models=kept, owned_package=False, models_block=False, previous_models=None, files={})
atomic_json(self.state_path, state)
else:
self.state_path.unlink(missing_ok=True)
def disable(self, clients):
state = self.load()
if not self.state_path.exists():
if not self.dry_run:
self.migrate(state)
if state['owned_package']:
self.run(['npm', 'uninstall', '--global', PACKAGE])
self.state_path.unlink(missing_ok=True)
return
if self.dry_run:
self.say('DRYRUN disable QMD local search')
return
state['clients'] = [c for c in state['clients'] if c not in clients]
if state['clients']:
atomic_json(self.state_path, state)
return
self.remove_all(state)
self.say('QMD local search removed.')
def benchmark(self):
state = self.load()
if self.dry_run:
self.say('DRYRUN benchmark QMD local search on GPU, then CPU')
return True
try:
self.install(state)
except (ValueError, OSError, subprocess.SubprocessError, json.JSONDecodeError) as error:
print(f'QMD local search is not suitable for this computer: {error}')
try:
self.remove_all(state)
except (ValueError, OSError, subprocess.SubprocessError, json.JSONDecodeError) as cleanup:
print(f'QMD cleanup was incomplete: {cleanup}')
return False
results = []
devices = (['gpu'] if self.detect_gpu() else []) + ['cpu']
for device in devices:
env = dict(os.environ)
env.pop('QMD_FORCE_CPU' if device == 'gpu' else 'QMD_LLAMA_GPU', None)
env['QMD_LLAMA_GPU' if device == 'gpu' else 'QMD_FORCE_CPU'] = 'auto' if device == 'gpu' else '1'
try:
result = json.loads(self.node('qmd-benchmark.mjs', '--device', device, env=env, timeout=DEVICE_TIMEOUT))
if not isinstance(result, dict):
raise ValueError('benchmark returned a non-object result')
except (ValueError, subprocess.SubprocessError) as error:
result = {'device': device, 'passed': False, 'error': str(error)[:200]}
results.append(result)
print(f'QMD benchmark {device.upper()}: ' + ('passed' if result.get('passed') else 'failed')
+ f" (search {result.get('querySeconds', '-')} s, embedding {result.get('chunksPerSecond', '-')} chunks/s)")
if result.get('passed'):
break
device = decide(results)
if device is None:
self.remove_all(state)
print('QMD local search is not suitable for this computer; removed what the setup installed.')
return False
state['device'] = device
atomic_json(self.state_path, state)
return True
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('action', choices=('sync', 'benchmark'))
parser.add_argument('--root', required=True, type=Path)
parser.add_argument('--home', required=True, type=Path)
parser.add_argument('--client', choices=('codex', 'claude', 'both'))
parser.add_argument('--enabled', choices=('true', 'false'))
parser.add_argument('--dry-run', action='store_true')
parser.add_argument('--update', action='store_true')
parser.add_argument('--summary', action='store_true')
args = parser.parse_args()
setup = Setup(args.home.resolve(), args.root.resolve(), run, args.dry_run, args.summary)
if args.action == 'benchmark':
raise SystemExit(0 if setup.benchmark() else 2)
if args.client is None or args.enabled is None:
parser.error('sync requires --client and --enabled')
clients = ['codex', 'claude'] if args.client == 'both' else [args.client]
(setup.enable if args.enabled == 'true' else setup.disable)(clients)
if __name__ == '__main__':
try:
main()
except (OSError, ValueError, json.JSONDecodeError, subprocess.SubprocessError) as error:
print(f'QMD local search: {error}', file=sys.stderr)
raise SystemExit(1)
@@ -1,18 +1,17 @@
#!/usr/bin/env bash
sync_semantic_retrieval() {
sync_qmd() {
local root=$1 client=$2 home_path=$3 enabled=$4 dry_run=$5 update=$6 summary=${7:-false}
[ "$enabled" = true ] || [ -f "$home_path/.my-ai-configuration/semantic-retrieval.json" ] || return 0
local arguments=("$root/scripts/lib/semantic-retrieval.py" sync --root "$root" --home "$home_path" --client "$client" --enabled "$enabled")
local arguments=("$root/scripts/lib/qmd.py" sync --root "$root" --home "$home_path" --client "$client" --enabled "$enabled")
[ "$dry_run" != true ] || arguments+=(--dry-run)
[ "$update" != true ] || arguments+=(--update)
[ "$summary" != true ] || arguments+=(--summary)
python3 "${arguments[@]}"
}
test_semantic_retrieval_device() {
test_qmd_device() {
local root=$1 home_path=$2 summary=${3:-false}
local arguments=("$root/scripts/lib/semantic-retrieval.py" benchmark --root "$root" --home "$home_path")
local arguments=("$root/scripts/lib/qmd.py" benchmark --root "$root" --home "$home_path")
[ "$summary" != true ] || arguments+=(--summary)
local status
if python3 "${arguments[@]}" >&2; then return 0; else status=$?; fi
+3
View File
@@ -7,6 +7,8 @@ import stat
import tempfile
from pathlib import Path
from personal_instructions import refresh
def remove_instruction(path: Path, instruction: str) -> bool:
if not path.exists():
@@ -75,6 +77,7 @@ def main() -> int:
print(f'Instruction removed for {target}.')
else:
print(f'Instruction not found for {target}.')
refresh(home, target)
return 0
-18
View File
@@ -1,18 +0,0 @@
function Sync-SemanticRetrieval {
param([Parameter(Mandatory=$true)][string]$RepositoryRoot,[Parameter(Mandatory=$true)][string]$HomePath,[ValidateSet('Codex','Claude','Both')][Parameter(Mandatory=$true)][string]$Client,[Parameter(Mandatory=$true)][bool]$Enabled,[switch]$DryRun,[switch]$Update,[switch]$Summary)
$statePath = Join-Path $HomePath '.my-ai-configuration/semantic-retrieval.json'
if (-not $Enabled -and -not (Test-Path -LiteralPath $statePath)) { return }
$arguments = @((Join-Path $PSScriptRoot 'semantic-retrieval.py'), 'sync', '--root', $RepositoryRoot, '--home', $HomePath, '--client', $Client.ToLowerInvariant(), '--enabled', $Enabled.ToString().ToLowerInvariant())
if ($DryRun) { $arguments += '--dry-run' }; if ($Update) { $arguments += '--update' }
if ($Summary) { $arguments += '--summary' }
& python @arguments
if ($LASTEXITCODE -ne 0) { throw 'Semantic retrieval reconciliation failed.' }
}
function Test-SemanticRetrievalDevice {
param([Parameter(Mandatory=$true)][string]$RepositoryRoot,[Parameter(Mandatory=$true)][string]$HomePath,[switch]$Summary)
& python (Join-Path $PSScriptRoot 'semantic-retrieval.py') benchmark --root $RepositoryRoot --home $HomePath $(if ($Summary) { '--summary' }) | Write-Host
if ($LASTEXITCODE -eq 0) { return $true }
if ($LASTEXITCODE -eq 2) { return $false }
throw 'Semantic retrieval benchmark failed.'
}
-230
View File
@@ -1,230 +0,0 @@
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
import tempfile
FILES = ('server.py', 'benchmark.py', 'requirements.txt')
CUDA_INDEX = 'https://download.pytorch.org/whl/cu126'
MODEL_PRELOAD_TIMEOUT = 900
def digest(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
def python_path(target):
return target / ('Scripts/python.exe' if os.name == 'nt' else 'bin/python')
def ensure_runtime(target, dry_run):
if dry_run:
return python_path(target / '.venv')
venv = target / '.venv'
if not venv.exists():
subprocess.run([sys.executable, '-m', 'venv', str(venv)], check=True, timeout=180)
python = python_path(venv)
if not python.is_file():
raise ValueError('Semantic retrieval virtual environment is incomplete.')
arguments = [str(python), '-m', 'pip', 'install', '--disable-pip-version-check', '-r', str(target / 'requirements.txt')]
if shutil.which('nvidia-smi'):
try:
subprocess.run([*arguments, '--extra-index-url', CUDA_INDEX], check=True, timeout=900)
except subprocess.CalledProcessError:
print('Semantic retrieval: CUDA runtime installation failed; falling back to CPU.', file=sys.stderr)
subprocess.run(arguments, check=True, timeout=900)
else:
subprocess.run(arguments, check=True, timeout=900)
return python
def ensure_models(python, target):
subprocess.run([str(python), str(target / 'server.py'), 'preload', '--model-cache', str(target / 'model-cache')],
check=True, timeout=MODEL_PRELOAD_TIMEOUT)
def load_state(path):
if not path.exists():
return {'version': 1, 'clients': [], 'files': {}}
state = json.loads(path.read_text(encoding='utf-8'))
if state.get('version') != 1 or not isinstance(state.get('clients'), list) or not isinstance(state.get('files'), dict):
raise ValueError('Invalid semantic retrieval ownership state.')
if any(client not in ('codex', 'claude') for client in state['clients']):
raise ValueError('Invalid semantic retrieval client state.')
if any(name not in FILES or not isinstance(value, str) or len(value) != 64 or any(character not in '0123456789abcdef' for character in value) for name, value in state['files'].items()):
raise ValueError('Invalid semantic retrieval file state.')
return state
def save_state(path, state, dry_run):
if not dry_run:
path.parent.mkdir(parents=True, exist_ok=True)
temporary = None
try:
with tempfile.NamedTemporaryFile(mode='w', encoding='utf-8', dir=path.parent, delete=False) as stream:
temporary = Path(stream.name)
json.dump(state, stream, indent=2)
stream.write('\n')
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, path)
finally:
if temporary is not None:
temporary.unlink(missing_ok=True)
def check_runtime(target, state):
if target.is_symlink():
raise ValueError('Semantic retrieval directory must not be a symbolic link.')
if not target.exists():
return
if not target.is_dir():
raise ValueError('Semantic retrieval target is not a directory.')
for name, expected in state['files'].items():
path = target / name
if path.is_symlink() or not path.is_file() or digest(path) != expected:
raise ValueError(f'Owned semantic retrieval file changed: {name}')
allowed = set(state['files']) | {'.venv', 'model-cache', '__pycache__', 'data'}
for path in target.iterdir():
if path.is_symlink() or path.name not in allowed:
raise ValueError(f'Foreign or linked semantic retrieval path: {path.name}')
if path.name == 'data' and (not path.is_dir() or any(
child.is_symlink() or not (
child.is_file() and (child.name in ('daemon.key', 'daemon.starting') or re.fullmatch(
r'[0-9a-f]{64}\.sqlite3(?:-(?:wal|shm|journal))?|daemon-[0-9a-f]{12}\.starting', child.name))
or child.is_socket() and re.fullmatch(r'daemon-[0-9a-f]{16}\.sock', child.name))
for child in path.iterdir())):
raise ValueError('Foreign or linked semantic retrieval path: data')
if path.name == '__pycache__' and (not path.is_dir() or any(
child.is_symlink() or not child.is_file() or not re.fullmatch(
r'(?:server|benchmark)\.[A-Za-z0-9_-]+(?:\.opt-\d+)?\.pyc', child.name)
for child in path.iterdir())):
raise ValueError('Foreign or linked semantic retrieval path: __pycache__')
def sync(root, home, clients, enabled, dry_run, update, summary=False):
base = home / '.my-ai-configuration'
target = base / 'semantic-retrieval'
state_path = base / 'semantic-retrieval.json'
state = load_state(state_path)
clients = list(dict.fromkeys(clients))
if base.exists() and base.is_symlink():
raise ValueError('Semantic retrieval state directory must not be a symbolic link.')
if not enabled:
owned = [client for client in state['clients'] if client in clients]
if dry_run:
if not summary:
print('DRYRUN disable semantic retrieval: ' + (', '.join(owned) if owned else 'nothing owned'))
return
state['clients'] = [client for client in state['clients'] if client not in clients]
if state['clients']:
save_state(state_path, state, False)
return
if not state['files']:
state_path.unlink(missing_ok=True)
return
check_runtime(target, state)
if target.exists() and target.is_dir():
shutil.rmtree(target)
state_path.unlink(missing_ok=True)
return
check_runtime(target, state)
if target.exists() and not state['files']:
raise ValueError('Semantic retrieval directory exists without setup ownership; refusing to overwrite it.')
if dry_run:
if not summary:
print('DRYRUN enable semantic retrieval: Qwen3-Embedding-0.6B + Qwen3-Reranker-0.6B')
return
target.mkdir(parents=True, exist_ok=True)
source = root / 'shared' / 'retrieval'
for name in FILES:
shutil.copy2(source / name, target / name)
state['files'][name] = digest(target / name)
save_state(state_path, state, False)
python = ensure_runtime(target, False)
ensure_models(python, target)
# Sandboxed clients may write only to data/; it cannot be created from inside the sandbox.
(target / 'data').mkdir(exist_ok=True)
state['clients'] = list(dict.fromkeys(state['clients']))
save_state(state_path, state, False)
for client in clients:
if client not in state['clients']:
state['clients'].append(client)
save_state(state_path, state, False)
save_state(state_path, state, False)
def benchmark(root, home, dry_run, summary=False):
base = home / '.my-ai-configuration'
target = base / 'semantic-retrieval'
state_path = base / 'semantic-retrieval.json'
if dry_run:
if not summary:
print('DRYRUN benchmark semantic retrieval: Qwen3-Embedding-0.6B + Qwen3-Reranker-0.6B')
return True
state = load_state(state_path)
source = root / 'shared' / 'retrieval'
if state['files']:
check_runtime(target, state)
elif target.is_symlink():
raise ValueError('Semantic retrieval directory must not be a symbolic link.')
if target.exists() and not state['files']:
if not all((target / name).is_file() and digest(target / name) == digest(source / name) for name in FILES):
raise ValueError('Semantic retrieval directory exists without setup ownership; refusing to overwrite it.')
state['files'] = {name: digest(target / name) for name in FILES}
check_runtime(target, state)
target.mkdir(parents=True, exist_ok=True)
for name in FILES:
shutil.copy2(source / name, target / name)
state['files'][name] = digest(target / name)
python = ensure_runtime(target, False)
result = subprocess.run([str(python), str(target / 'benchmark.py'), '--model-cache', str(target / 'model-cache')], check=True,
stdout=subprocess.PIPE, text=True, timeout=300)
outcome = json.loads(result.stdout)
if not isinstance(outcome, dict) or not isinstance(outcome.get('recommended'), bool):
raise ValueError('Semantic retrieval benchmark returned an invalid result.')
if outcome['recommended'] or state['clients']:
save_state(state_path, state, False)
else:
check_runtime(target, state)
shutil.rmtree(target)
state_path.unlink(missing_ok=True)
if not summary:
print('Semantic retrieval warm-model check: ' + ('passed' if outcome['recommended'] else 'failed') + f" (embedding {outcome['embeddingSeconds']}s, reranking {outcome['rerankingSeconds']}s; excludes model startup and indexing)")
return outcome['recommended']
def main():
parser = argparse.ArgumentParser()
parser.add_argument('action', choices=('sync', 'benchmark'))
parser.add_argument('--root', required=True, type=Path)
parser.add_argument('--home', required=True, type=Path)
parser.add_argument('--client', choices=('codex', 'claude', 'both'))
parser.add_argument('--enabled', choices=('true', 'false'))
parser.add_argument('--dry-run', action='store_true')
parser.add_argument('--update', action='store_true')
parser.add_argument('--summary', action='store_true')
args = parser.parse_args()
if args.action == 'benchmark':
recommended = benchmark(args.root.resolve(), args.home.resolve(), args.dry_run, args.summary)
raise SystemExit(0 if recommended else 2)
if args.client is None or args.enabled is None:
parser.error('sync requires --client and --enabled')
clients = ('codex', 'claude') if args.client == 'both' else (args.client,)
sync(args.root.resolve(), args.home.resolve(), clients, args.enabled == 'true', args.dry_run, args.update, args.summary)
print('SEMANTIC RETRIEVAL: reconciliation complete' + (' (dry run)' if args.dry_run else ''))
if __name__ == '__main__':
try:
main()
except (OSError, ValueError, json.JSONDecodeError, subprocess.SubprocessError) as error:
print(f'Semantic retrieval: {error}', file=sys.stderr)
raise SystemExit(1)
+5
View File
@@ -0,0 +1,5 @@
// Loads QMD's yaml dependency for tests, including hoisted installations.
import { importQmdDependency } from '../../../shared/qmd/qmd-lib.mjs';
const YAML = await importQmdDependency('yaml');
export const parseDocument = YAML.parseDocument;
export default YAML;
+12
View File
@@ -2,6 +2,18 @@
param([switch]$Summary)
$ErrorActionPreference = 'Stop'
$root = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent
$doctorPath = Join-Path $root 'scripts/commands/doctor.ps1'
$doctorAst = [System.Management.Automation.Language.Parser]::ParseFile($doctorPath, [ref]$null, [ref]$null)
$probe = $doctorAst.Find({ param($node) $node -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $node.Name -eq 'Invoke-DoctorCommand' }, $true)
if (-not $probe) { throw 'Doctor command probe is missing.' }
Invoke-Expression $probe.Extent.Text
foreach ($code in @(0, 7)) {
$result = Invoke-DoctorCommand -Command 'python' -Arguments @('-c', "import sys; print('fixture 1.0'); print('fixture warning', file=sys.stderr); sys.exit($code)")
if ($result.ExitCode -ne $code -or ($result.Output -join ' ') -notmatch 'fixture warning' -or ($result.Output -join ' ') -notmatch 'fixture 1.0') { throw 'Doctor must capture native warnings and preserve the exit code.' }
if ($ErrorActionPreference -ne 'Stop') { throw 'Doctor command probe changed the caller error preference.' }
}
$missing = Invoke-DoctorCommand -Command 'ai-config-missing-doctor-test-command'
if ($missing.ExitCode -eq 0) { throw 'Doctor must not report a missing command as successful.' }
$doctorSummaryOutput = & (Join-Path $root 'scripts/commands/doctor.ps1') -Summary
if (@($doctorSummaryOutput | Where-Object { $_ -match '^PASS ' }).Count -ne 0) { throw 'Doctor summary mode must not print individual PASS lines.' }
if (@($doctorSummaryOutput | Where-Object { $_ -match '^Doctor: PASS \| \d+ checks$' }).Count -ne 1) { throw "Doctor summary mode must print one 'Doctor: PASS | N checks' line: $($doctorSummaryOutput -join '; ')" }
+16
View File
@@ -8,6 +8,22 @@ while [ $# -gt 0 ]; do
esac
done
root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)
(
codex() { printf 'fixture warning\n' >&2; printf 'fixture 1.0\n'; return "$DOCTOR_TEST_CODE"; }
export -f codex
for DOCTOR_TEST_CODE in 0 7; do
export DOCTOR_TEST_CODE
if probe_output=$(bash "$root/scripts/commands/doctor.sh" --summary); then probe_status=0; else probe_status=$?; fi
if [ "$DOCTOR_TEST_CODE" -eq 0 ]; then
test "$probe_status" -eq 0
printf '%s\n' "$probe_output" | grep -Eq '^Doctor: PASS \| [0-9]+ checks$'
else
test "$probe_status" -eq 1
printf '%s\n' "$probe_output" | grep -Fq 'FAIL codex --version failed (exit 7): fixture warning'
printf '%s\n' "$probe_output" | grep -Eq '^Doctor: FAIL \| [0-9]+ checks$'
fi
done
)
doctor_summary_output=$(bash "$root/scripts/commands/doctor.sh" --summary)
if printf '%s\n' "$doctor_summary_output" | grep -q '^PASS '; then
printf 'Doctor summary mode must not print individual PASS lines.\n' >&2
+16
View File
@@ -17,6 +17,22 @@ try {
if (Test-AllManifestsPresent @($installed, $missing)) { throw 'Update guard must fail when any selected destination is not installed.' }
if (-not (Test-AllManifestsPresent @($installed))) { throw 'Update guard must pass when every selected destination is installed.' }
# QMD prompt: Auto maps device suitability; No never calls the benchmark.
. (Join-Path $root 'scripts/lib/install-options.ps1')
$originalIn = [Console]::In
try {
function Test-QmdDevice { param($RepositoryRoot, $HomePath) return $false }
[Console]::SetIn([IO.StringReader]::new("a`n"))
if ((Read-SemanticRetrievalOption -Default $false -RepositoryRoot $root -HomePath $work) -ne $false) { throw 'Auto unsuitable must yield false' }
function Test-QmdDevice { param($RepositoryRoot, $HomePath) return $true }
[Console]::SetIn([IO.StringReader]::new("a`n"))
if ((Read-SemanticRetrievalOption -Default $false -RepositoryRoot $root -HomePath $work) -ne $true) { throw 'Auto suitable must yield true' }
function Test-QmdDevice { throw 'must not run' }
[Console]::SetIn([IO.StringReader]::new("n`n"))
if ((Read-SemanticRetrievalOption -Default $true -RepositoryRoot $root -HomePath $work) -ne $false) { throw 'No must yield false' }
} finally { [Console]::SetIn($originalIn) }
if ((Get-Content -Raw (Join-Path $root 'scripts/lib/install-options.ps1')) -notmatch 'Enable local QMD search models \(Qwen3 embedding, Qwen3 reranker, QMD query expansion\)\? \[') { throw 'prompt text changed' }
if ($Summary) { Write-Output 'Tests: PASS | install guards' } else { Write-Output 'PASS install guards: already-installed and not-installed detection' }
} finally {
Remove-Item $work -Recurse -Force -ErrorAction SilentlyContinue
+14
View File
@@ -25,4 +25,18 @@ any_manifest_present "$installed" "$missing" || { printf 'Expected an installed
! all_manifests_present "$installed" "$missing" || { printf 'Update guard must fail when any selected destination is not installed.\n' >&2; exit 1; }
all_manifests_present "$installed" || { printf 'Update guard must pass when every selected destination is installed.\n' >&2; exit 1; }
# QMD prompt: Auto maps benchmark exit codes; No never calls the benchmark.
(
home_path=$(mktemp -d)
. "$root/scripts/lib/install-options.sh"
test_qmd_device() { return 1; }
[ "$(printf 'a\n' | read_semantic_retrieval_option false 2>/dev/null)" = false ] || { echo 'Auto unsuitable must yield false' >&2; exit 1; }
test_qmd_device() { return 0; }
[ "$(printf 'a\n' | read_semantic_retrieval_option false 2>/dev/null)" = true ] || { echo 'Auto suitable must yield true' >&2; exit 1; }
test_qmd_device() { echo 'must not run' >&2; exit 9; }
[ "$(printf 'n\n' | read_semantic_retrieval_option true 2>/dev/null)" = false ] || { echo 'No must yield false' >&2; exit 1; }
printf 'n\n' | read_semantic_retrieval_option true 2>&1 >/dev/null | grep -q 'Enable local QMD search models' || { echo 'prompt text changed' >&2; exit 1; }
rm -rf -- "$home_path"
)
if [ "$summary" = true ]; then printf 'Tests: PASS | install guards\n'; else printf 'PASS install guards: already-installed and not-installed detection\n'; fi
+79
View File
@@ -11,12 +11,28 @@ import unittest
ROOT = Path(__file__).resolve().parent.parent.parent
sys.path.insert(0, str(ROOT / 'scripts/lib'))
SPEC = importlib.util.spec_from_file_location('install_options', ROOT / 'scripts/lib/install-options.py')
OPTIONS = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(OPTIONS)
class InstallOptionsTests(unittest.TestCase):
def test_codex_update_replaces_old_sandbox_mode_and_preserves_user_settings(self):
current = ('sandbox_mode = "workspace-write"\nmodel = "custom-model"\n'
'[windows]\nsandbox = "elevated"\n'
'[sandbox_workspace_write]\nnetwork_access = false\n'
'[mcp_servers.demo]\ncommand = "demo"\n')
managed = (ROOT / 'adapters/codex/config/config.toml').read_text(encoding='utf-8')
result = tomllib.loads(OPTIONS.merge_toml(current, managed))
self.assertEqual(result['sandbox_mode'], 'danger-full-access')
self.assertEqual(result['approval_policy'], 'on-request')
self.assertEqual(result['approvals_reviewer'], 'auto_review')
self.assertEqual(result['model'], 'custom-model')
self.assertEqual(result['mcp_servers']['demo'], {'command': 'demo'})
def test_merge_toml_preserves_local_plugin_table_indentation(self):
plugin_state = (' [marketplaces.ponytail]\n'
' source_type = "git"\n'
@@ -73,6 +89,69 @@ class InstallOptionsTests(unittest.TestCase):
subprocess.run([sys.executable, script, 'merge-toml', '--current', str(current), '--managed', str(managed_path), '--output', str(output)], check=True)
self.assertIn('# Größe', output.read_text(encoding='utf-8'))
def test_semantic_retrieval_disabled_removes_qmd_session_hook(self):
with tempfile.TemporaryDirectory() as directory:
settings = Path(directory) / 'settings.json'
settings.write_text(json.dumps({'hooks': {'SessionStart': [
{'hooks': [{'type': 'command', 'command': 'bash "/c/hooks/scripts/show-session-state-pointer.sh"'}]},
{'hooks': [{'type': 'command', 'command': 'bash "/c/hooks/scripts/qmd-warm.sh"'}]}]}}), encoding='utf-8')
result = json.loads(OPTIONS.filter_options(settings, semantic_retrieval_enabled=False))
commands = [hook['command'] for group in result['hooks']['SessionStart'] for hook in group['hooks']]
self.assertEqual(commands, ['bash "/c/hooks/scripts/show-session-state-pointer.sh"'])
def test_semantic_retrieval_disabled_removes_codex_session_hook(self):
with tempfile.TemporaryDirectory() as directory:
config = Path(directory) / 'config.toml'
config.write_text('[features]\nhooks = true\n\n[[hooks.SessionStart]]\n[[hooks.SessionStart.hooks]]\ntype = "command"\n'
'command = \'bash "/c/hooks/scripts/qmd-warm.sh"\'\ncommand_windows = \'pwsh "/c/hooks/scripts/Start-QmdWarm.ps1"\'\ntimeout = 4\n',
encoding='utf-8')
result = OPTIONS.filter_options(config, semantic_retrieval_enabled=False)
self.assertNotIn('qmd-warm', result)
self.assertNotIn('hooks.SessionStart', result)
def test_instruction_line_only_with_semantic_search(self):
text = 'Line one.\nWhen `semantic-search` is installed, run it first.\nLine three.\n'
self.assertEqual(OPTIONS.filter_instructions(text, False), 'Line one.\nLine three.\n')
self.assertEqual(OPTIONS.filter_instructions(text, True), text)
def test_qmd_warm_match_is_anchored(self):
self.assertTrue(OPTIONS.is_qmd_warm({'command': 'bash "/c/hooks/scripts/qmd-warm.sh"'}))
self.assertTrue(OPTIONS.is_qmd_warm({'command_windows': 'pwsh "C:\hooks\Start-QmdWarm.ps1"'}))
self.assertFalse(OPTIONS.is_qmd_warm({'command': 'bash "/c/hooks/scripts/my-qmd-warm.sh"'}))
def test_merge_json_keeps_user_session_hook_and_drops_qmd_when_disabled(self):
user = {'type': 'command', 'command': 'bash "/home/me/my-start.sh"'}
pointer = {'type': 'command', 'command': 'bash "/c/hooks/scripts/show-session-state-pointer.sh"'}
qmd = {'type': 'command', 'command': 'bash "/c/hooks/scripts/qmd-warm.sh"'}
current = {'hooks': {'SessionStart': [{'hooks': [user]}]}}
managed = {'hooks': {'SessionStart': [{'hooks': [pointer]}, {'hooks': [qmd]}]}}
once = OPTIONS.merge_json(current, managed)
twice = OPTIONS.merge_json(once, managed)
self.assertEqual(twice['hooks']['SessionStart'], [{'hooks': [pointer]}, {'hooks': [qmd]}, {'hooks': [user]}])
disabled = OPTIONS.merge_json(twice, {'hooks': {'SessionStart': [{'hooks': [pointer]}]}})
commands = [hook['command'] for group in disabled['hooks']['SessionStart'] for hook in group['hooks']]
self.assertEqual(commands, [pointer['command'], user['command']])
removed = OPTIONS.merge_json(twice, {'hooks': {}})
self.assertEqual(removed['hooks']['SessionStart'], [{'hooks': [user]}])
def test_merge_toml_hook_groups_do_not_grow_and_user_hook_survives(self):
managed = ('[features]\nhooks = true\n\n[[hooks.SessionStart]]\n[[hooks.SessionStart.hooks]]\ntype = "command"\n'
'command = \'bash "/c/hooks/scripts/qmd-warm.sh"\'\n\n[[hooks.Stop]]\n[[hooks.Stop.hooks]]\ntype = "command"\n'
'command = \'bash "/c/hooks/flashbang.sh"\'\n')
user = '\n[[hooks.SessionStart]]\n[[hooks.SessionStart.hooks]]\ntype = "command"\ncommand = \'bash "/home/me/my-start.sh"\'\n'
text = managed + user
for _ in range(3):
text = OPTIONS.merge_toml(text, managed)
hooks = tomllib.loads(text)['hooks']
self.assertEqual(len(hooks['Stop']), 1)
self.assertEqual(len(hooks['SessionStart']), 2)
commands = [hook['command'] for group in hooks['SessionStart'] for hook in group['hooks']]
self.assertEqual(sorted(commands), ['bash "/c/hooks/scripts/qmd-warm.sh"', 'bash "/home/me/my-start.sh"'])
disabled = OPTIONS.merge_toml(text, '[features]\nhooks = true\n')
hooks = tomllib.loads(disabled)['hooks']
self.assertNotIn('Stop', hooks)
self.assertEqual([hook['command'] for group in hooks['SessionStart'] for hook in group['hooks']], ['bash "/home/me/my-start.sh"'])
def test_filter_preserves_other_claude_hooks_and_status(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'settings.json'
+8 -20
View File
@@ -260,27 +260,15 @@ class ManagedExtensionTests(unittest.TestCase):
self.manager(dry_run=True, summary=True).sync([plugin(), skill()], ['codex'], {'Example', 'Humanizer'})
self.assertEqual('EXTENSIONS: reconciliation complete (dry run)\n', output.getvalue())
def test_qmd_installs_once_as_a_cli_without_registering_mcp(self):
entry = dict(name='QMD', codex_method='qmd', codex_source='@tobilu/qmd', codex_skill='qmd', codex_plugin='-', codex_marketplace='-', claude_plugin='qmd@qmd', claude_marketplace='tobi/qmd')
self.manager().sync([entry], ['codex'], {'QMD'})
self.assertEqual([['npm', 'install', '--global', '@tobilu/qmd']], self.commands)
self.manager().sync([], ['codex'], set())
self.assertEqual(['npm', 'uninstall', '--global', '@tobilu/qmd'], self.commands[-1])
def test_manifest_has_no_qmd_extension(self):
rows = (root / 'adapters/plugins.tsv').read_text(encoding='utf-8').splitlines()
self.assertFalse(any(row.split(' ')[0] == 'QMD' for row in rows[1:]))
def test_qmd_installs_once_for_both_clients(self):
entry = dict(name='QMD', codex_method='qmd', codex_source='@tobilu/qmd', codex_skill='qmd', codex_plugin='-', codex_marketplace='-', claude_plugin='qmd@qmd', claude_marketplace='tobi/qmd')
self.manager().sync([entry], ['claude', 'codex'], {'QMD'})
self.assertEqual(1, self.commands.count(['npm', 'install', '--global', '@tobilu/qmd']))
self.assertIn(['claude', 'plugin', 'install', 'qmd@qmd', '--scope', 'user'], self.commands)
self.assertNotIn(['codex', 'mcp', 'add', 'qmd', '--', 'qmd', 'mcp'], self.commands)
def test_owned_qmd_uses_npm_update_instead_of_reinstalling(self):
entry = dict(name='QMD', codex_method='qmd', codex_source='@tobilu/qmd', codex_skill='qmd', codex_plugin='-', codex_marketplace='-', claude_plugin='qmd@qmd', claude_marketplace='tobi/qmd')
self.manager().sync([entry], ['codex'], {'QMD'})
self.commands.clear()
with patch.object(extensions.Manager, 'global_packages', return_value={'@tobilu/qmd'}):
self.manager(update=True).sync([entry], ['codex'], {'QMD'})
self.assertEqual([['npm', 'update', '--global', '@tobilu/qmd']], self.commands)
def test_legacy_qmd_record_still_validates(self):
(self.home / '.my-ai-configuration').mkdir()
(self.home / '.my-ai-configuration/extensions.json').write_text(json.dumps({'version': 1, 'resources': [
{'client': 'shared', 'name': 'QMD', 'kind': 'qmd', 'package': '@tobilu/qmd', 'clients': ['codex']}]}))
self.manager() # must not raise
def test_managed_mcporter_updates_when_requested(self):
self.manager().sync([mcporter()], ['codex'], {'MCPorter'})
+157
View File
@@ -0,0 +1,157 @@
import os
import hashlib
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import unittest
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / 'scripts/lib'))
from personal_instructions import CLIENT_FILES, START, refresh, render
class PersonalInstructionTests(unittest.TestCase):
def setUp(self):
directory = ROOT / '.ai-session' / 'test-personal-instructions'
directory.mkdir(parents=True, exist_ok=True)
self.temporary = tempfile.TemporaryDirectory(dir=directory)
self.addCleanup(self.temporary.cleanup)
self.home = Path(self.temporary.name)
self.overlays = self.home / '.my-ai-configuration/instructions'
self.overlays.mkdir(parents=True)
self.environment = dict(os.environ, TEST_HOME=str(self.home))
for name in ('TMP', 'TEMP', 'TMPDIR'):
self.environment[name] = str(self.home)
def run_command(self, *command):
result = subprocess.run(command, cwd=ROOT, env=self.environment, capture_output=True, text=True, timeout=30)
self.assertEqual(0, result.returncode, result.stdout + result.stderr)
return result.stdout
def test_missing_empty_and_whitespace_instructions_have_no_rule(self):
text = '# Global Instructions\n\nKeep me.\n'
for client in CLIENT_FILES:
self.assertEqual(text, render(text, self.home, client))
for content in ('', '\ufeff \n'):
(self.overlays / f'{client}.md').write_text(content, encoding='utf-8')
self.assertEqual(text, render(text, self.home, client))
def test_client_specific_rule_is_immediate_idempotent_and_removed_when_empty(self):
text = '# Global Instructions\n\nKeep me.\n'
overlay = self.overlays / 'codex.md'
overlay.write_text('Always use the release process.\n', encoding='utf-8')
rendered = render(text, self.home, 'codex')
self.assertIn('Read `~/.my-ai-configuration/instructions/codex.md` immediately', rendered)
self.assertIn('including after compaction', rendered)
self.assertEqual(rendered, render(rendered, self.home, 'codex'))
self.assertEqual(text, render(text, self.home, 'claude'))
overlay.unlink()
self.assertEqual(text, render(rendered, self.home, 'codex'))
def test_legacy_loader_removed_and_unrelated_bytes_preserved(self):
for client, name in CLIENT_FILES.items():
document = self.home / f'.{client}' / name
document.parent.mkdir()
reference = f'~/.my-ai-configuration/instructions/{client}.md'
legacy = f"Read `{reference}` first when present; it is the user's direct instruction. System and developer requirements still apply."
original = '\ufeff# Global Instructions\r\n\r\nKeep ĂĽ.\r\n'.encode('utf-8')
document.write_bytes(original.replace(b'Keep', (legacy + '\r\n\r\nKeep').encode()))
refresh(self.home, client)
self.assertEqual(original, document.read_bytes())
(self.overlays / f'{client}.md').write_text('Use English.\n', encoding='utf-8')
refresh(self.home, client)
(self.overlays / f'{client}.md').unlink()
refresh(self.home, client)
self.assertEqual(original, document.read_bytes())
def test_refresh_does_not_create_an_uninstalled_client(self):
(self.overlays / 'codex.md').write_text('Use English.\n', encoding='utf-8')
refresh(self.home, 'codex')
self.assertFalse((self.home / '.codex').exists())
def test_refresh_updates_only_matching_manifest_hashes(self):
document = self.home / '.codex/AGENTS.md'
document.parent.mkdir()
original = b'# Global Instructions\n'
document.write_bytes(original)
manifest = document.parent / '.ai-config-manifest.tsv'
manifest.write_text(f'path\tsha256\nAGENTS.md\t{hashlib.sha256(original).hexdigest()}\n', encoding='utf-8')
(self.overlays / 'codex.md').write_text('Use English.\n', encoding='utf-8')
refresh(self.home, 'codex')
self.assertIn(hashlib.sha256(document.read_bytes()).hexdigest(), manifest.read_text())
unchanged_manifest = manifest.read_bytes()
document.write_bytes(document.read_bytes() + b'User edit.\n')
(self.overlays / 'codex.md').unlink()
refresh(self.home, 'codex')
self.assertEqual(unchanged_manifest, manifest.read_bytes())
self.assertEqual(original + b'User edit.\n', document.read_bytes())
def test_linked_client_directory_is_rejected(self):
elsewhere = self.home / 'elsewhere'
elsewhere.mkdir()
document = elsewhere / 'AGENTS.md'
document.write_text('Keep me.\n', encoding='utf-8')
try:
(self.home / '.codex').symlink_to(elsewhere, target_is_directory=True)
except OSError:
self.skipTest('Directory symlinks are unavailable.')
with self.assertRaises(ValueError):
refresh(self.home, 'codex')
self.assertEqual('Keep me.\n', document.read_text())
def test_add_remove_and_managed_sync_in_both_shells(self):
powershell = shutil.which('powershell') or shutil.which('pwsh')
bash = Path('C:/Program Files/Git/bin/bash.exe') if os.name == 'nt' else shutil.which('bash')
for shell, executable in (('powershell', powershell), ('bash', bash)):
if not executable or not Path(executable).is_file():
continue
with self.subTest(shell=shell):
source = self.home / f'source-{shell}'
source.mkdir()
for name in CLIENT_FILES.values():
(source / name).write_text('# Global Instructions\n\nKeep me.\n', encoding='utf-8')
self.environment['TEST_SOURCE'] = str(source)
def sync(client, dry=False):
self.environment['TEST_CLIENT'] = client
if shell == 'powershell':
return self.run_command(str(executable), '-NoProfile', '-Command',
". ./scripts/lib/manifest.ps1; Sync-ManagedDestination -Source $env:TEST_SOURCE "
"-Destination (Join-Path $env:TEST_HOME ('.' + $env:TEST_CLIENT)) -Stamp test -Summary "
+ ('-DryRun' if dry else ''))
return self.run_command(str(executable), '-c',
'export PATH="/usr/bin:$PATH"; python3() { python "$@"; }; '
'. ./scripts/lib/manifest.sh; sync_managed_destination "$TEST_SOURCE" '
'"$TEST_HOME/.$TEST_CLIENT" test root bash pwsh ' + ('true' if dry else 'false') + ' true')
def change(action):
if shell == 'powershell':
self.run_command(str(executable), '-NoProfile', '-File', str(ROOT / f'scripts/commands/{action}-instruction.ps1'),
'-Client', 'Both', '-Instruction', 'Use English.', '-HomePath', str(self.home))
else:
self.run_command(str(executable), '-c',
'export PATH="/usr/bin:$PATH"; source scripts/commands/' + action + '-instruction.sh '
'--client both --instruction "Use English." --home "$TEST_HOME"')
for client, name in CLIENT_FILES.items():
sync(client)
self.assertNotIn(START, (self.home / f'.{client}' / name).read_text(encoding='utf-8-sig'))
change('add')
for client, name in CLIENT_FILES.items():
document = self.home / f'.{client}' / name
self.assertEqual(1, document.read_text(encoding='utf-8-sig').count(START))
before = document.read_bytes()
sync(client, dry=True)
self.assertEqual(before, document.read_bytes())
sync(client)
self.assertEqual(1, document.read_text(encoding='utf-8-sig').count(START))
change('remove')
for client, name in CLIENT_FILES.items():
self.assertNotIn(START, (self.home / f'.{client}' / name).read_text(encoding='utf-8-sig'))
if __name__ == '__main__':
unittest.main()
+31
View File
@@ -0,0 +1,31 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { passes, fixtureDocuments, describeDevices, embedThroughput } from '../../shared/qmd/qmd-benchmark.mjs';
test('both thresholds must pass', () => {
assert.equal(passes({ querySeconds: 5, chunksPerSecond: 20 }, {}), true);
assert.equal(passes({ querySeconds: 5.01, chunksPerSecond: 50 }, {}), false);
assert.equal(passes({ querySeconds: 1, chunksPerSecond: 19.9 }, {}), false);
});
test('thresholds are overridable', () => {
assert.equal(passes({ querySeconds: 8, chunksPerSecond: 5 }, { QMD_BENCHMARK_MAX_QUERY_SECONDS: '10', QMD_BENCHMARK_MIN_CHUNKS_PER_SECOND: '4' }), true);
});
test('device description lists every GPU and the thread count', () => {
assert.equal(describeDevices({ gpu: 'cuda', devices: ['RTX 4090', 'RTX 3090'], threads: 64 }), 'CUDA: RTX 4090, RTX 3090; CPU threads: 64');
assert.equal(describeDevices({ gpu: false, devices: [], threads: 16 }), 'no GPU; CPU threads: 16');
});
test('fixture is deterministic and large enough for throughput', () => {
const a = fixtureDocuments(100);
assert.equal(a.length, 100);
assert.deepEqual(a, fixtureDocuments(100));
assert.ok(a.every((d) => d.text.length > 2500));
});
test('embed throughput rejects errors and empty runs', () => {
assert.deepEqual(embedThroughput({ chunksEmbedded: 200, durationMs: 4000, errors: 0 }), { chunksPerSecond: 50, ok: true });
assert.equal(embedThroughput({ chunksEmbedded: 200, durationMs: 4000, errors: 1 }).ok, false);
assert.equal(embedThroughput({ chunksEmbedded: 0, durationMs: 4000, errors: 0 }).ok, false);
});
+217
View File
@@ -0,0 +1,217 @@
// scripts/tests/test-qmd-daemon.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { existsSync, mkdtempSync, rmSync, utimesSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { acquireStartLock, createServer, gitFingerprint, ignoreGlobs, keepModelsLoaded } from '../../shared/qmd/qmd-daemon.mjs';
import { collectionName } from '../../shared/qmd/qmd-lib.mjs';
const defaultHits = () => [{ displayPath: `${collectionName('/r')}/src/a.ts`, body: 'x\nretry here', bestChunkPos: 2, bestChunk: 'retry here', score: 0.91 }];
function fakeStore(updateGate, hits = defaultHits) {
const calls = [];
return {
calls,
listCollections: async () => [{ name: 'notes', includeByDefault: true }, ...calls.filter((c) => c[0] === 'add').map((c) => ({ name: c[1] }))],
getDefaultCollectionNames: async () => ['notes', 'repo-aaaaaaaaaaaa', 'repo-notes'],
addCollection: async (name, opts) => calls.push(['add', name, opts]),
update: async (opts) => { calls.push(['update', opts]); await updateGate; return { indexed: 1 }; },
embed: async (opts) => { calls.push(['embed', opts]); return { chunksEmbedded: 1 }; },
search: async (opts) => { calls.push(['search', opts]); return hits(); },
};
}
async function start(t, overrides = {}, { updateGate, hits } = {}) {
const store = fakeStore(updateGate, hits);
let clock = 0;
const idle = [];
const daemon = createServer({
store, token: 't0k', idleMs: 1000, now: () => clock, onIdle: () => idle.push(clock),
extractSnippet: () => ({ line: 2, snippet: 'retry here' }),
git: () => 'fingerprint-1', excludeCollection: async () => {}, recordCollection: () => {},
ignored: () => [], persistIgnore: async () => {}, ...overrides,
});
await new Promise((done) => daemon.server.listen(0, '127.0.0.1', done));
t.after(() => { daemon.server.closeAllConnections(); daemon.server.close(); });
const port = daemon.server.address().port;
const call = (path, body, token = 't0k') => fetch(`http://127.0.0.1:${port}${path}`, {
method: body === undefined ? 'GET' : 'POST', headers: { connection: 'close', 'x-qmd-token': token, 'content-type': 'application/json' },
body: body === undefined ? undefined : JSON.stringify(body) });
const raw = (path, body) => fetch(`http://127.0.0.1:${port}${path}`, { method: 'POST', headers: { connection: 'close', 'x-qmd-token': 't0k' }, body });
return { daemon, store, call, raw, setClock: (v) => { clock = v; }, idle };
}
const wait = (ms) => new Promise((done) => setTimeout(done, ms));
const gateOf = () => { let release; const gate = new Promise((done) => { release = done; }); return { gate, release }; };
test('rejects requests without the token', async (t) => {
const s = await start(t);
assert.equal((await s.call('/health', undefined, 'wrong')).status, 403);
});
test('health identifies the service', async (t) => {
const s = await start(t);
assert.equal((await (await s.call('/health')).json()).service, 'ai-config-qmd');
});
test('search adds the repo collection once, refreshes on change, and filters other repos', async (t) => {
const s = await start(t);
const body = await (await s.call('/search', { repo: '/r', query: 'where are retries', limit: 3 })).json();
assert.deepEqual(body.results, [{ path: 'src/a.ts', line: 2, score: 0.91, snippet: 'retry here' }]);
const search = s.store.calls.find((c) => c[0] === 'search')[1];
assert.equal(search.query, 'where are retries');
assert.ok(search.collections.includes('notes'));
assert.ok(!search.collections.includes('repo-aaaaaaaaaaaa'));
assert.ok(search.collections.includes('repo-notes'), 'only hash-named repo collections are filtered');
assert.equal(search.collections.filter((n) => /^repo-[0-9a-f]{12}$/.test(n)).length, 1);
assert.equal(s.store.calls.filter((c) => c[0] === 'add').length, 1);
assert.equal(s.store.calls.find((c) => c[0] === 'add')[2].path, resolve('/r'));
await s.call('/search', { repo: '/r', query: 'again', limit: 3 });
assert.equal(s.store.calls.filter((c) => c[0] === 'update').length, 1, 'unchanged fingerprint skips update');
});
test('rejects a missing query', async (t) => {
const s = await start(t);
assert.equal((await s.call('/search', { repo: '/r' })).status, 400);
});
test('idle check calls onIdle only after the idle window', async (t) => {
const s = await start(t);
s.setClock(500); s.daemon.checkIdle(); assert.deepEqual(s.idle, []);
s.setClock(1500); s.daemon.checkIdle(); assert.deepEqual(s.idle, [1500]);
});
test('a changed fingerprint triggers update and embed again', async (t) => {
let fingerprint = 'one';
const s = await start(t, { git: () => fingerprint });
await s.call('/search', { repo: '/r', query: 'q' });
fingerprint = 'two';
await s.call('/search', { repo: '/r', query: 'q' });
assert.equal(s.store.calls.filter((c) => c[0] === 'update').length, 2);
assert.equal(s.store.calls.filter((c) => c[0] === 'embed').length, 2);
});
test('fingerprint changes when an already dirty file is edited again', (t) => {
const dir = mkdtempSync(join(tmpdir(), 'qmd-fp-'));
t.after(() => rmSync(dir, { recursive: true, force: true }));
const git = (...args) => execFileSync('git', ['-C', dir, '-c', 'user.name=t', '-c', 'user.email=t@t', ...args], { stdio: 'ignore' });
git('init', '-q');
writeFileSync(join(dir, 'a.ts'), 'one');
git('add', '.');
git('commit', '-qm', 'init');
writeFileSync(join(dir, 'a.ts'), 'two');
const first = gitFingerprint(dir);
writeFileSync(join(dir, 'a.ts'), 'three!');
utimesSync(join(dir, 'a.ts'), new Date(), new Date(Date.now() + 5000));
assert.notEqual(gitFingerprint(dir), first);
});
test('a failing exclude does not stop the search and is retried', async (t) => {
let attempts = 0;
const recorded = [];
const s = await start(t, {
excludeCollection: async () => { attempts++; if (attempts === 1) throw new Error('boom'); },
recordCollection: (name) => recorded.push(name),
});
const first = await s.call('/search', { repo: '/r', query: 'q' });
assert.equal(first.status, 200);
assert.equal((await first.json()).results.length, 1);
assert.deepEqual(recorded, []);
await s.call('/search', { repo: '/r', query: 'q' });
assert.equal(attempts, 2);
assert.equal(recorded.length, 1);
});
test('concurrent searches share one refresh', async (t) => {
const { gate, release } = gateOf();
const s = await start(t, {}, { updateGate: gate });
const both = Promise.all([s.call('/search', { repo: '/r', query: 'a' }), s.call('/search', { repo: '/r', query: 'b' })]);
await wait(100);
release();
await both;
assert.equal(s.store.calls.filter((c) => c[0] === 'update').length, 1);
});
test('idle check waits for in-flight requests', async (t) => {
const { gate, release } = gateOf();
const s = await start(t, {}, { updateGate: gate });
const pending = s.call('/search', { repo: '/r', query: 'a' });
await wait(100);
s.setClock(5000); s.daemon.checkIdle(); assert.deepEqual(s.idle, []);
release();
await pending;
});
test('malformed or non-object bodies give 400 and limit is clamped', async (t) => {
const s = await start(t);
assert.equal((await s.raw('/search', '{nope')).status, 400);
assert.equal((await s.raw('/search', '[1]')).status, 400);
assert.equal((await s.raw('/search', 'null')).status, 400);
await s.call('/search', { repo: '/r', query: 'q', limit: 9999 });
await s.call('/search', { repo: '/r', query: 'q', limit: 'x' });
const limits = s.store.calls.filter((c) => c[0] === 'search').map((c) => c[1].limit);
assert.deepEqual(limits, [20, 5]);
});
test('non-repo hits become qmd:// URIs', async (t) => {
const s = await start(t, {}, { hits: () => [{ displayPath: 'notes/todo.md', body: 'x', bestChunkPos: 0, bestChunk: 'x', score: 0.5 }] });
const body = await (await s.call('/search', { repo: '/r', query: 'q' })).json();
assert.equal(body.results[0].path, 'qmd://notes/todo.md');
});
test('refresh, stop, 404 and 403 on POST routes', async (t) => {
const s = await start(t);
assert.deepEqual(await (await s.call('/refresh', { repo: '/r' })).json(), { collection: collectionName('/r'), updated: true });
assert.equal((await s.call('/nothing', { repo: '/r' })).status, 404);
assert.equal((await s.call('/search', { repo: '/r', query: 'q' }, 'wrong')).status, 403);
assert.deepEqual(await (await s.call('/stop', {})).json(), { stopping: true });
assert.equal(s.idle.length, 1);
});
test('keepModelsLoaded disables QMD unloading models after 5 idle minutes', () => {
const llm = { inactivityTimeoutMs: 5 * 60 * 1000, disposeModelsOnInactivity: true };
keepModelsLoaded({ internal: { llm } });
assert.deepEqual(llm, { inactivityTimeoutMs: 0, disposeModelsOnInactivity: false });
});
test('ignoreGlobs turns git ignored paths into escaped QMD ignore globs', () => {
const escape = (path) => path.replace(/[()[\]]/g, '\\$&');
assert.deepEqual(ignoreGlobs('dist/\0a b.ts\0out (1)/\0x[1].md\0', escape), ['dist/**', 'a b.ts', 'out \\(1\\)/**', 'x\\[1\\].md']);
assert.deepEqual(ignoreGlobs('', escape), []);
});
test('git ignored paths reach addCollection and are persisted only when they change', async (t) => {
let ignore = ['dist/**'];
let fingerprint = 'one';
const persisted = [];
const s = await start(t, { ignored: () => ignore, git: () => fingerprint, persistIgnore: async (name, list) => persisted.push([name, list]) });
await s.call('/search', { repo: '/r', query: 'q' });
assert.deepEqual(s.store.calls.find((c) => c[0] === 'add')[2].ignore, ['dist/**']);
assert.deepEqual(persisted, [[collectionName('/r'), ['dist/**']]]);
const order = s.store.calls.map((c) => c[0]);
assert.ok(order.indexOf('add') < order.indexOf('update'));
fingerprint = 'two';
await s.call('/search', { repo: '/r', query: 'q' });
assert.equal(persisted.length, 1, 'an unchanged list is not written again');
ignore = ['dist/**', 'tmp/**'];
fingerprint = 'three';
await s.call('/search', { repo: '/r', query: 'q' });
assert.deepEqual(persisted[1], [collectionName('/r'), ['dist/**', 'tmp/**']]);
assert.equal(s.store.calls.filter((c) => c[0] === 'update').length, 3);
});
test('the start lock admits one launcher and is taken over after 2 minutes', (t) => {
const dir = mkdtempSync(join(tmpdir(), 'qmd-lock-'));
t.after(() => rmSync(dir, { recursive: true, force: true }));
const lock = join(dir, 'daemon.lock');
const release = acquireStartLock(lock);
assert.equal(typeof release, 'function');
assert.equal(acquireStartLock(lock), null, 'a concurrent launch loses');
release();
assert.equal(existsSync(lock), false);
assert.equal(typeof acquireStartLock(lock), 'function');
assert.equal(acquireStartLock(lock, () => Date.now() + 60_000), null, 'a fresh lock holds');
assert.equal(typeof acquireStartLock(lock, () => Date.now() + 121_000), 'function', 'a stale lock is taken over');
});
+27
View File
@@ -0,0 +1,27 @@
// Real QMD, real models; run manually on a machine capable of downloading and running the models.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { mkdtempSync, mkdirSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { runQmdCli } from '../../shared/qmd/qmd-lib.mjs';
const search =resolve('shared/qmd/qmd-search.mjs');
test('wrapper finds the retry logic through the daemon and stops it', { timeout: 1_800_000 }, () => {
const home = mkdtempSync(join(tmpdir(), 'qmd-it-home-'));
const repo = join(home, 'repo');
mkdirSync(join(repo, 'src'), { recursive: true });
writeFileSync(join(repo, 'src', 'upload.ts'), 'export async function sendWithRetry(send) {\n for (let attempt = 1; attempt <= 3; attempt++) {\n try { return await send(); } catch (e) { if (!e.transient || attempt === 3) throw e; }\n }\n}\n');
writeFileSync(join(repo, 'src', 'price.ts'), 'export const formatCents = (c) => (c / 100).toFixed(2);\n');
execFileSync('git', ['init', '-q'], { cwd: repo });
execFileSync('git', ['add', '-A'], { cwd: repo });
const env = { ...process.env, HOME: home, USERPROFILE: home, XDG_CONFIG_HOME: join(home, '.config'), XDG_CACHE_HOME: process.env.QMD_TEST_CACHE || join(home, '.cache'), QMD_FORCE_CPU: '1' };
execFileSync('node', ['shared/qmd/qmd-config.mjs', 'set-models'], { env });
runQmdCli(['pull'], { env, timeout: 1_700_000, stdio: 'inherit' }); // download models first so the search only measures indexing and query
const results =JSON.parse(execFileSync('node', [search, '--root', repo, '--query', 'Where are failed uploads retried after temporary errors?'], { env, encoding: 'utf8', timeout: 1_700_000 }));
assert.equal(results[0].path.replaceAll('\\', '/'), 'src/upload.ts');
execFileSync('node', [search, '--stop'], { env });
});
+137
View File
@@ -0,0 +1,137 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { cpSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
import { createRequire } from 'node:module';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import * as YAML from './fixtures/yaml-shim.mjs';
import { collectionName, deviceEnv, idleMinutes, qmdConfigPath, qmdPackageDir, writePrivateJson, writeTextAtomic, readJson, runQmdCli, MODELS } from '../../shared/qmd/qmd-lib.mjs';
import { setModels, unsetModels, previousModels, restoreModels, setCollectionIgnore } from '../../shared/qmd/qmd-config.mjs';
for (const layout of ['nested', 'hoisted']) {
test(`config CLI and YAML shim load ${layout} dependencies`, (t) => {
const root = mkdtempSync(join(tmpdir(), 'qmd dependencies '));
t.after(() => rmSync(root, { recursive: true, force: true }));
const pkg = join(root, 'node_modules', '@tobilu', 'qmd');
mkdirSync(pkg, { recursive: true });
const require = createRequire(join(qmdPackageDir(), 'package.json'));
const source = dirname(require.resolve('yaml/package.json'));
cpSync(source, join(layout === 'nested' ? pkg : root, 'node_modules', 'yaml'), { recursive: true });
const env = { ...process.env, QMD_PACKAGE_DIR: pkg, QMD_CONFIG_DIR: join(root, 'config') };
const output = execFileSync(process.execPath, [fileURLToPath(new URL('../../shared/qmd/qmd-config.mjs', import.meta.url)), 'set-models'], { env, encoding: 'utf8' });
assert.deepEqual(JSON.parse(output), { previous: null });
assert.deepEqual(YAML.default.parse(readFileSync(join(env.QMD_CONFIG_DIR, 'index.yml'), 'utf8')).models, MODELS);
const shim = new URL('./fixtures/yaml-shim.mjs', import.meta.url).href;
const parsed = execFileSync(process.execPath, ['--input-type=module', '-e', `import YAML from ${JSON.stringify(shim)}; console.log(JSON.stringify(YAML.parse('answer: 42')));`], { env, encoding: 'utf8' });
assert.deepEqual(JSON.parse(parsed), { answer: 42 });
});
}
test('collection name is stable, prefixed, and path-safe', () => {
const a = collectionName('/tmp/My Repo/ĂĽ');
assert.match(a, /^repo-[0-9a-f]{12}$/);
assert.equal(a, collectionName('/tmp/My Repo/ĂĽ'));
assert.notEqual(a, collectionName('/tmp/My Repo/u'));
});
test('device env forces CPU or enables GPU auto', () => {
assert.equal(deviceEnv('cpu', { QMD_LLAMA_GPU: 'cuda' }).QMD_FORCE_CPU, '1');
assert.equal(deviceEnv('cpu', { QMD_LLAMA_GPU: 'cuda' }).QMD_LLAMA_GPU, undefined);
assert.equal(deviceEnv('gpu', { QMD_FORCE_CPU: '1' }).QMD_LLAMA_GPU, 'auto');
assert.equal(deviceEnv('gpu', { QMD_FORCE_CPU: '1' }).QMD_FORCE_CPU, undefined);
});
test('device env never narrows GPUs or threads', () => {
for (const device of ['gpu', 'cpu']) {
const env = deviceEnv(device, {});
for (const key of ['CUDA_VISIBLE_DEVICES', 'GGML_VK_VISIBLE_DEVICES', 'HIP_VISIBLE_DEVICES', 'OMP_NUM_THREADS']) assert.equal(env[key], undefined);
}
});
test('idle minutes default to 30 and reject nonsense', () => {
assert.equal(idleMinutes({}), 30);
assert.equal(idleMinutes({ QMD_IDLE_MINUTES: '5' }), 5);
assert.equal(idleMinutes({ QMD_IDLE_MINUTES: '-1' }), 30);
});
test('config path honors QMD_CONFIG_DIR then XDG_CONFIG_HOME', () => {
assert.equal(qmdConfigPath({ QMD_CONFIG_DIR: '/c' }, '/h'), join('/c', 'index.yml'));
assert.equal(qmdConfigPath({ XDG_CONFIG_HOME: '/x' }, '/h'), join('/x', 'qmd', 'index.yml'));
assert.equal(qmdConfigPath({}, '/h'), join('/h', '.config', 'qmd', 'index.yml'));
});
test('private json round-trips', () => {
const file = join(mkdtempSync(join(tmpdir(), 'qmd-lib-')), 'a', 'b.json');
writePrivateJson(file, { x: 1 });
assert.deepEqual(readJson(file, null), { x: 1 });
assert.equal(readJson(file + '.missing', 'fallback'), 'fallback');
});
test('setModels keeps collections and comments, unsetModels removes only models', () => {
const original = '# mine\ncollections:\n notes:\n path: /n # keep\n';
const withModels = setModels(original, YAML);
assert.match(withModels, /# mine/);
assert.match(withModels, /path: \/n # keep/);
assert.match(withModels, new RegExp(MODELS.embed.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')));
const restored = unsetModels(withModels, YAML);
assert.doesNotMatch(restored, /models:/);
assert.match(restored, /path: \/n # keep/);
});
test('setModels on an empty file creates only the models block', () => {
const text = setModels('', YAML);
assert.match(text, /^models:/m);
});
test('runQmdCli runs the CLI entry of QMD_PACKAGE_DIR without a shell', () => {
const dir = mkdtempSync(join(tmpdir(), 'qmd-pkg-'));
mkdirSync(join(dir, 'dist', 'cli'), { recursive: true });
writeFileSync(join(dir, 'dist', 'cli', 'qmd.js'), 'console.log(JSON.stringify(process.argv.slice(2)));\n');
const previous = process.env.QMD_PACKAGE_DIR;
process.env.QMD_PACKAGE_DIR = dir;
try {
assert.deepEqual(JSON.parse(runQmdCli(['search', 'a "b" & c'])), ['search', 'a "b" & c']);
} finally {
if (previous === undefined) delete process.env.QMD_PACKAGE_DIR; else process.env.QMD_PACKAGE_DIR = previous;
}
});
test('previousModels captures a user models block, null when absent', () => {
assert.deepEqual(previousModels('models:\n embed: mine\ncollections: {}\n', YAML), { embed: 'mine' });
assert.equal(previousModels('models: custom\n', YAML), 'custom');
assert.deepEqual(previousModels('models: [a, b]\n', YAML), ['a', 'b']);
assert.equal(previousModels('collections: {}\n', YAML), null);
assert.equal(previousModels('', YAML), null);
});
test('restoreModels writes the previous block back and keeps comments and other keys', () => {
const original = '# keep\ncollections:\n a: {}\nmodels:\n embed: mine\n';
const replaced = setModels(original, YAML);
const restored = restoreModels(replaced, YAML, { embed: 'mine' });
assert.match(restored, /# keep/);
assert.match(restored, /embed: mine/);
assert.match(restored, /collections:/);
assert.doesNotMatch(restored, /Qwen/);
});
test('writeTextAtomic replaces the file through a temp file in the same directory', () => {
const dir = join(mkdtempSync(join(tmpdir(), 'qmd-lib-')), 'cfg');
const file = join(dir, 'index.yml');
writeTextAtomic(file, 'a: 1\n');
writeTextAtomic(file, 'b: 2\n');
assert.equal(readFileSync(file, 'utf8'), 'b: 2\n');
assert.deepEqual(readdirSync(dir), ['index.yml']);
});
test('setCollectionIgnore writes the ignore list of one collection and keeps the rest', () => {
const original = '# mine\ncollections:\n notes:\n path: /n # keep\n repo-aaaaaaaaaaaa:\n path: /r\n includeByDefault: false\n';
const withIgnore = setCollectionIgnore(original, YAML, 'repo-aaaaaaaaaaaa', ['dist/**', 'a.ts']);
assert.match(withIgnore, /# mine/);
assert.match(withIgnore, /path: \/n # keep/);
assert.deepEqual(YAML.default.parse(withIgnore).collections['repo-aaaaaaaaaaaa'], { path: '/r', includeByDefault: false, ignore: ['dist/**', 'a.ts'] });
const cleared = setCollectionIgnore(withIgnore, YAML, 'repo-aaaaaaaaaaaa', []);
assert.deepEqual(YAML.default.parse(cleared).collections['repo-aaaaaaaaaaaa'], { path: '/r', includeByDefault: false });
assert.equal(setCollectionIgnore(original, YAML, 'repo-bbbbbbbbbbbb', ['x']), null, 'an unknown collection is not created');
});
+111
View File
@@ -0,0 +1,111 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { EventEmitter } from 'node:events';
import { mkdtempSync, rmSync, writeFileSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { callDaemon, runSearch, startDaemon, toCliResults } from '../../shared/qmd/qmd-search.mjs';
import { collectionName } from '../../shared/qmd/qmd-lib.mjs';
const hit = [{ path: 'src/a.ts', line: 3, score: 0.9, snippet: 's' }];
test('uses the daemon when it answers', async () => {
const results = await runSearch({ root: '/My Repo/ĂĽ', query: 'q', topK: 5,
daemon: async (path, body) => { assert.equal(path, '/search'); assert.equal(body.repo, '/My Repo/ĂĽ'); return { results: hit }; },
startDaemon: async () => assert.fail('must not start'), cli: async () => assert.fail('must not fall back') });
assert.deepEqual(results, hit);
});
test('starts the daemon once when the first call fails, then retries', async () => {
let calls = 0; let started = 0;
const results = await runSearch({ root: '/r', query: 'q', topK: 5,
daemon: async () => { calls += 1; if (calls === 1) throw Object.assign(new Error('ECONNREFUSED'), { notRunning: true }); return { results: hit }; },
startDaemon: async () => { started += 1; }, cli: async () => assert.fail('must not fall back') });
assert.deepEqual(results, hit);
assert.equal(started, 1);
});
test('falls back to the CLI when the daemon cannot start', async () => {
const results = await runSearch({ root: '/r', query: 'q', topK: 2,
daemon: async () => { throw Object.assign(new Error('stale'), { notRunning: true }); }, startDaemon: async () => { throw new Error('spawn failed'); },
cli: async (args) => { assert.ok(args.includes('--json')); return [{ file: `qmd://${collectionName('/r')}/src/b.ts`, line: 7, score: 0.5, snippet: 'b' }]; } });
assert.deepEqual(results, [{ path: 'src/b.ts', line: 7, score: 0.5, snippet: 'b' }]);
});
test('CLI results keep foreign collections as qmd URIs', () => {
assert.deepEqual(toCliResults([{ file: 'qmd://notes/x.md', line: 1, score: 0.4, snippet: 'n' }], 'repo-x'),
[{ path: 'qmd://notes/x.md', line: 1, score: 0.4, snippet: 'n' }]);
});
const cliHit = (root) => async () => [{ file: `qmd://${collectionName(root)}/src/b.ts`, line: 7, score: 0.5, snippet: 'b' }];
const cliExpected = [{ path: 'src/b.ts', line: 7, score: 0.5, snippet: 'b' }];
test('a live daemon that answers with an error does not trigger a second daemon', async () => {
const results = await runSearch({ root: '/r', query: 'q', topK: 2,
daemon: async () => { throw new Error('daemon answered 500'); },
startDaemon: async () => assert.fail('must not start'), cli: cliHit('/r') });
assert.deepEqual(results, cliExpected);
});
test('a daemon reply without results falls back to the CLI', async () => {
const results = await runSearch({ root: '/r', query: 'q', topK: 2,
daemon: async () => ({}), startDaemon: async () => assert.fail('must not start'), cli: cliHit('/r') });
assert.deepEqual(results, cliExpected);
});
test('a daemon failure after a successful start falls back to the CLI', async () => {
let calls = 0;
const results = await runSearch({ root: '/r', query: 'q', topK: 2,
daemon: async () => { calls += 1; throw calls === 1 ? Object.assign(new Error('down'), { notRunning: true }) : new Error('daemon answered 500'); },
startDaemon: async () => {}, cli: cliHit('/r') });
assert.deepEqual(results, cliExpected);
assert.equal(calls, 2);
});
test('startDaemon rejects quickly when the child exits immediately', async () => {
const started = Date.now();
await assert.rejects(startDaemon({ spawnChild: () => {
const child = new EventEmitter(); child.unref = () => {}; child.pid = 1;
setImmediate(() => child.emit('exit', 1));
return child;
} }), /exited with code 1/);
assert.ok(Date.now() - started < 5000);
});
test('startDaemon rejects when the child cannot be spawned', async () => {
await assert.rejects(startDaemon({ spawnChild: () => {
const child = new EventEmitter(); child.unref = () => {}; child.pid = 1;
setImmediate(() => child.emit('error', new Error('spawn failed')));
return child;
} }), /spawn failed/);
});
function tempDaemonFile(t, value) {
const dir = mkdtempSync(join(tmpdir(), 'qmd-search-'));
t.after(() => rmSync(dir, { recursive: true, force: true }));
const file = join(dir, 'daemon.json');
if (value) writeFileSync(file, JSON.stringify(value));
return file;
}
test('startDaemon resolves when its child loses the start race to a running daemon', async (t) => {
const file = tempDaemonFile(t, { port: 4000, token: 'other', pid: 999 });
await startDaemon({ file, spawnChild: () => {
const child = new EventEmitter(); child.unref = () => {}; child.pid = 1;
setImmediate(() => child.emit('exit', 0));
return child;
} });
});
test('a 403 means a stale token: daemon.json is deleted and one new daemon is started', async (t) => {
const file = tempDaemonFile(t, { port: 4000, token: 'stale', pid: 999 });
const fetchImpl = async (url, options) => (options.headers['x-qmd-token'] === 'fresh'
? { ok: true, status: 200, json: async () => ({ results: hit }) } : { ok: false, status: 403 });
let started = 0;
const results = await runSearch({ root: '/r', query: 'q', topK: 5,
daemon: (path, body) => callDaemon(path, body, { file, fetchImpl }),
startDaemon: async () => { started += 1; assert.equal(existsSync(file), false); writeFileSync(file, JSON.stringify({ port: 4001, token: 'fresh', pid: 998 })); },
cli: async () => assert.fail('must not fall back') });
assert.deepEqual(results, hit);
assert.equal(started, 1);
});
+362
View File
@@ -0,0 +1,362 @@
import importlib.util
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tempfile
import unittest
from unittest import mock
ROOT = Path(__file__).resolve().parents[2]
spec = importlib.util.spec_from_file_location('qmd_setup', ROOT / 'scripts/lib/qmd.py')
qmd = importlib.util.module_from_spec(spec)
spec.loader.exec_module(qmd)
class FakeRunner:
def __init__(self, installed=False, gpu='cuda', gpu_result=None, cpu_result=None, previous=None, fail_pull=False, list_output=None,
node='v22.4.0', fail=(), on_call=None):
self.calls, self.installed, self.gpu = [], installed, gpu
self.previous, self.fail_pull, self.list_output = previous, fail_pull, list_output
self.node, self.fail, self.on_call = node, fail, on_call
self.results = {'gpu': gpu_result, 'cpu': cpu_result}
def __call__(self, arguments, env=None, timeout=None, check=True):
self.calls.append(list(arguments))
joined = ' '.join(arguments)
if self.on_call:
self.on_call(arguments)
if any(part in joined for part in self.fail):
raise ValueError(f'{joined} failed')
if arguments[:2] == ['node', '--version']:
return self.node
if arguments[-1:] == ['set-models']:
return json.dumps({'previous': self.previous}) + chr(10)
if self.fail_pull and arguments[:2] == ['qmd', 'pull']:
raise ValueError('pull failed')
if 'npm list' in joined and self.list_output is not None:
return self.list_output
if 'npm list' in joined:
return json.dumps({'dependencies': {'@tobilu/qmd': {}} if self.installed else {}})
if 'npm install' in joined:
self.installed = True
if 'npm uninstall' in joined:
self.installed = False
if '--detect' in joined:
return json.dumps({'gpu': self.gpu, 'devices': ['GPU 0', 'GPU 1'] if self.gpu else [], 'threads': 32})
if '--device' in joined:
device = arguments[arguments.index('--device') + 1]
return json.dumps(self.results[device])
return ''
class SetupTest(unittest.TestCase):
def setUp(self):
self.home = Path(tempfile.mkdtemp())
self.addCleanup(shutil.rmtree, self.home, ignore_errors=True)
environment = mock.patch.dict(os.environ)
environment.start()
self.addCleanup(environment.stop)
os.environ.pop('XDG_CACHE_HOME', None)
(self.home / '.cache/qmd/models').mkdir(parents=True)
def setup(self, runner, dry_run=False):
return qmd.Setup(self.home, ROOT, runner, dry_run=dry_run, summary=True)
def state(self):
return json.loads((self.home / '.my-ai-configuration/qmd.json').read_text(encoding='utf-8'))
def test_decide_prefers_gpu_then_cpu(self):
self.assertEqual(qmd.decide([{'device': 'gpu', 'passed': True}, {'device': 'cpu', 'passed': True}]), 'gpu')
self.assertEqual(qmd.decide([{'device': 'gpu', 'passed': False}, {'device': 'cpu', 'passed': True}]), 'cpu')
self.assertIsNone(qmd.decide([{'device': 'gpu', 'passed': False}, {'device': 'cpu', 'passed': False}]))
self.assertIsNone(qmd.decide([]))
def test_node_version(self):
self.assertTrue(qmd.node_version_ok('v22.0.0'))
self.assertFalse(qmd.node_version_ok('v20.11.1'))
self.assertFalse(qmd.node_version_ok('garbage'))
def test_enable_installs_package_scripts_and_records_ownership(self):
runner = FakeRunner()
self.setup(runner).enable(['claude'])
state = self.state()
self.assertTrue(state['owned_package'])
self.assertEqual(state['device'], 'gpu')
self.assertTrue(state['models_block'])
self.assertTrue((self.home / '.my-ai-configuration/qmd/qmd-search.mjs').is_file())
self.assertIn(['npm', 'install', '--global', '@tobilu/qmd@2.8.3'], runner.calls)
def test_enable_keeps_foreign_package_unowned(self):
runner = FakeRunner(installed=True)
self.setup(runner).enable(['claude'])
self.assertFalse(self.state()['owned_package'])
self.assertFalse(any('npm install' in ' '.join(c) for c in runner.calls))
def test_disable_never_removes_foreign_package(self):
runner = FakeRunner(installed=True)
setup = self.setup(runner)
setup.enable(['claude'])
setup.disable(['claude'])
self.assertFalse(any('npm uninstall' in ' '.join(c) for c in runner.calls))
self.assertFalse((self.home / '.my-ai-configuration/qmd.json').exists())
def test_disable_removes_owned_models_and_collections(self):
runner = FakeRunner()
setup = self.setup(runner)
setup.enable(['claude'])
state = self.state()
model = self.home / '.cache/qmd/models' / 'qwen3-embedding.gguf'
model.write_text('x')
state['owned_models'] = ['qwen3-embedding.gguf']
(self.home / '.my-ai-configuration/qmd.json').write_text(json.dumps(state))
(self.home / '.my-ai-configuration/qmd/collections.json').write_text(json.dumps(['repo-abcdefabcdef']))
setup.disable(['claude'])
self.assertFalse(model.exists())
self.assertIn(['qmd', 'collection', 'remove', 'repo-abcdefabcdef'], runner.calls)
self.assertIn(['npm', 'uninstall', '--global', '@tobilu/qmd'], runner.calls)
def test_benchmark_unsuitable_cleans_up(self):
runner = FakeRunner(gpu=False, cpu_result={'device': 'cpu', 'passed': False, 'querySeconds': 9, 'chunksPerSecond': 3})
self.assertFalse(self.setup(runner).benchmark())
self.assertFalse((self.home / '.my-ai-configuration/qmd.json').exists())
self.assertIn(['npm', 'uninstall', '--global', '@tobilu/qmd'], runner.calls)
self.assertFalse(any('--device' in c and 'gpu' in c for c in runner.calls), 'no GPU means no GPU run')
def test_benchmark_falls_back_to_cpu(self):
runner = FakeRunner(gpu='vulkan', gpu_result={'device': 'gpu', 'passed': False, 'querySeconds': 7, 'chunksPerSecond': 30},
cpu_result={'device': 'cpu', 'passed': True, 'querySeconds': 4, 'chunksPerSecond': 25})
self.assertTrue(self.setup(runner).benchmark())
self.assertEqual(self.state()['device'], 'cpu')
def test_migration_removes_python_retrieval_and_adopts_ledger_package(self):
base = self.home / '.my-ai-configuration'
(base / 'semantic-retrieval/.venv').mkdir(parents=True)
(base / 'semantic-retrieval.json').write_text('{"version": 1, "clients": [], "files": {}}')
(base / 'extensions.json').write_text(json.dumps({'version': 1, 'resources': [
{'client': 'shared', 'name': 'QMD', 'kind': 'qmd', 'package': '@tobilu/qmd', 'clients': ['codex']},
{'client': 'claude', 'name': 'QMD', 'kind': 'plugin', 'selector': 'qmd@qmd'}]}))
runner = FakeRunner(installed=True)
self.setup(runner).enable(['claude'])
self.assertFalse((base / 'semantic-retrieval').exists())
self.assertFalse((base / 'semantic-retrieval.json').exists())
ledger = json.loads((base / 'extensions.json').read_text())
self.assertEqual([r['kind'] for r in ledger['resources']], ['plugin'], 'plugin record stays so reconcile uninstalls qmd@qmd')
self.assertTrue(self.state()['owned_package'], 'ledger-owned package is adopted')
def test_previous_models_are_restored_on_disable(self):
runner = FakeRunner(previous={'embed': 'mine'})
setup = self.setup(runner)
setup.enable(['claude'])
self.assertEqual(self.state()['previous_models'], {'embed': 'mine'})
setup.disable(['claude'])
self.assertIn(['node', str(self.home / '.my-ai-configuration/qmd/qmd-config.mjs'), 'restore-models', '{"embed": "mine"}'], runner.calls)
self.assertFalse(any(c[-1:] == ['unset-models'] for c in runner.calls))
def test_models_block_removed_when_none_existed(self):
runner = FakeRunner()
setup = self.setup(runner)
setup.enable(['claude'])
self.assertIsNone(self.state()['previous_models'])
setup.disable(['claude'])
self.assertTrue(any(c[-1:] == ['unset-models'] for c in runner.calls))
self.assertFalse(any('restore-models' in c for c in runner.calls))
def ledger(self, base):
base.mkdir(parents=True, exist_ok=True)
(base / 'extensions.json').write_text(json.dumps({'version': 1, 'resources': [
{'client': 'shared', 'name': 'QMD', 'kind': 'qmd', 'package': '@tobilu/qmd', 'clients': ['codex']}]}))
def test_disable_without_state_uninstalls_adopted_package(self):
base = self.home / '.my-ai-configuration'
self.ledger(base)
runner = FakeRunner(installed=True)
self.setup(runner).disable(['claude'])
self.assertIn(['npm', 'uninstall', '--global', '@tobilu/qmd'], runner.calls)
self.assertFalse((base / 'qmd.json').exists())
def test_ownership_is_persisted_before_ledger_rewrite(self):
base = self.home / '.my-ai-configuration'
self.ledger(base)
real = qmd.atomic_json
def failing(path, value):
if path.name == 'extensions.json':
raise OSError('disk full')
real(path, value)
with mock.patch.object(qmd, 'atomic_json', failing):
with self.assertRaises(OSError):
self.setup(FakeRunner(installed=True)).enable(['claude'])
self.assertTrue(self.state()['owned_package'])
self.assertEqual(len(json.loads((base / 'extensions.json').read_text())['resources']), 1)
def test_symlinked_script_directory_is_refused(self):
base = self.home / '.my-ai-configuration'
base.mkdir(parents=True)
target = self.home / 'elsewhere'
target.mkdir()
try:
(base / 'qmd').symlink_to(target, target_is_directory=True)
except OSError:
self.skipTest('symlinks unavailable')
with self.assertRaises(ValueError):
self.setup(FakeRunner()).enable(['claude'])
self.assertEqual(list(target.iterdir()), [])
def test_non_dict_benchmark_result_counts_as_failure(self):
runner = FakeRunner(gpu=False, cpu_result=['unexpected'])
self.assertFalse(self.setup(runner).benchmark())
def test_package_installed_parses_listing(self):
self.assertTrue(self.setup(FakeRunner(installed=True)).package_installed())
self.assertFalse(self.setup(FakeRunner()).package_installed())
def test_package_installed_rejects_unreliable_listings(self):
for output in ('', ' ', 'not json', '{"error": {"code": "ELSPROBLEMS"}}', '{}', '[]'):
with self.assertRaises(ValueError, msg=repr(output)):
self.setup(FakeRunner(list_output=output)).package_installed()
self.assertTrue(self.setup(FakeRunner(list_output='{"dependencies": {"@tobilu/qmd": {}}, "error": {}}')).package_installed())
self.assertFalse(self.setup(FakeRunner(list_output='{"dependencies": {}}')).package_installed())
def test_non_mapping_previous_models_are_restored_verbatim(self):
runner = FakeRunner(previous='custom.gguf')
setup = self.setup(runner)
setup.enable(['claude'])
self.assertEqual(self.state()['previous_models'], 'custom.gguf')
setup.disable(['claude'])
self.assertTrue(any(c[-2:] == ['restore-models', '"custom.gguf"'] for c in runner.calls))
def test_dry_run_changes_nothing(self):
runner = FakeRunner()
self.setup(runner, dry_run=True).enable(['claude'])
self.assertFalse((self.home / '.my-ai-configuration/qmd.json').exists())
self.assertFalse(any('npm install' in ' '.join(c) for c in runner.calls))
def scripts_dir(self):
return self.home / '.my-ai-configuration/qmd'
def test_benchmark_with_too_old_node_is_unsuitable(self):
runner = FakeRunner(node='v20.11.1', cpu_result={'device': 'cpu', 'passed': True})
self.assertFalse(self.setup(runner).benchmark())
self.assertFalse((self.home / '.my-ai-configuration/qmd.json').exists())
self.assertFalse(any('npm install' in ' '.join(c) for c in runner.calls))
def test_benchmark_npm_install_failure_is_unsuitable(self):
runner = FakeRunner(fail=('npm install',), cpu_result={'device': 'cpu', 'passed': True})
self.assertFalse(self.setup(runner).benchmark())
self.assertFalse((self.home / '.my-ai-configuration/qmd.json').exists())
self.assertFalse(any('--device' in c for c in runner.calls))
def test_benchmark_download_failure_cleans_up_what_it_created(self):
runner = FakeRunner(fail_pull=True, cpu_result={'device': 'cpu', 'passed': True})
self.assertFalse(self.setup(runner).benchmark())
self.assertTrue(any(c[-1:] == ['unset-models'] for c in runner.calls))
self.assertIn(['npm', 'uninstall', '--global', '@tobilu/qmd'], runner.calls)
self.assertFalse(self.scripts_dir().exists())
self.assertFalse((self.home / '.my-ai-configuration/qmd.json').exists())
def test_benchmark_detect_crash_runs_cpu_only(self):
runner = FakeRunner(fail=('--detect',), gpu_result={'device': 'gpu', 'passed': True}, cpu_result={'device': 'cpu', 'passed': True})
self.assertTrue(self.setup(runner).benchmark())
self.assertEqual(self.state()['device'], 'cpu')
self.assertFalse(any('--device' in c and 'gpu' in c for c in runner.calls))
def test_enable_with_detect_crash_uses_cpu(self):
self.setup(FakeRunner(fail=('--detect',))).enable(['claude'])
self.assertEqual(self.state()['device'], 'cpu')
def legacy_install(self):
legacy = self.home / '.my-ai-configuration/semantic-retrieval'
python = legacy / '.venv' / ('Scripts/python.exe' if os.name == 'nt' else 'bin/python')
python.parent.mkdir(parents=True)
python.write_text('')
(legacy / 'server.py').write_text('')
return legacy, python
def test_migration_stops_the_legacy_daemon_before_removal(self):
legacy, python = self.legacy_install()
seen = []
runner = FakeRunner(installed=True, on_call=lambda a: seen.append(legacy.exists()) if 'server.py' in ' '.join(a) else None)
self.setup(runner).enable(['claude'])
self.assertIn([str(python), str(legacy / 'server.py'), 'stop', '--data-dir', str(legacy / 'data'),
'--model-cache', str(legacy / 'model-cache')], runner.calls)
self.assertEqual(seen, [True], 'stop runs while the legacy directory still exists')
self.assertFalse(legacy.exists())
def test_failing_legacy_stop_does_not_block_migration(self):
legacy, _ = self.legacy_install()
self.setup(FakeRunner(installed=True, fail=('server.py',))).enable(['claude'])
self.assertFalse(legacy.exists())
def test_removal_waits_for_the_daemon_to_exit_before_deleting_models(self):
runner = FakeRunner()
setup = self.setup(runner)
setup.enable(['claude'])
(self.scripts_dir() / 'daemon.json').write_text(json.dumps({'port': 1, 'token': 't', 'pid': 4242}))
checks = []
def alive(pid):
checks.append((pid, any('--stop' in c for c in runner.calls), any('npm uninstall' in ' '.join(c) for c in runner.calls)))
return len(checks) < 3
with mock.patch.object(qmd, 'process_alive', alive):
setup.disable(['claude'])
self.assertEqual(checks, [(4242, True, False)] * 3, 'the PID is polled after --stop and before the uninstall')
self.assertIn(['npm', 'uninstall', '--global', '@tobilu/qmd'], runner.calls)
def test_process_alive(self):
self.assertTrue(qmd.process_alive(os.getpid()))
child = subprocess.Popen([sys.executable, '-c', 'pass'])
child.wait()
self.assertFalse(qmd.process_alive(child.pid))
def test_undeletable_model_keeps_state_for_a_retry(self):
runner = FakeRunner()
setup = self.setup(runner)
setup.enable(['claude'])
state = self.state()
models = self.home / '.cache/qmd/models'
(models / 'locked.gguf').write_text('x')
(models / 'free.gguf').write_text('x')
state['owned_models'] = ['free.gguf', 'locked.gguf']
(self.home / '.my-ai-configuration/qmd.json').write_text(json.dumps(state))
real = Path.unlink
def unlink(path, missing_ok=False):
if path.name == 'locked.gguf':
raise PermissionError('in use')
real(path, missing_ok=missing_ok)
with mock.patch.object(qmd.Path, 'unlink', unlink):
setup.disable(['claude'])
state = self.state()
self.assertEqual(state['owned_models'], ['locked.gguf'])
self.assertFalse(state['owned_package'])
self.assertFalse(state['models_block'])
self.assertFalse((models / 'free.gguf').exists())
self.assertIn(['npm', 'uninstall', '--global', '@tobilu/qmd'], runner.calls)
self.assertFalse(self.scripts_dir().exists())
runner.calls.clear()
setup.disable(['claude'])
self.assertFalse((models / 'locked.gguf').exists())
self.assertFalse((self.home / '.my-ai-configuration/qmd.json').exists())
self.assertFalse(any('npm uninstall' in ' '.join(c) for c in runner.calls), 'a retry never uninstalls a later foreign package')
def test_failing_models_block_restore_does_not_stop_removal(self):
runner = FakeRunner(fail=('unset-models',))
setup = self.setup(runner)
setup.enable(['claude'])
setup.disable(['claude'])
self.assertIn(['npm', 'uninstall', '--global', '@tobilu/qmd'], runner.calls)
self.assertFalse(self.scripts_dir().exists())
self.assertFalse((self.home / '.my-ai-configuration/qmd.json').exists())
def test_models_dir_honors_xdg_cache_home(self):
os.environ['XDG_CACHE_HOME'] = str(self.home / 'xdg')
self.assertEqual(self.setup(FakeRunner()).models, self.home / 'xdg' / 'qmd' / 'models')
os.environ['XDG_CACHE_HOME'] = ''
self.assertEqual(self.setup(FakeRunner()).models, self.home / '.cache' / 'qmd' / 'models')
if __name__ == '__main__':
unittest.main()
+41
View File
@@ -0,0 +1,41 @@
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { mkdtempSync, writeFileSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { withLock } from '../../shared/qmd/qmd-warm.mjs';
test('a fresh lock skips the second warm-up and is released afterwards', async () => {
const lock = join(mkdtempSync(join(tmpdir(), 'qmd-warm-')), 'repo.lock');
let runs = 0;
await withLock(lock, async () => {
runs += 1;
await withLock(lock, async () => { runs += 1; });
});
assert.equal(runs, 1);
assert.equal(existsSync(lock), false);
});
test('a stale lock older than 30 minutes is replaced', async () => {
const lock = join(mkdtempSync(join(tmpdir(), 'qmd-warm-')), 'repo.lock');
writeFileSync(lock, String(Date.now() - 31 * 60_000));
let runs = 0;
await withLock(lock, async () => { runs += 1; });
assert.equal(runs, 1);
});
test('an empty or non-numeric lock counts as stale', async () => {
for (const content of ['', 'garbage']) {
const lock = join(mkdtempSync(join(tmpdir(), 'qmd-warm-')), 'repo.lock');
writeFileSync(lock, content);
let runs = 0;
await withLock(lock, async () => { runs += 1; });
assert.equal(runs, 1);
}
});
test('a run never removes a lock that another run took over', async () => {
const lock = join(mkdtempSync(join(tmpdir(), 'qmd-warm-')), 'repo.lock');
await withLock(lock, async () => { writeFileSync(lock, 'other-run'); });
assert.equal(existsSync(lock), true);
});
-548
View File
@@ -1,548 +0,0 @@
import importlib.util
from array import array
from concurrent.futures import ThreadPoolExecutor
import os
from pathlib import Path
import re
import socket
import subprocess
from tempfile import TemporaryDirectory
from types import SimpleNamespace
import time
import unittest
import urllib.request
from unittest.mock import patch
ROOT = Path(__file__).resolve().parent.parent.parent
SPEC = importlib.util.spec_from_file_location('retrieval_server', ROOT / 'shared/retrieval/server.py')
MODULE = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MODULE)
def tokenize(text, **kwargs):
offsets = [match.span() for match in re.finditer(r'\S+', text)]
return {'offset_mapping': offsets, 'input_ids': list(range(len(offsets)))}
def embedding():
return array('f', [1.0] + [0.0] * (MODULE.EMBEDDING_DIMENSIONS - 1))
class VectorBatch:
def __init__(self, blob):
self.values = array('f', blob)
def reshape(self, *shape):
return self
def __matmul__(self, vector):
width = MODULE.EMBEDDING_DIMENSIONS
return [sum(a * b for a, b in zip(self.values[offset:offset + width], vector))
for offset in range(0, len(self.values), width)]
class ApproximateIndex:
def __init__(self, **kwargs):
self.vectors = {}
def add(self, keys, vectors):
width = MODULE.EMBEDDING_DIMENSIONS
for position, key in enumerate(keys):
start = position * width
self.vectors[key] = self.vectors.get(key, []) + list(vectors.values[start:start + width])
def search(self, vector, count):
matches = sorted(((key, sum(a * b for a, b in zip(values, vector)))
for key, values in self.vectors.items()), key=lambda item: (-item[1], item[0]))[:count]
return [SimpleNamespace(key=key, distance=1.0 - score) for key, score in matches]
def fake_numpy():
return SimpleNamespace(frombuffer=lambda blob, **kwargs: VectorBatch(blob), float32=None)
def fake_usearch():
module = SimpleNamespace(Index=ApproximateIndex)
return SimpleNamespace(index=module), module
def initialize_repository(root):
subprocess.run(['git', '-C', str(root), 'init', '--quiet'], check=True)
def huggingface_available():
try:
request = urllib.request.Request('https://huggingface.co', method='HEAD')
with urllib.request.urlopen(request, timeout=5):
pass
except OSError:
return False
return True
def huggingface_network_unavailable(error):
while error:
if isinstance(error, socket.gaierror):
return True
error = error.__cause__ or error.__context__
return False
class RetrievalServerTests(unittest.TestCase):
def test_huggingface_availability_handles_connection_failure(self):
with patch('urllib.request.urlopen', side_effect=OSError):
self.assertFalse(huggingface_available())
def test_huggingface_network_failure_is_detected_from_cause_chain(self):
error = OSError('model download failed')
error.__cause__ = socket.gaierror('name resolution failed')
self.assertTrue(huggingface_network_unavailable(error))
self.assertFalse(huggingface_network_unavailable(OSError('model is missing')))
def test_ci_disables_xet_for_model_downloads(self):
content = (ROOT / '.gitea/workflows/ci.yml').read_text(encoding='utf-8')
self.assertIn("HF_HUB_DISABLE_XET: '1'", content)
def test_client_smoke_builds_packages(self):
content = (ROOT / '.gitea/workflows/ci.yml').read_text(encoding='utf-8')
self.assertIn('name: Build packages\n run: bash scripts/commands/build.sh --summary', content)
def test_runtime_python_uses_adjacent_virtualenv(self):
with TemporaryDirectory() as temporary:
script = Path(temporary) / 'semantic-retrieval' / 'server.py'
runtime = script.parent / '.venv' / 'Scripts' / 'python.exe'
runtime.parent.mkdir(parents=True)
runtime.touch()
self.assertEqual(MODULE.runtime_python(script, 'C:/Python/python.exe', 'nt'), runtime.resolve())
def test_windows_runtime_handoff_keeps_multi_word_arguments(self):
calls = []
with patch.object(MODULE, 'runtime_python', return_value=Path('C:/runtime/python.exe')), patch.object(MODULE.sys, 'argv', ['server.py', 'search', '--query', 'two words']), patch.object(MODULE.subprocess, 'run', side_effect=lambda command: calls.append(command) or SimpleNamespace(returncode=3)), self.assertRaises(SystemExit) as stopped:
MODULE.use_runtime_python('nt')
self.assertEqual(stopped.exception.code, 3)
self.assertEqual(calls[0][2:], ['search', '--query', 'two words'])
def test_daemon_identity_changes_with_code_version(self):
with TemporaryDirectory() as temporary:
data_dir = Path(temporary)
with patch.object(MODULE, 'code_version', return_value='aaaaaaaaaaaa'):
old_address, old_lock = MODULE.daemon_address(data_dir), MODULE.daemon_lock_path(data_dir)
with patch.object(MODULE, 'code_version', return_value='bbbbbbbbbbbb'):
self.assertNotEqual(MODULE.daemon_address(data_dir), old_address)
self.assertNotEqual(MODULE.daemon_lock_path(data_dir), old_lock)
def test_run_direct_runs_in_process_when_pipes_are_forbidden(self):
def action(action, root, data_dir, model_cache, query, top_k, output):
output.put((True, [query, top_k]))
with patch.object(MODULE, 'pipes_available', return_value=False), patch.object(MODULE, 'run_action', action):
self.assertEqual(MODULE.run_direct('search', Path('.'), Path('.'), Path('.'), 'two words', 3), ['two words', 3])
def test_model_kwargs_use_bfloat16_only_on_supporting_gpus(self):
bfloat16 = object()
for available, supported, expected in [(True, True, {'dtype': bfloat16}), (True, False, {}), (False, True, {})]:
cuda = SimpleNamespace(is_available=lambda available=available: available, is_bf16_supported=lambda supported=supported: supported)
with patch.dict('sys.modules', torch=SimpleNamespace(cuda=cuda, bfloat16=bfloat16)):
self.assertEqual(MODULE.model_kwargs(), expected)
def test_query_prompt_matches_qwen_format(self):
self.assertTrue(MODULE.QUERY_PROMPT.startswith('Instruct: '))
self.assertTrue(MODULE.QUERY_PROMPT.endswith('\nQuery:'))
def test_preferred_device_uses_cuda_with_cpu_fallback(self):
with patch.dict('sys.modules', torch=SimpleNamespace(cuda=SimpleNamespace(is_available=lambda: True))):
self.assertEqual(MODULE.preferred_device(), 'cuda')
with patch.dict('sys.modules', torch=SimpleNamespace(cuda=SimpleNamespace(is_available=lambda: False))):
self.assertEqual(MODULE.preferred_device(), 'cpu')
def test_preload_models_warms_embedding_and_reranking_from_model_cache(self):
with TemporaryDirectory() as temporary:
cache = Path(temporary) / 'model-cache'
calls = []
encoder = SimpleNamespace(encode=lambda values, **kwargs: calls.append(('encode', values, kwargs)) or [embedding()])
reranker = SimpleNamespace(predict=lambda pairs, **kwargs: calls.append(('predict', pairs, kwargs)) or [0.0])
with patch.object(MODULE, 'shared_encoder', return_value=encoder), \
patch.object(MODULE, 'shared_reranker', return_value=reranker), \
patch.dict(os.environ, {'HF_HOME': 'other-cache'}):
getattr(MODULE, 'preload_models', lambda _: None)(cache)
self.assertEqual(str(cache.resolve()), os.environ['HF_HOME'])
self.assertTrue(cache.is_dir())
self.assertEqual([
('encode', ['Verify semantic retrieval models.'], {'prompt_name': 'query'}),
('predict', [('Verify semantic retrieval models.', 'Verify semantic retrieval models.')], {'show_progress_bar': False}),
], calls)
@unittest.skipUnless(os.environ.get('QWEN_TEST_MODEL_CACHE'), 'Set QWEN_TEST_MODEL_CACHE to run the real-model check.')
def test_real_models(self):
if not huggingface_available():
self.skipTest('huggingface.co DNS is unavailable')
os.environ['HF_HOME'] = os.environ['QWEN_TEST_MODEL_CACHE']
with TemporaryDirectory() as temporary:
root = Path(temporary)
initialize_repository(root)
(root / 'authentication.md').write_text('Password reset tokens are created by create_reset_token. The reset_password endpoint sends the user a password reset email.', encoding='utf-8')
(root / 'weather.md').write_text('The weather service fetches temperature and rainfall forecasts for a city.', encoding='utf-8')
index = MODULE.Index(root, root / 'data')
self.addCleanup(index.db.close)
try:
tokenizer = index.encoder().tokenizer
for source in ['alpha beta gamma ' * 450, 'Grüße 世界 🙂 ' * 300]:
windows = list(MODULE.chunks(source, 'sample.txt', tokenizer))
self.assertGreater(len(windows), 1)
self.assertTrue(all(len(tokenizer(value, add_special_tokens=False)['input_ids']) <= 400
for _, value in windows))
self.assertTrue(all('\ufffd' not in value for _, value in windows))
for query, expected in [('Where are password reset tokens created?', 'authentication.md'),
('Which service provides rainfall forecasts?', 'weather.md')]:
started = time.monotonic()
results = index.search(query, 2)
self.assertEqual(results[0]['path'], expected)
self.assertEqual(len(results), 2)
self.assertGreater(results[0]['score'], results[1]['score'])
print(f'Qwen real search: {expected}, {time.monotonic() - started:.2f}s, embedding device {index.encoder().device}, reranker device {index.reranker().device}', flush=True)
except OSError as error:
if huggingface_network_unavailable(error):
self.skipTest('Hugging Face network is unavailable')
raise
def test_isolation_freshness_and_incremental_encoding(self):
with TemporaryDirectory() as temporary:
root = Path(temporary)
first, second = root / 'first', root / 'second'
first.mkdir()
second.mkdir()
initialize_repository(first)
initialize_repository(second)
file = first / 'a.md'
file.write_text('alpha', encoding='utf-8')
(second / 'a.md').write_text('other', encoding='utf-8')
indexes = [MODULE.Index(repository, root / 'data') for repository in (first, second)]
encoded = []
def encode(values, **kwargs):
self.assertEqual(kwargs.get('prompt_name'), 'query' if values == ['query'] else None)
encoded.extend(values)
return [embedding() for _ in values]
for index in indexes:
index.encoder = lambda: SimpleNamespace(encode=encode, tokenizer=tokenize)
index.rerank = lambda query, rows, limit: rows[:limit]
numpy = fake_numpy()
usearch, usearch_index = fake_usearch()
try:
with patch.dict('sys.modules', numpy=numpy, usearch=usearch, **{'usearch.index': usearch_index}):
self.assertEqual(indexes[0].search('query', 5)[0]['text'], 'alpha')
self.assertEqual(indexes[1].search('query', 5)[0]['text'], 'other')
encoded.clear()
with patch.object(Path, 'read_bytes', side_effect=AssertionError('unchanged files must not be read')):
self.assertEqual(indexes[0].search('query', 5)[0]['text'], 'alpha')
with ThreadPoolExecutor(max_workers=1) as executor:
self.assertEqual(executor.submit(indexes[0].search, 'query', 5).result()[0]['text'], 'alpha')
self.assertEqual(encoded, [])
file.write_text('bravo', encoding='utf-8')
self.assertEqual(indexes[0].search('query', 5)[0]['text'], 'bravo')
file.rename(first / 'renamed.md')
self.assertEqual(indexes[0].search('query', 5)[0]['path'], 'renamed.md')
(first / 'renamed.md').unlink()
encoded.clear()
self.assertEqual(indexes[0].search('query', 5), [])
self.assertEqual(indexes[0].search('query', 5), [])
self.assertEqual(encoded, [])
self.assertEqual(indexes[1].search('query', 5)[0]['text'], 'other')
finally:
for index in indexes:
index.db.close()
def test_failed_refresh_preserves_previous_index(self):
with TemporaryDirectory() as temporary:
root = Path(temporary)
initialize_repository(root)
(root / 'a.md').write_text('before', encoding='utf-8')
index = MODULE.Index(root, root / 'data')
index.encoder = lambda: SimpleNamespace(encode=lambda values, **kwargs: [embedding() for _ in values], tokenizer=tokenize)
try:
index.rebuild()
(root / 'a.md').write_text('after', encoding='utf-8')
(root / 'b.md').write_text('fail', encoding='utf-8')
def encode(values, **kwargs):
if any(value.endswith('\n\nfail') for value in values):
raise RuntimeError('encoder failed')
return [embedding() for _ in values]
index.encoder = lambda: SimpleNamespace(encode=encode, tokenizer=tokenize)
with self.assertRaisesRegex(RuntimeError, 'encoder failed'):
index.rebuild()
self.assertEqual(index.db.execute('select path, text from chunks').fetchall(), [('a.md', 'before')])
self.assertEqual(index.db.execute('select document from lexical').fetchall(),
[('Path: a.md\nSymbol: <module>\n\nbefore',)])
finally:
index.db.close()
def test_refresh_batches_embeddings_for_changed_files(self):
with TemporaryDirectory() as temporary:
root = Path(temporary)
initialize_repository(root)
(root / 'first.md').write_text('first', encoding='utf-8')
(root / 'second.md').write_text('second', encoding='utf-8')
index = MODULE.Index(root, root / 'data')
encoded = []
def encode(values, **kwargs):
encoded.append(list(values))
return [embedding() for _ in values]
index.encoder = lambda: SimpleNamespace(encode=encode, tokenizer=tokenize)
try:
index.rebuild()
self.assertEqual(len(encoded), 1)
self.assertEqual(len(encoded[0]), 2)
finally:
index.db.close()
def test_scan_prunes_ignored_directories_and_warns_at_limit(self):
with TemporaryDirectory() as temporary:
root = Path(temporary)
initialize_repository(root)
index = MODULE.Index(root, root / 'data')
(root / 'visible.md').write_text('visible', encoding='utf-8')
(root / '.gitignore').write_text('secrets.json\n**/private/\n', encoding='utf-8')
(root / 'secrets.json').write_text('{"token":"hidden"}', encoding='utf-8')
private = root / 'nested' / 'private'
private.mkdir(parents=True)
(private / 'hidden.md').write_text('hidden', encoding='utf-8')
for name in ('node_modules', '.git', 'generated', 'data'):
directory = root / name
directory.mkdir(exist_ok=True)
(directory / 'hidden.md').write_text('hidden', encoding='utf-8')
try:
self.assertEqual(index.files(), [(root / 'visible.md').resolve()])
filenames = ['visible.md'] * 2001
with patch.object(MODULE.subprocess, 'run', return_value=SimpleNamespace(stdout=b'visible.md\0' * 2001)), \
patch.object(MODULE.sys, 'stderr') as stderr:
self.assertEqual(len(index.files()), 2000)
self.assertIn('stopped scanning after 2000', stderr.write.call_args_list[0].args[0])
finally:
index.db.close()
def test_status_reports_index_freshness_without_creating_an_index(self):
with TemporaryDirectory() as temporary:
root, data = Path(temporary), Path(temporary) / 'data'
initialize_repository(root)
source = root / 'module.py'
source.write_text('value = 1\n', encoding='utf-8')
status = MODULE.index_status(root, data)
self.assertEqual((status['indexable_files'], status['indexed_files'], status['stale_files']), (1, 0, 1))
self.assertFalse(status['models_loaded'])
self.assertFalse(MODULE.index_database(root, data).exists())
index = MODULE.Index(root, data)
try:
metadata = source.stat()
with index.db:
index.db.execute('insert into files values (?, ?, ?, ?, ?)',
('module.py', 'digest', metadata.st_size, metadata.st_mtime_ns, metadata.st_ctime_ns))
finally:
index.db.close()
self.assertEqual(MODULE.index_status(root, data)['stale_files'], 0)
source.write_text('value = 22\n', encoding='utf-8')
self.assertEqual(MODULE.index_status(root, data)['stale_files'], 1)
def test_max_files_is_configurable_via_environment(self):
with TemporaryDirectory() as temporary:
root = Path(temporary)
initialize_repository(root)
index = MODULE.Index(root, root / 'data')
try:
with patch.dict(os.environ, {MODULE.MAX_FILES_ENV: '3'}):
self.assertEqual(MODULE.read_max_files(), 3)
with patch.dict(os.environ, {MODULE.MAX_FILES_ENV: 'not-a-number'}):
with self.assertRaises(ValueError):
MODULE.read_max_files()
with patch.dict(os.environ, {}, clear=True):
self.assertEqual(MODULE.read_max_files(), MODULE.DEFAULT_MAX_FILES)
(root / 'visible.md').write_text('visible', encoding='utf-8')
index.max_files = 3
for number in range(5):
(root / f'{number}.md').write_text('visible', encoding='utf-8')
with patch.object(MODULE.sys, 'stderr'):
self.assertEqual(len(index.files()), 3)
finally:
index.db.close()
def test_daemon_address_is_deterministic_and_platform_specific(self):
with TemporaryDirectory() as temporary:
data_dir = Path(temporary)
first = MODULE.daemon_address(data_dir)
second = MODULE.daemon_address(data_dir)
self.assertEqual(first, second)
if os.name == 'nt':
self.assertTrue(first.startswith(r'\\.\pipe\ai-config-retrieval-'))
else:
self.assertTrue(first.startswith(str(data_dir)))
def test_shared_encoder_and_reranker_are_cached_module_singletons(self):
with patch.object(MODULE, '_shared_encoder', None), patch.object(MODULE, '_shared_reranker', None), \
patch.object(MODULE, 'preferred_device', lambda: 'cpu'), patch.object(MODULE, 'model_kwargs', dict):
sentinel = SimpleNamespace()
with patch.dict('sys.modules', sentence_transformers=SimpleNamespace(
SentenceTransformer=lambda *args, **kwargs: sentinel,
CrossEncoder=lambda *args, **kwargs: sentinel)):
self.assertIs(MODULE.shared_encoder(), sentinel)
self.assertIs(MODULE.shared_encoder(), MODULE.shared_encoder())
self.assertIs(MODULE.shared_reranker(), sentinel)
self.assertIs(MODULE.shared_reranker(), MODULE.shared_reranker())
def test_rerank_uses_cross_encoder_scores(self):
index = MODULE.Index.__new__(MODULE.Index)
class Reranker:
def predict(self, pairs, **kwargs):
self.pairs = pairs
return [0.1, 0.8, 0.5]
reranker = Reranker()
index.reranker = lambda: reranker
rows = [
{'path': 'a.md', 'symbol': 'A', 'score': 0.9, 'text': 'first'},
{'path': 'b.md', 'symbol': 'B', 'score': 0.8, 'text': 'second'},
{'path': 'c.md', 'symbol': 'C', 'score': 0.7, 'text': 'third'},
]
result = index.rerank('query', rows, 2)
self.assertEqual([item['path'] for item in result], ['b.md', 'c.md'])
self.assertEqual(reranker.pairs, [('query', 'Path: a.md\nSymbol: A\n\nfirst'),
('query', 'Path: b.md\nSymbol: B\n\nsecond'),
('query', 'Path: c.md\nSymbol: C\n\nthird')])
def test_token_windows_have_exact_overlap_and_no_redundant_tail(self):
for size, lengths in [(0, []), (400, [400]), (401, [400, 65]), (736, [400, 400]), (800, [400, 400, 128])]:
words = [f'token{number}' for number in range(size)]
result = list(MODULE.chunks(' '.join(words), 'plain.txt', tokenize))
windows = [content.split() for _, content in result]
self.assertEqual([len(window) for window in windows], lengths)
for first, second in zip(windows, windows[1:]):
self.assertEqual(first[-64:], second[:64])
if windows:
reconstructed = windows[0] + [word for window in windows[1:] for word in window[64:]]
self.assertEqual(reconstructed, words)
def test_python_symbols_include_decorators_and_restore_parent(self):
source = 'class Service:\n @cached\n def load(self):\n return 1\n\nvalue = 2\n\nasync def refresh():\n return value\n'
result = list(MODULE.sections(source, 'service.py'))
self.assertEqual([symbol for symbol, _ in result], ['Service', 'Service.load', '<module>', 'refresh'])
self.assertIn('@cached', result[1][1])
self.assertEqual(''.join(content for _, content in result), source)
broken = 'def incomplete(\n' + 'word ' * 450
self.assertEqual(len(list(MODULE.chunks(broken, 'broken.py', tokenize))), 2)
def test_markdown_sections_ignore_fenced_headings(self):
source = '# Setup\nIntro\n## Install\n```sh\n# not a heading\n```\nSteps\n## Run\nStart\n'
result = list(MODULE.chunks(source, 'guide.md', tokenize))
self.assertEqual([symbol for symbol, _ in result], ['Setup', 'Setup / Install', 'Setup / Run'])
self.assertIn('# not a heading', result[1][1])
def test_common_declarations_and_config_sections(self):
for path, source, symbols in [
('service.ts', 'export function load() {\n return 1;\n}\nexport const save = () => 2;\n', ['load', 'save']),
('service.cs', 'public class Service\n{\n public void Load()\n {\n }\n}\n', ['Service', 'Load']),
('setup.ps1', 'function Get-Result {\n return 1\n}\n', ['Get-Result']),
('build.sh', 'quote() {\n echo "$1"\n}\nfor shell in a b; do\n quote "$shell"\ndone\n', ['quote', '<module>']),
('schema.sql', 'CREATE TABLE users (id int);\nCREATE OR ALTER PROCEDURE dbo.GetUser AS SELECT 1;\n', ['users', 'dbo.GetUser']),
('Main.kt', 'fun main() {\n println(1)\n}\n', ['main']),
('config.toml', '[server]\nport = 1\n[client]\nport = 2\n', ['server', 'client']),
]:
self.assertEqual([symbol for symbol, _ in MODULE.sections(source, path)], symbols)
def test_rrf_rewards_agreement_and_breaks_ties_deterministically(self):
self.assertEqual(MODULE.reciprocal_rank_fusion([1, 2, 3], [4, 2, 3]), [2, 3, 1, 4])
self.assertEqual(MODULE.reciprocal_rank_fusion([], [4, 2]), [4, 2])
def test_dense_candidates_converts_vector_keys_for_sqlite(self):
class VectorKey:
def __int__(self):
return 9
with TemporaryDirectory() as temporary:
root = Path(temporary)
index = MODULE.Index(root, root / 'data')
index.vector_index = SimpleNamespace(search=lambda vector, count: [SimpleNamespace(key=VectorKey())])
usearch, usearch_index = fake_usearch()
try:
with patch.dict('sys.modules', numpy=fake_numpy(), usearch=usearch,
**{'usearch.index': usearch_index}):
candidates = index.dense_candidates(embedding())
self.assertEqual(candidates, [9])
self.assertIs(type(candidates[0]), int)
finally:
index.db.close()
def test_hybrid_candidates_and_document_metadata(self):
with TemporaryDirectory() as temporary:
root = Path(temporary)
initialize_repository(root)
for number in range(110):
(root / f'{number:03}.md').write_text(f'# Symbol{number}\ncontent {number}', encoding='utf-8', newline='\n')
index = MODULE.Index(root, root / 'data')
documents = []
def encode(values, **kwargs):
if kwargs.get('prompt_name') != 'query':
documents.extend(values)
return [embedding() for _ in values]
pairs = []
index.encoder = lambda: SimpleNamespace(encode=encode, tokenizer=tokenize)
index.reranker = lambda: SimpleNamespace(predict=lambda values, **kwargs:
pairs.extend(values) or [float('109.md' in document) for _, document in values])
numpy = fake_numpy()
original_fusion = MODULE.reciprocal_rank_fusion
rankings = []
def fuse(*values):
rankings.extend(values)
return original_fusion(*values)
try:
usearch, usearch_index = fake_usearch()
with patch.dict('sys.modules', numpy=numpy, usearch=usearch, **{'usearch.index': usearch_index}), \
patch.object(MODULE, 'reciprocal_rank_fusion', fuse):
results = index.search('109.md Symbol109 content', 20)
self.assertEqual([len(ranking) for ranking in rankings], [50, 50])
self.assertEqual(len(pairs), 50)
self.assertEqual(len(results), 5)
self.assertEqual(results[0]['path'], '109.md')
self.assertEqual(results[0]['symbol'], 'Symbol109')
self.assertIn('Path: 109.md\nSymbol: Symbol109\n\n# Symbol109\ncontent 109', documents)
self.assertTrue(all(document in documents for _, document in pairs))
with patch.dict('sys.modules', numpy=numpy, usearch=usearch, **{'usearch.index': usearch_index}):
self.assertEqual(len(index.search('" OR * () : -')), 5)
self.assertEqual(len(index.search('***')), 5)
(root / '109.md').unlink()
index.search('Symbol109')
self.assertEqual(index.db.execute('select count(*) from lexical where lexical match ?', ('"Symbol109"',)).fetchone()[0], 0)
finally:
index.db.close()
def test_old_cache_is_preserved_but_not_reused(self):
with TemporaryDirectory() as temporary:
root = Path(temporary)
data = root / 'data'
data.mkdir()
identity = os.path.normcase(str(root.resolve())) + '\n' + MODULE.MODEL
old = data / (MODULE.hashlib.sha256(identity.encode()).hexdigest() + '.sqlite3')
old.write_bytes(b'old cache')
index = MODULE.Index(root, data)
try:
self.assertNotEqual(Path(index.db.execute('pragma database_list').fetchone()[2]), old)
self.assertEqual(old.read_bytes(), b'old cache')
finally:
index.db.close()
if __name__ == '__main__':
unittest.main()
-163
View File
@@ -1,163 +0,0 @@
import hashlib
import importlib.util
import io
import json
from contextlib import redirect_stdout
from pathlib import Path
import py_compile
import subprocess
import tempfile
import unittest
from unittest.mock import patch
ROOT = Path(__file__).resolve().parent.parent.parent
SPEC = importlib.util.spec_from_file_location('semantic_retrieval', ROOT / 'scripts/lib/semantic-retrieval.py')
MODULE = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MODULE)
class SemanticRetrievalTests(unittest.TestCase):
def test_sync_does_not_register_mcp(self):
with tempfile.TemporaryDirectory() as directory:
home = Path(directory) / 'home'
output = io.StringIO()
with redirect_stdout(output):
MODULE.sync(ROOT, home, ['codex', 'claude'], True, True, False)
self.assertNotIn('mcp', output.getvalue().lower())
def test_runtime_prefers_cuda_packages_for_nvidia_gpu(self):
with tempfile.TemporaryDirectory() as directory:
target = Path(directory)
python = MODULE.python_path(target / '.venv')
python.parent.mkdir(parents=True)
python.touch()
with patch.object(MODULE.shutil, 'which', return_value='nvidia-smi'), patch.object(MODULE.subprocess, 'run') as run:
MODULE.ensure_runtime(target, False)
self.assertEqual(run.call_args.args[0][-2:], ['--extra-index-url', MODULE.CUDA_INDEX])
def test_runtime_falls_back_to_cpu_packages_when_cuda_install_fails(self):
with tempfile.TemporaryDirectory() as directory:
target = Path(directory)
python = MODULE.python_path(target / '.venv')
python.parent.mkdir(parents=True)
python.touch()
failure = MODULE.subprocess.CalledProcessError(1, 'pip')
with patch.object(MODULE.shutil, 'which', return_value='nvidia-smi'), patch.object(MODULE.subprocess, 'run', side_effect=[failure, None]) as run:
MODULE.ensure_runtime(target, False)
self.assertEqual(run.call_count, 2)
self.assertNotIn('--extra-index-url', run.call_args.args[0])
def test_sync_preloads_models_after_installing_runtime(self):
with tempfile.TemporaryDirectory() as directory:
home = Path(directory) / 'home'
target = home / '.my-ai-configuration/semantic-retrieval'
python = MODULE.python_path(target / '.venv')
with patch.object(MODULE, 'ensure_runtime', return_value=python), \
patch.object(MODULE.subprocess, 'run') as run:
MODULE.sync(ROOT, home, ['codex'], True, False, False)
self.assertEqual([[
str(python), str(target / 'server.py'), 'preload', '--model-cache', str(target / 'model-cache')
]], [call.args[0] for call in run.call_args_list])
def test_summary_dry_run_omits_details_without_writes(self):
with tempfile.TemporaryDirectory() as directory:
home = Path(directory) / 'home'
for enabled in (True, False):
output = io.StringIO()
with redirect_stdout(output):
MODULE.sync(ROOT, home, ['codex'], enabled, True, False, summary=True)
self.assertEqual(output.getvalue(), '')
self.assertFalse(home.exists())
def test_disable_removes_owned_runtime_without_cli_registration(self):
with tempfile.TemporaryDirectory() as directory:
home = Path(directory) / 'home'
target = home / '.my-ai-configuration/semantic-retrieval'
target.mkdir(parents=True)
for name in MODULE.FILES:
(target / name).write_text(name)
state_path = target.parent / 'semantic-retrieval.json'
state_path.write_text(json.dumps({'version': 1, 'clients': ['codex'], 'files': {
name: hashlib.sha256((target / name).read_bytes()).hexdigest() for name in MODULE.FILES}}))
MODULE.sync(ROOT, home, ['codex'], False, False, False)
self.assertFalse(target.exists())
self.assertFalse(state_path.exists())
def test_update_accepts_python_cache_but_rejects_foreign_cache_content(self):
with tempfile.TemporaryDirectory() as directory:
home = Path(directory) / 'home'
target = home / '.my-ai-configuration/semantic-retrieval'
target.mkdir(parents=True)
files = {}
for name in MODULE.FILES:
content = (ROOT / 'shared/retrieval' / name).read_bytes()
(target / name).write_bytes(content)
files[name] = hashlib.sha256(content).hexdigest()
MODULE.save_state(target.parent / 'semantic-retrieval.json', {'version': 1, 'clients': ['codex'], 'files': files}, False)
py_compile.compile(str(target / 'server.py'), doraise=True)
cache = target / '__pycache__'
with patch.object(MODULE, 'ensure_runtime', return_value=Path('python')), patch.object(MODULE, 'ensure_models'):
MODULE.sync(ROOT, home, ['codex'], True, False, True)
(cache / 'foreign.txt').write_text('keep')
with self.assertRaisesRegex(ValueError, '__pycache__'):
MODULE.sync(ROOT, home, ['codex'], True, False, True)
def test_install_search_update_and_remove_owned_data(self):
with tempfile.TemporaryDirectory() as directory:
home = Path(directory) / 'home'
repository = Path(directory) / 'repository'
repository.mkdir()
subprocess.run(['git', '-C', str(repository), 'init', '--quiet'], check=True)
target = home / '.my-ai-configuration/semantic-retrieval'
state_path = target.parent / 'semantic-retrieval.json'
with patch.object(MODULE, 'ensure_runtime', return_value=Path('python')), \
patch.object(MODULE, 'ensure_models'):
MODULE.sync(ROOT, home, ['codex'], True, False, False)
installed = importlib.util.spec_from_file_location('installed_retrieval', target / 'server.py')
server = importlib.util.module_from_spec(installed)
installed.loader.exec_module(server)
index = server.Index(repository, target / 'data')
try:
self.assertEqual(index.search('query'), [])
finally:
index.db.close()
self.assertEqual(len(list((target / 'data').glob('*.sqlite3'))), 1)
server.daemon_authkey(target / 'data')
MODULE.sync(ROOT, home, ['codex'], True, False, True)
self.assertTrue((target / 'data').is_dir())
foreign = target / 'data' / 'foreign.txt'
foreign.write_text('keep')
with self.assertRaisesRegex(ValueError, 'data'):
MODULE.sync(ROOT, home, ['codex'], False, False, False)
self.assertTrue(foreign.exists())
foreign.unlink()
MODULE.sync(ROOT, home, ['codex'], False, False, False)
self.assertFalse(target.exists())
self.assertFalse(state_path.exists())
def test_negative_benchmark_keeps_active_runtime(self):
with tempfile.TemporaryDirectory() as directory:
home = Path(directory) / 'home'
target = home / '.my-ai-configuration/semantic-retrieval'
target.mkdir(parents=True)
state_path = target.parent / 'semantic-retrieval.json'
files = {}
for name in MODULE.FILES:
content = (ROOT / 'shared/retrieval' / name).read_bytes()
(target / name).write_bytes(content)
files[name] = hashlib.sha256(content).hexdigest()
MODULE.save_state(state_path, {'version': 1, 'clients': ['codex'], 'files': files}, False)
result = type('Result', (), {'stdout': '{"recommended": false, "embeddingSeconds": 1, "rerankingSeconds": 1}'})()
with patch.object(MODULE, 'ensure_runtime', return_value=Path('python')), \
patch.object(MODULE.subprocess, 'run', return_value=result):
self.assertFalse(MODULE.benchmark(ROOT, home, False, True))
self.assertTrue(target.is_dir())
self.assertEqual(MODULE.load_state(state_path)['clients'], ['codex'])
if __name__ == '__main__':
unittest.main()
+33 -3
View File
@@ -207,15 +207,45 @@ class SessionStateHookTests(unittest.TestCase):
with self.subTest(shell=shell):
environment = dict(os.environ)
if shell == 'powershell':
environment['LOCALAPPDATA'] = ''
command = [executable, '-NoProfile', '-ExecutionPolicy', 'Bypass', '-File',
str(ROOT / 'shared/hooks/flashbang.ps1')]
environment['FLASHBANG_TEST_SCRIPT'] = str(ROOT / 'shared/hooks/flashbang.ps1')
command = [executable, '-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command',
"function Add-Type { throw 'GUI runtime unavailable' }; & $env:FLASHBANG_TEST_SCRIPT"]
else:
environment['PATH'] = ''
command = [executable, str(ROOT / 'shared/hooks/flashbang.sh')]
result = subprocess.run(command, env=environment, capture_output=True, text=True, timeout=20)
self.assertEqual(result.returncode, 0, result.stderr)
@unittest.skipUnless(os.name == 'nt', 'Windows GUI runtime required')
def test_flashbang_cache_is_optional(self):
executable = shutil.which('powershell') or shutil.which('pwsh')
if not executable:
self.skipTest('PowerShell unavailable')
# Exercise native initialization, stopping at the UI boundary without flashing the screen.
probe = """
function New-Object {
param([string]$TypeName)
if ($TypeName -eq 'System.Windows.Forms.Form' -and ('Flashbang.Native' -as [type])) {
[Console]::WriteLine('OVERLAY_READY')
}
throw 'Stop at UI boundary'
}
& $env:FLASHBANG_TEST_SCRIPT
"""
with tempfile.TemporaryDirectory(prefix='.flashbang-test-', dir=ROOT) as directory:
blocked = Path(directory) / 'not-a-directory'
blocked.write_text('keep')
for cache in ('', str(blocked), directory):
with self.subTest(cache=cache):
environment = dict(os.environ, LOCALAPPDATA=cache,
FLASHBANG_TEST_SCRIPT=str(ROOT / 'shared/hooks/flashbang.ps1'))
result = subprocess.run([executable, '-NoProfile', '-ExecutionPolicy', 'Bypass',
'-Command', probe], env=environment, capture_output=True,
text=True, timeout=20)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn('OVERLAY_READY', result.stdout, 'Flashbang exited before creating the overlay')
self.assertEqual(blocked.read_text(), 'keep')
if __name__ == '__main__':
unittest.main()
+8 -2
View File
@@ -46,6 +46,11 @@ foreach ($shell in @('powershell','bash')) {
if (@(Get-ChildItem "$package/agents" -File).Count -ne $sourceAgentCount) { throw "Missing agents in $package" }
$expectedReviewerShellTool = if ($shell -eq 'powershell') { 'PowerShell' } else { 'Bash' }
if ($client -eq 'claude' -and (Get-Content (Join-Path $package 'agents/reviewer.md') -Raw) -notmatch "(?m)^tools: Read,Grep,Glob,$expectedReviewerShellTool\r?`$") { throw "Reviewer capability profile is missing in $package" }
if ($client -eq 'claude') {
foreach ($role in @('architect','researcher')) {
if ((Get-Content (Join-Path $package "agents/$role.md") -Raw) -notmatch "(?m)^tools: Read,Grep,Glob,WebFetch,WebSearch,$expectedReviewerShellTool\r?`$") { throw "$role research shell capability is missing in $package" }
}
}
$expectedSkillCount = if ($client -eq 'claude') { $sourceSkillCount + $ruleSkillCount } else { $sourceSkillCount }
if (@(Get-ChildItem "$package/skills" -Filter SKILL.md -Recurse).Count -ne $expectedSkillCount) { throw "Missing skills in $package" }
$securityReview = if ($client -eq 'claude') { 'skills/security-review/SKILL.md' } else { 'skills/reviews/security-review/SKILL.md' }
@@ -56,10 +61,11 @@ foreach ($shell in @('powershell','bash')) {
if ($shell -eq 'powershell' -and $content -match 'WindowStyle\s+Hidden') { throw "PowerShell hook hides the terminal in $package" }
if ($client -eq 'codex') {
$events = @([regex]::Matches($content, '(?m)^\[\[hooks\.([^.\]]+)\]\]\s*$') | ForEach-Object { $_.Groups[1].Value })
if ($events.Count -ne 1 -or $events[0] -ne 'Stop' -or $content -match 'flashbang-if-input' -or $content -match '(?m)^async\s*=\s*true\s*$') { throw "Codex finish hook is incorrect in $package" }
if ((($events | Sort-Object) -join ',') -ne 'SessionStart,Stop' -or $content -match 'flashbang-if-input' -or $content -match '(?m)^async\s*=\s*true\s*$') { throw "Codex finish hook is incorrect in $package" }
if ($content -notmatch 'approvals_reviewer\s*=\s*"auto_review"') { throw "Codex auto review is missing in $package" }
if ($content -notmatch '(?m)^sandbox_mode\s*=\s*"danger-full-access"\s*$') { throw "Codex full access is missing in $package" }
if ($content -notmatch '(?m)^max_depth\s*=\s*1\s*$') { throw "Codex agent depth limit is missing in $package" }
if ($content -notmatch 'status_line\s*=\s*\["model", "reasoning", "approval-mode", "project-name", "git-branch", "context-window-size", "context-used", "used-tokens"\]') { throw "Codex status line is incorrect in $package" }
if ($content -notmatch 'status_line\s*=\s*\["model", "reasoning", "fast-mode", "approval-mode", "five-hour-limit", "weekly-limit", "project-name", "git-branch", "context-window-size", "context-used", "used-tokens"\]') { throw "Codex status line is incorrect in $package" }
} elseif ((($settings.hooks.PSObject.Properties.Name | Sort-Object) -join ',') -ne 'PreCompact,SessionStart,Stop' -or $content -match 'flashbang-if-input' -or $settings.hooks.Stop[0].hooks[0].async) {
throw "Claude finish hook is incorrect in $package"
} elseif ($content -notmatch '"defaultMode"\s*:\s*"auto"') {
+8 -2
View File
@@ -52,6 +52,11 @@ for shell in powershell bash; do
expected_reviewer_shell_tool=Bash
[ "$shell" != powershell ] || expected_reviewer_shell_tool=PowerShell
if [ "$client" = claude ]; then grep -q "^tools: Read,Grep,Glob,$expected_reviewer_shell_tool\$" "$package/agents/reviewer.md"; fi
if [ "$client" = claude ]; then
for role in architect researcher; do
grep -q "^tools: Read,Grep,Glob,WebFetch,WebSearch,$expected_reviewer_shell_tool\$" "$package/agents/$role.md" || { printf '%s research shell capability is missing in %s\n' "$role" "$package"; exit 1; }
done
fi
expected_skill_count=$source_skill_count
[ "$client" != claude ] || expected_skill_count=$((source_skill_count + rule_skill_count))
test "$(find "$package/skills" -name SKILL.md | wc -l)" -eq "$expected_skill_count"
@@ -74,12 +79,13 @@ for shell in powershell bash; do
grep -q 'bash .*flashbang.sh' "$package/$file"
fi
if [ "$client" = codex ]; then
[ "$(sed -n 's/^\[\[hooks\.\([^].]*\)\]\]$/\1/p' "$package/$file")" = Stop ]
[ "$(sed -n 's/^\[\[hooks\.\([^].]*\)\]\]$/\1/p' "$package/$file" | sort)" = $'SessionStart\nStop' ]
! grep -q 'flashbang-if-input' "$package/$file"
! grep -Eq '^async[[:space:]]*=[[:space:]]*true[[:space:]]*$' "$package/$file"
grep -q 'approvals_reviewer[[:space:]]*=[[:space:]]*"auto_review"' "$package/$file"
grep -Eq '^sandbox_mode[[:space:]]*=[[:space:]]*"danger-full-access"[[:space:]]*$' "$package/$file"
grep -Eq '^max_depth[[:space:]]*=[[:space:]]*1[[:space:]]*$' "$package/$file"
grep -Fq 'status_line = ["model", "reasoning", "approval-mode", "project-name", "git-branch", "context-window-size", "context-used", "used-tokens"]' "$package/$file"
grep -Fq 'status_line = ["model", "reasoning", "fast-mode", "approval-mode", "five-hour-limit", "weekly-limit", "project-name", "git-branch", "context-window-size", "context-used", "used-tokens"]' "$package/$file"
else
python3 -c 'import json,sys; settings=json.load(open(sys.argv[1], encoding="utf-8-sig")); assert set(settings["hooks"]) == {"PreCompact", "SessionStart", "Stop"}' "$package/$file"
! grep -q 'flashbang-if-input' "$package/$file"
@@ -21,6 +21,10 @@ class UserCustomizationTests(unittest.TestCase):
self.assertEqual(0, result.returncode, result.stderr)
self.assertIn('with-credential.ps1', (ROOT / 'generated/codex-powershell/AGENTS.md').read_text())
self.assertIn('with-credential.sh', (ROOT / 'generated/claude-bash/CLAUDE.md').read_text())
for client, name in (('codex', 'AGENTS.md'), ('claude', 'CLAUDE.md')):
for shell in ('powershell', 'bash'):
self.assertNotIn('.my-ai-configuration/instructions/',
(ROOT / 'generated' / f'{client}-{shell}' / name).read_text())
def test_add_instruction_creates_client_overlays_outside_managed_configuration(self):
powershell = shutil.which('powershell') or shutil.which('pwsh')
+1 -1
View File
@@ -1,3 +1,3 @@
name: architect
description: Technical architecture and design
description: Compare designs and recommend technical boundaries and interfaces before implementation
role: architect
+4
View File
@@ -1,3 +1,7 @@
# Architect
Start from the parent's requirements and references. Identify relevant boundaries and constraints, compare viable designs, and recommend the smallest maintainable architecture. Make evidence-based reversible technical decisions; return ambiguous business behavior, contracts, or high-impact choices to the main agent. Do not invent requirements or broadly explore unrelated code. Use the shared orchestration handoff protocol.
Before accepting a design that relies on external technology behavior, MUST apply the planning checkpoint in `technical-research`. Carry supporting evidence or unresolved assumptions into the handoff.
Use shell commands for inspection and required research tools, including semantic search and MCPorter. Do not modify implementation code unless assigned.
+1 -1
View File
@@ -1,3 +1,3 @@
name: implementer
description: Execute engineering work
description: Implement scoped features and fixes with proportionate tests and self-review
role: implementer
@@ -2,4 +2,6 @@
Understand requirements and existing code, implement the smallest complete solution, follow applicable rules, and choose proportionate verification. Inspect your changes to the degree needed for their risk; a separate or full review is not automatic. Preserve unrelated user work.
For external technology behavior, MUST apply the implementation checkpoint in `technical-research` before coding and its verification checkpoint before reporting completion. Inspect evidence supplied with the plan and reuse it only when it matches actual dependencies and configuration.
Stay within assigned ownership, start at supplied references, and inspect additional context only when needed. Return changes, evidence, and unresolved risks using the shared orchestration handoff protocol; the parent owns shared task state.
+1 -1
View File
@@ -1,3 +1,3 @@
name: researcher
description: Repository and technical investigation
description: Investigate repository behavior, external documentation, and unknowns with source evidence
role: researcher
+2
View File
@@ -1,3 +1,5 @@
# Researcher
Start with the parent's question and references. Inspect only code, patterns, documentation, and constraints needed to answer it; expand when evidence requires it. Return conclusions, concrete references, and unknowns using the shared orchestration handoff protocol. Avoid modifying implementation code by default.
Use shell commands for inspection and required research tools, including semantic search and MCPorter.
+1 -1
View File
@@ -1,3 +1,3 @@
name: reviewer
description: Independent quality review
description: Review diffs for correctness, security, and contract defects without running tests or builds
role: reviewer
+4 -2
View File
@@ -1,7 +1,9 @@
# Reviewer
Provide static and semantic correctness review using the acceptance criteria and actual git diff. Inspect changed symbols first, then direct callers, dependencies, tests, and contracts needed to judge correctness. Expand for shared state, configuration, persistence, security, generated code, or public API effects; avoid broad repository exploration by default. Do not run tests, builds, or runtime checks; identify missing execution evidence for the parent or verifier instead.
Provide static and semantic correctness review using the acceptance criteria and actual git diff. Inspect changed symbols first, then direct callers, dependencies, tests, and contracts needed to judge correctness. Expand for shared state, configuration, persistence, security, generated code, or public API effects; avoid broad repository exploration by default.
Use shell commands to inspect diffs and run required research tools, including semantic search and MCPorter. Do not run tests, builds, or runtime checks; reuse supplied execution evidence and report gaps to the parent or verifier.
For each finding, report SEVERITY (blocking, important, or minor), LOCATION, IMPACT, EVIDENCE, and SCOPE (introduced, pre-existing, or uncertain). Do not report stylistic preferences or pre-existing issues unless they materially affect the requested change. Prioritize actionable correctness defects, keep unrelated cleanup out of scope, and use blocking only for security, data-loss, broken contracts, or deterministic failures.
Check logic, regressions, architecture, maintainability, security, and contract consistency. Do not execute checks or duplicate a verifier's evidence. Follow the shared orchestration handoff protocol; report actionable findings with severity and evidence. Do not modify implementation code unless assigned.
Check logic, regressions, architecture, maintainability, security, and contract consistency. Follow the shared orchestration handoff protocol. Do not modify implementation code unless assigned.
+1 -1
View File
@@ -1,3 +1,3 @@
name: verifier
description: Focused verification of completed work
description: Run focused tests, builds, and reproductions to verify acceptance criteria and report execution evidence
role: verifier
+2
View File
@@ -2,4 +2,6 @@
Provide execution evidence against acceptance criteria. Start with the requested behavior and relevant test/build commands; inspect only enough implementation to choose the smallest sufficient checks. Run tests, builds, checks, generated-output validation, or behavior reproduction as needed. Reuse valid earlier results instead of repeating checks. Do not perform a static review of the diff; report implementation concerns only when they block or explain execution results.
For criteria relying on external technology behavior, MUST apply the verification checkpoint in `technical-research`. Inspect supplied evidence and version applicability; retrieve only missing evidence. Keep documentation support separate from execution results and mark unsupported criteria unverified.
Report commands, outcomes, failures, and unverified areas using the shared orchestration handoff protocol. Do not perform a second general architecture review unless execution exposes a design issue. Do not change implementation code unless assigned.
-2
View File
@@ -1,7 +1,5 @@
# Global Instructions
Read `~/.my-ai-configuration/instructions/__CLIENT__.md` first when present; it is the user's direct instruction. System and developer requirements still apply.
For credentials, run the child command through `~/.my-ai-configuration/bin/with-credential.__SHELL__` with the client and credential key. Never expose credentials in commands, output, logs, or files.
## Loading rules
+3 -1
View File
@@ -2,6 +2,8 @@
This directory contains registered hooks and optional utilities. They do not bypass client permission prompts or grant access.
Both clients register `Stop` (`flashbang`) for finish notifications. Claude also registers `PreCompact` (`Record-Compact`/`record-compact.sh`) and `SessionStart` (`Show-SessionStatePointer`/`show-session-state-pointer.sh`). PreCompact records its timestamp under the installed configuration root. Both session hooks point to an existing workspace `.ai-session/state.json`, using event `cwd` or the process working directory. The main agent owns that state; hooks never write it or print its contents. Missing state is a successful no-op.
When Flashbang is enabled, both clients register `Stop` (`flashbang`) for finish notifications. Claude also registers `PreCompact` (`Record-Compact`/`record-compact.sh`) and `SessionStart` (`Show-SessionStatePointer`/`show-session-state-pointer.sh`). PreCompact records its timestamp under the installed configuration root. Both session hooks point to an existing workspace `.ai-session/state.json`, using event `cwd` or the process working directory. The main agent owns that state; hooks never write it or print its contents. Missing state is a successful no-op.
`Validate-CommandSafety`/`validate-command-safety.sh` and `Invoke-PostChangeVerification`/`invoke-post-change-verification.sh` are not registered or invoked automatically. The safety utility checks selected command-text patterns; the verification utility runs the repository build only when explicitly invoked. `Test-SessionConfig`/`test-session-config.sh` checks repository inputs during a build.
When local QMD search is enabled, both clients also register a `SessionStart` warm-up hook (`Start-QmdWarm.ps1`/`qmd-warm.sh`). It starts the installed Node wrapper in the background from the process working directory, only inside a Git repository. Warm-up errors are recorded in `~/.my-ai-configuration/qmd/warm.log`; they do not fail the session. Disabling local search removes the hook registration.
+12 -12
View File
@@ -66,27 +66,27 @@ $nativeSource = @'
if (-not ('Flashbang.Native' -as [type])) {
# Keyed by host runtime: an assembly emitted by Windows PowerShell (.NET
# Framework) is not guaranteed to load under pwsh (.NET), and vice versa.
$cacheDir = Join-Path $env:LOCALAPPDATA 'claude-flashbang'
$runtimeTag = '{0}{1}' -f $PSVersionTable.PSEdition, $PSVersionTable.PSVersion.Major
$cacheDll = Join-Path $cacheDir "Flashbang.Native.$runtimeTag.dll"
$loaded = $false
try {
$cacheDir = Join-Path $env:LOCALAPPDATA 'claude-flashbang'
$runtimeTag = '{0}{1}' -f $PSVersionTable.PSEdition, $PSVersionTable.PSVersion.Major
$cacheDll = Join-Path $cacheDir "Flashbang.Native.$runtimeTag.dll"
$loaded = $false
if (Test-Path -LiteralPath $cacheDll) {
try { Add-Type -Path $cacheDll; $loaded = $true } catch { $loaded = $false }
}
if (Test-Path -LiteralPath $cacheDll) {
try { Add-Type -Path $cacheDll; $loaded = $true } catch { $loaded = $false }
}
if (-not $loaded) {
try {
if (-not $loaded) {
if (-not (Test-Path -LiteralPath $cacheDir)) {
New-Item -ItemType Directory -Path $cacheDir -Force | Out-Null
}
Add-Type -Namespace 'Flashbang' -Name 'Native' -MemberDefinition $nativeSource `
-OutputAssembly $cacheDll
Add-Type -Path $cacheDll
} catch {
# Cache unavailable (locked file, no write access) - compile in-process.
Add-Type -Namespace 'Flashbang' -Name 'Native' -MemberDefinition $nativeSource
}
} catch {
# Missing profile environment or unusable cache must not suppress the flash.
Add-Type -Namespace 'Flashbang' -Name 'Native' -MemberDefinition $nativeSource
}
}
Loaded 100 of 146 files, more files were not shown because too many files have changed in this diff. Show more