fraujulian/MyAiConfiguration
1
0 Fork 0
Code Issues Pull Requests Actions Activity

perf(retrieval): use bfloat16 on supporting GPUs

Browse Source 
Load both models in bfloat16 when a CUDA GPU supports it and keep float32 on
CPUs. Document the daemon versioning and the indexed file types.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Julian lechnerandClaude Opus 5.5 committed 2026-10-01 13:13:07 +02:00
1 parent f134c50d39
commit 739d2f7801
3 files changed
+19 -4
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/plugins.md
+1 -1
View File
@@ -20,7 +20,7 @@ On Windows, installation and update repair the known Caveman 1.0.0 hook launcher
Normal installation and update ask whether to enable Flashbang, apply the custom status line, and enable the optional Qwen3-Embedding-0.6B semantic retrieval layer. Retrieval uses a per-user virtual environment and index and is invoked through the installed `server.py search` CLI, disabled by default. Quickupdate reuses the saved shell, client, status line, Flashbang, retrieval, and extension choices without opening menus. Disabling the status line removes its custom setting and setup-owned Claude script; disabling retrieval removes its setup-owned runtime and index. Only successful runs save selections; dry runs do not update extensions or retrieval. Install and update do not modify the Codex or Claude CLI installations.
The `server.py search`/`rebuild` CLI transparently reuses a background daemon (spawned automatically on first use per `--data-dir`) that keeps the embedding and reranking models loaded in memory, so only the first query after a cold start or an idle period pays the model-loading cost; every call falls back to loading the models directly in-process if the daemon cannot be reached or started. The daemon exits on its own after 30 minutes of inactivity. Search requires a Git repository and indexes tracked files plus untracked files that Git does not ignore. Indexing skips files larger than 512 KB and, per repository, stops after 2000 indexable files by default; set `SEMANTIC_RETRIEVAL_MAX_FILES` to raise that cap for larger monorepos. When the cap is reached, the CLI prints an explicit warning to stderr rather than silently omitting files.
The `server.py search`/`rebuild` CLI transparently reuses a background daemon (spawned automatically on first use per `--data-dir` and code version, so an updated `server.py` never reuses an older daemon) that keeps the embedding and reranking models loaded in memory, so only the first query after a cold start or an idle period pays the model-loading cost; every call falls back to loading the models directly in-process if the daemon cannot be reached or started. The daemon exits on its own after 30 minutes of inactivity. Search requires a Git repository and indexes tracked files plus untracked files that Git does not ignore. Indexing covers common source, markup, style, SQL, XAML, project, and configuration file types, uses bfloat16 on GPUs that support it, and skips files larger than 512 KB and, per repository, stops after 2000 indexable files by default; set `SEMANTIC_RETRIEVAL_MAX_FILES` to raise that cap for larger monorepos. When the cap is reached, the CLI prints an explicit warning to stderr rather than silently omitting files.
Plugin installation requires the corresponding client CLI, network access, and any authentication required by the plugin. Context7 may open an OAuth flow. Codex or plugin hooks may require a trust review in `/hooks`. The scripts never install plugins during a build or doctor run and never install them when `--dry-run` or `-DryRun` is used.