perf(retrieval): use bfloat16 on supporting GPUs

Load both models in bfloat16 when a CUDA GPU supports it and keep float32 on
CPUs. Document the daemon versioning and the indexed file types.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/plugins.md

@@ -20,7 +20,7 @@ On Windows, installation and update repair the known Caveman 1.0.0 hook launcher
 
 Normal installation and update ask whether to enable Flashbang, apply the custom status line, and enable the optional Qwen3-Embedding-0.6B semantic retrieval layer. Retrieval uses a per-user virtual environment and index and is invoked through the installed `server.py search` CLI, disabled by default. Quickupdate reuses the saved shell, client, status line, Flashbang, retrieval, and extension choices without opening menus. Disabling the status line removes its custom setting and setup-owned Claude script; disabling retrieval removes its setup-owned runtime and index. Only successful runs save selections; dry runs do not update extensions or retrieval. Install and update do not modify the Codex or Claude CLI installations.
 
-The `server.py search`/`rebuild` CLI transparently reuses a background daemon (spawned automatically on first use per `--data-dir`) that keeps the embedding and reranking models loaded in memory, so only the first query after a cold start or an idle period pays the model-loading cost; every call falls back to loading the models directly in-process if the daemon cannot be reached or started. The daemon exits on its own after 30 minutes of inactivity. Search requires a Git repository and indexes tracked files plus untracked files that Git does not ignore. Indexing skips files larger than 512 KB and, per repository, stops after 2000 indexable files by default; set `SEMANTIC_RETRIEVAL_MAX_FILES` to raise that cap for larger monorepos. When the cap is reached, the CLI prints an explicit warning to stderr rather than silently omitting files.
+The `server.py search`/`rebuild` CLI transparently reuses a background daemon (spawned automatically on first use per `--data-dir` and code version, so an updated `server.py` never reuses an older daemon) that keeps the embedding and reranking models loaded in memory, so only the first query after a cold start or an idle period pays the model-loading cost; every call falls back to loading the models directly in-process if the daemon cannot be reached or started. The daemon exits on its own after 30 minutes of inactivity. Search requires a Git repository and indexes tracked files plus untracked files that Git does not ignore. Indexing covers common source, markup, style, SQL, XAML, project, and configuration file types, uses bfloat16 on GPUs that support it, and skips files larger than 512 KB and, per repository, stops after 2000 indexable files by default; set `SEMANTIC_RETRIEVAL_MAX_FILES` to raise that cap for larger monorepos. When the cap is reached, the CLI prints an explicit warning to stderr rather than silently omitting files.
 
 Plugin installation requires the corresponding client CLI, network access, and any authentication required by the plugin. Context7 may open an OAuth flow. Codex or plugin hooks may require a trust review in `/hooks`. The scripts never install plugins during a build or doctor run and never install them when `--dry-run` or `-DryRun` is used.
 

scripts/tests/test-retrieval-server.py

@@ -133,6 +133,13 @@ class RetrievalServerTests(unittest.TestCase):
                 self.assertNotEqual(MODULE.daemon_address(data_dir), old_address)
                 self.assertNotEqual(MODULE.daemon_lock_path(data_dir), old_lock)
 
+    def test_model_kwargs_use_bfloat16_only_on_supporting_gpus(self):
+        bfloat16 = object()
+        for available, supported, expected in [(True, True, {'dtype': bfloat16}), (True, False, {}), (False, True, {})]:
+            cuda = SimpleNamespace(is_available=lambda available=available: available, is_bf16_supported=lambda supported=supported: supported)
+            with patch.dict('sys.modules', torch=SimpleNamespace(cuda=cuda, bfloat16=bfloat16)):
+                self.assertEqual(MODULE.model_kwargs(), expected)
+
     def test_query_prompt_matches_qwen_format(self):
         self.assertTrue(MODULE.QUERY_PROMPT.startswith('Instruct: '))
         self.assertTrue(MODULE.QUERY_PROMPT.endswith('\nQuery:'))
@@ -350,7 +357,7 @@ class RetrievalServerTests(unittest.TestCase):
 
     def test_shared_encoder_and_reranker_are_cached_module_singletons(self):
         with patch.object(MODULE, '_shared_encoder', None), patch.object(MODULE, '_shared_reranker', None), \
-             patch.object(MODULE, 'preferred_device', lambda: 'cpu'):
+             patch.object(MODULE, 'preferred_device', lambda: 'cpu'), patch.object(MODULE, 'model_kwargs', dict):
             sentinel = SimpleNamespace()
             with patch.dict('sys.modules', sentence_transformers=SimpleNamespace(
                     SentenceTransformer=lambda *args, **kwargs: sentinel,

shared/retrieval/server.py

@@ -179,6 +179,12 @@ def preferred_device():
     return "cuda" if torch.cuda.is_available() else "cpu"
 
 
+def model_kwargs():
+    """Use bfloat16 on GPUs that support it; CPUs keep float32, where bfloat16 is often slower."""
+    import torch
+    return {"dtype": torch.bfloat16} if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else {}
+
+
 def read_max_files():
     value = os.environ.get(MAX_FILES_ENV)
     if value is None:
@@ -203,7 +209,8 @@ def shared_encoder():
         if _shared_encoder is None:
             from sentence_transformers import SentenceTransformer
             _shared_encoder = SentenceTransformer(MODEL, revision=MODEL_REVISION, truncate_dim=EMBEDDING_DIMENSIONS,
-                                                   prompts={"query": QUERY_PROMPT}, device=preferred_device())
+                                                   prompts={"query": QUERY_PROMPT}, device=preferred_device(),
+                                                   model_kwargs=model_kwargs())
         return _shared_encoder
 
 
@@ -213,7 +220,8 @@ def shared_reranker():
         if _shared_reranker is None:
             from sentence_transformers import CrossEncoder
             _shared_reranker = CrossEncoder(RERANKER_MODEL, revision=RERANKER_MODEL_REVISION, prompts={"query": RETRIEVAL_INSTRUCTION},
-                                             default_prompt_name="query", device=preferred_device())
+                                             default_prompt_name="query", device=preferred_device(),
+                                             model_kwargs=model_kwargs())
         return _shared_reranker