Commit Graph
2 Commits
Author SHA1 Message Date
Julian lechnerandClaude Opus 5.5 e1493d9758 feat(benchmarks): compare sessions with and without semantic search
Add the setup-no-search arm for Claude, generated filler modules, an optional
warm index outside the timing, and a searches count per run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 13:27:57 +02:00
Julian lechnerandClaude Opus 5.5 89bee27dd2 feat(benchmarks): add setup versus baseline A/B runner
Run tasks with and without the installed setup and compare pass rate, rework,
duration, tokens, cost, tool calls, and workflow checks; exclude runs that end
with a client or API error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 12:45:41 +02:00