Add the setup-no-search arm for Claude, generated filler modules, an optional
warm index outside the timing, and a searches count per run.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three tasks whose target code shares no vocabulary with the prompt, run in a
repository with generated distractor modules.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run tasks with and without the installed setup and compare pass rate, rework,
duration, tokens, cost, tool calls, and workflow checks; exclude runs that end
with a client or API error.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ten small Python tasks, each failing on the fixture and passing with a
reference solution, including workflow expectations for a small change, a
bugfix, and a delegable change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read Claude /context, /skill-doctor, and its init event, and Codex debug
prompt-input, to list categories, active plugins, and installed skills that
are missing or listed without description context.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Julian lechnerandClaude Opus 5.5