test(benchmarks): add concept-search tasks

Three tasks whose target code shares no vocabulary with the prompt, run in a
repository with generated distractor modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Julian lechnerandClaude Opus 5.5 committed 2026-10-01 13:27:56 +02:00
1 parent a8f6f01f2f
commit 00a44f50b0
19 files changed
+106

No files matched your search

@@ -0,0 +1,9 @@
{
"prompt": "Invoice totals are sometimes one cent too low, for example 1.005 shows as 1.00. Round half amounts up to the next cent.",
"generate": {
"filler_modules": 250
},
"workflow": {
"kind": "concept search in a large repository"
}
}