test(benchmarks): add concept-search tasks
Three tasks whose target code shares no vocabulary with the prompt, run in a repository with generated distractor modules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a8f6f01f2f
commit
00a44f50b0
19 files changed
+106
No files matched your search
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"prompt": "Uploads that fail with a temporary network error are never retried. Make at most three attempts in total before giving up with the last error; other errors must not be retried.",
|
||||
"generate": {
|
||||
"filler_modules": 250
|
||||
},
|
||||
"workflow": {
|
||||
"kind": "concept search in a large repository"
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user