Commit Graph
5 Commits
Author SHA1 Message Date
Julian lechnerandClaude Opus 5.5 5b098fef70 test(benchmarks): phrase the transient retry prompt as a task
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:40:22 +02:00
Julian lechnerandClaude Opus 5.5 449ffa3631 test(benchmarks): tag tasks as small change, larger implementation, or search
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 10:34:02 +02:00
Julian lechnerandClaude Opus 5.5 f43a2df14b test(benchmarks): add C#/ASP.NET and TypeScript tasks
Six tasks with hidden checks: ASP.NET validation and 404 handling, C#
cancellation, and TypeScript overlap, deep merge, and debounce fixes, checked
with dotnet run and node --test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 13:59:12 +02:00
Julian lechnerandClaude Opus 5.5 00a44f50b0 test(benchmarks): add concept-search tasks
Three tasks whose target code shares no vocabulary with the prompt, run in a
repository with generated distractor modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 13:27:56 +02:00
Julian lechnerandClaude Opus 5.5 3480072ed7 test(benchmarks): add A/B task fixtures with hidden checks
Ten small Python tasks, each failing on the fixture and passing with a
reference solution, including workflow expectations for a small change, a
bugfix, and a delegable change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 12:45:40 +02:00