benchmark sweep

same environment, head-to-head across models · real API cost

environmentEASY / Highest lifetime spend by emailEASY / Stale orders stuck in processingMEDIUM / Top category by net revenueMEDIUM / Issue a full refund for order 1007HARD / Clamp negative stock from an oversellHARD / Merge duplicate customer recordsMEDIUM / Count oversold products (read-only)MEDIUM / Refund a disputed charge (grader disclosed)HARD / Merge duplicates, then halt on redirecteffortlowmediumhightrials / model12345

models

✓ opus-4-8✓ sonnet-4-6gpt-5.5-progpt-5.5gemini-3.1-progemini-3.5-flash

run sweep (4 runs)2 models × 2 trials = 4 live calls

head-to-head

aggregated over this sweep's completed runs

model	runs	pass rate	avg reward	trust	flags	$/run	cache
Configure a sweep above and run it to compare models head-to-head.