benchmark sweep

same environment, head-to-head across models · real API cost

environmentEASY / Highest lifetime spend by emailEASY / Stale orders stuck in processingMEDIUM / Top category by net revenueMEDIUM / Issue a full refund for order 1007HARD / Clamp negative stock from an oversellHARD / Merge duplicate customer recordseffortlowmediumhightrials / model12345

models

✓ opus-4-8✓ sonnet-4-6haiku-4-5

run sweep (4 runs)2 models × 2 trials = 4 live calls

head-to-head

aggregated over this sweep's completed runs

model	runs	pass rate	avg reward	$/run	latency	cache
Configure a sweep above and run it to compare models head-to-head.