Coding tasks (agent with a harness)
Fix or build something in one of our repos; a task passes when its check script passes. 14 shared tasks x 3 runs.
Best go-to: gaents/claude-opus (incumbent stays): the top callable route (86%, interval 72%-93%; $0.9086 capacity per passing run). The cheaper gaents/glm-flash ($0.0703 cash per passing run) does not qualify: mergeable as-is (blind review of passing diffs) 44% is below gaents/claude-opus's interval 70%-100%.
Rule (setting models.go_to_rule, agent-chosen, unruled; Adam sets): Start from the top-scoring route you can call as one gaents alias (the incumbent). A cheaper callable route becomes the go-to only if, on EVERY measured quality column of the task, its score is inside or above the incumbent's 95% interval. If a cheaper route falls below on any column, the incumbent stays and the line says which column. Cost per correct output is ranked on cash plus capacity at list price; the two are shown in separate columns.
Ranked by cash plus capacity per passing run (lowest first; rows with no known cost last); cash and capacity are shown in separate columns. Correct / n is for the first quality column (runs). Source: the swarm agent's coding head-to-head r2-r4 (2026-11-09), three runs per task.
| # | Route | Call it as | runs whose check passes [95% interval] | mergeable as-is (blind review of passing diffs) [95% interval] | Correct / n | Cash per correct | Capacity per correct (not cash) | Price now in/out $/1M |
|---|---|---|---|---|---|---|---|---|
| 1 | glm-flash (opencode harness) | gaents/glm-flash |
79% [64%-88%] | 44% [19%-73%] 4/9 | 33 / 42 | $0.0703 list price (opencode's count); billed at most $2.70 (shared-key upper bound) |
not in source | $0.1741 / $0.5802 |
| 2 | claude-opus (Claude Code harness) | gaents/claude-opus |
86% [72%-93%] | 100% [70%-100%] 9/9 | 36 / 42 | not in source | $0.9086 | $4.6420 / $23.2100 |
Read this with
- Model and harness are measured together (Claude Code vs opencode).
- Mergeable as-is is Judge's blind review of one passing run per task on the 9 tasks both pass (n=9, one Opus reviewer; coding-diff-review-2026-11-17). Its interval is Wilson 95% computed here.
- glm-flash took about 3.1x the wall time.
- Claude cost is API-equivalent capacity on a subscription login; glm-flash cost is list price (billed at most $2.70, a shared-key upper bound).
The measured runs paid the provider's price (or used capacity), not gaents' price; "price now" is what gaents charges today (/v1/models at 2026-11-19T06:07:16Z). Rows dated: 2026-11-09. Every number: /models/models.json.