Industry placement (family level)
Place a company in an industry family from its evidence. A placement is correct when it matches a blind review by six Opus reviewers.
Best go-to: gaents/qwen-flash (family precision 0.77, family recall 0.72; $1.204 cash per 1,000 correct family placements): inside the interval of the top callable route gaents/claude-haiku (0.83, interval 0.77-0.88; its cost: not recomputable at the corrected Haiku price: Pixel's race logged the CLI's total_cost_usd per call (its Haiku call log, usd_notional) and no token counts) on every measured quality column. The cheaper gaents/deepseek ($1.174 cash per 1,000 correct family placements) does not qualify: family precision 0.66 is below gaents/claude-haiku's interval 0.77-0.88; family recall 0.59 is below gaents/claude-haiku's interval 0.70-0.82.
Rule (setting models.go_to_rule, agent-chosen, unruled; Adam sets): Start from the top-scoring route you can call as one gaents alias (the incumbent). A cheaper callable route becomes the go-to only if, on EVERY measured quality column of the task, its score is inside or above the incumbent's 95% interval. If a cheaper route falls below on any column, the incumbent stays and the line says which column. Cost per correct output is ranked on cash plus capacity at list price; the two are shown in separate columns.
Ranked by cash plus capacity per 1,000 correct family placements (lowest first; rows with no known cost last); cash and capacity are shown in separate columns. Correct / n is for the first quality column (graded answers). Source: Pixel's BESTOF-PLACE race 2026-10-24..26 (race_results.tsv, race_cost.tsv, race_combos.txt).
| # | Route | Call it as | family precision [95% interval] | family recall [95% interval] | Correct / n | Cash per 1,000 correct | Capacity per 1,000 correct (not cash) | Price now in/out $/1M |
|---|---|---|---|---|---|---|---|---|
| 1 | deepseek | gaents/deepseek |
0.66 [0.58-0.72] | 0.59 [0.52-0.66] 117/198 | 117 / 178 | $1.174 | not in source | $0.3017 / $0.4410 |
| 2 | qwen-flash | gaents/qwen-flash |
0.77 [0.71-0.83] | 0.72 [0.66-0.78] 143/198 | 143 / 185 | $1.204 | not in source | $0.1741 / $0.5454 |
| 3 | kimi (no gaents alias) | not one alias | 0.78 [0.67-0.86] | 0.66 [0.55-0.75] 56/85 | 56 / 72 | $8.645 | not in source | - |
| 4 | grok | gaents/grok |
0.80 [0.58-0.92] | 0.53 [0.36-0.70] 16/30 | 16 / 20 | $13.78 | not in source | $1.4506 / $2.9013 |
| 5 | claude-sonnet | gaents/claude-sonnet |
0.79 [0.73-0.84] | 0.73 [0.66-0.78] 144/198 | 144 / 182 | not in source | $54.99 | $2.3210 / $11.6050 |
| 6 | claude-haiku and claude-sonnet agree (two calls) | not one alias | 0.92 [0.86-0.96] | 0.65 [no interval] | 129 / 140 | not in source | not in source | - |
| 7 | claude-haiku and deepseek agree (two calls) | not one alias | 0.92 [0.85-0.95] | 0.49 [no interval] | 97 / 106 | not in source | not in source | - |
| 8 | qwen-flash and deepseek agree (two calls) | not one alias | 0.92 [0.85-0.95] | 0.49 [no interval] | 97 / 106 | not in source | not in source | - |
| 9 | qwen-flash and claude-haiku agree (two calls) | not one alias | 0.91 [0.85-0.95] | 0.67 [no interval] | 132 / 145 | not in source | not in source | - |
| 10 | claude-haiku | gaents/claude-haiku |
0.83 [0.77-0.88] | 0.76 [0.70-0.82] 151/198 | 151 / 182 | not in source | not in source not recomputable at the corrected Haiku price: Pixel's race logged the CLI's total_cost_usd per call (its Haiku call log, usd_notional) and no token counts |
$0.1161 / $0.5802 |
Read this with
- Opus is the answer key, not an arm: nothing here measures a route against Opus (Pixel).
- n=228 companies gives roughly +/-0.06-0.10; strict scoring understates every arm (16 of 31 disagreements defensible both ways) (Pixel).
- kimi ran on 114 companies and grok on 48, not 228.
- Cost is the family stage only. Claude arms ran on a subscription login: capacity, in its own column. Haiku's placement capacity is not shown: the race logged the CLI's dollar figure but no token counts, so it cannot be recomputed at the corrected Haiku price. Two-call combinations have no cost in the source.
- Recall intervals are Wilson 95% computed here from Pixel's counts (family_right / truth_family_decidable); precision intervals are Pixel's.
The measured runs paid the provider's price (or used capacity), not gaents' price; "price now" is what gaents charges today (/v1/models at 2026-11-19T06:07:16Z). Rows dated: 2026-10-26T06:30:13Z, 2026-10-26T06:30:24Z. Every number: /models/models.json.