Two brains, one Texas library. Here's exactly where each wins.

Choose your engine

Same battery, same items, four engines. The private tier runs on hardware you control and is priced by contract; the three paid tiers are current OpenRouter models at their published list rates. Every percentage below is a measured score on the items described under the table — not a vendor claim.

Same 300 bait moments and 300 statute items for every engine (seeded draw, seed 20260709, deterministic scoring). Prices are OpenRouter list rates read from their API on 2026-08-21; the private tier runs on our own hardware and is quoted by contract.

What this run cost us: $0.00 of real OpenRouter billing across all three paid tiers — each completion priced by OpenRouter’s own reported charge, not an estimate. The private tier cost $0 in tokens: it is our hardware. Provider reasoning was turned off for every paid tier (a live coaching card has a seconds-level budget; thinking modes blow it), which is why the cross-check row — the same Opus 4.8 measured earlier through Anthropic’s API with adaptive thinking on — is shown beside it rather than merged into it.

The benchmark — every test, both engines, one table

Both engines run identical inputs against the same full Texas library (615,705 opinions + 307 TX Code sections) and are scored by the same deterministic rubric. The leader in each row is bold; rows where the result spans both columns are engine-independent — they ship identically with either brain.

The full-library finding: moving from a small test index to the full library made the test harder — real, on-point Texas case law in the window tempts a model to coach through moments where the correct move is silence. Discipline, not fluency, separates the engines.

The coaching-LLM axes run per-engine. Claude Opus 4.8 ran a random seeded n=300 sample (600 completions); the private studycase-14b runs the full set on the live brain pod (Qwen2.5-14B-AWQ). For a true apples-to-apples read, the table above scores the 14b on Opus’s exact same 300 items (not the full set) — identical inputs, identical N; the head-to-head then reads by 95% Wilson intervals (non-overlapping = a real difference).

The citation-gate finding: our source-reading gate let 0 of 2,000 fake citations through (the industry-default string check let 720 of 2,000 through, 36.0%, and missed all 638 disguised case swaps). The price of that zero: the strict gate holds 56.2% of good citations for re-verification instead of silently trusting them — by design.

On-prem will be published on this same battery, versioned and dated, before it ships. No numbers claimed until then.

Difficulty was pre-registered. The generator and seed reproduce every item and its label, fixed before either model was scored. Citation corruption families: genuine: 2,000 · sibling indexed: 638 · page delta: 651 · reporter swap: 316 · vol delta: 395. Confidence intervals computed per Anthropic's 'Adding Error Bars to Evals' (Miller, 2024).

The honest read

Under real retrieval both tiers improve across every axis. 14B’s edge is the coach axis (more directive next-steps: 1.68 vs 1.32); Opus leads on no-invention (2.0 vs 1.933). Correct-issue is now a dead tie (1.92/1.92). The gap is a coaching-directness vs invention-discipline tradeoff, not a capability chasm.