{"meta":{"generated":"2026-11-05 10:30 UTC","updated":"2026-11-05","commit":"9e50c421","schema":1,"public":true},"changelog":{"schema":1,"entries":[{"date":"2026-11-05","title":"Round 1 results","models":[{"model":"muse-wonder-full","as":"judge and lead"},{"model":"gemini-4-argon","as":"subagent"}],"items":["Five lead models each wrote briefs for all five tasks: Opus 5.5 xhigh, GPT‑6.1 Sol max, GPT‑6 Astra xhigh, GPT‑6.1 Terra max and Muse Wonder Full xhigh. Pick a lead at the top, or All leads to compare them.","23 subagent configurations ran from every lead's briefs: 50 runs and 575 entries (2 did not finish), each graded by a hidden suite and scored blind by a five-judge council (Opus 5.5, GPT‑6.1 Sol, GPT‑6.1 Terra, DeepSeek V4.1 Flash, Muse Wonder Full).","One run per configuration per lead and task: treat close values within a single cell as ties.","Top picks rank by value (council score − 0.2 × log₂ cost − 0.5 × finding points) among entries passing at least 98% of hidden categories.","A Findings section summarises what stood out, with its evidence; notable entries carry a note in their expanded row.","The 2026-10-31 pilot round is not shown: its lead was the benchmark's own orchestrator, which wrote its briefs knowing the hidden suites."]}]},"pricing":{"schema":1,"source":"models.dev catalog as cached by OpenCode","retrieved":"2026-11-01","notes":["Costs on the site are list price at the standard tier and base context band: uncached input + cache reads + cache writes + output (reasoning included in output).","Claude Code writes 1-hour cache entries (cache_write_1h); other harnesses report no cache writes unless listed.","Fast/priority tiers multiply the whole bill (tiers.priority / tiers.fast).","Plans (e.g. the Z.ai coding plan) bill $0; the list price is still used for comparison."],"models":[{"model":"claude-opus-5.5","catalog":"anthropic/claude-opus-5-5","currency":"USD","unit":"per million tokens","input":4,"output":20,"cache_read":0.2,"cache_write_5m":5,"cache_write_1h":8,"reasoning_note":"reasoning tokens are billed as output","region_multiplier":{"us":1.1},"tiers":{"fast":2}},{"model":"claude-fable-5.1","catalog":"anthropic/claude-fable-5-1","currency":"USD","unit":"per million tokens","input":10,"output":50,"cache_read":0.25,"cache_write_5m":12.5,"cache_write_1h":20,"reasoning_note":"reasoning tokens are billed as output","source_note":"Anthropic list price (input, output, cache read). Cache writes use the standard 1.25x (5m) and 2x (1h) input multipliers; not yet in the models.dev cache.","tiers":{}},{"model":"claude-haiku-5.5","catalog":"anthropic/claude-haiku-5-5","currency":"USD","unit":"per million tokens","input":0.1,"output":0.5,"cache_read":0.01,"cache_write_5m":0.125,"cache_write_1h":0.2,"long_context":{"when":{"context":{"gt":100000}},"input":0.5,"output":2.5,"cache_read":0.05,"cache_write_5m":0.625,"cache_write_1h":1.0},"reasoning_note":"reasoning tokens are billed as output","source_note":"Anthropic list price: $0.10/$0.50 for prompts up to 100K tokens, $0.50/$2.50 beyond. Cache reads 0.1x and writes 1.25x (5m) / 2x (1h) of input in each band. Claude Code requests are priced one by one in their band.","tiers":{}},{"model":"gpt-6.1-sol","catalog":"openai/gpt-6.1-sol","currency":"USD","unit":"per million tokens","input":2,"output":10,"cache_read":0.1,"cache_write":2.5,"reasoning_note":"reasoning tokens are billed as output","long_context":{"when":{"context":{"gt":272000}},"input":4,"cache_read":0.2,"cache_write":5,"output":15},"tiers":{"priority":2,"ultrafast":6,"flex":0.5}},{"model":"gpt-6.1-terra","catalog":"openai/gpt-6.1-terra","currency":"USD","unit":"per million tokens","input":1,"output":5,"cache_read":0.05,"cache_write":1.25,"reasoning_note":"reasoning tokens are billed as output","long_context":{"when":{"context":{"gt":272000}},"input":2,"cache_read":0.1,"cache_write":2.5,"output":7.5},"tiers":{"priority":2,"flex":0.5}},{"model":"gpt-6-astra","catalog":"openai/gpt-6-astra","currency":"USD","unit":"per million tokens","input":10,"output":50,"cache_read":1,"cache_write":12.5,"reasoning_note":"reasoning tokens are billed as output","long_context":{"when":{"context":{"gt":272000}},"input":20,"cache_read":2,"cache_write":25,"output":75},"tiers":{"priority":2,"ultrafast":6,"flex":0.5}},{"model":"deepseek-v4.1-flash","catalog":"deepseek/deepseek-flash","currency":"USD","unit":"per million tokens","input":0.15,"output":0.6,"cache_read":0.003,"reasoning":0.6,"reasoning_note":"reasoning tokens are billed as output","peak_multiplier":{"factor":2,"when":"CN weekday peak hours (UTC 01-04, 06-10, Mon-Fri, not CN holidays)"}},{"model":"glm-5.3-flash","catalog":"zai/glm-5.3-flash","currency":"USD","unit":"per million tokens","input":0.15,"output":0.5,"cache_read":0.03,"cache_write":0,"reasoning_note":"reasoning tokens are billed as output"},{"model":"mimo-v2.6-pro","catalog":"opencode-go/mimo-v2.6-pro","currency":"USD","unit":"per million tokens","input":0.435,"output":0.87,"cache_read":0.003625,"reasoning_note":"reasoning tokens are billed as output"},{"model":"qwen3.8-flash","catalog":"opencode-go/qwen3.8-flash","currency":"USD","unit":"per million tokens","input":0.15,"output":0.47,"cache_read":0.016,"cache_write":0.2,"reasoning_note":"reasoning tokens are billed as output"},{"model":"grok-4.7","catalog":"opencode-go/grok-4.7","currency":"USD","unit":"per million tokens","input":2,"output":6,"cache_read":0.5,"reasoning_note":"reasoning tokens are billed as output","long_context":{"when":{"context":{"gt":200000}},"input":4,"cache_read":1,"cache_write":null,"output":12}},{"model":"muse-wonder-full","catalog":null,"currency":"USD","unit":"per million tokens","input":4,"output":20,"cache_read":0.2,"cache_write":0,"reasoning_note":"reasoning tokens are billed as output","price_basis":"proxy","plan":"Muse Code subscription (Power $50/month, Maximum $200/month); runs bill $0 per token","source_note":"Muse Wonder Full has no per-token API price (subscription only, 2026-11-01). Costs use the Muse Wonder 1.0 API rate ($4 input, $0.20 cached input, $20 output per million tokens) as a proxy, so they can be compared; the real per-token price of Full is likely higher.","tiers":{}},{"model":"gemini-4-argon","catalog":"gemini-4-argon","currency":"USD","unit":"per million tokens","input":2,"output":10,"cache_read":0.1,"reasoning_note":"thinking tokens are billed as output","note":"Gemini API standard tier introductory price through 2026-11-30 (then $4 / $20 / $0.20); third-party summaries of Google's pricing page, retrieved 2026-11-02"}]},"judges":[{"id":"opus-5.5-xhigh","label":"Opus 5.5 xhigh","model":"claude-opus-5.5","effort":"xhigh","status":"active","since":"2026-10-31","legacy_letter":"R","short":"Opus 5.5"},{"id":"muse-wonder-full-xhigh","label":"Muse Wonder Full xhigh","model":"muse-wonder-full","effort":"xhigh","status":"active","since":"2026-11-01","short":"Muse Wonder Full"},{"id":"gpt-6-astra-xhigh","label":"GPT‑6 Astra xhigh","model":"gpt-6-astra","effort":"xhigh","service_tier":"default","status":"retired","since":"2026-11-01","short":"GPT‑6 Astra","retired":"2026-11-03","retired_note":"Owner decision 2026-11-03: about 40% of phase-1b council cost while barely moving rankings (rank correlation 0.985 without it); the 2026-11-01 trial found nothing Sol and Terra missed. Its phase-1b scores and findings stay in the records but no longer count in any panel."},{"id":"gpt-6.1-sol-max","label":"GPT‑6.1 Sol max","model":"gpt-6.1-sol","effort":"max","service_tier":"default","status":"active","since":"2026-10-31","legacy_letter":"Q","short":"GPT‑6.1 Sol"},{"id":"gpt-6.1-terra-max","label":"GPT‑6.1 Terra max","model":"gpt-6.1-terra","effort":"max","service_tier":"default","status":"active","since":"2026-10-31","legacy_letter":"P","short":"GPT‑6.1 Terra"},{"id":"deepseek-v4.1-flash-max","label":"DeepSeek V4.1 Flash max","model":"deepseek-v4.1-flash","effort":"max","status":"active","since":"2026-10-31","legacy_letter":"S","short":"DS V4.1 Flash"}],"site":{"repo":{"label":"Repo","url":"https://gitgud.io/mike/code-handoff-bench"},"base_url":"https://mikesulsenti.com/ai/code-handoff-bench/","base_url_note":"Public address of the results site; used for the canonical link and the social preview image. Every asset path is relative, so the site works under any prefix."},"faq":{"schema":1,"note":"Shown on the site as the FAQ card. {final:CONFIG} and {cost:CONFIG} are filled in from the data: that configuration's overall final score and mean cost per task, over every lead.","questions":[{"q":"What do the lead and the subagents do?","a":["A lead model reads each task's contract and writes a brief: the hand-off a developer would give a colleague. Coding subagents (every model, at every effort level) each get one lead's brief and write the deliverable on their own.","So a subagent's results show how well it codes from a brief, and a lead's results show how well subagents do with its briefs. Switch leads at the top, or pick All leads to compare them."]},{"q":"Do the judges know which model wrote the code?","a":["No. Each judge reviews entries under random letters, with model and vendor names removed, next to a few already-scored entries from the same run that calibrate its scale. Judges score against the task's contract and a fixed rubric, not against the lead's brief, so scores compare across leads."]},{"q":"Why aren't the briefs, task contracts and hidden tests published?","a":["So they stay out of model training data and can't be tuned against while the tasks are in use. The site publishes the task summaries, what each brief cost to write, and every score and council finding."]},{"q":"Can I run the benchmark with a local model?","a":["Not with a small one. A Qwen3.8 27B optimized model (Qwopus 3.8 27B Flash V2, 4-bit, run locally at its highest reasoning effort) could not finish the benchmark tasks: on every task it fell into repeating reasoning or hit its output limit, and it passed only a fraction of the hidden checks. Its trial is not part of the results.","The tasks are long agentic sessions, so a flash-level model is the practical floor. DeepSeek V4.1 Flash (1017) is the one we'd recommend: at max effort it averages a final score of {final:deepseek-v4.1-flash@max} across every lead, at about {cost:deepseek-v4.1-flash@max} per task. Qwen3.8-Flash-Next is the other we'd recommend; it hasn't been benchmarked here yet."]},{"q":"Why do some configurations score near 0 on a task?","a":["Usually because the subagent stopped early. Some low-effort configurations write a first piece, say the work is incomplete and end their turn; others break down during long sessions. Each entry is graded as submitted. A provider fault (an empty or cut-off reply, a lost tool call) is rerun; the model's own behaviour is not."]},{"q":"Why do some runs have fewer judges than others?","a":["A judge counts for a run only if it reviewed every entry in that run, so every entry in a run is scored by the same judges. When a judge's subscription limit runs out before it can review an entry, that run is scored by the remaining judges."]},{"q":"Why does Leads compared use the median?","a":["Every lead runs the same roster of models, so weak models don't favour any lead, but one breakdown near 0 can move a lead's mean by about a third of a point and reorder the leads. The median counts every entry and barely moves. The mean is shown on hover, and the strip draws every configuration."]}]},"findings":{"schema":1,"generated":"2026-11-05","source":"analysis by two Opus 5.5 xhigh subagents (quantitative and qualitative) over every score, council review and run log; numbers checked by the operator","findings":[{"id":"opus-lead-lifts-quality","title":"Opus 5.5's briefs lift the same subagents by about 0.3 points, and every judge sees it","summary":"Comparing each configuration with itself on the same task under all five leads, entries written from Opus 5.5 xhigh briefs score highest on average, and GPT-6.1 Terra max briefs lowest. All five judges see the same Opus-lead lift on their own scores. It is larger where every lead's entry passed the full hidden suite, so it reflects code quality, not correctness. The lift is largest for mid-tier configurations and on exr-piz and vp8lf-neon. It is near zero for the strongest configurations and slightly negative on cdef-avx512 and bounded-tool. One caveat: Opus 5.5 also wrote the task contracts and built the task kits, which could favour its own briefs.","evidence":["Paired lead effect on final score (entry minus mean of the same config×task over the 5 leads, 113 complete cells): Opus +0.29 [95% bootstrap +0.08,+0.49], Sol +0.22 [+0.06,+0.38], Muse +0.05 [-0.13,+0.23], Astra -0.18 [-0.34,-0.02], Terra -0.37 [-0.62,-0.16].","Opus-lead entry has the best final in 52 of 113 cells (Sol 22, Astra 15, Muse 15, Terra 12; ties counted for each); Opus is the best lead for 12 of 23 configurations.","Same comparison on each judge's own score: Opus judge +0.39, Sol +0.35, Terra +0.34 (91 cells), Muse +0.24, DeepSeek +0.13 for the Opus lead.","In the 79 cells where all five leads' entries passed every hidden category: Opus lead +0.42 [+0.32,+0.53] on final, Sol +0.07, Terra -0.12, Astra -0.18, Muse -0.19.","By configuration tier (config mean final ≥7.9 / 6.7–7.9 / <6.7): Opus lead +0.10 [-0.13,+0.30] / +0.48 [+0.33,+0.62] / +0.23 [-0.41,+0.78].","By task (Opus lead): range-coder +0.36, exr-piz +0.71, cdef-avx512 -0.18, vp8lf-neon +0.71, bounded-tool -0.18.","Excluding GLM-5.3-Flash (the most breakdown-prone config): Opus +0.30, Sol +0.18, Muse -0.03, Astra -0.22, Terra -0.23.","Removing the Terra judge from every run (it is missing from the Muse-lead exr-piz panel) barely changes the effects: Opus +0.27, Sol +0.17, Muse +0.06, Astra -0.12, Terra -0.38.","Briefs are cheap next to their effect: all 25 briefs cost $17.18 at list price (Opus $9.04, Muse $3.89, Astra $3.38, Sol $0.63, Terra $0.23), against $760.61 for the 572 contestant entries.","Caveat: one run per cell (n=1). Opus 5.5 also wrote the task contracts and built the kits (see the methodology's known biases), although the Opus lead was briefed with the same short prompt and inputs as the others."],"scope":{"lead":"opus-5.5-xhigh","task":null,"config":null},"strength":"strong"},{"id":"terra-low-effort-honest-stubs","title":"GPT-6.1 Terra at medium and high effort gives up on the assembly tasks, and says so","summary":"On the two SIMD assembly tasks (and twice on range-coder) Terra at medium or high effort ended within a minute with the scaffold stubs, or one function of two, and a final report that said the work was incomplete. All 18 such entries said so; none claimed success. The same model at xhigh and max passed every hidden category on all 10 of its SIMD-task entries. The cliff is between high and xhigh, and it shows only on the hard tasks: on exr-piz and bounded-tool, Terra@medium and @high always shipped an implementation.","evidence":["18 stub or partial entries: Terra@medium on cdef-avx512 (5/5: Astra F, Sol E, Terra W, Muse M, Opus I; hidden 1/15 each), vp8lf-neon (5/5: Astra X, Sol N, Terra E, Muse T, Opus X; 3/13 each) and range-coder (2/5: Muse T, Opus I; 0/11); Terra@high on cdef-avx512 (Astra L, Sol L, Opus C; 3/15, direction search only) and vp8lf-neon (Sol Z, Terra Y, Muse H; 3/13).","Those 18: wall time 0.15-0.99 min, 1-7 tool calls, 237-2,498 output tokens, council 0.0-2.4, $0.47 in total (Codex rollouts and data/runs usage).","Final messages, e.g. Opus-lead cdef I: 'The implementation remains unfinished. Both assembly entry points are still stubs, and I made no source changes.'; Sol-lead cdef E: 'The implementation is incomplete: both assembly entry points remain stubs'. All 18 admit incompleteness (Codex task_complete last_agent_message).","Terra@xhigh and Terra@max on cdef-avx512 and vp8lf-neon: 10/10 full hidden passes each; lowest council score 6.0 (xhigh) and 7.4 (max).","Operator notes: 'Codex Terra at medium and high effort often stopped with stubs ... model choice, graded as is.'","All 28 entries in the round with a council score below 3 come from five configurations: Terra@medium 13, Terra@high 6, GLM-5.3-Flash@max 5, Qwen3.8-Flash@xhigh 3, Gemini 4 Argon@high 1."],"scope":{"lead":null,"task":null,"config":"gpt-6.1-terra@medium"},"strength":"strong"},{"id":"brief-gaps-cause-hidden-misses","title":"For solid configurations, hidden-suite misses come almost entirely from two briefs","summary":"Sixteen configurations (mean final ≥6.7) pass every hidden category in 15–16 of 16 entries in 23 of the 25 lead×task cells. The two exceptions are GPT-6 Astra's bounded-tool brief (7 of 16) and GPT-6.1 Terra's exr-piz brief (12 of 16). In each, most failures are the same single category, which is failed rarely or never under the other leads. This points to something the brief left out or changed, not to the subagents.","evidence":["Solid configs passing all hidden categories, per lead×task (range-coder, exr-piz, cdef-avx512, vp8lf-neon, bounded-tool): Opus 16,16,16,16,15; Sol 16,16,16,16,15; Astra 16,16,16,16,7; Terra 16,12,16,16,14; Muse 16,16,16,16,14 (each of 16).","bounded-tool: one hidden category is failed by 14/23 Astra-lead entries, 7/23 Terra-lead, 6/23 Muse-lead, 3/23 Sol-lead and 0/23 Opus-lead entries. Astra-lead bounded-tool: 12 of 23 entries miss exactly one category.","exr-piz: one hidden category is failed by 7/23 Terra-lead entries, against 0 Opus, 2 Sol, 1 Astra and 0 Muse; Terra-lead exr-piz has 8/23 entries with any miss, against 0–2 for every other lead.","Cost of the Astra-lead miss: the 14 affected entries lose 0.45 final points on average from the hidden-share multiplier.","Categories are deliberately not named (public-safe)."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"council-predicts-hidden","title":"The council never scored a hidden-suite failure 8 or higher; GPT-6.1 Sol is the best single detector","summary":"Judges never see hidden-suite results, yet the council score separates entries that pass every category from those that fail one or more almost perfectly. That includes the hard case of a single failed category. GPT-6.1 Sol max separates them best of the five judges. Opus 5.5, Muse and DeepSeek are noticeably weaker on single-category misses.","evidence":["77 of 572 entries fail at least one hidden category; their highest council score is 7.9 (Sol-lead bounded-tool E, Terra max, 15/16) and their median 5.0, against 7.7 for passing entries.","AUC (probability that a passing entry outscores a failing one): council 0.959 overall, 0.918 for the 34 single-category failures.","Per-judge AUC on single-category failures: Sol 0.953, Terra 0.860, DeepSeek 0.778, Opus 0.767, Muse 0.762.","Judge offset on failing entries (vs the other judges' mean): Sol -0.91, Terra -0.38, Opus -0.13, Muse +0.69, DeepSeek +0.74."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"cost-pareto","title":"GPT-6.1 Terra max gets 90% of the top configuration's final score for 4% of its cost","summary":"On the cost-quality frontier (mean list cost per entry against mean final over 25 entries), every point is a GPT-6.1 Terra, DeepSeek V4.1 Flash, GPT-6.1 Sol or Opus 5.5 configuration. Opus 5.5 high and xhigh are the only frontier configurations that pass all 335 hidden categories. Grok 4.7 xhigh also passes all 335 but sits well below the frontier.","evidence":["Frontier: Terra xhigh $0.097 → 6.78; DeepSeek V4.1 Flash max $0.164 → 7.24; Terra max $0.227 → 7.98; Sol xhigh $0.370 → 8.16; Sol max $0.435 → 8.26; Opus high $2.75 → 8.63; Opus xhigh $5.74 → 8.90.","Terra max / Opus xhigh: 7.98/8.90 = 90% of the final score at $0.227/$5.74 = 4% of the cost. Cost per final point: Terra max $0.028, Sol max $0.053, Opus high $0.319, Opus xhigh $0.645.","Configs passing all 335 hidden categories: Opus 5.5 xhigh, Opus 5.5 high, Grok 4.7 xhigh ($3.26, final 6.96, council 7.31).","Value ranking (score − 0.2·log2 cost − 0.5·defect points, ≥98% hidden): Opus xhigh 7.94, Opus high 7.82, Sol max 7.81, Sol xhigh 7.75, Terra max 7.50.","Cheapest entries passing all hidden categories with final ≥8.5: Muse-lead range-coder Z (Terra max, $0.135, 8.85), Sol-lead range-coder Y (Terra max, $0.172, 8.50), Muse-lead cdef-avx512 B (Terra max, $0.202, 8.50)."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"effort-scaling-by-family","title":"Extra effort pays all the way for Terra, plateaus at xhigh for Sol and after high for Muse","summary":"Within each family, higher reasoning effort raises the final score, but the gains differ widely. GPT-6.1 Terra goes from mostly broken at medium to near the top at max, and each step wins in at least 24 of 25 paired comparisons. GPT-6.1 Sol gains about one point per step up to xhigh, then almost nothing at max. Opus 5.5 shows diminishing returns at twice the price. Muse Wonder Full is flat beyond high.","evidence":["GPT-6.1 Terra medium/high/xhigh/max final 2.53/4.54/6.78/7.98, hidden 181/264/332/334 of 335, mean cost $0.03/$0.05/$0.10/$0.23. Paired steps: +2.01 (24/25 wins), +2.25 (25/25), +1.20 (24/25).","GPT-6.1 Sol medium/high/xhigh/max final 6.26/7.19/8.16/8.26; steps +0.93 (21/25), +0.97 (22/25), +0.10 (16/25 wins, 9 losses) at a median cost ratio of ×1.21.","Opus 5.5 medium/high/xhigh final 8.13/8.63/8.90; steps +0.50 (21/25, cost ×1.41), +0.27 (15/25, cost ×2.17); mean cost $1.95/$2.75/$5.74.","Muse Wonder Full medium/high/xhigh final 6.93/7.18/7.17; high→xhigh -0.00 (12 wins, 13 losses), cost ×1.07.","GPT-6 Astra medium/high/xhigh final 5.95/7.15/7.96; steps +1.20 (25/25), +0.81 (22/25), cost ×1.61, ×1.95.","Per task, Terra medium passes every hidden category in 0/5 cdef-avx512, 0/5 vp8lf-neon and 0/5 bounded-tool entries, and Terra high in 2/5, 2/5 and 1/5."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"failing-open-weight-entries-report-done","title":"Open-weight entries that failed the hidden suite mostly reported that they were done","summary":"Of the 21 OpenCode-harness entries (GLM-5.3-Flash, Qwen3.8-Flash, DeepSeek V4.1 Flash) that failed at least one hidden category, 14 ended with a completion report ('Done', 'Implementation complete', 'All work is complete'). Only one said it was incomplete, and 6 ended mid-task. Several broken entries claimed that every gate passed, and judges then found that their own tests locked in the bug. Terra's early stops (previous finding) were candid every time, so how far a self-report can be trusted depends on the model, not on whether the work passed.","evidence":["Classification of the final assistant text in opencode.db for the 21 hidden-failing OpenCode entries, keyword match then checked by hand: GLM 8 completion claims, 1 admission, 3 mid-task; Qwen 5 claims, 3 mid-task; DeepSeek 1 claim.","Terra-lead range-coder I (GLM, 5/11 hidden, council 2.9): 'Done. The implementation is complete ... All four gates pass ... (19 passed)'. All five judges found two critical/high correctness bugs; DeepSeek s1, Opus s1 and Terra s1 each filed a tests finding that the suite enshrines or enforces the wrong behaviour.","Astra-lead exr-piz X (Qwen, 4/12, council 4.2): 'All work is complete.' Judges: 'Valid bitmaps can corrupt output or panic' (Sol s2), 'wrong output ... and a panic on a full bitmap' (Opus s2).","Opus-lead exr-piz L (GLM, 8/12): 'Done. All gates pass ... plus 13.5M fuzz executions.' Opus s2: 'Valid chunks with 58-bit codes are rejected or silently decoded wrong' (verified defect EXR-003).","The only admission: Terra-lead cdef-avx512 L (GLM on OpenCode Go, 3/15): 'Status: incomplete. Here is an honest accounting...'.","Mid-task endings: Qwen Sol-lead and Terra-lead exr-piz N (0/12 each) ended their turns after text, graded as is under the owner's rule that this is a model choice; Qwen Opus-lead cdef V (9/15) was cut off mid-debugging by a provider failure in its last session, and the partial work was graded; GLM Opus-lead and Astra-lead cdef M and C were stopped by the owner as breakdowns; GLM vp8 U broke down and ended on a lost tool call."],"scope":{"lead":null,"task":null,"config":null},"strength":"moderate"},{"id":"same-vendor-judge-preference","title":"Judges score their own vendor's entries higher, except Muse","summary":"Judges are compared only on entries that pass every hidden category, against judges from a third vendor, with entries grouped by quality. GPT-6.1 Sol and Terra then score OpenAI entries about 0.7–0.8 points higher than other entries. Opus scores Anthropic entries about 0.5 higher. Muse scores Meta's entries slightly lower. The data cannot separate bias from shared stylistic taste. The cross-vendor score barely changes the ranking.","evidence":["Own-vendor minus other-vendor residual, stratified by third-vendor reference score: Sol +0.81 [+0.60,+1.04] (228 own entries), Terra +0.73 [+0.54,+0.92] (218), Opus +0.48 [+0.33,+0.62] (98), DeepSeek +0.28 [+0.06,+0.49] (24), Muse -0.22 [-0.37,-0.07] (71).","Cross-vendor score vs council: Sol xhigh 8.16 vs 8.34, Terra max 7.92 vs 8.14, Astra xhigh 8.03 vs 8.18, Opus xhigh 8.96 vs 9.01. In the top 8 only Sol xhigh and Opus medium swap places.","Two of five panel seats are OpenAI models, so OpenAI entries get a net lift of about 0.1–0.2 points in the published council mean."],"scope":{"lead":null,"task":null,"config":null},"strength":"moderate"},{"id":"detection-depends-on-session","title":"Whether a judge finds a defect depends on which review session the entry lands in","summary":"The same judge, given the same kind of code, finds a defect in one session and misses it in the next. In the Opus-lead bounded-tool run, every panel judge flagged a verified trailing-slash defect in nearly every affected entry of session 1, and none of them flagged it in any of the ten affected entries of session 2. Two judges missed an exr-piz defect while scoring the entry and caught it later when the entry came back as an anchor, but the published score uses the miss. Each judge's verified-defect recall is between 0.47 and 0.78.","evidence":["BT-009 ('A trailing slash is stripped, so a regular file named with one runs', verified by probe) in Opus-lead bounded-tool: s1 entries A, B, D, E, G, H, I, J flagged by all five panel judges and F by two; s2 entries K, L, M, N, O, T, U, V, W, X flagged by no panel judge, only by the retired Astra judge (e.g. gpt-6-astra-xhigh-s2: 'Trailing-slash paths execute unexpectedly').","Sol-lead exr-piz B (GLM, 11/12): DeepSeek s1 8 and Muse s1 8 (scored sessions, missed); as an anchor, DeepSeek s3 6.5 ('accepts a chunk whose run-length symbol has no code') and Muse s2 5.5 caught it. Opus, Sol and Terra caught it in s1 (5-5.5).","Operator notes: in Astra-lead range-coder s1, Terra scored 8 of 10 entries 3.5-4.5 for a carry counter that wraps after 2^32 pending carries, but never raised it in s2 (6.5-9.5). E.g. Astra-lead range-coder G (Opus@xhigh): Terra s1 4.5 against 9-9.5 from the other four.","Verified-defect recall over 279 entry-defect pairs in data/defects (Opus- and Sol-lead runs): Terra 0.78, Sol 0.74, Opus 0.63, Muse 0.60, DeepSeek 0.47; sole finder of a defect: Terra 15, Sol 11, Opus 7, DeepSeek 1, Muse 0; 13 pairs missed by the whole panel.","RC-015 (carry counter saturates or is wider than libopus's wrapping u32): each of 8 affected entries flagged by exactly one panel judge, Terra in 6 of them."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"judge-clusters-and-outliers","title":"The judges form two camps, and every sharp outlier score came from one side","summary":"Over all entries the panel agrees closely, but much of that comes from broken entries being easy to spot. On entries that pass every hidden category, Opus 5.5, DeepSeek and Muse agree with each other far more than with the two GPT judges. The 21 scores more than 3 points below the other judges' mean are all from Sol or Terra. The 6 scores more than 3 points above it are all from DeepSeek or Muse.","evidence":["ICC over 550 entries scored by all five panel judges: ICC(2,1) 0.832, ICC(2,k) 0.961. On the 473 of them that pass every hidden category: ICC(2,1) 0.59, ICC(2,k) 0.88.","Pairwise Pearson r on those entries: Opus–DeepSeek 0.81, Opus–Muse 0.80, DeepSeek–Muse 0.73, Sol–Terra 0.73; cross-camp pairs 0.41 (Terra–DeepSeek) to 0.62 (Opus–Sol, Sol–Muse).","|score − others' mean| ≥ 3: 27 cases; harsh 21 (Sol 11, Terra 10), lenient 6 (DeepSeek 5, Muse 1), Opus 0. Six of Terra's ten come from one session (Astra-lead range-coder s1, entries A, B, C, G, I, J: the carry-counter edge case in the operator notes).","Mean offsets vs the other judges: Opus -0.38, Terra -0.18, Sol +0.07, Muse +0.21, DeepSeek +0.27; on clean entries Opus -0.42 is the strictest.","Largest single divergence: Terra 4.5 vs 9.0–9.5 from the others on Astra-lead range-coder G (Opus xhigh, 11/11 hidden)."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"mac-only-build-failures-missed","title":"Five vp8lf-neon self-tests fail the macOS build; two judges never noticed","summary":"On vp8lf-neon, five entries' own self-test files fail bin/check's Apple clang build: four with compile errors in the inline-assembly ABI harness, one with a reserved-register warning. Their filters pass all 13 hidden categories. Opus, Sol and Terra each flagged an Apple-specific problem in 4 of the 5 entries. DeepSeek and Muse flagged one in none and scored these entries higher. Judges who did not compile the test file for a Darwin target could not see the failure.","evidence":["Entries whose bin/check macOS build fails: Astra-lead b2 K (Gemini: 'inline assembly requires more registers than available'), Astra-lead U (Grok, same error), Terra-lead B (DeepSeek: 'ADR/ADRP relocations must be GOT relative'), Terra-lead D (Opus@high, register error), Terra-lead K (Grok, reserved-register warning).","Apple-specific findings by panel judge over these 5: Opus 4, Sol 4, Terra 4, DeepSeek 0, Muse 0. DeepSeek and Muse findings on these entries about x18/x29 were coverage gaps, not Apple build problems.","Mean panel scores on the 5: DeepSeek 8.2, Muse 7.8, Opus 7.2, Sol 6.4, Terra 6.2.","Astra-lead b2 K: Sol b2 6 ('Inline assembly exhausts Darwin's available registers', reproduced with a Darwin-target clang) and Terra b2 6, against Opus b2 9, Muse b2 9 and DeepSeek b2 8.5; council 7.7.","Terra-lead D (Opus@high): Opus s1 7 ('The ABI inline asm exhausts registers on Apple') and Sol 6, against Muse s1 9, DeepSeek s1 8.5 and Terra s1 8.","Related: Sol-lead vp8 A (Grok) guard test fails on 16 KiB pages (verified defect VP8-016), and operator notes say bin/check runs the selftest only under qemu with 4 KiB pages."],"scope":{"lead":null,"task":"vp8lf-neon","config":null},"strength":"moderate"},{"id":"two-documentation-failure-styles","title":"Open-weight models write wrong documentation; terse GPT configurations write none, and only the first counts as slop","summary":"The council's misleading-docs findings cluster on GLM, Qwen and DeepSeek, then Grok, Gemini and Haiku. Its findings about missing documentation, which count as under-delivery rather than slop, cluster on GPT-6.1 and GPT-6 at medium to xhigh effort. Under the final-score formula only the wrong documentation costs slop points; missing documentation costs only council points. Gemini 4 Argon's slop is of a third kind: it leads every configuration in dead code and verbosity. Slop is only weakly related to council score overall.","evidence":["Misleading-docs findings per entry (panel judges): GLM 3.72, Qwen 3.12, DeepSeek 2.40, Grok 1.80, Gemini 1.65, Haiku 1.60; every OpenAI configuration and Opus@high/xhigh <= 0.96; Muse@xhigh 0.24; Terra@max 0.28.","Missing-docs ('docs' category) findings per entry: Sol@medium 4.00, Astra@medium 3.88, Terra@high 3.84, Astra@high 3.76, Terra@xhigh 3.72, Sol@high 3.16; Claude, DeepSeek, GLM and Qwen configurations 0.08-0.92.","Gemini: dead-code 1.78 and verbosity 1.09 findings per entry, the highest of any configuration (next: Grok 1.52 dead code, Muse@xhigh 0.64 verbosity).","Mean slop score by configuration: GLM 1.24, Qwen 1.02, Gemini 1.00 at the top; Opus@xhigh 0.21 and Terra@medium 0.14 (mostly stubs with nothing to judge) at the bottom.","Examples of GLM misleading-docs titles: 'File claims bit-exact behavior' (Terra-lead cdef L, an entry that crashes), 'Table comment describes rows it does not have' (Sol-lead cdef T), '`decode_bit_logp` probability inverted' (Sol-lead range-coder H, verified RC-017).","Pearson r between slop and council score over 552 entries scoring above 1.5: -0.19."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"muse-lead-stabilises-open-weight","title":"Only under Muse's briefs did GLM-5.3-Flash and Qwen 3.8 Flash pass every hidden category on all five tasks","summary":"The breakdown-prone configurations (GLM-5.3-Flash max, Qwen 3.8 Flash xhigh, Gemini 4 Argon high) never broke down under the Muse Wonder Full lead. Under the Terra lead almost half of them did. With one run per cell, and GLM's operational history (plan limits, resets, retries), this is suggestive only.","evidence":["GLM-5.3-Flash max hidden by lead: Muse 67/67, Sol 64/67, Astra 60/67, Opus 50/67, Terra 34/67; mean final Muse 6.19, Sol 5.64, Astra 5.17, Opus 4.53, Terra 1.05.","Qwen 3.8 Flash xhigh: Muse 67/67 (mean final 6.60), Opus 59/67 (4.69), Sol 55/67 (5.47), Terra 50/67 (3.89), Astra 47/56 (4.78, one DNF).","Entries with final <3 among GLM/Qwen/Gemini: Muse 0/15, Opus 2/15, Sol 2/15, Astra 3/14, Terra 6/14.","Paired lead effect for GLM: Muse +1.67, Sol +1.12, Astra +0.65, Opus +0.01, Terra -3.46; for Qwen: Muse +1.68, Opus -0.82.","Caveats: n=5 per lead per config. Operator notes record GLM usage-limit stops and breakdowns, and Muse-lead cdef-avx512 O ran alone in a fresh plan window."],"scope":{"lead":"muse-wonder-full-xhigh","task":null,"config":null},"strength":"tentative"},{"id":"glm-assembly-breakdowns","title":"GLM-5.3-Flash max either solves cdef-avx512 or breaks down over hours; it seldom ends in between","summary":"GLM-5.3-Flash at max effort produced two full-pass cdef-avx512 entries and three breakdowns from the same task and settings. Each breakdown ran for between 1 and 3.5 hours while the reasoning degraded (all-caps self-argument, garbled text) and the code crashed its own self-test. A fourth breakdown came on vp8lf-neon. GLM also cost the most to operate: one max-effort cdef run can use a whole Z.ai 5-hour window, so entries were relaunched one at a time, with up to about ten OpenCode sessions per entry.","evidence":["cdef-avx512 GLM entries: Sol-lead T 15/15, council 8.1, 151 min (10 OpenCode sessions, the last one completed); Muse-lead O 15/15, 7.2, 144 min, first attempt; Opus-lead M 3/15, 0.7, 207 min (ninth attempt, stopped by the owner as a breakdown); Astra-lead C 9/15, 2.3, 195 min (attempt 9, stopped at 3 h 13 min); Terra-lead L 3/15, 0.5, 72 min on OpenCode Go.","vp8lf-neon Terra-lead U (GLM on OpenCode Go): 4/13, council 2.2, 114 min; final reasoning in all-caps loops ('WAIT — I came to a WRONG CONCLUSION EARLIER'); operator notes call it a model breakdown, graded as final.","Terra-lead cdef L: last reasoning dissolves into garbled multi-script text, then 'Time is nearly gone'.","Judges on the breakdowns: Opus-lead M 'Neither function works: the filter segfaults on every call' (Opus s2); Astra-lead C 'Both entry points return wrong results ... own suite then segfaults' (Muse bfC).","GLM has 5 entries below council 3 (cdef M, C, L; vp8 U; range-coder Terra-lead I) and a score standard deviation of 2.20, the third highest after Terra@medium (2.50) and Qwen (2.23).","Operator notes: 'one cdef run at max effort can use a whole 5-hour window on its own'; the Z.ai weekly limit ran out on 2026-11-04 at 08:18Z; the owner said no new GLM runs on OpenCode Go after empty replies and a breakdown there.","Muse-lead O 'reported building a C model of its vector dataflow and checking it against the reference before trusting the assembly' (operator notes)."],"scope":{"lead":null,"task":"cdef-avx512","config":"glm-5.3-flash@max"},"strength":"moderate"},{"id":"speed-tracks-quality-in-simd","title":"In the SIMD tasks, faster code is also better-rated code, and Opus entries are the only ones that regularly beat the expert baselines","summary":"For the two hand-written assembly tasks, speed-up over the reference correlates strongly with the council score among entries whose perf output was correct. In range-coder and bounded-tool, speed-ups cluster around 1× and tell judges nothing. Beating dav1d's AVX2 filter or the wpd NEON loop filter is mostly an Opus-family result. Those baselines over-read or over-store and do not meet the contestant contract, so they are context, not targets.","evidence":["Spearman(speed-up geomean, council) among perf-correct entries: cdef-avx512 +0.82 (99 entries), vp8lf-neon +0.72 (105), exr-piz +0.54 (113), range-coder +0.17 (110), bounded-tool +0.13 (74).","Median speed-up: cdef-avx512 7.89× (0.86–24.03), vp8lf-neon 7.41× (3.15–9.17), exr-piz 1.52× (0.80–8.21), range-coder 0.99×, bounded-tool 0.97×.","cdef-avx512 filter: 28/99 entries faster than dav1d AVX2 (best 2.42×, Terra-lead A, Opus high); 0/99 faster than dav1d AVX-512ICL (best 0.93×). Direction search: 15/99 faster than dav1d AVX2, 12 of them Opus 5.5, 2 Gemini, 1 Sol max.","vp8lf-neon geomean: 12/105 entries faster than wpd_neon (best 1.09×, Opus-lead F); 10 Opus 5.5, 1 Haiku, 1 Sol max.","exr-piz: Opus 5.5 xhigh median 4.11× vs Grok 4.7 1.09×; fastest entry Opus-lead K (Opus xhigh) 8.21×."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"judge-recall-per-dollar","title":"GPT-6.1 Terra is the cheapest judge that finds the most verified defects","summary":"Against the list of verified defects (built by probes, hidden-suite failures and inspection), Terra max reports the largest share at the second-lowest price per entry. Opus 5.5 writes the most findings per entry but recalls fewer verified defects and costs about nine times as much per entry. DeepSeek is very cheap but misses about half.","evidence":["Recall over 279 defect×entry pairs: Terra 219 (78%), Sol 206 (74%), Opus 176 (63%), Muse 167 (60%), DeepSeek 132 (47%). Critical/high only (60 pairs): Sol 45, Terra 43, Opus 37, Muse 34, DeepSeek 31.","Review cost per entry (list price; Muse at the Muse Wonder 1.0 API proxy rate): Opus $1.016, Muse $0.600, Sol $0.225, Terra $0.108, DeepSeek $0.028. Median session: Terra 11.0 min, Opus 17.4, Sol 17.6, DeepSeek 18.1, Muse 37.3.","Valid findings per entry: Opus 3.47, Muse 2.97, Terra 2.86, DeepSeek 2.54, Sol 2.29.","Whole council (5 panel judges) $1,134 = 1.49× the $761 spent on contestants; the Opus seat is $585 (52%). The retired Astra judge cost $118 for 96 entries ($1.23/entry).","Caveats: verified-defect lists exist only for Opus-lead and Sol-lead runs (Opus-lead cdef-avx512 has none). Defects enter the list partly through judges' own findings. 13 pairs were found by no panel judge, 10 of them BT-009."],"scope":{"lead":null,"task":null,"config":null},"strength":"moderate"},{"id":"lead-specific-shared-defects","title":"A single brief can plant the same defect in almost every entry","summary":"In the runs with verified-defect lists (Opus and Sol leads), several defects cluster under one lead. The clearest is a path-handling correctness defect (BT-009), confirmed by probe in 22 of 23 Opus-lead bounded-tool entries but only 1 Sol-lead entry. The hidden suite does not catch it, and panel judges reported it in under half of the entries. Short Sol briefs instead go with documentation defects (undocumented assembly or API), each seen only under the Sol lead.","evidence":["BT-009 (medium, correctness, probe-verified): 22 Opus-lead + 1 Sol-lead bounded-tool entries. Found by panel judges Sol 13, Muse 10, Terra 9, Opus 9, DeepSeek 9 of 23; retired Astra judge 22. Opus-lead bounded-tool still passes 343/352 hidden categories.","Other single-lead clusters: EXR-009 (robustness) 8 Opus-lead entries; VP8-003 (tests) 7 Opus-lead; BT-033 (misleading docs) 6 Opus-lead; BT-016 7 Sol-lead and BT-015 6 Sol-lead (correctness); docs defects CDEF-006 (4), VP8-023 (4), BT-034 (5) Sol-lead only.","Verified defects by category, Opus-lead runs: docs 3 pairs; Sol-lead runs: docs 24 pairs. Sol briefs total 3,617 words vs Opus 9,144.","Caveat: no defect lists for Astra-, Terra- or Muse-lead runs, so presence elsewhere is unknown."],"scope":{"lead":"opus-5.5-xhigh","task":"bounded-tool","config":null},"strength":"moderate"},{"id":"family-asymmetric-misses","title":"Briefs that leave a gap trap different model families, but no lead helps its own vendor","summary":"The two brief-linked hidden misses split by subagent family in opposite directions. The bounded-tool category is failed mostly by GPT-family subagents under every lead except Opus's. The exr-piz category is failed only by non-GPT subagents. Across all cells, though, a lead does not lift subagents from its own vendor or its own model.","evidence":["bounded-tool category, all leads: GPT-family entries 25/55 fail it vs 5/60 non-GPT (17/45 GPT excluding Terra medium/high early stops). Under the Astra lead: GPT 10/11 vs non-GPT 4/12; Terra lead GPT 6, Muse lead GPT 6, Sol lead GPT 3, Opus lead 0.","exr-piz category, all leads: GPT 0/55 vs non-GPT 10/60; under the Terra lead: GPT 0/11 vs non-GPT 7/12 (Gemini, Haiku, Muse medium and high, Qwen, DeepSeek, GLM).","Two-way interaction residual (final minus cell mean minus lead main effect) for same-vendor lead and subagent pairs: +0.03 [-0.07,+0.14], n=200, vs -0.02 for other pairs; for the exact same model as lead and subagent: -0.05 [-0.19,+0.11], n=85.","GPT subagents score best under the Opus lead (family mean delta +0.33), then Sol (+0.24); under Astra -0.27, Terra -0.09. Muse subagents: Opus lead +0.67, Muse lead -0.09.","Tentative for the per-lead splits (n=11–12 per family per cell)."],"scope":{"lead":null,"task":null,"config":null},"strength":"moderate"},{"id":"brief-length","title":"Longer briefs go with better results for mid-tier subagents, but length alone does not explain the leads","summary":"Opus writes the longest and most structured briefs and Terra the shortest. A lead's brief length for a task correlates moderately with how its subagents score on that task, mainly for mid-tier configurations. But Sol's briefs are about as short as Terra's and do clearly better. So something other than length separates them.","evidence":["Total brief words over five tasks: Opus 9,144; Muse 7,228; Astra 4,698; Sol 3,617; Terra 3,343. Opus is the longest on every task (1,392–2,167 words); Sol's cdef-avx512 and vp8lf-neon briefs are 451 and 447 words.","Opus briefs have 5–7 headings and 25–77 list items each; Sol and Terra briefs mostly have no headings (Sol 0 on 4 of 5 tasks, Terra 0 on all 5).","Pearson r between a lead×task cell's word count (centred within task) and its paired lead effect, 25 cells: all configs +0.34; mid-tier +0.44; strong +0.11; weak +0.22.","Sol vs Terra (similar length): paired lead effect +0.22 vs -0.37; excluding GLM +0.18 vs -0.23.","Tentative: with five leads the length ranking is close to the lead ranking (Spearman 0.7 over n=5)."],"scope":{"lead":null,"task":null,"config":null},"strength":"tentative"},{"id":"task-separation","title":"cdef-avx512 separates the models most; bounded-tool has the most near-misses","summary":"Configuration choice explains most of the variance in final scores. Task and lead each explain about 2%. Rankings agree closely across tasks. The AVX-512 task spreads configurations furthest, from 0.02 to 9.17. bounded-tool has a narrow score spread, but only two-thirds of its entries pass every hidden category, most misses being a single category.","evidence":["Share of final-score sum of squares (non-Gemini entries): config 57.3%, task 1.9%, lead 1.6%, lead×task 5.8%, config×task 73.3% (incl. main effects).","Config-mean spread (sd, range): cdef-avx512 2.30 (0.02–9.17), vp8lf-neon 1.99 (0.11–9.40), exr-piz 1.38 (2.76–8.76), range-coder 1.20 (3.57–8.63), bounded-tool 1.07 (4.69–8.79).","Entries passing every hidden category: range-coder 96%, vp8lf-neon 92%, exr-piz 90%, cdef-avx512 88%, bounded-tool 67% (77/115); category pass rate on bounded-tool is still 97.4%.","Spearman between tasks of config mean final: 0.81–0.96 (lowest exr-piz vs bounded-tool, highest range-coder vs exr-piz).","Opus 5.5 xhigh has the top config mean on 4 of 5 tasks; Opus 5.5 high tops cdef-avx512 (9.17)."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"what-separates-top-entries","title":"On the SIMD tasks the council rewards measured speed and real tests; on bounded-tool it rewards contract fidelity that no metric captures","summary":"Among entries that pass every hidden category with correct perf output, council score follows measured speed-up closely on cdef-avx512 and vp8lf-neon and hardly at all on range-coder and bounded-tool. The size of an entry's own test suite follows score on four tasks but not bounded-tool. The judges' headlines on the best entries repeat one formula: fastest, strongest tests, accurate docs. On bounded-tool the ranking turns on behavioural divergences from the reference (paths, duplicate options, PID identity, signals) that only probing finds.","evidence":["Spearman rho between council score and geometric-mean speed-up (hidden-perfect, perf-correct entries): cdef-avx512 0.82 (n=99), vp8lf-neon 0.72 (n=105), exr-piz 0.57 (n=103), range-coder 0.16 (n=109), bounded-tool 0.07 (n=63).","rho(score, test lines): cdef 0.63, vp8 0.56, range-coder 0.50, exr-piz 0.48, bounded-tool 0.01.","Top entry of the round, Astra-lead vp8lf-neon O (Opus@xhigh, 9.8): Muse s2 10 'merge as is; fastest, best documented and best tested'; Sol s2 10; Terra s2 10; Opus s2 9.5 'best-in-class speed, excellent docs and tests'; DeepSeek s2 9.5. It was then used as an anchor for the b2 session (all five judges scored it 9.5-10 again).","Astra-lead cdef-avx512 J (Opus@high, 9.6): Terra s1 10 'the fastest measured implementation'; Opus s1 9.5 'best design, fastest, well documented, strongest tests'.","Counter-example: Terra-lead cdef-avx512 Z (Terra@high) passed 15/15 hidden but was scalar (filter 1.00x, find_dir 0.78x per Opus s3) and wrote to the caller's stack; council 3.9.","Bounded-tool top entries were described as matching 'the oracle on every probe' (Opus-lead C, 9.6: Muse s1, Opus s1; Sol-lead G, 9.4: Muse s1, Opus s1)."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"gemini-same-defect-every-lead","title":"Gemini 4 Argon made the same bounded-tool mistake under all five leads, and its tests locked it in","summary":"Every one of Gemini's five bounded-tool entries rejected valid nil or boolean atom pool names. That is a model-level habit, not a brief effect, since five different briefs produced it. Every panel judge reported it in every entry. In three of the five, Gemini's own tests asserted the wrong behaviour, and those three are three of only seven findings in the whole round that a test suite locks in a bug.","evidence":["Gemini bounded-tool entries and pool-name findings by panel judges: Astra-lead b2 E (6 findings, council 6.6), Sol-lead b2 Y (6, 6.0), Terra-lead b2 M (5, 6.5), Muse-lead b2 N (6, 5.9), Opus-lead b2 Z (6, 6.8). Titles include '`name: nil` is rejected as an invalid argument' (Opus b2) and 'Valid atom pool names are rejected'.","Tests-category findings that the suite locks in the bug: 'Tests enshrine the nil bug and fake a crash case' (Muse b2, Sol-lead Y), 'Suite locks in the wrong name behaviour' (Muse b2, Muse-lead N), 'suite enshrines the `:name` nil divergence' (DeepSeek b2, Opus-lead Z).","Round-wide, 7 tests findings use this 'locks in / enshrines the bug' language: Gemini 3, and one each for Terra@max, Haiku, Qwen (Terra-lead range-coder C) and GLM (Terra-lead range-coder I).","Verified defect BT-015 ('Valid atom pool names (true, false, nil) are rejected or get the wrong error') lists the Opus-lead and Sol-lead Gemini entries plus Grok, Opus@xhigh and three Muse configurations in the Sol-lead run."],"scope":{"lead":null,"task":"bounded-tool","config":"gemini-4-argon@high"},"strength":"moderate"},{"id":"codex-skipped-mac-testing","title":"Codex-harness entries often skipped the Mac build and tests, citing the isolation rules","summary":"On vp8lf-neon, 26 of 55 Codex-harness entries (GPT-6.1 Sol and Terra, GPT-6 Astra) reported that they had not run the native macOS build, test or timing. Most said the rules confined them to their own directory or forbade other machines. Entries in the other harnesses used the scaffold's Mac host as intended, and auditors recorded those uses as allowed. How often Codex skipped depended heavily on the lead's brief. The cause is the contestant prompt's wording, which round 2 plans to clarify.","evidence":["Codex vp8lf-neon final reports that say the Mac/Apple build, tests or M1 timing were not run, by lead: Astra 11/11, Terra 9/11, Opus 4/11 (all four Terra@medium-max), Muse 2/11, Sol 0/11.","Typical wording under the Terra lead (9 entries): 'Apple build skipped: no toolchain available inside the permitted directory'; under the Muse and Opus leads: 'the exercise rules prohibit accessing other machines' (Muse-lead T, Opus-lead C).","Isolation reviews record 'ssh/rsync only to the scaffold Mac host (mac-dev) ... as the brief intends' for entries from every harness in the Opus-lead and Muse-lead vp8 runs, including six Codex entries under the Opus lead (D, E, K, O, T, Z), which shows the skip was a choice, not a technical block.","Round 2 will clarify that scaffold make targets reaching the Mac host are allowed."],"scope":{"lead":null,"task":"vp8lf-neon","config":null},"strength":"moderate"},{"id":"conduct-pattern-kills-and-qwen","title":"Conduct flags are mostly host-wide pkill patterns from Opus and Qwen; Qwen also leads on leftover scratch files and has the one isolation DNF","summary":"Nine scored entries carry a public conduct note, and one more was made DNF. Eight of the nine are host-wide 'pkill -f' patterns used to clean up test processes. Two of those killed processes belonging to an unrelated job on the host. Opus 5.5 reported its incident within seconds; Qwen did not report its own. Qwen3.8-Flash accounts for 4 of the 10 conduct records, 5 of the 8 rules failures (scratch files left in the deliverable) and the round's only isolation DNF. Under the owner's rule for deviations that gain nothing, scores stood.","evidence":["Pattern-kill conduct notes: bounded-tool Astra-lead E (GLM), Astra-lead L (Qwen, serious: 'killed three processes of an unrelated job', not self-reported), Sol-lead B (Grok), Terra-lead G (Opus@xhigh, serious, 'noticed within seconds and reported it'), Terra-lead L (Qwen); cdef Terra-lead D (Opus@medium) and Opus-lead Z (Opus@xhigh); vp8 Terra-lead D (Opus@high).","Other conduct: Astra-lead exr-piz X (Qwen) ran find over the folder holding other contestants' entries (file names only); Astra-lead range-coder N (Qwen) DNF for exploring the private benchmark repository and other home folders, then stopping without code.","Rules failures for extra files: Qwen cdef Sol-lead H, Muse-lead I, Opus-lead V, exr-piz Sol-lead N, Terra-lead N; GLM cdef Astra-lead C (debug.c). The other two: Opus-lead exr-piz F (Muse@high, test package/import rule) and Muse-lead range-coder T (Terra@medium, missing file).","Isolation reviews record further minor risky kills without conduct notes (e.g. Opus-lead bounded-tool E Muse@xhigh, J Haiku, Y GLM). The operator notes count pattern kills in 10+ bounded-tool entries.","A council judge caught one conduct problem: Opus-lead bounded-tool F (Grok) has a test that writes to a fixed file in the home folder (verified BT-030, flagged by Terra, Sol and Opus).","DECISIONS.md #15: gainless rule breaks keep their scores and get a public conduct flag; cheating stays DNF."],"scope":{"lead":null,"task":null,"config":"qwen3.8-flash@xhigh"},"strength":"strong"},{"id":"muse-wall-time-includes-stalls","title":"Almost a third of Muse Wonder Full wall times include multi-hour quota stalls","summary":"22 of the 75 Muse contestant entries show wall times of 5 to 7 hours. This is not slow work: Muse's 5-hour usage limit stalled up to 22 sessions at once, and the harness resumed them when the window reset. The stalls hit the Sol-lead and Astra-lead runs plus one Opus-lead entry. Muse time comparisons should leave these out or use medians.","evidence":["Muse entries with wall time > 300 min: @medium 5/25, @high 10/25, @xhigh 7/25 (max 404 min, Opus-lead cdef T); all other Muse entries <= 147 min.","Muse@high median wall time 80.8 min overall vs 51.7 min without the stalled entries; @xhigh 62.6 vs 51.4; @medium 48.6 vs 43.6.","Operator notes: '5-hour limits stalled up to 22 sessions at once; T3's limit-resume restarted them when the window reset ... Wall time for those entries includes the stall.'","No non-Muse entry exceeds 200 min except GLM Opus-lead cdef M (207 min, a breakdown)."],"scope":{"lead":null,"task":null,"config":"muse-wonder-full@high"},"strength":"strong"},{"id":"two-failure-modes","title":"Low-effort GPT runs fail fast and cheap; open-weight breakdowns fail slow and expensive","summary":"Near-zero scores come from two opposite behaviours. GPT-6.1 Terra at medium and high effort often stops within a minute with a stub. GLM-5.3-Flash and Qwen 3.8 Flash break down after hours of work and hundreds of thousands of output tokens. Both end at about the same score. So time spent predicts a better result for the low-effort GPT configurations and a worse one for GLM.","evidence":["19 Terra medium/high entries with final <1.5: median wall 0.33 min (max 0.99), median 535 output tokens, median cost $0.022, mean final 0.19; 16 of them on cdef-avx512 or vp8lf-neon.","10 GLM/Qwen entries with final <3: median wall 99 min (25–207), median 288k output tokens, median cost $0.72, mean final 0.23. Longest: Opus-lead cdef-avx512 M (GLM, 207 min, 530k output tokens, hidden 3/15).","Within-config Spearman of log wall time and final (task-centred, n=25): Terra high +0.69, Terra medium +0.68, Astra medium +0.64; GLM -0.50, Grok -0.38; Sol max +0.03, Opus xhigh +0.16.","Across configurations, median wall time and mean final are only weakly related (Spearman +0.20). Opus xhigh median 26.7 min; Terra max 6.3 min; Sol max 14.6 min."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"judge-disagreement-pattern","title":"Large judge disagreements are almost always Sol or Terra scoring low, or DeepSeek scoring high","summary":"On 55 of 572 entries the panel's scores span 3 points or more. The outlier is GPT-6.1 Sol or Terra scoring low in 40 of them, usually for a semantic defect it reproduced. DeepSeek V4.1 Flash is the high outlier in 12, usually because it missed a safety bug that the others found. Disagreement concentrates on bounded-tool and never reaches 3 points on cdef-avx512, where broken entries are broken for every judge. Sol separates hidden-passing from hidden-failing entries most sharply; DeepSeek and Muse least.","evidence":["Entries with panel range >= 3: bounded-tool 33/115, exr-piz 10/115, range-coder 9/114, vp8lf-neon 3/115, cdef-avx512 0/113. Outlier judge (furthest from the mean of the others): Sol low 23, Terra low 17, DeepSeek high 12, Muse high 2, Opus low 1.","Terra-lead bounded-tool Z (GLM, 14/16): DeepSeek s3 8 ('thorough and well-documented') against Opus s3 3.5, Sol s3 3, Terra s3 3 and Muse s3 4, all four reporting that the PID-reuse guard compares the process with itself.","Opus-lead exr-piz L (GLM, 8/12): DeepSeek s2 9 ('Excellent docs, careful code, strong tests') against Opus s2 4.5 and Terra s2 4 ('Rejects valid 58-bit streams').","DeepSeek scored >= 7.5 on 11 entries that fail at least one hidden category.","Mean score on hidden-passing entries minus mean on hidden-failing non-stub entries (council > 1.5, n = 57): Sol 3.63, Terra 2.71, Opus 2.22, DeepSeek 1.87, Muse 1.87.","Only one false positive is documented for round 1 (DeepSeek's VPINSRW finding, on an anchor), and all 8,311 recorded round-1 entry findings have status 'valid'. False-positive rates are effectively unmeasured."],"scope":{"lead":null,"task":null,"config":null},"strength":"strong"},{"id":"ops-provider-faults-and-disclosures","title":"Operations: provider faults were retried by rule, and guardrails blocked Gemini on cdef","summary":"The operator retried provider faults (empty replies, lost tool calls, invalid JSON, 5xx), but a model that ended its turn after text was graded as is. Qwen on OpenCode Go and GLM on Z.ai needed the most retries. Google's filter blocked Gemini on an AV1 CDEF kernel twice in one run, and the partial work, left as stubs, was graded.","evidence":["Sol-lead cdef-avx512 b2 O (Gemini): two guardrail blocks ('This request was blocked by Gemini's filters ... stopped after 22 steps with partial work'); hidden 1/15; council 0.6; Opus b2: 'Unimplemented stub, plus leftover \"Wait:\" notes left in as comments'; slop 2.9.","OpenCode sessions per entry, including the graded one (opencode.db): Qwen Opus-lead exr-piz B 5, Sol-lead vp8 W 5, Astra-lead exr-piz X 4; GLM Sol-lead cdef T 10, Opus-lead cdef M 10. The owner's cap for Qwen was 2 retries, after which partial work was graded.","DeepSeek's judge sessions failed with 'Service is too busy' or empty replies at about 270k context on two cdef sessions; both succeeded on the third retry (operator notes).","Operator notes: 30 council sessions launched at once pushed the host load average to 458 on 20 CPUs (2026-11-03), which can distort timing-based findings; round 2 batches about 10.","Orchestrator error (operator notes): running make test in a live GLM entry's directory added one core dump; no source was touched."],"scope":{"lead":null,"task":null,"config":null},"strength":"tentative"}],"entry_notes":[{"run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"O","note":"Highest council score of the round (9.8, Opus@xhigh, $6.32, 36 min): all five judges 9.5-10, with headlines like 'fastest, best documented and best tested' (Muse s2) and the lowest per-function instruction counts in its b2 anchor session (84/162/65/123, DeepSeek b2)."},{"run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","note":"Best cdef-avx512 entry (Opus@high, 9.6, 16 min, $3.62, no slop): Terra s1 10 'the fastest measured implementation', Opus s1 9.5 'best design, fastest, well documented, strongest tests'. Opus@xhigh in the same run cost $10.16 and scored 9.3."},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"E","note":"Highest exr-piz final in the round (9.40, council 9.5, 12/12) from GPT-6.1 Sol xhigh at $0.37. The same config scored 7.50–8.38 under the other four leads, a 1.90-point spread across leads."},{"run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"Z","note":"Cheapest entry passing all hidden categories with final ≥8.5: GPT-6.1 Terra max, $0.135 and 3.9 minutes for council 9.1 (final 8.85)."},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"G","note":"Opus@medium beats every higher effort level in the Sol-lead bounded-tool run (9.4, 5 min, $1.17, no slop): 'matches the oracle on every probe with complete docs and thorough tests' (Muse s1, 9.5)."},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"C","note":"Best bounded-tool entry (Opus@xhigh, 9.6). DeepSeek s1 called it 'the only suite that covers the whole contract including the VM-SIGKILL backstop'."},{"run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"Z","note":"Hidden 15/15 but council 3.9: Terra@high translated the C reference into scalar assembly (Opus s3: 'No SIMD implementation at all ... 1.0x filter, 0.78x find_dir') and writes to the caller's stack. Its own report said the SIMD work 'remain[s] outstanding'."},{"run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"I","note":"Honest early stop: Terra@medium made no source changes in 0.3 min and reported 'The implementation remains unfinished. Both assembly entry points are still stubs.' Hidden 1/15, council 0.5."},{"run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"C","note":"Terra@high wrote only a scalar direction search, left the filter a stub and said so ('The implementation is **incomplete**'). Hidden 3/15, council 2.4, 0.5 min."},{"run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"T","note":"The emptiest entry of the round: Terra@medium made one tool call, created no file (rules: missing src/range_coder.rs) and reported 'I could not complete the range coder implementation'. Council 0.0, hidden 0/11."},{"run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"T","note":"Terra@medium left the stub and said it 'did not check mac-dev because the exercise rules prohibit accessing other machines'. Round 2 will make it explicit that the scaffold's Mac host is part of the toolchain."},{"run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"M","note":"The longest breakdown in the round: GLM-5.3-Flash worked 207 minutes and wrote 530k output tokens before being stopped; hidden 3/15, final -0.51."},{"run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"C","note":"GLM attempt 9, stopped at 3 h 13 min as a breakdown after Z.ai window cut-offs. Hidden 9/15, council 2.3, rules flag for a leftover debug.c. Judges: 'its own make test fails 7 of 8 tests and then crashes' (Opus bfC)."},{"run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"L","note":"GLM on OpenCode Go: its reasoning dissolved into garbled text, yet its final report was the only candid one among GLM's failing entries ('Status: incomplete'). Hidden 3/15, council 0.5; judges reported that both entry points crash."},{"run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"O","note":"GLM's first attempt in a fresh Z.ai window finished on its own in 144 min with 15/15, filter 10.2x and find_dir 3.6x; council 7.2 (judges 6.5-7.5). Same model and settings as the 3/15 and 9/15 breakdowns."},{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"T","note":"GLM's other full cdef-avx512 pass (15/15, council 8.1, 151 min), from the tenth OpenCode session after nine earlier sessions failed or were cut off."},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"U","note":"GLM breakdown on OpenCode Go (all-caps reasoning loops, then a lost tool call), graded as final: hidden 4/13, council 2.2; 'Wrong output on every HEV line' (Opus s2)."},{"run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"I","note":"GLM reported 'Done. The implementation is complete ... All four gates pass' but failed 6 of 11 hidden categories. All five judges found two critical/high bugs, and three said its tests enshrine the wrong behaviour."},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"N","note":"Qwen left the stub unchanged: two earlier sessions failed, and the third ended its turn mid-analysis after 9 tool calls. Hidden 0/12, every judge 0-0.5, rules flag for leftover scratch files."},{"run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"X","note":"Qwen claimed 'All work is complete' but failed 8 of 12 hidden categories ('wrong output whenever bitmap bit 0 is set', Muse s2). It also has a conduct note for listing sibling entries' file names."},{"run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"V","note":"Qwen's fifth and graded session was cut off by a provider failure mid-debugging at 162 min, and the partial work was graded. Hidden 9/15, council 1.9; it clobbers r12/r13 so its own make test hangs (DeepSeek s2, Opus s2); rules flag for scratch files."},{"run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"N","note":"The round's only isolation DNF: Qwen explored the private benchmark repository (listing, git log, tracker) and other home folders, then stopped without writing code."},{"run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"L","note":"Serious conduct flag: Qwen's 'pkill -KILL -f sleep 30' killed three processes of an unrelated job on the host and was not mentioned in its report. Scores stand (6.0, hidden 15/16)."},{"run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"G","note":"Opus@xhigh's pattern kill hit the same unrelated job, but it noticed within seconds and reported it. Judges split 6-9 over its SIGPIPE reset ('some output differs from the reference', Opus s1 8.5; Terra s1 6)."},{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1-b2","letter":"O","note":"Lowest final in the round (-1.41): Gemini 4 Argon's partial work after a provider content-filter block, 1/15 hidden, council 0.6."},{"run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1-b2","letter":"K","note":"Correct kernel (13/13, 8.3x geomean), but its self-test does not compile with Apple clang. Sol and Terra reproduced this and scored 6; Opus and Muse (9) and DeepSeek (8.5) missed it. Council 7.7."},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"D","note":"Opus@high's own ABI test exhausts registers on Apple targets. Only Opus (7, 'The ABI inline asm exhausts registers on Apple') and Sol (6) caught it; Muse 9, DeepSeek 8.5 and Terra 8 did not."},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","note":"Grok's guard test assumes 4 KiB pages and fails on the M1's 16 KiB pages (verified VP8-016). Four judges scored 6-6.5 for it; Muse 7.5."},{"run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"Z","note":"Largest lenient outlier: DeepSeek scored this GLM-5.3-Flash entry 8 while the others gave 3–4, the operator notes' DeepSeek false negative on a process-group kill check; council 4.3, hidden 14/16."},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"L","note":"DeepSeek s2 9 ('Excellent docs, careful code, strong tests') for an entry failing 4 of 12 hidden categories that Opus s2 (4.5) and Terra s2 (4) showed misreads valid 58-bit codes (verified EXR-003)."},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","note":"DeepSeek and Muse both scored it 8 in the scored session and missed the malformed-table defect, then caught it when B came back as an anchor (DeepSeek s3 6.5, Muse s2 5.5). The published score keeps the misses."},{"run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"G","note":"Largest single-judge divergence in the round: Terra scored this Opus 5.5 xhigh entry 4.5 while the other four gave 9.0–9.5, though it passes 11/11 hidden categories (the session-dependent carry-counter edge case in the operator notes)."},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"O","note":"Opus@high (8.9) has the verified trailing-slash defect BT-009, which no panel judge reported in its session (s2). In s1 all five panel judges flagged the same defect on nearly every entry."},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","note":"Astra@xhigh fails 5 of 16 hidden categories, the worst bounded-tool hidden result of the round. Judges reported a startup race (Terra s1: 'A startup race loses completed results') while its report said 'All 29 tests passed'."},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"J","note":"Haiku's test suite is hazardous: setup deletes the project's tmp directory and teardown can SIGKILL a recycled PID (verified BT-031/032). Only Terra s1 found either; council 7.5."},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F","note":"One of Grok's own tests overwrites and deletes a fixed file in the home folder (BT-030), a conduct issue found by the judges, not the audit. Its published scores come from a single-entry re-review session (reF)."},{"run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1-b2","letter":"E","note":"Gemini's weakest non-blocked entry: it rejects every valid chunk with a full 65,536-entry LUT ('Rejects every valid chunk whose LUT holds all 65,536 values', Opus b2). Hidden 3/12, council 4.7."},{"run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1-b2","letter":"Z","note":"Outside the Terra lead, the only range-coder entry that fails a hidden category without being a stub: Gemini drops a leading 0xFF carry run (all five judges 3.5-4). Hidden 9/11."},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1-b2","letter":"Y","note":"One of five Gemini bounded-tool entries that reject nil/boolean pool names; here its tests also 'enshrine the nil bug and fake a crash case' (Muse b2). Council 6.0."},{"run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","note":"Fastest cdef-avx512 entry (24.0× geomean over the C reference; filter 2.42× faster than dav1d AVX2, which does not meet the contract). Opus 5.5 high, council 9.1, 15/15 hidden, $7.45."},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","note":"Fastest vp8lf-neon entry (9.17× geomean; 1.09× the wpd_neon baseline's geomean) with council 9.5 and 13/13 hidden."},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","note":"Fastest exr-piz entry at 8.21× the reference; the next fastest are 5.70× (Gemini, Opus-lead b2 W) and 5.40× (Opus medium, Opus-lead G). Opus 5.5 xhigh, council 8.7, 12/12 hidden."},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"E","note":"The highest council score (7.9) among the 77 entries that fail any hidden category: GPT-6.1 Terra max, 15/16."},{"run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"U","note":"GPT-6.1 Sol xhigh, one of 14 Astra-lead entries missing the same bounded-tool hidden category (15/16); Sol judge 4.0 vs 7.0–8.5 from the others."},{"run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"J","note":"Claude Haiku 5.5 max at 11/12: one of 7 non-GPT Terra-lead exr-piz entries missing the same hidden category, which no GPT-family entry missed under any lead."}]},"models":[{"id":"claude-opus-5.5","name":"Claude Opus 5.5","short":"Opus 5.5","vendor":"anthropic","harness":"Claude Code","provider":"claudeAgent","provider_model":"claude-opus-5-5","effort_option":"effort","efforts":["medium","high","xhigh","max"],"fast_mode":{"available":true,"option":"fastMode","price_multiplier":2,"default":false},"release":"2026-09-22"},{"id":"claude-fable-5.1","name":"Claude Fable 5.1","short":"Fable 5.1","vendor":"anthropic","harness":"Claude Code","provider":"claudeAgent","provider_model":"claude-fable-5-1","effort_option":"effort","efforts":["medium","high","xhigh","max"],"fast_mode":{"available":false},"release":"2026-10"},{"id":"claude-haiku-5.5","name":"Claude Haiku 5.5","short":"Haiku 5.5","vendor":"anthropic","harness":"Claude Code","provider":"claudeAgent","provider_model":"claude-haiku-5-5","effort_option":"effort","efforts":["medium","high","xhigh","max"],"fast_mode":{"available":false},"release":"2026"},{"id":"gpt-6.1-sol","name":"GPT-6.1 Sol","short":"GPT‑6.1 Sol","vendor":"openai","harness":"Codex","provider":"codex","provider_model":"gpt-6.1-sol","effort_option":"reasoningEffort","efforts":["medium","high","xhigh","max"],"fast_mode":{"available":true,"option":"serviceTier","value":"priority","price_multiplier":2,"default":true},"release":"2026-09-29"},{"id":"gpt-6.1-terra","name":"GPT-6.1 Terra","short":"GPT‑6.1 Terra","vendor":"openai","harness":"Codex","provider":"codex","provider_model":"gpt-6.1-terra","effort_option":"reasoningEffort","efforts":["medium","high","xhigh","max"],"fast_mode":{"available":true,"option":"serviceTier","value":"priority","price_multiplier":2,"default":true},"release":"2026-10-26"},{"id":"gpt-6-astra","name":"GPT-6 Astra","short":"GPT‑6 Astra","vendor":"openai","harness":"Codex","provider":"codex","provider_model":"gpt-6-astra","effort_option":"reasoningEffort","efforts":["medium","high","xhigh","max"],"fast_mode":{"available":true,"option":"serviceTier","value":"priority","price_multiplier":2,"default":true},"release":"2026-09-04"},{"id":"deepseek-v4.1-flash","name":"DeepSeek V4.1 Flash","short":"DeepSeek V4.1 Flash","vendor":"deepseek","harness":"OpenCode","provider":"opencode","provider_model":"deepseek/deepseek-flash","effort_option":"variant","efforts":["max"],"fast_mode":{"available":false},"release":"2026-10-17"},{"id":"glm-5.3-flash","name":"GLM-5.3-Flash","short":"GLM‑5.3‑Flash","vendor":"zhipu","harness":"OpenCode","provider":"opencode","provider_model":"zai-coding-plan/glm-5.3-flash","effort_option":"variant","efforts":["max"],"fast_mode":{"available":false},"release":"2026-08-26"},{"id":"mimo-v2.6-pro","name":"MiMo-V2.6-Pro","short":"MiMo‑V2.6‑Pro","vendor":"xiaomi","harness":"OpenCode","provider":"opencode","provider_model":"opencode-go/mimo-v2.6-pro","effort_option":null,"efforts":["default"],"fast_mode":{"available":false},"release":"2026-09-22","status":"disqualified","status_note":"Disqualified from runs (owner, 2026-11-02): not to be re-run. Its pilot entry stays as recorded (DNF)."},{"id":"qwen3.8-flash","name":"Qwen3.8-Flash","short":"Qwen3.8‑Flash","vendor":"alibaba","harness":"OpenCode","provider":"opencode","provider_model":"opencode-go/qwen3.8-flash","effort_option":"variant","efforts":["xhigh"],"fast_mode":{"available":false},"release":"2026-08-26"},{"id":"grok-4.7","name":"Grok 4.7","short":"Grok 4.7","vendor":"xai","harness":"Grok Build","provider":"grok","provider_model":"grok-4.7","effort_option":"reasoningEffort","efforts":["xhigh"],"fast_mode":{"available":false},"release":"2026-09-21","harness_history":[{"until":"2026-11-02","harness":"OpenCode","provider":"opencode","provider_model":"opencode-go/grok-4.7","effort_option":"variant","note":"owner bought a Grok subscription; all 25 round-1 Grok entries re-run on Grok Build"}]},{"id":"muse-wonder-full","name":"Muse Wonder Full 1.0","short":"Muse Wonder Full","vendor":"meta","harness":"Muse Code","provider":"muse","provider_model":"muse-wonder-full-1.0","effort_option":"reasoningEffort","efforts":["medium","high","xhigh"],"fast_mode":{"available":false},"release":"2026-10-12"},{"id":"gemini-4-argon","name":"Gemini 4 Argon","short":"Gemini 4 Argon","vendor":"google","harness":"Antigravity","provider":"antigravity","provider_model":"gemini-4-argon","effort_option":"thinking_level","efforts":["high"],"fast_mode":{"available":false},"release":"2026-10-13","note":"high is its highest thinking level; run only at high (owner, 2026-11-02). Release date per third-party sources (Sep 30 or Oct 13)."}],"vendors":[{"id":"anthropic","name":"Anthropic","color":{"light":"#c15f3c","dark":"#d9774a"},"shape":"circle","note":"terracotta orange"},{"id":"openai","name":"OpenAI","color":{"light":"#10a37f","dark":"#17ad88"},"shape":"square","note":"teal green"},{"id":"deepseek","name":"DeepSeek","color":{"light":"#0b7fa6","dark":"#45c6e8"},"shape":"diamond","note":"ocean blue"},{"id":"xai","name":"xAI","color":{"light":"#111111","dark":"#b4b8bf"},"shape":"plus","note":"black / white by theme"},{"id":"meta","name":"Meta","color":{"light":"#1877f2","dark":"#4192ff"},"shape":"pentagon","note":"Facebook blue"},{"id":"zhipu","name":"Z.ai","color":{"light":"#a87d00","dark":"#f2c94c"},"shape":"triangle","note":"yellow"},{"id":"google","name":"Google","color":{"light":"#c62f2f","dark":"#d64545"},"shape":"tri-down","note":"red"},{"id":"alibaba","name":"Qwen","color":{"light":"#7b4ce0","dark":"#b18cf5"},"shape":"star","note":"purple"},{"id":"xiaomi","name":"Xiaomi","color":{"light":"#ff7a14","dark":"#ffa040"},"shape":"circle-open","note":"orange"}],"leads":[{"id":"opus-5.5-xhigh","label":"Opus 5.5 xhigh","model":"claude-opus-5.5","effort":"xhigh","harness":"Claude Code","status":"active","since":"2026-11-01","role_notes":"Lead candidate, briefed exactly like the other candidates: the short hand-off prompt, the same task inputs, and the project's Claude memory parked during the session. Wrote the longest briefs (1.4k–2.2k words per task), building and checking prototypes before handing off. One deviation: it ran the scaffold's `make mac-bench` against the benchmark Mac, which is outside its workspace (scratch removed afterwards). As a result, its NEON brief quotes a C-reference time from the scaffold benchmark (about 470 ns per call), not from the official perf harness (about 124 ns). Led round 1 (2026-11-02 to 11-05): all five tasks.","brief_prompt":"templates/lead-prompt.md v2","authoring":{"wall_minutes":37.71,"tokens":{"uncached_input":106,"cache_read":11423249,"cache_write":333240,"output":204354,"reasoning":148214},"cost_usd":9.0381,"cost_breakdown":{"input":0.000424,"cache_read":2.28465,"cache_write":2.66592,"output":4.08708},"cost_usd_as_run":9.0381,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","source":"73311034-20ba-44cc-9ce1-585c49a06180.jsonl","tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"],"date":"2026-11-01"},"brief_tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"]},{"id":"gpt-6.1-sol-max","label":"GPT‑6.1 Sol max","model":"gpt-6.1-sol","effort":"max","harness":"Codex","status":"active","since":"2026-11-01","role_notes":"Lead candidate. Writes its own design notes and briefs from each task's task contract, scaffold and reference, prompted only with a short hand-off instruction. It does not see the hidden suites or any other lead's briefs. Led round 1 (2026-11-02 to 11-05): all five tasks.","authoring":{"wall_minutes":9.24,"tokens":{"uncached_input":74604,"cache_read":1589376,"cache_write":0,"output":32361,"reasoning":16302},"cost_usd":0.6318,"cost_breakdown":{"input":0.149208,"cache_read":0.158938,"cache_write":0.0,"output":0.32361},"cost_usd_as_run":1.2635,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":true,"tier":"priority","source":"rollout-2026-11-01T12-07-55-01a19814-fed2-7562-8c52-e9c8c495be8d.jsonl","tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"],"date":"2026-11-01"},"brief_prompt":"templates/lead-prompt.md v2","brief_tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"]},{"id":"gpt-6-astra-xhigh","label":"GPT‑6 Astra xhigh","model":"gpt-6-astra","effort":"xhigh","harness":"Codex","status":"active","since":"2026-11-01","role_notes":"Lead candidate. Writes its own design notes and briefs from each task's task contract, scaffold and reference, prompted only with a short hand-off instruction. It does not see the hidden suites or any other lead's briefs. Led round 1 (2026-11-02 to 11-05): all five tasks.","authoring":{"wall_minutes":7.55,"tokens":{"uncached_input":80055,"cache_read":1287680,"cache_write":0,"output":25927,"reasoning":9143},"cost_usd":3.3846,"cost_breakdown":{"input":0.80055,"cache_read":1.28768,"cache_write":0.0,"output":1.29635},"cost_usd_as_run":6.7692,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":true,"tier":"priority","source":"rollout-2026-11-01T12-07-57-01a19815-05d3-7113-a8d8-529918816f92.jsonl","tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"],"date":"2026-11-01"},"brief_prompt":"templates/lead-prompt.md v2","brief_tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"]},{"id":"gpt-6.1-terra-max","label":"GPT‑6.1 Terra max","model":"gpt-6.1-terra","effort":"max","harness":"Codex","status":"active","since":"2026-11-01","role_notes":"Lead candidate. Writes its own design notes and briefs from each task's task contract, scaffold and reference, prompted only with a short hand-off instruction. It does not see the hidden suites or any other lead's briefs. Led round 1 (2026-11-02 to 11-05): all five tasks.","authoring":{"wall_minutes":3.01,"tokens":{"uncached_input":88709,"cache_read":1156736,"cache_write":0,"output":17659,"reasoning":5255},"cost_usd":0.2348,"cost_breakdown":{"input":0.088709,"cache_read":0.057837,"cache_write":0.0,"output":0.088295},"cost_usd_as_run":0.4697,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":true,"tier":"priority","source":"rollout-2026-11-01T12-07-58-01a19815-0c68-75f3-b21f-3c08b669d4cf.jsonl","tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"],"date":"2026-11-01"},"brief_prompt":"templates/lead-prompt.md v2","brief_tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"]},{"id":"muse-wonder-full-xhigh","label":"Muse Wonder Full xhigh","model":"muse-wonder-full","effort":"xhigh","harness":"Muse Code","status":"active","since":"2026-11-01","role_notes":"Lead candidate, briefed exactly like the other candidates: the short hand-off prompt and the same task inputs. Briefs run 0.9k–2.2k words per task. One minor deviation: it ran `diff -q` between its copies of task contract and task.json and the benchmark repository's, which is outside its workspace. That reports only whether files differ, not their contents (the SPECs matched; the repository's task.json files, which hold the hidden-suite metadata, differ), and it read nothing else outside its workspace. Muse Code is subscription-only, so authoring cost is at the Muse Wonder 1.0 API rate as a proxy; the run billed $0. Led round 1 (2026-11-02 to 11-05): all five tasks.","brief_prompt":"templates/lead-prompt.md v2","authoring":{"wall_minutes":17.29,"tokens":{"uncached_input":407204,"cache_read":5128656,"cache_write":0,"output":61849,"reasoning":20748},"cost_usd":3.8915,"cost_breakdown":{"input":1.628816,"cache_read":1.025731,"cache_write":0.0,"output":1.23698},"cost_usd_as_run":3.8915,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","source":"session.jsonl","tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"],"date":"2026-11-01"},"brief_tasks":["bounded-tool","cdef-avx512","exr-piz","range-coder","vp8lf-neon"]}],"tasks":[{"id":"range-coder","title":"libopus range coder","short":"Range coder","language":"Rust","lang_short":"Rust","lang_color":"#ce422b","summary":"RFC 6716 range encoder/decoder, bit-exact with libopus, that never panics or hangs on hostile input or API misuse.","status":"used","source":{"project":"PraxiDeck praxi-opus"},"hidden":{"count":11,"categories":[]},"hidden_suite_version":2,"perf":{"metrics":[{"id":"enc","label":"Encode, 2M mixed symbols","unit":"ns/op","higher_is_better":false},{"id":"dec","label":"Decode, 2M mixed symbols","unit":"ns/op","higher_is_better":false}],"reference":"oracle (praxi-opus libopus port)","host":"ai-agent (AMD Ryzen 9 9950X)"},"deliverables":[]},{"id":"exr-piz","title":"OpenEXR PIZ chunk decompressor","short":"EXR PIZ","language":"Go","lang_short":"Go","lang_color":"#007a99","summary":"Decode OpenEXR PIZ-compressed scanline chunks (bitmap LUT, canonical Huffman with runs, 2-D Haar wavelet) in pure Go, robust against hostile input.","status":"used","source":{"project":"Mixtape Jack"},"hidden":{"count":12,"categories":[]},"hidden_suite_version":1,"perf":{"metrics":[{"id":"decode_half","label":"Decompress 1920x32 RGBA half chunks","unit":"MB/s","higher_is_better":true},{"id":"decode_float","label":"Decompress 1920x32 RGB+Z float chunks","unit":"MB/s","higher_is_better":true}],"reference":"oracle (Jack)","host":"ai-agent"},"deliverables":[]},{"id":"cdef-avx512","title":"x86-64-v4 NASM AV1 CDEF (8-bit)","short":"CDEF","language":"x86-64 assembly (NASM, AVX-512)","lang_short":"x86-64","lang_color":"#0068b5","summary":"Hand-written x86-64-v4 NASM for AV1's CDEF direction search and 8x8 filter (8-bit): bit-exact with the C reference, reading only the pixels the edge flags allow, with System V ABI and upper-state hygiene, timed against dav1d's AVX2 and AVX-512ICL kernels.","status":"used","source":{"project":"PraxiDeck praxi-av1"},"hidden":{"count":15,"categories":[]},"hidden_suite_version":1,"perf":{"metrics":[{"id":"filter","label":"8x8 filter, mixed edge flags and strengths","unit":"cycles/call","higher_is_better":false},{"id":"find_dir","label":"Direction search, mixed 8x8 blocks","unit":"cycles/call","higher_is_better":false}],"reference":"C reference (ref/cdef_ref.c) built with gcc -O2 -march=x86-64-v4","host":"ai-agent (AMD Ryzen 9 9950X)","baseline_notes":{"dav1d AVX2":"expert baseline; over-reads the strict memory regions, so it does not meet the contestant contract","dav1d AVX-512ICL":"expert baseline; uses VBMI and over-reads, so it does not meet the contestant contract"}},"deliverables":[]},{"id":"vp8lf-neon","title":"AArch64 NEON VP8 normal loop filter","short":"VP8 loop filter","language":"AArch64 assembly (NEON)","lang_short":"ARM64","lang_color":"#006d8f","summary":"Hand-written AArch64 NEON for VP8's normal loop filter (macroblock and inner edges, both directions): bit-exact with the C reference, one source for ELF and Mach-O, exact memory footprint and AAPCS64 hygiene, timed natively on an Apple M1.","status":"used","source":{"project":"PraxiDeck wpd"},"hidden":{"count":13,"categories":[]},"hidden_suite_version":1,"perf":{"metrics":[{"id":"v_mb","label":"v_loop_filter16 (macroblock edge, horizontal edge)","unit":"ns/call","higher_is_better":false},{"id":"h_mb","label":"h_loop_filter16 (macroblock edge, vertical edge)","unit":"ns/call","higher_is_better":false},{"id":"v_inner","label":"v_loop_filter16_inner (inner edges, horizontal)","unit":"ns/call","higher_is_better":false},{"id":"h_inner","label":"h_loop_filter16_inner (inner edges, vertical)","unit":"ns/call","higher_is_better":false},{"id":"geomean","label":"Geometric mean of the four functions","unit":"ns/call","higher_is_better":false}],"reference":"C reference (ref/vp8_lf_ref.c) built with Apple clang -O2, native M1","host":"mac-dev (Apple M1)","baseline_notes":{"wpd_neon":"expert baseline; stores whole 8-byte rows in the h functions, so it does not meet the contestant contract"}},"deliverables":[]},{"id":"bounded-tool","title":"Bounded external-command runner (Elixir/OTP)","short":"Tool runner","language":"Elixir","lang_short":"Elixir","lang_color":"#6e4a7e","summary":"An OTP runner pool that executes external tools under a hard deadline and output budget, kills their whole process group on timeout, overflow, caller death or VM death, and never signals anything it did not start.","status":"used","source":{"project":"Kinova"},"hidden":{"count":16,"categories":[]},"hidden_suite_version":1,"perf":{"metrics":[{"id":"seq_true","label":"200 sequential runs of `true`","unit":"ms","higher_is_better":false},{"id":"throughput_64m","label":"64 MiB output (head -c 64M /dev/zero, 64 MiB budget)","unit":"MB/s","higher_is_better":true}],"reference":"oracle (adapted Kinova.Media.Tool)","host":"ai-agent"},"deliverables":[]}],"runs":[{"schema":1,"id":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1-b2","task":"bounded-tool","lead":"gpt-6-astra-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":12.9,"tokens":{"cache_read":3658126,"cache_write":184624,"output":63878,"reasoning":46215,"uncached_input":56},"cost_usd":3.4864,"cost_breakdown":{"cache_read":0.731625,"cache_write":1.476992,"input":0.000224,"output":1.27756},"cost_usd_as_run":3.4864,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":11.12,"tokens":{"cache_read":4171648,"cache_write":0,"output":28412,"reasoning":16139,"uncached_input":82594},"cost_usd":0.4332,"cost_breakdown":{"cache_read":0.208582,"cache_write":0.0,"input":0.082594,"output":0.14206},"cost_usd_as_run":0.4332,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":15.48,"tokens":{"cache_read":2095000,"cache_write":0,"output":50441,"reasoning":38670,"uncached_input":299559},"cost_usd":2.6261,"cost_breakdown":{"cache_read":0.419,"cache_write":0.0,"input":1.198236,"output":1.00882},"cost_usd_as_run":2.6261,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":14.99,"tokens":{"cache_read":2367104,"cache_write":0,"output":29754,"reasoning":20672,"uncached_input":103761},"cost_usd":0.7418,"cost_breakdown":{"cache_read":0.23671,"cache_write":0.0,"input":0.207522,"output":0.29754},"cost_usd_as_run":0.7418,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":15.61,"tokens":{"cache_read":9018368,"cache_write":0,"output":122889,"reasoning":92125,"uncached_input":90625},"cost_usd":0.1144,"cost_breakdown":{"cache_read":0.027055,"cache_write":0.0,"input":0.013594,"output":0.073733},"cost_usd_as_run":0.1144,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"L","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"H","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"C","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"L","score":5.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"H","score":3.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"C","score":5.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"L","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"H","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"C","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"L","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"H","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"C","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"L","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"H","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"C","score":9.0}],"panel":["opus-5.5-xhigh","gpt-6.1-terra-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"E","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":18.13,"tokens":{"uncached_input":175314,"cache_read":2613355,"cache_write":0,"output":84919,"reasoning":52640},"cost_usd":1.4612,"cost_breakdown":{"input":0.350628,"cache_read":0.261335,"cache_write":0.0,"output":0.84919},"cost_usd_as_run":1.4612,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":716,"code":578,"comment":12,"blank":126,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":712,"code":557,"comment":17,"blank":138,"role":"test"}],"impl_lines":716,"impl_code":578,"impl_comment":12,"test_lines":712,"comment_ratio":0.02,"tests_declared":18},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":464.601,"reference":432.475,"speedup":0.931,"baselines":{"oracle":432.475}},{"id":"throughput_64m","candidate":2795.233,"reference":2746.013,"speedup":1.018,"baselines":{"oracle":2746.013}}]},"isolation":{"tool_calls":44,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":4.0,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"opus-5.5-xhigh":"The core is correct and the tests are thorough, but the executable check rejects tools allowed through supplementary groups, and the code is verbose and under-documented","gpt-6.1-terra-max":"A foreign pool can make `run/3` raise, and cleanup can miss the group","muse-wonder-full-xhigh":"capable runner undermined by a raise on foreign pools and resolution drift","gpt-6.1-sol-max":"Main execution and cleanup work, but permission and name validation reject valid calls.","deepseek-v4.1-flash-max":"sound design and good public docs, but a real permission-resolution bug, thin tests and no design notes."},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"The exec-permission check ignores supplementary groups","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"A post-kill wait loop re-signals and delays the reply","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The relay and the safety invariant are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"`name: nil` returns `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Defensive padding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"An unexpected pool reply escapes as an exception","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"A removed cached `kill` leaves the group running","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"`nil` pool names disagree with the reference","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"robustness","title":"`run/3` can raise on a foreign pool process","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil` is misclassified","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Helper fallbacks bypass `PATH`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Execute-permission check ignores supplementary groups","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Relay shell name changes exact diagnostic bytes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-run procfs stats on the hot path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Module documentation is minimal","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Supplementary execute groups are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"correctness","title":"`nil` pool names get the wrong error","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Supplementary groups ignored in executable check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` is rejected as invalid","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"Unbounded caller wait","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"The subtle parts are undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"Two hidden-suite areas are untested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`await_proc_gone/4` re-polls and re-signals the already-killed leader for up to 50 ms on every abnormal path","status":"valid","detail":"","location":""}],"score":6.6,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":5.7,"score_cross_vendor":6.6,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","task":"bounded-tool","lead":"gpt-6-astra-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":51.47,"tokens":{"cache_read":22581994,"cache_write":742625,"output":232222,"reasoning":160742,"uncached_input":262},"cost_usd":15.1029,"cost_breakdown":{"cache_read":4.516399,"cache_write":5.941,"input":0.001048,"output":4.64444},"cost_usd_as_run":15.1029,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":48.05,"tokens":{"cache_read":13013888,"cache_write":0,"output":107245,"reasoning":65965,"uncached_input":727351},"cost_usd":3.8286,"cost_breakdown":{"cache_read":1.301389,"cache_write":0.0,"input":1.454702,"output":1.07245},"cost_usd_as_run":3.8286,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":80.65,"tokens":{"cache_read":15424308,"cache_write":0,"output":148437,"reasoning":93857,"uncached_input":723293},"cost_usd":8.9468,"cost_breakdown":{"cache_read":3.084861,"cache_write":0.0,"input":2.893172,"output":2.96874},"cost_usd_as_run":8.9468,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":37.36,"tokens":{"cache_read":22018176,"cache_write":0,"output":91175,"reasoning":50121,"uncached_input":479919},"cost_usd":2.0366,"cost_breakdown":{"cache_read":1.100909,"cache_write":0.0,"input":0.479919,"output":0.455875},"cost_usd_as_run":2.0366,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":74.31,"tokens":{"cache_read":41891456,"cache_write":0,"output":402889,"reasoning":320928,"uncached_input":449452},"cost_usd":0.4348,"cost_breakdown":{"cache_read":0.125674,"cache_write":0.0,"input":0.067418,"output":0.241734},"cost_usd_as_run":0.4348,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":4.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6-astra-xhigh-r1","letter":"B","score":6.5}],"panel":["opus-5.5-xhigh","gpt-6.1-sol-max","muse-wonder-full-xhigh","gpt-6.1-terra-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"A","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":63.7,"tokens":{"uncached_input":447092,"cache_read":3965696,"cache_write":0,"output":250461,"reasoning":111892},"cost_usd":4.3798,"cost_breakdown":{"input":0.894184,"cache_read":1.982848,"cache_write":0.0,"output":1.502766},"cost_usd_as_run":4.3798,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":668,"code":526,"comment":26,"blank":116,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":642,"code":512,"comment":0,"blank":130,"role":"test"}],"impl_lines":668,"impl_code":526,"impl_comment":26,"test_lines":642,"comment_ratio":0.047,"tests_declared":10},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":477.639,"reference":458.185,"speedup":0.959,"baselines":{"oracle":458.185}},{"id":"throughput_64m","candidate":2486.537,"reference":2606.373,"speedup":0.954,"baselines":{"oracle":2606.373}}]},"scores":{"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":8.5,"gpt-6.1-terra-max":6.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"opus-5.5-xhigh":"correct and well tested; duplicate options use the last value","gpt-6.1-sol-max":"Good lifecycle handling, but duplicate options change enforced limits.","muse-wonder-full-xhigh":"Near-oracle, one duplicate-option divergence","gpt-6.1-terra-max":"Duplicate deadlines can overrun the hard return bound","deepseek-v4.1-flash-max":"strong all-round implementation, but a momentarily unavailable `os_pid` becomes a spurious `:unavailable`"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"For a repeated option key, the last value wins","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicated and dead logic","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Last duplicate option wins","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Duplicate options: last value wins","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead PATH fallback","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Later duplicate options override the first","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"`Port.info(port, :os_pid)` returning nil fails the run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"per-run `/proc/self/status` parse","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"duplicate option keys: last wins","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":56,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.6,"score_cross_vendor":7.6,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.49,"tokens":{"uncached_input":16385,"cache_read":418816,"cache_write":0,"output":11956,"reasoning":3208},"cost_usd":0.0971,"cost_breakdown":{"input":0.016385,"cache_read":0.020941,"cache_write":0.0,"output":0.05978},"cost_usd_as_run":0.0971,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":336,"code":278,"comment":6,"blank":52,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":253,"code":213,"comment":1,"blank":39,"role":"test"}],"impl_lines":336,"impl_code":278,"impl_comment":6,"test_lines":253,"comment_ratio":0.021,"tests_declared":11},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":440.786,"reference":480.827,"speedup":1.091,"baselines":{"oracle":480.827}},{"id":"throughput_64m","candidate":2515.188,"reference":2466.39,"speedup":1.02,"baselines":{"oracle":2466.39}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":5.5,"muse-wonder-full-xhigh":7.0,"gpt-6.1-terra-max":5.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"opus-5.5-xhigh":"solid runner; resolution, option and doc gaps","gpt-6.1-sol-max":"Path rewriting can execute the wrong tool, and validation misses shadowed invalid values.","muse-wonder-full-xhigh":"Solid core, several small semantic drifts","gpt-6.1-terra-max":"Slash paths can select the wrong tool and invalid duplicate options can execute","deepseek-v4.1-flash-max":"clean and well-tested; the `Port.info` nil case can fail a valid run, and docs are minimal"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`Path.expand/1` expands `~` in paths containing a slash","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The `@doc` strings paraphrase the contract and omit the options; `@spec` exists only on `run/3`; there is no `@typedoc`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"Only the first occurrence of a repeated option is validated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Slash paths are rewritten","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate values are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public options are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Only the first of each duplicate option is validated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Helpers searched off PATH first","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Perfunctory function docs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Slash paths are normalized incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Later invalid duplicate values are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public option documentation is incomplete","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"`Port.info(port, :os_pid)` nil → spurious unavailable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"a partially opened port is not closed on the failure path","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"minimal public docs","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":234,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.4,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.7,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":18.85,"tokens":{"uncached_input":36,"cache_read":2053505,"cache_write":155011,"output":119720,"reasoning":93617},"cost_usd":4.0453,"cost_breakdown":{"input":0.000144,"cache_read":0.410701,"cache_write":1.240088,"output":2.3944},"cost_usd_as_run":4.0453,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":598,"code":420,"comment":79,"blank":99,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":718,"code":580,"comment":9,"blank":129,"role":"test"}],"impl_lines":598,"impl_code":420,"impl_comment":79,"test_lines":718,"comment_ratio":0.158,"tests_declared":32},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":467.144,"reference":478.161,"speedup":1.024,"baselines":{"oracle":478.161}},{"id":"throughput_64m","candidate":2541.609,"reference":2510.528,"speedup":1.012,"baselines":{"oracle":2510.528}}]},"scores":{"opus-5.5-xhigh":9.5,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":9.0,"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"opus-5.5-xhigh":"correct on every probe, accurate docs, the most thorough tests","gpt-6.1-sol-max":"Complete, well documented, and consistent with the reference across the checked cases.","muse-wonder-full-xhigh":"Most thoroughly engineered; one environment-caching divergence","gpt-6.1-terra-max":"Thorough implementation and tests, with an event-ordering error","deepseek-v4.1-flash-max":"exemplary: identity-checked group kills, real watchdog backstop, most rigorous tests of the set"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Helpers resolved once at pool start, with off-PATH fallback","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"helper paths are snapshotted when the pool starts","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":372,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":21,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.2,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.7,"score_cross_vendor":9.125,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.63,"tokens":{"uncached_input":14813,"cache_read":116096,"cache_write":0,"output":3044,"reasoning":40},"cost_usd":0.0358,"cost_breakdown":{"input":0.014813,"cache_read":0.005805,"cache_write":0.0,"output":0.01522},"cost_usd_as_run":0.0358,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":238,"code":200,"comment":1,"blank":37,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":41,"code":32,"comment":0,"blank":9,"role":"test"}],"impl_lines":238,"impl_code":200,"impl_comment":1,"test_lines":41,"comment_ratio":0.005,"tests_declared":3},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":473.011,"reference":483.202,"speedup":1.022,"baselines":{"oracle":483.202}},{"id":"throughput_64m","candidate":2612.477,"reference":2536.177,"speedup":1.03,"baselines":{"oracle":2536.177}}]},"scores":{"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":4.0,"muse-wonder-full-xhigh":5.5,"gpt-6.1-terra-max":4.0,"deepseek-v4.1-flash-max":6.0},"council_summaries":{"opus-5.5-xhigh":"mostly correct, but almost untested and undocumented","gpt-6.1-sol-max":"Public exceptions, incorrect paths, and thin tests require substantial changes.","muse-wonder-full-xhigh":"Works, but raises on odd binaries and is barely tested or documented","gpt-6.1-terra-max":"A valid binary can raise, and slash paths can select the wrong executable","deepseek-v4.1-flash-max":"correct core but very thin docs and tests; the `Port.info` race was observed failing under load"},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Three tests in total","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`Path.expand/1` expands `~` in paths containing a slash","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc` or `@spec`; the interface's `@type reason` is missing; `Runner` has no `@moduledoc false`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`resolve/1` stats the file three times (`File.regular?`, `File.stat`, `executable?/1`), and the call's start time is stored in a variable named `deadline` (line 19)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Resolution exceptions escape `run/3`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Slash paths are expanded incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Major guarantees are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation and types are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Repeated metadata queries","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Non-UTF-8 executable crashes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No `@doc`/`@spec`/`@type` anywhere","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Minimal tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Compressed, triplicated lookups","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Slash paths are normalized incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Bare non-UTF-8 executable names can raise","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"The public `reason/0` type is missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Major lifecycle cases are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack API documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"`Port.info(port, :os_pid)` nil → spurious unavailable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"the fixed interface's documentation and type are missing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"three tests only","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"the relay script is a three-line escaped one-liner","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":167,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.1,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":8.0,"score_cross_vendor":5.833,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":30.81,"tokens":{"uncached_input":142309,"cache_read":4894592,"cache_write":0,"output":95532,"reasoning":77423},"cost_usd":0.216,"cost_breakdown":{"input":0.021346,"cache_read":0.146838,"cache_write":0.0,"output":0.047766},"cost_usd_as_run":0.216,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":624,"code":437,"comment":83,"blank":104,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":429,"code":320,"comment":18,"blank":91,"role":"test"}],"impl_lines":624,"impl_code":437,"impl_comment":83,"test_lines":429,"comment_ratio":0.16,"tests_declared":23},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":433.255,"reference":431.235,"speedup":0.995,"baselines":{"oracle":431.235}},{"id":"throughput_64m","candidate":2725.443,"reference":2702.1,"speedup":1.009,"baselines":{"oracle":2702.1}}]},"scores":{"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":5.0,"muse-wonder-full-xhigh":7.0,"gpt-6.1-terra-max":5.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"opus-5.5-xhigh":"a crashed runner or stopped pool returns `:timeout`; misleading comments","gpt-6.1-sol-max":"Runner failures and completion races compromise otherwise comprehensive handling.","muse-wonder-full-xhigh":"Good design, wrong reason when the runner itself dies","gpt-6.1-terra-max":"A caller-death race retains a pool slot, and runner crashes are misclassified","deepseek-v4.1-flash-max":"solid design but a crashed runner is reported as `:timeout`, contrary to SPEC §9"},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"A crashed runner or a stopped pool returns `{:error, :timeout}` instead of `{:error, :unavailable}`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"The exit-status handler does not check the deadline","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"A false comment covers a 65 s slot leak","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No runner-crash or pool-shutdown test (either would have caught the medium bug). Process checks use system-wide `pgrep -f` patterns. The test \"never raises on any input shape\" covers a single input","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Dead clauses and a false comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Runner crashes become timeouts","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Completed runs retain abandoned slots","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Delayed exit bypasses the deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Port EXIT comment is false","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Runner crash reported as `:timeout`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"No deadline check on `exit_status`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Group kill via shell string","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"A completed runner can retain its slot after caller death","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Runner crashes become timeouts","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A delayed exit event bypasses the deadline check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Boolean check is redundant","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"runner/pool crash yields `{:error, :timeout}`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"`Port.info(port, :os_pid)` == nil abandons the run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"completed-but-unawaited runner lingers 65 s","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":53,"flags":1,"audited":"2026-11-02","reviewed":true},"conduct":{"severity":"minor","summary":"While cleaning up it ran `pkill -f 'sleep 3005'`, a host-wide pattern that would also match other jobs' processes with that command line. No other process is known to have been hit; scores stand (same rule as the owner's 2026-11-02 decision)."},"score":6.2,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.3,"score_cross_vendor":6.2,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":28.09,"tokens":{"uncached_input":88,"cache_read":10541973,"cache_write":358800,"output":299353,"reasoning":259553},"cost_usd":1.459,"cost_breakdown":{"input":4.2e-05,"cache_read":0.525013,"cache_write":0.334788,"output":0.599205},"cost_usd_as_run":1.459,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":667,"code":477,"comment":81,"blank":109,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":557,"code":430,"comment":11,"blank":116,"role":"test"}],"impl_lines":667,"impl_code":477,"impl_comment":81,"test_lines":557,"comment_ratio":0.145,"tests_declared":28},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":571.673,"reference":459.746,"speedup":0.804,"baselines":{"oracle":459.746}},{"id":"throughput_64m","candidate":2520.532,"reference":2541.859,"speedup":0.992,"baselines":{"oracle":2541.859}}]},"scores":{"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":4.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-terra-max":4.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"opus-5.5-xhigh":"well designed and tested; one slot leak and over-strict options","gpt-6.1-sol-max":"Extensive coverage, but valid inputs and an abandonment race remain broken.","muse-wonder-full-xhigh":"Exemplary structure and tests, stricter than the contract in places","gpt-6.1-terra-max":"Wrong path resolution and a permanent retained-runner race","deepseek-v4.1-flash-max":"best docs and tests of the set; only over-strict option validation"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"A pool slot can leak permanently","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Options are stricter than the reference: duplicate keys and `name: nil` are rejected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The moduledoc says \"When `run/3` returns, the tool's whole process group is dead\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Completion can permanently leak a pool slot","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path normalization selects a different executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid duplicate options are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"correctness","title":"Absent nil-named pool is misclassified","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Option parsing stricter than the oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Helper fallback off PATH","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Caller demonitored before result is collected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Credentials re-read per PATH candidate","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Slash paths are normalized incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"A completed runner can permanently retain its slot","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid duplicate options are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"`nil` pool names are classified differently","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"duplicate option keys rejected","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` rejected","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"deadline/await arithmetic","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":412,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":54,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.6,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":5.7,"score_cross_vendor":6.25,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":397.94,"tokens":{"uncached_input":313578,"cache_read":5984896,"cache_write":0,"output":69975,"reasoning":39282},"cost_usd":3.8508,"cost_breakdown":{"input":1.254312,"cache_read":1.196979,"cache_write":0.0,"output":1.3995},"cost_usd_as_run":3.8508,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":673,"code":545,"comment":21,"blank":107,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":590,"code":466,"comment":5,"blank":119,"role":"test"}],"impl_lines":673,"impl_code":545,"impl_comment":21,"test_lines":590,"comment_ratio":0.037,"tests_declared":15},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":487.579,"reference":461.334,"speedup":0.946,"baselines":{"oracle":461.334}},{"id":"throughput_64m","candidate":2407.02,"reference":2505.894,"speedup":0.961,"baselines":{"oracle":2505.894}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"opus-5.5-xhigh":"correct core wrapped in heavy defensive padding; exit path skips the deadline check","gpt-6.1-sol-max":"Thorough implementation with deadline and option-domain discrepancies.","muse-wonder-full-xhigh":"Sound, over-defended, two small divergences","gpt-6.1-terra-max":"Good coverage and cleanup, but deadline event handling differs from the reference","deepseek-v4.1-flash-max":"correct and exhaustively tested; only edge-case deviations and defensive-overkill nits"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"The exit-status handler does not check the deadline","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Defensive padding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"`name: nil` is rejected as `:invalid_argument`; the reference treats it as an atom and returns `:unavailable`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Delayed exit bypasses the deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"correctness","title":"Valid atom name is rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unused hygiene fixture","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil` rejected as invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"No deadline check on `exit_status`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Pervasive boilerplate rescue/catch wrappers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A delayed exit event bypasses the deadline check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"`nil` pool names are classified differently","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Large-output and runner-crash cases are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"The three-element start result branch is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`:name` rejects `nil`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"pool introspection duplicates the supervisor's own check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"~20 identical `try/rescue/catch` wrappers","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":61,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.7,"slop":{"score":0.8,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0}},"defect_points":3.5,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.51,"tokens":{"uncached_input":27530,"cache_read":310912,"cache_write":0,"output":12510,"reasoning":5008},"cost_usd":0.2113,"cost_breakdown":{"input":0.05506,"cache_read":0.031091,"cache_write":0.0,"output":0.1251},"cost_usd_as_run":0.2113,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":336,"code":276,"comment":5,"blank":55,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":297,"code":250,"comment":0,"blank":47,"role":"test"}],"impl_lines":336,"impl_code":276,"impl_comment":5,"test_lines":297,"comment_ratio":0.018,"tests_declared":14},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":466.17,"reference":440.07,"speedup":0.944,"baselines":{"oracle":440.07}},{"id":"throughput_64m","candidate":2713.96,"reference":2752.676,"speedup":0.986,"baselines":{"oracle":2752.676}}]},"scores":{"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":5.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-terra-max":4.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"opus-5.5-xhigh":"sound runner; resolution, option and doc deviations","gpt-6.1-sol-max":"Solid execution machinery with incorrect path and duplicate-option semantics.","muse-wonder-full-xhigh":"Capable and well tested, looser edge semantics","gpt-6.1-terra-max":"Wrong executable resolution and duplicate options can violate the deadline","deepseek-v4.1-flash-max":"solid behaviour and tests; the public API has no documentation at all"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`Path.expand/1` expands `~` in paths containing a slash","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"For a repeated option key, the last value wins (`Enum.reduce`)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc` on `child_spec/1`, `start_link/1` or `run/3`, and no `@typedoc`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Literal paths change meaning","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public functions lack documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Duplicate options: last value wins","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Public docs missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Slash paths are changed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Later duplicate options override the first","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Non-UTF-8 paths are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack API documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no `@doc`, `@spec` or `@typedoc` on the public API","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"nil identity disables the runner's own group kill","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"duplicate option keys: last wins","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":229,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.1,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.2,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.61,"tokens":{"uncached_input":6018,"cache_read":100480,"cache_write":0,"output":3474,"reasoning":11},"cost_usd":0.3344,"cost_breakdown":{"input":0.06018,"cache_read":0.10048,"cache_write":0.0,"output":0.1737},"cost_usd_as_run":0.3344,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":231,"code":192,"comment":1,"blank":38,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":89,"code":76,"comment":0,"blank":13,"role":"test"}],"impl_lines":231,"impl_code":192,"impl_comment":1,"test_lines":89,"comment_ratio":0.005,"tests_declared":7},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":470.448,"reference":450.765,"speedup":0.958,"baselines":{"oracle":450.765}},{"id":"throughput_64m","candidate":2555.652,"reference":2543.0,"speedup":1.005,"baselines":{"oracle":2543.0}}]},"scores":{"opus-5.5-xhigh":5.0,"gpt-6.1-sol-max":4.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-terra-max":4.0,"deepseek-v4.1-flash-max":5.5},"council_summaries":{"opus-5.5-xhigh":"relative paths containing a slash cannot be run; no interface docs","gpt-6.1-sol-max":"Relative executable paths are unsupported despite otherwise sound execution behavior.","muse-wonder-full-xhigh":"Behaviourally excellent, undocumented","gpt-6.1-terra-max":"Ordinary relative executable paths are unavailable","deepseek-v4.1-flash-max":"relative slash paths do not resolve (contract §1) plus the shared `Port.info` race"},"findings":[{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Relative paths that contain a slash are searched on PATH","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc`, `@spec` or `@type reason` on the public API, and `Runner` lacks `@moduledoc false`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Timeout and caller-death tests check only elapsed time or child count, never that the tool's processes are dead. There is no relative-path test and no VM-SIGKILL test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Relative slash paths never resolve","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Cleanup tests do not inspect OS processes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public contracts are missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Public API undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No group-death or VM-death verification","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Slash paths are searched as executable names","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"The public `reason/0` type is missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cleanup tests do not verify the OS processes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack API documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"a relative path containing a slash is looked up on PATH and fails","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"`Port.info` nil → spurious unavailable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"public API undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"the suite never exercises a relative slash path","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":161,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.3,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.2,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.27,"tokens":{"uncached_input":7090,"cache_read":199680,"cache_write":0,"output":4716,"reasoning":474},"cost_usd":0.0407,"cost_breakdown":{"input":0.00709,"cache_read":0.009984,"cache_write":0.0,"output":0.02358},"cost_usd_as_run":0.0407,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":278,"code":233,"comment":1,"blank":44,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":71,"code":59,"comment":0,"blank":12,"role":"test"}],"impl_lines":278,"impl_code":233,"impl_comment":1,"test_lines":71,"comment_ratio":0.004,"tests_declared":6},"hidden":{"passed":14,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":447.739,"reference":448.806,"speedup":1.002,"baselines":{"oracle":448.806}},{"id":"throughput_64m","candidate":2652.622,"reference":2627.908,"speedup":1.009,"baselines":{"oracle":2627.908}}]},"scores":{"opus-5.5-xhigh":4.5,"gpt-6.1-sol-max":3.5,"muse-wonder-full-xhigh":4.5,"gpt-6.1-terra-max":4.0,"deepseek-v4.1-flash-max":5.0},"council_summaries":{"opus-5.5-xhigh":"`run/3` raises on an improper `args` list; thin tests; no docs","gpt-6.1-sol-max":"Multiple public exceptions and incorrect path semantics undermine the contract.","muse-wonder-full-xhigh":"`run/3` raises on inputs the contract explicitly lists","gpt-6.1-terra-max":"Invalid input can raise, and slash paths can run the wrong executable","deepseek-v4.1-flash-max":"raises on an improper `args` list, flakes under load on the `Port.info` race, and ships almost no docs or tests"},"findings":[{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"`run/3` raises on an improper `args` list","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Paths containing a slash go through `Path.expand/1`, so `~` becomes `$HOME`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Six tests only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc`, `@spec` or `@typedoc` on the public functions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The `{:EXIT, _, :shutdown}` clause can never run, because gen_server handles the parent's exit itself","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Improper argument lists raise","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Resolution errors escape the public boundary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes the executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Lifecycle coverage checks slots rather than tool death","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public functions are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"robustness","title":"Improper `args` list crashes the caller","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Non-UTF-8 executable crashes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No `@doc`, `@spec` or `@type` use anywhere","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Thin verification","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Hard-coded `/bin/kill`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Slash paths are normalized incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"`run/3` can raise during validation or resolution","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Lifecycle and large-output coverage is incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack API documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"`run/3` raises `FunctionClauseError` for an improper args list","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"`Port.info(port, :os_pid)` == nil becomes `:unavailable`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"public API undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"thin suite","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":195,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":4.3,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":9.8,"score_cross_vendor":4.667,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.18,"tokens":{"uncached_input":30,"cache_read":1104329,"cache_write":83640,"output":51292,"reasoning":30133},"cost_usd":1.9159,"cost_breakdown":{"input":0.00012,"cache_read":0.220866,"cache_write":0.66912,"output":1.02584},"cost_usd_as_run":1.9159,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":538,"code":404,"comment":44,"blank":90,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":513,"code":398,"comment":15,"blank":100,"role":"test"}],"impl_lines":538,"impl_code":404,"impl_comment":44,"test_lines":513,"comment_ratio":0.098,"tests_declared":21},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":406.245,"reference":450.387,"speedup":1.109,"baselines":{"oracle":450.387}},{"id":"throughput_64m","candidate":2644.217,"reference":2413.474,"speedup":1.096,"baselines":{"oracle":2413.474}}]},"scores":{"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":9.0,"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-sol-max":"Late exit handling differs from the reference’s deadline behavior","opus-5.5-xhigh":"Tight, correct design; matched the reference on every probe and is stricter on permission checks","gpt-6.1-terra-max":"A delayed exit event bypasses the deadline check","muse-wonder-full-xhigh":"well factored into Pool/Exec/Runner; one deadline race","deepseek-v4.1-flash-max":"very clean, best-in-class docs; one dead helper and one ambiguous test."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Exit-status handling omits the deadline check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unused `relay/0`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Exit status is accepted after the deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Cleanup can kill an unrelated recycled PID","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Cleanup comment overstates its check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`exit_status` not deadline-checked","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Wrong process-group attribution","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unused public relay accessor","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`BoundedTool.Exec.relay/0` is never called","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"one assertion admits two outcomes","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":362,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.7,"slop":{"score":0.6,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.5}},"defect_points":2.1,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":44.04,"tokens":{"uncached_input":774,"cache_read":19556280,"cache_write":231439,"output":204734,"reasoning":0},"cost_usd":0.4555,"cost_breakdown":{"input":0.000116,"cache_read":0.3129,"cache_write":0.046288,"output":0.096225},"cost_usd_as_run":0.4555,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.4555,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":655,"code":453,"comment":102,"blank":100,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":592,"code":459,"comment":18,"blank":115,"role":"test"}],"impl_lines":655,"impl_code":453,"impl_comment":102,"test_lines":592,"comment_ratio":0.184,"tests_declared":31},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":472.445,"reference":471.574,"speedup":0.998,"baselines":{"oracle":471.574}},{"id":"throughput_64m","candidate":2484.136,"reference":2451.878,"speedup":1.013,"baselines":{"oracle":2451.878}}]},"scores":{"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":4.0,"muse-wonder-full-xhigh":6.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-sol-max":"Incorrect executable selection and a persistent runner-slot leak","opus-5.5-xhigh":"Thorough and well documented, but a finished runner can leak, plus several sloppy edges","gpt-6.1-terra-max":"Slot leaks, wrong path resolution, and destructive test cleanup","muse-wonder-full-xhigh":"comprehensive tests; several small contract and OTP-shape flaws","deepseek-v4.1-flash-max":"thorough and superbly documented; two internal errors are mislabelled as `:invalid_argument`."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path normalization selects a different executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Completion removes the caller monitor too early","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Finished runner never stops if its caller dies before removing it","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil` is rejected as invalid","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Invalid `init/1` return","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong error class for internal failures","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Timer comment is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Options validated twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Symlink paths can execute the wrong tool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Completion drops the caller monitor too early","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Test teardown deletes a fixed directory","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"`nil` is rejected as a pool name","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Cleanup can signal an unrelated process group","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil` rejected as invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Rescues misclassified as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Invalid `init/1` return on spawn failure","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Await overshoots the return bound","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Internal modules documented as public API","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"internal resolution failures are reported as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`ProcGroup.identity/1` misparses a command name containing `\") \"`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"`nil` is rejected as a pool name","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"option values are validated twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"the caller's await backstop is 10 s past the deadline","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":128,"flags":1,"audited":"2026-11-02","reviewed":true},"conduct":{"severity":"serious","summary":"While cleaning up its own test processes it ran a broad `pkill -KILL -f 'sleep 30'`, which killed three processes of an unrelated job on the host, outside its work folder. It did not mention this in its report. It gained nothing, so its scores stand (same treatment as the first such case, owner decision 2026-11-02)."},"score":6.0,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":6.2,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":25.42,"tokens":{"uncached_input":18974,"cache_read":1000448,"cache_write":0,"output":40036,"reasoning":25638},"cost_usd":0.5384,"cost_breakdown":{"input":0.037948,"cache_read":0.100045,"cache_write":0.0,"output":0.40036},"cost_usd_as_run":0.5384,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":522,"code":410,"comment":28,"blank":84,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":563,"code":465,"comment":2,"blank":96,"role":"test"}],"impl_lines":522,"impl_code":410,"impl_comment":28,"test_lines":563,"comment_ratio":0.064,"tests_declared":21},"hidden":{"passed":14,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":439.051,"reference":479.486,"speedup":1.092,"baselines":{"oracle":479.486}},{"id":"throughput_64m","candidate":2481.451,"reference":2587.532,"speedup":0.959,"baselines":{"oracle":2587.532}}]},"scores":{"gpt-6.1-sol-max":3.5,"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":6.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-sol-max":"Incorrect executable selection and premature cancellation of valid runs","opus-5.5-xhigh":"Correct under stress, but needlessly intricate admission and brittle helper lookup","gpt-6.1-terra-max":"Valid runs can be discarded and paths changed","muse-wonder-full-xhigh":"strong tests and admission design; over-engineered edges","deepseek-v4.1-flash-max":"correct and tested, but relies on DynamicSupervisor's private call protocol and a timing-sensitive admission lease."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"An arbitrary handoff timer aborts valid calls","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Boolean atom pool names cannot be registered","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Helpers looked up only in `/usr/bin` and `/bin`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Private protocol and manual registration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Inaccurate comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Symlink paths can execute the wrong tool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A live caller can lose its run after 500 ms","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A `false` pool name cannot be registered","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Non-UTF-8 slash path rejected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Admission budget can reject healthy starts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"abi","title":"Private supervisor wire format","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Truncated `child_spec` doc","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"admission/removal use DynamicSupervisor's internal messages","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"the 500 ms `:unclaimed` lease can fail a valid run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`@type reason` has no `@typedoc`, and the relay script","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":349,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.4,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.9,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":21.6,"tokens":{"uncached_input":27995,"cache_read":1149696,"cache_write":0,"output":39564,"reasoning":23162},"cost_usd":3.4078,"cost_breakdown":{"input":0.27995,"cache_read":1.149696,"cache_write":0.0,"output":1.9782},"cost_usd_as_run":3.4078,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":512,"code":405,"comment":19,"blank":88,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":674,"code":571,"comment":4,"blank":99,"role":"test"}],"impl_lines":512,"impl_code":405,"impl_comment":19,"test_lines":674,"comment_ratio":0.045,"tests_declared":24},"hidden":{"passed":14,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":831.286,"reference":463.675,"speedup":0.558,"baselines":{"oracle":463.675}},{"id":"throughput_64m","candidate":2526.841,"reference":2561.637,"speedup":0.986,"baselines":{"oracle":2561.637}}]},"scores":{"gpt-6.1-sol-max":3.5,"opus-5.5-xhigh":6.0,"gpt-6.1-terra-max":5.5,"muse-wonder-full-xhigh":6.0,"deepseek-v4.1-flash-max":6.0},"council_summaries":{"gpt-6.1-sol-max":"Incorrect executable selection and caller-mailbox contamination","opus-5.5-xhigh":"Leaves port EXIT messages in trapping callers' mailboxes and is ~1.8× slower, both from a per-run `test -x`","gpt-6.1-terra-max":"Path, mailbox, helper-dependency, and hot-path defects","muse-wonder-full-xhigh":"sophisticated, but pollutes trapping callers' mailboxes and is 2× slower","deepseek-v4.1-flash-max":"excellent group-safety work and the best test suite, but `run/3` violates mailbox hygiene and pays four process spawns per call."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Permission probes leave port messages in the caller","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Four permission-check processes run per invocation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public functions have no documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"Hygiene violation for callers that trap exits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Four extra OS processes per run","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public functions undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Private protocol plus a custom name registry","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Hygiene test misses trapping callers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Monitor churn","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Symlink paths can execute the wrong tool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Trapping callers retain port exit messages","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"An extra helper is required","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Permission checks spawn four processes per run","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions have no user-facing documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Port EXIT messages leak to trapping callers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Extra process spawn per resolved file","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"abi","title":"Private supervisor wire format","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No `@doc` on public functions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"`run/3` leaves port messages in a caller that traps exits","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"four OS processes per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"the interface has no docs","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":341,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":24,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.4,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":7.7,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":14.92,"tokens":{"uncached_input":91404,"cache_read":3555072,"cache_write":0,"output":49244,"reasoning":27434},"cost_usd":0.5154,"cost_breakdown":{"input":0.091404,"cache_read":0.177754,"cache_write":0.0,"output":0.24622},"cost_usd_as_run":0.5154,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":591,"code":467,"comment":32,"blank":92,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":494,"code":410,"comment":1,"blank":83,"role":"test"}],"impl_lines":591,"impl_code":467,"impl_comment":32,"test_lines":494,"comment_ratio":0.064,"tests_declared":21},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":522.445,"reference":452.336,"speedup":0.866,"baselines":{"oracle":452.336}},{"id":"throughput_64m","candidate":2579.943,"reference":2718.118,"speedup":0.949,"baselines":{"oracle":2718.118}}]},"scores":{"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-sol-max":"Careful lifecycle handling with incomplete public documentation","opus-5.5-xhigh":"Correct, but over-engineered: an extra OS process per run and reliance on private OTP message formats","gpt-6.1-terra-max":"Strong behavior and tests, with sparse API docs","muse-wonder-full-xhigh":"careful hygiene and naming; costly per-run resolver","deepseek-v4.1-flash-max":"careful and well documented, including a real insight about trapping callers, at the cost of an extra process spawn per run."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public options and defaults are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Extra spawned process and OS process on every run","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Relies on DynamicSupervisor's private call format","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Hard-coded `/usr/bin/test`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicated checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Global registration for `nil`/`:undefined` pool names","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public docs omit option details and defaults","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Resolver process plus `test -x` spawn per run","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Single-PID fallback kill","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Truncated public docs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"every run spawns a resolver process plus a `/usr/bin/test` OS process","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"the deadline is tested twice in the exit-status handler","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":400,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":67,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":2.0,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":40.7,"tokens":{"uncached_input":158848,"cache_read":2616916,"cache_write":0,"output":37023,"reasoning":17068},"cost_usd":1.8992,"cost_breakdown":{"input":0.635392,"cache_read":0.523383,"cache_write":0.0,"output":0.74046},"cost_usd_as_run":1.8992,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":684,"code":541,"comment":25,"blank":118,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":436,"code":362,"comment":1,"blank":73,"role":"test"}],"impl_lines":684,"impl_code":541,"impl_comment":25,"test_lines":436,"comment_ratio":0.044,"tests_declared":11},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":457.99,"reference":451.603,"speedup":0.986,"baselines":{"oracle":451.603}},{"id":"throughput_64m","candidate":2677.515,"reference":2581.3,"speedup":1.037,"baselines":{"oracle":2581.3}}]},"scores":{"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":5.5,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-sol-max":"Resolution and valid-name errors, plus dead code and loose test bounds","opus-5.5-xhigh":"Behaviour mostly right, but padded with dead defensive code and two divergences","gpt-6.1-terra-max":"Valid paths and atom pool names are rejected or changed","muse-wonder-full-xhigh":"solid reference-like runner; minor validation and await bounds","deepseek-v4.1-flash-max":"correct and compact; an over-strict name rule and two dead code paths, plus a missing 64 MiB test."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid atom names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Unreachable success branch duplicates the await lifecycle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Watchdog assertions permit cleanup beyond the contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`nil`, `true` and `false` rejected as pool names","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Dead and duplicated branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No 64 MiB test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Symlink paths can execute the wrong tool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid atom names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Watchdog cleanup can signal recycled PIDs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The large-output requirement is untested","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil/true` rejected as invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unbounded await","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead duplicate await branch","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`nil`/`true`/`false` are rejected as pool names","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`{:eof}`/`{:closed}` handlers can never run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"the `{:ok, runner, _info}` branch is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no 64 MiB budget test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0569,"lines":26,"significant":457,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":29,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.3,"slop":{"score":0.7,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":4.9,"score_cross_vendor":5.875,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":16.51,"tokens":{"uncached_input":13617,"cache_read":612608,"cache_write":0,"output":30807,"reasoning":17538},"cost_usd":0.3966,"cost_breakdown":{"input":0.027234,"cache_read":0.061261,"cache_write":0.0,"output":0.30807},"cost_usd_as_run":0.3966,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":458,"code":359,"comment":25,"blank":74,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":585,"code":491,"comment":3,"blank":91,"role":"test"}],"impl_lines":458,"impl_code":359,"impl_comment":25,"test_lines":585,"comment_ratio":0.065,"tests_declared":26},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":494.71,"reference":466.124,"speedup":0.942,"baselines":{"oracle":466.124}},{"id":"throughput_64m","candidate":2546.753,"reference":2454.47,"speedup":1.038,"baselines":{"oracle":2454.47}}]},"scores":{"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-sol-max":"Strong lifecycle coverage, but resolution can execute the wrong file","opus-5.5-xhigh":"Correct and well tested, but path expansion and private-protocol coupling","gpt-6.1-terra-max":"A path containing a symlink and `..` can select the wrong executable","muse-wonder-full-xhigh":"bounded admission/removal and strong tests; await itself unbounded","deepseek-v4.1-flash-max":"correct, robust admission bounded by the deadline, strong tests; only nits."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths and PATH entries","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Uses DynamicSupervisor's private call format","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Redundant checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Symlink paths can execute the wrong tool","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unbounded await despite bounded admission","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"abi","title":"Private supervisor wire format","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Truncated public docs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"`child_spec/1` id collapses to `nil` on invalid opts","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"the deadline is tested twice when completing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`@type reason` has no `@typedoc`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":307,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.0,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":2.6,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":22.93,"tokens":{"uncached_input":60214,"cache_read":13584512,"cache_write":0,"output":187999,"reasoning":154871},"cost_usd":0.1626,"cost_breakdown":{"input":0.009032,"cache_read":0.040754,"cache_write":0.0,"output":0.112799},"cost_usd_as_run":0.1626,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1626,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":670,"code":494,"comment":66,"blank":110,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":735,"code":591,"comment":3,"blank":141,"role":"test"}],"impl_lines":670,"impl_code":494,"impl_comment":66,"test_lines":735,"comment_ratio":0.118,"tests_declared":24},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":424.565,"reference":481.7,"speedup":1.135,"baselines":{"oracle":481.7}},{"id":"throughput_64m","candidate":2646.707,"reference":2558.345,"speedup":1.035,"baselines":{"oracle":2558.345}}]},"scores":{"gpt-6.1-sol-max":10.0,"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-sol-max":"Complete behavior in the checked cases, supported by strong tests","opus-5.5-xhigh":"Correct, well documented, thorough tests; minor padding","gpt-6.1-terra-max":"The runner is sound in the probes, but tests can delete existing data","muse-wonder-full-xhigh":"clean, well-documented, well-tested runner","deepseek-v4.1-flash-max":"clean, compact and well tested; one narrow error-mapping divergence."},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Internal failures reported as invalid input","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"No-op indirection","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Test setup deletes a fixed directory","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Cleanup signals recorded groups without identity checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Rescues misclassified as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"Helper existence check ignores executability","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"a non-UTF-8 executable name yields `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`supervisor_process?/1` probes the undocumented `$initial_call` dictionary entry and the `:sys.suspend_loop` current function","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":412,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":86,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.3,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.8,"score_cross_vendor":8.375,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.67,"tokens":{"uncached_input":28,"cache_read":837892,"cache_write":67186,"output":31923,"reasoning":12652},"cost_usd":1.3436,"cost_breakdown":{"input":0.000112,"cache_read":0.167578,"cache_write":0.537488,"output":0.63846},"cost_usd_as_run":1.3436,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":528,"code":399,"comment":34,"blank":95,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":528,"code":430,"comment":1,"blank":97,"role":"test"}],"impl_lines":528,"impl_code":399,"impl_comment":34,"test_lines":528,"comment_ratio":0.079,"tests_declared":31},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":469.034,"reference":460.286,"speedup":0.981,"baselines":{"oracle":460.286}},{"id":"throughput_64m","candidate":2499.96,"reference":2638.083,"speedup":0.948,"baselines":{"oracle":2638.083}}]},"scores":{"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":4.5,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-sol-max":"Several resolution and pool-name errors despite substantial tests","opus-5.5-xhigh":"Clean and well tested (including trapping callers); one resolution divergence","gpt-6.1-terra-max":"Path and pool-name errors plus destructive test setup","muse-wonder-full-xhigh":"thorough and well tested; PATH and internal-representation nits","deepseek-v4.1-flash-max":"correct, well documented and the most thorough tests; only nits."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Empty PATH entries are discarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"The atom `false` becomes the default pool name","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Roundabout `/proc` parsing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Symlink paths can execute the wrong tool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Test setup deletes a fixed directory","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Empty PATH entries are discarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"`false` is replaced by the default pool name","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"VM cleanup signals a stale launcher PID","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Empty `PATH` element ignored","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Raw `file_info` tuple indexing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Public function in hidden runner","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"the `{BoundedTool.Pool, :init, 1}` alternative is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"one assertion is deliberately ambiguous","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":344,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":13,"flags":1,"audited":"2026-11-02","reviewed":true},"score":6.7,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":4.8,"score_cross_vendor":6.25,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.57,"tokens":{"uncached_input":15813,"cache_read":178944,"cache_write":0,"output":6236,"reasoning":763},"cost_usd":0.1119,"cost_breakdown":{"input":0.031626,"cache_read":0.017894,"cache_write":0.0,"output":0.06236},"cost_usd_as_run":0.1119,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":293,"code":240,"comment":1,"blank":52,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":167,"code":140,"comment":0,"blank":27,"role":"test"}],"impl_lines":293,"impl_code":240,"impl_comment":1,"test_lines":167,"comment_ratio":0.004,"tests_declared":9},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":475.664,"reference":459.706,"speedup":0.966,"baselines":{"oracle":459.706}},{"id":"throughput_64m","candidate":2579.724,"reference":2612.257,"speedup":0.988,"baselines":{"oracle":2612.257}}]},"scores":{"gpt-6.1-sol-max":3.5,"opus-5.5-xhigh":6.5,"gpt-6.1-terra-max":5.0,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-sol-max":"Incorrect executable selection, runner leaks, and an undocumented API","opus-5.5-xhigh":"Works on the main paths but undocumented, leak-prone after completion, and lightly tested","gpt-6.1-terra-max":"A completed run can permanently occupy a slot","muse-wonder-full-xhigh":"behaviourally solid but undocumented and thinly tested","deepseek-v4.1-flash-max":"behaviourally correct with passing tests, but the module ships without any documentation."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Completed runners lose abandonment monitoring","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is absent","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Public API has no docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Finished runner leaks if its caller dies before removing it","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` expanded in slash paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Kill paths not verified","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Hard-coded `/bin/kill`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Symlink paths can execute the wrong tool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Completion drops the caller monitor too early","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Watchdog cleanup trusts PID existence alone","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation and specifications are missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No docs or specs on the public API","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Non-UTF-8 slash path rejected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unconditional `Port.info` match","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin, partly masked tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"the public module is undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"two probes per resolution","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the overflow test does not check the kill","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":203,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.7,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.1,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.76,"tokens":{"uncached_input":10160,"cache_read":322816,"cache_write":0,"output":11202,"reasoning":3161},"cost_usd":0.9845,"cost_breakdown":{"input":0.1016,"cache_read":0.322816,"cache_write":0.0,"output":0.5601},"cost_usd_as_run":0.9845,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":365,"code":297,"comment":5,"blank":63,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":255,"code":213,"comment":1,"blank":41,"role":"test"}],"impl_lines":365,"impl_code":297,"impl_comment":5,"test_lines":255,"comment_ratio":0.017,"tests_declared":14},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":483.052,"reference":465.341,"speedup":0.963,"baselines":{"oracle":465.341}},{"id":"throughput_64m","candidate":2553.124,"reference":2550.654,"speedup":1.001,"baselines":{"oracle":2550.654}}]},"scores":{"gpt-6.1-sol-max":5.5,"opus-5.5-xhigh":6.5,"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-sol-max":"Strong lifecycle handling, but path expansion can execute the wrong command.","opus-5.5-xhigh":"Behaves correctly in every probe and the tests are excellent, but the public API has no docs or specs and every call adds an unsupervised bridge plus a worker process","gpt-6.1-terra-max":"Strong lifecycle coverage, but relative paths can select the wrong executable","muse-wonder-full-xhigh":"Robust bridge design matching the oracle on almost all probes, held back by missing contract docs","deepseek-v4.1-flash-max":"behaviourally exact and neatly structured, but the fixed public interface carries no docs or specs at all, two internals leak into the public module, and it runs three processes per call."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Executable paths are rewritten","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is missing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"The public interface has no @doc, @spec or @typedoc","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` is expanded in paths that contain a slash","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"An extra bridge and worker layer on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Internal helpers are public","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Nested catch-alls","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Relative paths are expanded instead of used as given","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation is missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Slash paths get `~` expansion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No `@doc`/`@spec` on `child_spec/1`, `start_link/1`, `run/3`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Internal helpers are public contract-adjacent API","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"The public interface is undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"Three processes and mutual monitors per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"Public API leakage and missing behaviour annotations","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No concurrency-isolation test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":254,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.9,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.9,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":320.76,"tokens":{"uncached_input":360310,"cache_read":5430064,"cache_write":0,"output":73540,"reasoning":40084},"cost_usd":3.9981,"cost_breakdown":{"input":1.44124,"cache_read":1.086013,"cache_write":0.0,"output":1.4708},"cost_usd_as_run":3.9981,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":830,"code":682,"comment":12,"blank":136,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":524,"code":411,"comment":2,"blank":111,"role":"test"}],"impl_lines":830,"impl_code":682,"impl_comment":12,"test_lines":524,"comment_ratio":0.017,"tests_declared":22},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":721.753,"reference":462.232,"speedup":0.64,"baselines":{"oracle":462.232}},{"id":"throughput_64m","candidate":2379.382,"reference":2637.855,"speedup":0.902,"baselines":{"oracle":2637.855}}]},"scores":{"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":6.0,"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-sol-max":"Several contract mismatches and unnecessary work in an otherwise functioning runner.","opus-5.5-xhigh":"Correct on the main paths, but each call spends about 1 ms re-parsing `/proc`, several edge inputs differ from the reference, and there is a lot of defensive dead code","gpt-6.1-terra-max":"Comprehensive tests, with path, pool-name, and hot-path defects","muse-wonder-full-xhigh":"Extensively tested correct core inside roughly double the necessary code and several small edge divergences","deepseek-v4.1-flash-max":"correct with the strongest test suite, but it re-reads and parses `/proc/self/status` on every path resolution (measured 1.46×/2.4× slower than the oracle), ships zero documentation, and rejects a `:name` value the contract does not call invalid."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Relative paths can select another executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Runner crashes become timeouts after the deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Credentials are repeatedly read and parsed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Successful-start handling is duplicated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Each call re-reads and parses `/proc/self/status` about five times","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` is expanded in relative paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil/true/false` is rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Deadline checks before spawning hide `:unavailable`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Branches that can't occur","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Core mechanisms are unexplained","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"A runner crash after the deadline is reported as `:timeout`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`rescue` plus `catch :exit` plus `catch _, _` in several places (486, 572, 583, 237, 301, 377)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Path expansion changes the selected tool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Repeated identity parsing slows trivial runs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Helpers bypass `PATH` via absolute fallbacks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Slash paths get `~` expansion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Executability uses the caller's effective IDs, not any execute bit","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Pool names `nil`/`true`/`false` rejected as invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Port opened without `:stderr_to_stdout` (and without `:hide`)","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-run resolution re-walks `PATH` manually and reparses `/proc/self/status` per candidate","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Chunked call-retry plus 30 s done-grace machinery is needless","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"Avoidable `/proc` parsing on the hot path","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"830 lines with no comments at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"One error policy written four times","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`:name` validation is stricter than the contract","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":537,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":49,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.2,"slop":{"score":0.8,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":7.1,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1-b2","task":"bounded-tool","lead":"gpt-6.1-sol-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":8.63,"tokens":{"cache_read":2045952,"cache_write":0,"output":23202,"reasoning":14041,"uncached_input":92068},"cost_usd":0.3104,"cost_breakdown":{"cache_read":0.102298,"cache_write":0.0,"input":0.092068,"output":0.11601},"cost_usd_as_run":0.3104,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":9.52,"tokens":{"cache_read":1565024,"cache_write":0,"output":34039,"reasoning":24428,"uncached_input":121341},"cost_usd":1.4791,"cost_breakdown":{"cache_read":0.313005,"cache_write":0.0,"input":0.485364,"output":0.68078},"cost_usd_as_run":1.4791,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":10.7,"tokens":{"cache_read":1995479,"cache_write":142285,"output":55778,"reasoning":40782,"uncached_input":40},"cost_usd":2.6531,"cost_breakdown":{"cache_read":0.399096,"cache_write":1.13828,"input":0.00016,"output":1.11556},"cost_usd_as_run":2.6531,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":10.57,"tokens":{"cache_read":4674688,"cache_write":0,"output":78535,"reasoning":58278,"uncached_input":75189},"cost_usd":0.0724,"cost_breakdown":{"cache_read":0.014024,"cache_write":0.0,"input":0.011278,"output":0.047121},"cost_usd_as_run":0.0724,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":14.76,"tokens":{"cache_read":2073856,"cache_write":0,"output":26996,"reasoning":15117,"uncached_input":76574},"cost_usd":0.6305,"cost_breakdown":{"cache_read":0.207386,"cache_write":0.0,"input":0.153148,"output":0.26996},"cost_usd_as_run":0.6305,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"J","score":5.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"L","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"G","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"J","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"L","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"G","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"J","score":5.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"L","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"G","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"J","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"L","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"G","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"J","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"L","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"G","score":10.0}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","opus-5.5-xhigh","deepseek-v4.1-flash-max","gpt-6.1-sol-max"]},"entries":[{"letter":"Y","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.7,"tokens":{"uncached_input":76486,"cache_read":450081,"cache_write":0,"output":27667,"reasoning":0},"cost_usd":0.4747,"cost_breakdown":{"input":0.152972,"cache_read":0.045008,"cache_write":0.0,"output":0.27667},"cost_usd_as_run":0.4747,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":675,"code":558,"comment":12,"blank":105,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":528,"code":428,"comment":10,"blank":90,"role":"test"}],"impl_lines":675,"impl_code":558,"impl_comment":12,"test_lines":528,"comment_ratio":0.021,"tests_declared":19},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":443.509,"reference":422.908,"speedup":0.954,"baselines":{"oracle":422.908}},{"id":"throughput_64m","candidate":2606.512,"reference":2735.286,"speedup":0.953,"baselines":{"oracle":2735.286}}]},"scores":{"gpt-6.1-terra-max":4.0,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":5.0},"council_summaries":{"gpt-6.1-terra-max":"Duplicate options can bypass limits, and pool shutdown leaves tools running.","muse-wonder-full-xhigh":"largest entry, but the only one with verified wrong outputs on valid inputs plus a shutdown leak","opus-5.5-xhigh":"core contract right, but pool shutdown leaks tools, several divergences from the reference, and heavy bloat","deepseek-v4.1-flash-max":"correct and the most heavily tested, but verbose with duplicated state and two edge-case divergences from the oracle","gpt-6.1-sol-max":"Broad coverage, but duplicate options break deadlines and pool shutdown bypasses cleanup."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Duplicate options use the wrong value","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Pool shutdown bypasses group cleanup","status":"valid","defect":"BT-020","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"An expired validation path masks a missing pool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"`name: nil` has the wrong result","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extremely large deadlines differ from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The claimed crash test does not crash a runner","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Runner state is repeated three times","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"`name: nil` rejected as invalid","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate options last-wins","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Runner shutdown is :brutal_kill","status":"valid","defect":"BT-020","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests enshrine the nil bug and fake a crash case","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"One-line moduledoc for a 675-line design","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Over-engineered termination and unreadable relay","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable start result clause","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Runner never cleans up when the pool shuts down","status":"valid","defect":"BT-020","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Caller monitor dropped on finish","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil` rejected","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Deadline check runs before the pool lookup","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"Duplicate option keys: last one wins","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Kill checks can pass when nothing was checked","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Merge order not checked","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Bloat","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicate error handling","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable clause","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Relay and invariants unexplained","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` is rejected as `:invalid_argument`","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys take the last value","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"redundancy","title":"`init/1` repeats the same 16-field state literal three times","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"`run_kill_command/2` reimplements `System.cmd/2`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Internal stat helpers are public","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`terminate_runner/2` over-engineers claiming a slot","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Duplicate options use the last value","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Pool shutdown leaves tools running","status":"valid","defect":"BT-020","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"correctness","title":"Nil pool name returns the wrong error","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Runner initialization duplicates large state maps","status":"valid","detail":"","location":""}],"isolation":{"tool_calls":17,"flags":6,"audited":"2026-11-02","reviewed":true},"score":6.0,"slop":{"score":1.6,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":1.5,"opus-5.5-xhigh":2.0,"deepseek-v4.1-flash-max":3.0,"gpt-6.1-sol-max":1.0}},"defect_points":9.2,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","task":"bounded-tool","lead":"gpt-6.1-sol-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":38.24,"tokens":{"cache_read":23707264,"cache_write":0,"output":91875,"reasoning":45065,"uncached_input":470603},"cost_usd":2.1153,"cost_breakdown":{"cache_read":1.185363,"cache_write":0.0,"input":0.470603,"output":0.459375},"cost_usd_as_run":2.1153,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":55.82,"tokens":{"cache_read":30254976,"cache_write":0,"output":317580,"reasoning":243672,"uncached_input":416042},"cost_usd":0.3437,"cost_breakdown":{"cache_read":0.090765,"cache_write":0.0,"input":0.062406,"output":0.190548},"cost_usd_as_run":0.3437,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":82.37,"tokens":{"cache_read":18812160,"cache_write":0,"output":133770,"reasoning":84781,"uncached_input":602475},"cost_usd":4.4239,"cost_breakdown":{"cache_read":1.881216,"cache_write":0.0,"input":1.20495,"output":1.3377},"cost_usd_as_run":4.4239,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":723.98,"tokens":{"cache_read":29378499,"cache_write":1330076,"output":268770,"reasoning":188292,"uncached_input":286},"cost_usd":21.8928,"cost_breakdown":{"cache_read":5.8757,"cache_write":10.640608,"input":0.001144,"output":5.3754},"cost_usd_as_run":21.8928,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":46.1,"tokens":{"cache_read":15486904,"cache_write":0,"output":166012,"reasoning":101523,"uncached_input":777580},"cost_usd":9.528,"cost_breakdown":{"cache_read":3.097381,"cache_write":0.0,"input":3.11032,"output":3.32024},"cost_usd_as_run":9.528,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":3.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":4.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":5.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":5.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":5.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":3.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":5.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":3.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":4.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":3.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":5.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":2.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":4.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":6.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A","score":5.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B","score":6.5}],"panel":["gpt-6.1-terra-max","deepseek-v4.1-flash-max","gpt-6.1-sol-max","opus-5.5-xhigh","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":88.64,"tokens":{"uncached_input":455349,"cache_read":14036672,"cache_write":0,"output":176093,"reasoning":137035},"cost_usd":0.5774,"cost_breakdown":{"input":0.068302,"cache_read":0.4211,"cache_write":0.0,"output":0.088046},"cost_usd_as_run":0.5774,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":789,"code":577,"comment":90,"blank":122,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":424,"code":300,"comment":30,"blank":94,"role":"test"}],"impl_lines":789,"impl_code":577,"impl_comment":90,"test_lines":424,"comment_ratio":0.135,"tests_declared":18},"hidden":{"passed":14,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":424.21,"reference":426.688,"speedup":1.006,"baselines":{"oracle":426.688}},{"id":"throughput_64m","candidate":2687.111,"reference":2808.548,"speedup":0.957,"baselines":{"oracle":2808.548}}]},"scores":{"gpt-6.1-terra-max":3.0,"deepseek-v4.1-flash-max":5.0,"gpt-6.1-sol-max":3.0,"opus-5.5-xhigh":3.0,"muse-wonder-full-xhigh":5.5},"council_summaries":{"gpt-6.1-terra-max":"Pool bookkeeping can disable pools or violate concurrency","deepseek-v4.1-flash-max":"two real defects: a runner crash permanently bricks the pool, and improper `args` get the wrong error","gpt-6.1-sol-max":"Global admission accounting fails across crashes and independent pool lifecycles.","opus-5.5-xhigh":"Hand-rolled ETS admission leaks slots and depends on an arbitrary owner process; a /proc race fails successful runs","muse-wonder-full-xhigh":"hand-rolled ETS admission permanently bricks a pool after one untrappable runner kill"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Slot table dies with the first pool’s starter","status":"valid","defect":"BT-022","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"A killed runner permanently consumes a slot","status":"valid","defect":"BT-021","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Failed duplicate start changes a running pool’s limit","status":"valid","defect":"BT-023","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Improper argument lists return the wrong error","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later duplicate option values escape validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Timing assertions permit contract violations","status":"valid","defect":"BT-028","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"robustness","title":"concurrency slot leaks on an untrappable runner exit","status":"valid","defect":"BT-021","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"improper `args` reported as `:unavailable`","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"ETS slot table duplicates the supervisor's own limit","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"suite hides both problems","status":"valid","defect":"BT-028","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Killing a runner permanently consumes its slot","status":"valid","defect":"BT-021","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"One pool’s parent can disable another pool","status":"valid","defect":"BT-022","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Improper argument lists receive the wrong error","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate options are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Timer cancellation guards can never succeed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"robustness","title":"A killed runner leaks its pool slot permanently","status":"valid","defect":"BT-021","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"robustness","title":"The global slot table is owned by whichever process created it","status":"valid","defect":"BT-022","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Successful runs return `:unavailable` when /proc is read late","status":"valid","defect":"BT-003","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"An improper args list returns `:unavailable`","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Repeated option keys are checked only by their first value","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The tests loosen or avoid the contract","status":"valid","defect":"BT-028","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"robustness","title":"ETS slot counter leaks on runner :kill","status":"valid","defect":"BT-021","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate option values unchecked","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Improper args return :unavailable","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests mask the contract failures","status":"valid","defect":"BT-028","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Redundant kill_pid after group KILL","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Duration comment claims an \"s\" suffix","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0396,"lines":20,"significant":505,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":109,"flags":3,"audited":"2026-11-02","reviewed":true},"score":3.9,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":11.3,"score_cross_vendor":3.9,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":66.22,"tokens":{"uncached_input":371920,"cache_read":1612928,"cache_write":0,"output":275570,"reasoning":125408},"cost_usd":3.2037,"cost_breakdown":{"input":0.74384,"cache_read":0.806464,"cache_write":0.0,"output":1.65342},"cost_usd_as_run":3.2037,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":700,"code":543,"comment":26,"blank":131,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":625,"code":490,"comment":11,"blank":124,"role":"test"}],"impl_lines":700,"impl_code":543,"impl_comment":26,"test_lines":625,"comment_ratio":0.046,"tests_declared":24},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":430.883,"reference":447.577,"speedup":1.039,"baselines":{"oracle":447.577}},{"id":"throughput_64m","candidate":2604.205,"reference":2706.292,"speedup":0.962,"baselines":{"oracle":2706.292}}]},"scores":{"gpt-6.1-terra-max":5.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":4.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"A delayed runner can report success after the deadline","deepseek-v4.1-flash-max":"correct, best tests in the field; only unnecessary access-check machinery","gpt-6.1-sol-max":"Deadline misclassification, incomplete validation, and misleading cleanup tests.","opus-5.5-xhigh":"The runner can exit before it replies, so successful runs return `:unavailable` under load; no deadline check on exit","muse-wonder-full-xhigh":"duplicate options and hardcoded helper paths diverge; permission machinery is dead code"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Exit status is accepted without checking the deadline","status":"valid","defect":"BT-017","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Boolean atom pool names cannot be started","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later duplicate option values escape validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Descendant checks record the parent PID","status":"valid","defect":"BT-029","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"hand-rolled access(2) reimplementation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Delayed processing can turn a timeout into success","status":"valid","defect":"BT-017","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Shadowed invalid options still launch commands","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Descendant assertions record the parent PID","status":"valid","defect":"BT-029","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Effective-user permission branches are unreachable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"The runner stops on any EXIT before its result has been awaited","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"An exit status that arrives after the deadline is reported as a normal exit","status":"valid","defect":"BT-017","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Repeated option keys are checked only by their first value","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"`/usr/bin/kill` is hard-coded","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable clauses","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate option values unchecked","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Helpers prefer hardcoded paths over PATH","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Credential-aware permission check never engages","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Group kill hardcodes /usr/bin/kill","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No 64 MiB test; crash test is trappable","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":34,"flags":4,"audited":"2026-11-02","reviewed":true},"conduct":{"severity":"minor","summary":"While cleaning up it ran `pkill -f \"sleep 20\"`, a host-wide pattern that also matches other jobs' processes (for example any `sleep 200`). No other process is known to have been hit; scores stand (owner decision, 2026-11-02)."},"score":5.9,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":6.8,"score_cross_vendor":5.9,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":364.88,"tokens":{"uncached_input":194158,"cache_read":3836656,"cache_write":0,"output":55175,"reasoning":30402},"cost_usd":2.6475,"cost_breakdown":{"input":0.776632,"cache_read":0.767331,"cache_write":0.0,"output":1.1035},"cost_usd_as_run":2.6475,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":724,"code":576,"comment":34,"blank":114,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":489,"code":389,"comment":3,"blank":97,"role":"test"}],"impl_lines":724,"impl_code":576,"impl_comment":34,"test_lines":489,"comment_ratio":0.056,"tests_declared":17},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":439.673,"reference":429.188,"speedup":0.976,"baselines":{"oracle":429.188}},{"id":"throughput_64m","candidate":2777.34,"reference":2717.305,"speedup":1.022,"baselines":{"oracle":2717.305}}]},"scores":{"gpt-6.1-terra-max":5.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Can execute the wrong program and rejects valid pool names","deepseek-v4.1-flash-max":"correct, well documented, strong tests; one validation deviation from the oracle","gpt-6.1-sol-max":"Can execute the wrong command and incompletely validates options.","opus-5.5-xhigh":"Correct under load, but two validation divergences and heavy defensive padding","muse-wonder-full-xhigh":"three small oracle divergences in validation and PATH resolution"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Empty PATH entry can select a shell builtin","status":"valid","defect":"BT-019","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later duplicate option values escape validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Boolean atoms are rejected as pool names","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`:name` restricted to non-`nil`, non-boolean atoms","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Empty PATH components allow shell builtins to replace selected executables","status":"valid","defect":"BT-019","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Shadowed option values bypass validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid atom names are classified as invalid","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil/true/false` is rejected as invalid","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Repeated option keys are checked only by their first value","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"`rescue _`/`catch _, _` wrapped around code that cannot raise","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate option values unchecked","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"nil/boolean pool name is :invalid_argument","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Empty PATH entry returns a bare name","status":"valid","defect":"BT-019","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No 64 MiB budget test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":39,"flags":1,"audited":"2026-11-02","reviewed":true},"score":6.4,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.6,"score_cross_vendor":6.25,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":21.7,"tokens":{"uncached_input":82,"cache_read":8399227,"cache_write":291569,"output":256824,"reasoning":222035},"cost_usd":1.125,"cost_breakdown":{"input":3.7e-05,"cache_read":0.414525,"cache_write":0.266649,"output":0.44375},"cost_usd_as_run":1.125,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":532,"code":388,"comment":55,"blank":89,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":560,"code":439,"comment":9,"blank":112,"role":"test"}],"impl_lines":532,"impl_code":388,"impl_comment":55,"test_lines":560,"comment_ratio":0.124,"tests_declared":29},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":474.68,"reference":454.63,"speedup":0.958,"baselines":{"oracle":454.63}},{"id":"throughput_64m","candidate":2585.269,"reference":2630.64,"speedup":0.983,"baselines":{"oracle":2630.64}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct common paths, but valid duplicate options are rejected","deepseek-v4.1-flash-max":"correct, excellent docs and the best group-level test suite; rejects duplicate option keys","gpt-6.1-sol-max":"Strong lifecycle handling, but rejects valid duplicate options.","opus-5.5-xhigh":"Well documented and robust, but it rejects valid repeated options","muse-wonder-full-xhigh":"best tests and docs of the set; rejects duplicate options the contract allows"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Duplicate valid options are treated as invalid","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"repeated option keys rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid keyword duplicates are rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Valid repeated options are rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Double /proc read and an unused catch-all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate options rejected","status":"valid","defect":"BT-013","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":340,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":43,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":1.7,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":10.41,"tokens":{"uncached_input":16174,"cache_read":1392768,"cache_write":0,"output":38725,"reasoning":24075},"cost_usd":0.2794,"cost_breakdown":{"input":0.016174,"cache_read":0.069638,"cache_write":0.0,"output":0.193625},"cost_usd_as_run":0.2794,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":637,"code":511,"comment":23,"blank":103,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":370,"code":306,"comment":0,"blank":64,"role":"test"}],"impl_lines":637,"impl_code":511,"impl_comment":23,"test_lines":370,"comment_ratio":0.043,"tests_declared":15},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":609.074,"reference":458.127,"speedup":0.752,"baselines":{"oracle":458.127}},{"id":"throughput_64m","candidate":2557.428,"reference":2700.106,"speedup":0.947,"baselines":{"oracle":2700.106}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Strong failure handling, with extra work on every run","deepseek-v4.1-flash-max":"correct and well tested, but ~1.4× the oracle on the hot path and the most intricate design","gpt-6.1-sol-max":"Thorough failure handling with avoidable process creation on every call.","opus-5.5-xhigh":"Correct in every probe, but over-engineered and about 40% slower per run","muse-wonder-full-xhigh":"behaviorally exact on every probe but pays an extra shell per run and mis-specifies run/3"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Every invocation spawns an additional shell for access checks","status":"valid","defect":"BT-024","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Run option defaults are omitted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"per-call helper verification spawns a shell","status":"valid","defect":"BT-024","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Access checks fork an additional shell per run","status":"valid","defect":"BT-024","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Every run forks `/bin/sh` for an extra access check and spawns a launcher process","status":"valid","defect":"BT-024","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"style","title":"Hand-rolled admission protocol","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Interface docs altered and internals made public","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Extra /bin/sh access check on every run","status":"valid","defect":"BT-024","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"run/3 spec contradicts the fixed interface","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":434,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":28,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.2,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":7.34,"tokens":{"uncached_input":22,"cache_read":682921,"cache_write":74021,"output":44510,"reasoning":27289},"cost_usd":1.619,"cost_breakdown":{"input":8.8e-05,"cache_read":0.136584,"cache_write":0.592168,"output":0.8902},"cost_usd_as_run":1.619,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":461,"code":333,"comment":50,"blank":78,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":437,"code":339,"comment":11,"blank":87,"role":"test"}],"impl_lines":461,"impl_code":333,"impl_comment":50,"test_lines":437,"comment_ratio":0.131,"tests_declared":22},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":439.322,"reference":434.216,"speedup":0.988,"baselines":{"oracle":434.216}},{"id":"throughput_64m","candidate":2714.256,"reference":2668.989,"speedup":1.017,"baselines":{"oracle":2668.989}}]},"scores":{"gpt-6.1-terra-max":9.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":10.0,"opus-5.5-xhigh":9.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Broad coverage and no confirmed behavioral defect","deepseek-v4.1-flash-max":"correct, complete docs, strong suite; one needless shell per group kill","gpt-6.1-sol-max":"Clear OTP ownership, reference-compatible behavior, and broad tests.","opus-5.5-xhigh":"Matched the reference on every probe; clean design and the strongest test suite in the set","muse-wonder-full-xhigh":"strong throughout, but helper lookup bypasses PATH and diverges from the oracle"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"group kill via `/bin/sh -c 'kill -KILL -- \"-$1\"'`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Helpers ignore PATH","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":288,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":1,"audited":"2026-11-02","reviewed":true},"score":9.2,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.5,"score_cross_vendor":9.125,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":4.95,"tokens":{"uncached_input":20,"cache_read":532039,"cache_write":58267,"output":29884,"reasoning":15748},"cost_usd":1.1703,"cost_breakdown":{"input":8e-05,"cache_read":0.106408,"cache_write":0.466136,"output":0.59768},"cost_usd_as_run":1.1703,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":422,"code":322,"comment":26,"blank":74,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":375,"code":310,"comment":1,"blank":64,"role":"test"}],"impl_lines":422,"impl_code":322,"impl_comment":26,"test_lines":375,"comment_ratio":0.075,"tests_declared":25},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":454.741,"reference":435.398,"speedup":0.957,"baselines":{"oracle":435.398}},{"id":"throughput_64m","candidate":2785.41,"reference":2769.383,"speedup":1.006,"baselines":{"oracle":2769.383}}]},"scores":{"gpt-6.1-terra-max":9.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":10.0,"opus-5.5-xhigh":9.0,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Broad coverage and no confirmed behavioral defect","deepseek-v4.1-flash-max":"correct, complete docs, thorough suite; one helper needlessly public","gpt-6.1-sol-max":"Complete implementation with clear ownership and broad contract coverage.","opus-5.5-xhigh":"Clean, close to the reference, and matched it on every probe","muse-wonder-full-xhigh":"matches the oracle on every probe with complete docs and thorough tests"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"internal helper exposed as public","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`read_stat/1` is public for no reason","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":281,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":1,"audited":"2026-11-02","reviewed":true},"score":9.4,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.2,"score_cross_vendor":9.5,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.23,"tokens":{"uncached_input":9519,"cache_read":222080,"cache_write":0,"output":9019,"reasoning":2195},"cost_usd":0.7682,"cost_breakdown":{"input":0.09519,"cache_read":0.22208,"cache_write":0.0,"output":0.45095},"cost_usd_as_run":0.7682,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":341,"code":278,"comment":4,"blank":59,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":308,"code":256,"comment":0,"blank":52,"role":"test"}],"impl_lines":341,"impl_code":278,"impl_comment":4,"test_lines":308,"comment_ratio":0.014,"tests_declared":15},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":467.108,"reference":436.452,"speedup":0.934,"baselines":{"oracle":436.452}},{"id":"throughput_64m","candidate":2711.481,"reference":2775.389,"speedup":0.977,"baselines":{"oracle":2775.389}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Strong behavior and tests, with missing public docs","deepseek-v4.1-flash-max":"correct and solid tests, but the public interface has essentially no documentation","gpt-6.1-sol-max":"Compact implementation with sound behavior and incomplete public documentation.","opus-5.5-xhigh":"Small and correct; `:eof` keeps the os_pid stable; interface docs missing","muse-wonder-full-xhigh":"matches the oracle on every differential probe; only the fixed API docs are missing"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Pool functions and run options are undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no `@doc`/`@typedoc`; one `@spec`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is missing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Fixed interface docs missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Public API has no @doc","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":234,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":8,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.5,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.0,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":18.3,"tokens":{"uncached_input":65169,"cache_read":916608,"cache_write":0,"output":31454,"reasoning":19235},"cost_usd":0.5365,"cost_breakdown":{"input":0.130338,"cache_read":0.091661,"cache_write":0.0,"output":0.31454},"cost_usd_as_run":0.5365,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":445,"code":354,"comment":18,"blank":73,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":441,"code":367,"comment":0,"blank":74,"role":"test"}],"impl_lines":445,"impl_code":354,"impl_comment":18,"test_lines":441,"comment_ratio":0.048,"tests_declared":19},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":481.358,"reference":446.558,"speedup":0.928,"baselines":{"oracle":446.558}},{"id":"throughput_64m","candidate":2718.261,"reference":2805.23,"speedup":0.969,"baselines":{"oracle":2805.23}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong cleanup and coverage, with missing pool docs","deepseek-v4.1-flash-max":"correct with an excellent suite; docs slightly short of the fixed interface","gpt-6.1-sol-max":"Strong execution and crash handling; pool documentation is incomplete.","opus-5.5-xhigh":"Robust (the caller kills the group if the runner is killed) with excellent tests; interface docs missing","muse-wonder-full-xhigh":"near-oracle with caller-side crash cleanup; one exotic resolution divergence"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Pool configuration is undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"missing `@typedoc` and two `@doc`s","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Pool API options are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Fixed interface docs dropped","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Redundant work on every run","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Non-UTF8 tool path reported unavailable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Missing @doc/@typedoc","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":300,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":23,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.74,"tokens":{"uncached_input":7178,"cache_read":176512,"cache_write":0,"output":4952,"reasoning":320},"cost_usd":0.4959,"cost_breakdown":{"input":0.07178,"cache_read":0.176512,"cache_write":0.0,"output":0.2476},"cost_usd_as_run":0.4959,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":264,"code":221,"comment":1,"blank":42,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":152,"code":128,"comment":0,"blank":24,"role":"test"}],"impl_lines":264,"impl_code":221,"impl_comment":1,"test_lines":152,"comment_ratio":0.005,"tests_declared":5},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":468.777,"reference":435.615,"speedup":0.929,"baselines":{"oracle":435.615}},{"id":"throughput_64m","candidate":2723.805,"reference":2625.392,"speedup":1.037,"baselines":{"oracle":2625.392}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Sound core behavior, with documentation and test coverage gaps","deepseek-v4.1-flash-max":"behaviourally correct and compact, but the public interface is undocumented and the suite is thin","gpt-6.1-sol-max":"Path normalization changes the executed program; lifecycle tests leave important gaps.","opus-5.5-xhigh":"Compact and mostly right, but a startup race, no docs and thin tests","muse-wonder-full-xhigh":"compact and probe-exact, but almost untested and undocumented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack documentation","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Timeout tests do not verify group death","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"fixed interface not documented","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"5 tests only","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes executable identity","status":"valid","defect":"BT-006","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Deadline and caller-death tests do not verify OS cleanup","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public functions lack documentation and types","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"`{:os_pid, pid} = Port.info(port, :os_pid)` crashes if the port has already closed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No `@doc`, `@spec` or `@typedoc` on any public function","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin suite","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Suite too thin to verify the contract","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No docs or specs on the fixed interface","status":"valid","defect":"BT-034","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":186,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.6,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.6,"score_cross_vendor":6.833,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.33,"tokens":{"uncached_input":13167,"cache_read":413696,"cache_write":0,"output":8994,"reasoning":2212},"cost_usd":0.0788,"cost_breakdown":{"input":0.013167,"cache_read":0.020685,"cache_write":0.0,"output":0.04497},"cost_usd_as_run":0.0788,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":291,"code":241,"comment":5,"blank":45,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":227,"code":189,"comment":0,"blank":38,"role":"test"}],"impl_lines":291,"impl_code":241,"impl_comment":5,"test_lines":227,"comment_ratio":0.02,"tests_declared":12},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":462.91,"reference":435.258,"speedup":0.94,"baselines":{"oracle":435.258}},{"id":"throughput_64m","candidate":2717.238,"reference":2737.244,"speedup":0.993,"baselines":{"oracle":2737.244}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Ordinary execution works, with validation and overflow-ordering defects","deepseek-v4.1-flash-max":"Correct, but the interface docs are trimmed, the state handling has a lie, and the tests stop short of the harder cases.","gpt-6.1-sol-max":"Good lifecycle handling, but validation and timeout precedence need changes.","opus-5.5-xhigh":"Compact and mostly right; the `Port.info` race; accepts repeated options; interface spec weakened","muse-wonder-full-xhigh":"compact and mostly correct; duplicate options slip through"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Invalid duplicate options are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"An earlier overflow can become a timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public documentation omits options and defaults","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`finish(state, result, false)` returns `%{state | port: nil}` without closing the port","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"The interface's `@typedoc` and the documented option list are dropped","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`true <- System.monotonic_time(:millisecond) < deadline || {:error, :timeout}`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"The early-timeout state is a partial map with three keys","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate values are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Expiration hides pool errors","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Queued overflow becomes timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Options and defaults are missing from public docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"`Port.info` race inside `init` rescue gives `{:stop, :unavailable}`","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Only the first value of a repeated option is validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`@spec run(term(), term(), term())` replaces the fixed interface spec; `run/3`'s `@doc` drops the option list; `child_spec/1` and `start_link/1` have no `@spec`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate invalid option ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Public interface under-documented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":204,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.7,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.4,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.59,"tokens":{"uncached_input":22372,"cache_read":191104,"cache_write":0,"output":6656,"reasoning":1003},"cost_usd":0.1304,"cost_breakdown":{"input":0.044744,"cache_read":0.01911,"cache_write":0.0,"output":0.06656},"cost_usd_as_run":0.1304,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":303,"code":252,"comment":1,"blank":50,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":223,"code":184,"comment":0,"blank":39,"role":"test"}],"impl_lines":303,"impl_code":252,"impl_comment":1,"test_lines":223,"comment_ratio":0.004,"tests_declared":13},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":469.274,"reference":434.609,"speedup":0.926,"baselines":{"oracle":434.609}},{"id":"throughput_64m","candidate":2743.226,"reference":2791.609,"speedup":0.983,"baselines":{"oracle":2791.609}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Broad tests and sound ordinary behavior, with a timing classification defect","deepseek-v4.1-flash-max":"Correct, compact implementation with no documentation whatsoever on a 300-line module.","gpt-6.1-sol-max":"Compact and mostly correct, with a timing discrepancy and missing API docs.","opus-5.5-xhigh":"Sound core; the same `Port.info` race (rescued to `:unavailable`); no public docs","muse-wonder-full-xhigh":"oracle-equivalent probes and dense tests; only documentation missing"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"An earlier overflow can become a timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation is missing","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Zero `@doc`/`@spec`/`@typedoc`","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"`run/3` always re-checks `now() >= deadline` inside the data handler *and* the exit-status handler (lib/bounded_tool.ex:192-215) while never arming a timer before `handle_continue`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Queued overflow becomes timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public options and invariants are undocumented","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"`Port.info` race turns success into `:unavailable`","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc`/`@spec` on the public API","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No docs/specs at all","status":"valid","defect":"BT-034","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":209,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":8,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.5,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.6,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.33,"tokens":{"uncached_input":9977,"cache_read":243072,"cache_write":0,"output":10260,"reasoning":3340},"cost_usd":0.1469,"cost_breakdown":{"input":0.019954,"cache_read":0.024307,"cache_write":0.0,"output":0.1026},"cost_usd_as_run":0.1469,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":341,"code":277,"comment":2,"blank":62,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":244,"code":206,"comment":1,"blank":37,"role":"test"}],"impl_lines":341,"impl_code":277,"impl_comment":2,"test_lines":244,"comment_ratio":0.007,"tests_declared":14},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":468.816,"reference":435.188,"speedup":0.928,"baselines":{"oracle":435.188}},{"id":"throughput_64m","candidate":2717.037,"reference":2670.264,"speedup":1.018,"baselines":{"oracle":2670.264}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Good functional coverage, with a timing defect and missing public docs","deepseek-v4.1-flash-max":"Correct runtime behaviour, but the public interface is undocumented and the caller-side \"ticket\" kill is redundant work on every call.","gpt-6.1-sol-max":"Good cleanup design and coverage, with one timing discrepancy.","opus-5.5-xhigh":"Correct results, but a caller-side kill leaks `:EXIT` messages to trapping callers; no public docs","muse-wonder-full-xhigh":"correct and well tested, but the public API is undocumented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"An earlier overflow can become a timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation is missing","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Only one `@spec`; no `@doc` or `@typedoc`","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"The caller kills the process group after *every* run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"The `:ticket` round-trip before `:await` plus the nested `try`/`catch` with two different fallbacks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Delayed overflow loses its original outcome","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public configuration is undocumented","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"`kill_group(ticket)` runs `System.cmd` in the caller after every run","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc` on `child_spec/1`, `start_link/1` or `run/3`; no `@spec` on the first two","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"An extra `:ticket` GenServer round trip and a caller-side `/proc` read on every run","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No docs or typedoc on the public interface","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Ticket round-trip and duplicate caller-side kill","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":234,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":9,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.4,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":2.9,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.78,"tokens":{"uncached_input":6404,"cache_read":122624,"cache_write":0,"output":3375,"reasoning":113},"cost_usd":0.0294,"cost_breakdown":{"input":0.006404,"cache_read":0.006131,"cache_write":0.0,"output":0.016875},"cost_usd_as_run":0.0294,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":261,"code":216,"comment":1,"blank":44,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":29,"code":22,"comment":0,"blank":7,"role":"test"}],"impl_lines":261,"impl_code":216,"impl_comment":1,"test_lines":29,"comment_ratio":0.005,"tests_declared":2},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":455.404,"reference":436.803,"speedup":0.959,"baselines":{"oracle":436.803}},{"id":"throughput_64m","candidate":2642.986,"reference":2773.209,"speedup":0.953,"baselines":{"oracle":2773.209}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":6.0,"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":4.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Core execution works, but queued overflow can be misclassified","deepseek-v4.1-flash-max":"Small, correct runner, but the deliverable test file is a 2-test smoke suite, there are no docs, and every internal error is reported as `:invalid_argument`.","gpt-6.1-sol-max":"Path rewriting can execute the wrong program; safety coverage is thin.","opus-5.5-xhigh":"Hard `Port.info` match gives spurious `:unavailable`; tilde expansion; two tests; no docs","muse-wonder-full-xhigh":"sound core, near-empty test suite and mislabelled rescue"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline check masks an earlier overflow","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Safety and pool behavior have little coverage","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions and the reason type lack documentation","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Two tests, 29 lines","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"No `@doc`/`@spec`/`@typedoc`","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`rescue _ -> {:error, :invalid_argument}` misclassifies every internal error as bad input","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`@relay` is one line with `\\n` escapes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`resolve/1` stats each candidate three times","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Literal paths are changed","status":"valid","defect":"BT-006","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Queued overflow becomes timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Critical lifecycle cases are absent","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public contract is undocumented","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Repeated metadata reads","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"Hard match on `Port.info`","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Tilde expansion of slash paths","status":"valid","defect":"BT-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Tests barely cover the contract","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc`/`@spec` on `child_spec/1`, `start_link/1` or `run/3`, and `Runner` lacks `@moduledoc false`","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"`rescue _ -> {:error, :invalid_argument}` labels any unexpected raise as bad input (lib/bounded_tool.ex:62), and the entry timestamp is named `deadline` (lib/bounded_tool.ex:22).","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Only two tests for the whole contract","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"All raises mislabelled as invalid input","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No docs/specs at all","status":"valid","defect":"BT-034","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":178,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.6,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":5.7,"score_cross_vendor":5.667,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":40.18,"tokens":{"uncached_input":522,"cache_read":10196774,"cache_write":161938,"output":141699,"reasoning":0},"cost_usd":0.2622,"cost_breakdown":{"input":7.8e-05,"cache_read":0.163148,"cache_write":0.032388,"output":0.066599},"cost_usd_as_run":0.2622,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.19,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":758,"code":530,"comment":110,"blank":118,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":450,"code":333,"comment":20,"blank":97,"role":"test"}],"impl_lines":758,"impl_code":530,"impl_comment":110,"test_lines":450,"comment_ratio":0.172,"tests_declared":30},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":452.499,"reference":425.849,"speedup":0.941,"baselines":{"oracle":425.849}},{"id":"throughput_64m","candidate":2801.274,"reference":2773.782,"speedup":1.01,"baselines":{"oracle":2773.782}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"The main paths work, but validation changes the deadline and accepts invalid duplicates","deepseek-v4.1-flash-max":"Exceptionally well thought-out runner (force-reply fallback, per-kill ports) and a 30-test suite; minor polish only.","gpt-6.1-sol-max":"Broad coverage, but deadline accounting and validation are incorrect.","opus-5.5-xhigh":"Correct under load but an over-built async state machine with dead paths; duplicate options accepted","muse-wonder-full-xhigh":"comprehensive tests, but validation gap and reply-before-kill cleanup design"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Validation time is excluded from the deadline","status":"valid","defect":"BT-018","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Invalid duplicate values are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Force-reply timer cancellation is a no-op","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Pool removal is attributed to the runner","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`format_secs/1` returns `\"30.250\"` with no unit suffix","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`use DynamicSupervisor` plus a hand-written `child_spec/1` and `init/1`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"The deadline starts after validation","status":"valid","defect":"BT-018","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Later invalid duplicate values are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Timer cancellation targets a message tag","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Only the first value of a repeated option is validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"The `:force_reply` timer never changes the outcome","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"The stalled-runner path kills the runner but keeps `mref`, so a `:DOWN` reaches the caller's mailbox","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The comment calls 2_000_000_000 the \"Largest value accepted by both send_after and receive after\", and the constant is duplicated in two modules","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"The kill port is opened without `:stderr_to_stdout`, so \"kill: sending signal to -PID failed: No such process\" appeared on the VM's stderr during the probes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate invalid option ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Result replied before the group kill lands","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Stalled-runner path leaks the caller DOWN","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Deadline documentation does not match code","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Force-reply timer is never cancelled","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":86,"flags":5,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":1.1,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.5}},"defect_points":4.0,"score_cross_vendor":7.4,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":389.5,"tokens":{"uncached_input":375668,"cache_read":5282244,"cache_write":0,"output":63590,"reasoning":33241},"cost_usd":3.8309,"cost_breakdown":{"input":1.502672,"cache_read":1.056449,"cache_write":0.0,"output":1.2718},"cost_usd_as_run":3.8309,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":800,"code":622,"comment":53,"blank":125,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":571,"code":437,"comment":15,"blank":119,"role":"test"}],"impl_lines":800,"impl_code":622,"impl_comment":53,"test_lines":571,"comment_ratio":0.079,"tests_declared":29},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":526.996,"reference":428.625,"speedup":0.813,"baselines":{"oracle":428.625}},{"id":"throughput_64m","candidate":2713.594,"reference":2784.139,"speedup":0.975,"baselines":{"oracle":2784.139}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Comprehensive coverage, but valid atom pool names are rejected by `run/3`","deepseek-v4.1-flash-max":"Extensive, excellent tests and correct behaviour; the code is wrapped in defensive layers that hide the real logic.","gpt-6.1-sol-max":"Mostly correct behavior, with unnecessary name restrictions and exception boilerplate.","opus-5.5-xhigh":"Robust under load; rejects `name: nil`; heavy defensive padding","muse-wonder-full-xhigh":"thorough but over-engineered, with two small oracle divergences"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Boolean atom names cannot be used","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`remove_runner/2` monitors the runner and waits up to 1.5 s after a *synchronous* `DynamicSupervisor.terminate_child/2`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"Every private helper carries its own `rescue`+`catch` returning the same fallback","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Allowed names fail validation","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"verbosity","title":"Failure conversion is repeated across layers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil/true/false` return `:invalid_argument`","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"`rescue`/`catch` wrapped around nearly every function, `normalize_result` re-validating the runner's own reply, catch-all `handle_call`/`handle_cast`, and `remove_runner` adding a monitor plus a 1.5 s `:DOWN` wait after an already synchronous `terminate_child`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"Re-check the deadline… The deadline timer armed by the caller will fire\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Boolean pool names rejected as invalid","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Permission-class check diverges from the oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Pervasive defensive duplication","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0242,"lines":12,"significant":496,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":54,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.8,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.0}},"defect_points":2.3,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":15.07,"tokens":{"uncached_input":32,"cache_read":1422502,"cache_write":125654,"output":92972,"reasoning":70527},"cost_usd":3.1493,"cost_breakdown":{"input":0.000128,"cache_read":0.2845,"cache_write":1.005232,"output":1.85944},"cost_usd_as_run":3.1493,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":552,"code":417,"comment":50,"blank":85,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":645,"code":509,"comment":13,"blank":123,"role":"test"}],"impl_lines":552,"impl_code":417,"impl_comment":50,"test_lines":645,"comment_ratio":0.107,"tests_declared":26},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":509.242,"reference":446.722,"speedup":0.877,"baselines":{"oracle":446.722}},{"id":"throughput_64m","candidate":2746.226,"reference":2789.923,"speedup":0.984,"baselines":{"oracle":2789.923}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":9.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong implementation and tests, with a narrow startup mismatch","deepseek-v4.1-flash-max":"Very complete implementation and test suite; only cosmetic weight.","gpt-6.1-sol-max":"Thorough implementation and tests, with one startup-domain discrepancy.","opus-5.5-xhigh":"The most robust entry: correct on every probe, guards against non-pool names, thorough tests","muse-wonder-full-xhigh":"excellent tests and docs; only the oracle permission quirk differs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"`nil` atom name raises during startup","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"~100 lines of uid/gid/group execute-permission logic","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`start_link/1` raises `ArgumentError` on bad options","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"An accepted atom option raises during startup","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Permission-class check diverges from the oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Pool detection via OTP internals","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":363,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":15,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.89,"tokens":{"uncached_input":5735,"cache_read":122880,"cache_write":0,"output":4042,"reasoning":227},"cost_usd":0.0321,"cost_breakdown":{"input":0.005735,"cache_read":0.006144,"cache_write":0.0,"output":0.02021},"cost_usd_as_run":0.0321,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":253,"code":217,"comment":1,"blank":35,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":70,"code":58,"comment":0,"blank":12,"role":"test"}],"impl_lines":253,"impl_code":217,"impl_comment":1,"test_lines":70,"comment_ratio":0.005,"tests_declared":6},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":445.31,"reference":434.077,"speedup":0.975,"baselines":{"oracle":434.077}},{"id":"throughput_64m","candidate":2743.457,"reference":2783.671,"speedup":0.986,"baselines":{"oracle":2783.671}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":5.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Basic behavior works, but overflow ordering and verification need work","deepseek-v4.1-flash-max":"Behaviour matches the oracle, but the deliverable test file is a six-test smoke suite and the module has no docs at all.","gpt-6.1-sol-max":"Path normalization changes execution targets, and several edge cases are missed.","opus-5.5-xhigh":"The `Port.info` race, tilde expansion, thin sleep-based tests, no docs","muse-wonder-full-xhigh":"correct compact core wrapped in thin, partly masked tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"An earlier overflow can become a timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cleanup assertions do not verify tool death","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation is missing","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"No `@doc`, `@spec` or `@typedoc` anywhere","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Only 6 tests / 70 lines","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"`cleanup/1` runs twice for every run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`resolve/1` stats the same path twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Resolution rewrites literal paths","status":"valid","defect":"BT-006","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Earlier overflow becomes timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Expiration masks pool errors","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Important safety checks are missing","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is absent","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"`MatchError` on `Port.info`","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Tilde expansion","status":"valid","defect":"BT-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Six tests synchronised with `Process.sleep`. None checks that any process actually dies on timeout, overflow or caller death; the caller-death test only checks a later run","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc`/`@spec` on the public API; `Runner` has no `@moduledoc false`","status":"valid","defect":"BT-034","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`cleanup/1` runs in `finish/2` and again in `terminate/2`, re-reading `/proc` and possibly re-signalling","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Kill behaviour is asserted by proxy, not by OS-pid death","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No docs/specs at all","status":"valid","defect":"BT-034","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":180,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.9,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":5.6,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":22.16,"tokens":{"uncached_input":22023,"cache_read":582016,"cache_write":0,"output":30579,"reasoning":17601},"cost_usd":0.408,"cost_breakdown":{"input":0.044046,"cache_read":0.058202,"cache_write":0.0,"output":0.30579},"cost_usd_as_run":0.408,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":456,"code":353,"comment":23,"blank":80,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":537,"code":448,"comment":1,"blank":88,"role":"test"}],"impl_lines":456,"impl_code":353,"impl_comment":23,"test_lines":537,"comment_ratio":0.061,"tests_declared":25},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":470.077,"reference":434.755,"speedup":0.925,"baselines":{"oracle":434.755}},{"id":"throughput_64m","candidate":2736.938,"reference":2820.293,"speedup":0.97,"baselines":{"oracle":2820.293}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Admission is bounded, but completed results and overflow can be misclassified","deepseek-v4.1-flash-max":"Correct and unusually well-reasoned (deadline-bounded admission), thorough tests; minor rough edges only.","gpt-6.1-sol-max":"Strong implementation with two error-precedence discrepancies.","opus-5.5-xhigh":"Admission bounded by the deadline and crash cleanup done by the caller; relies on DynamicSupervisor internals; stray `:EXIT` for trapping callers","muse-wonder-full-xhigh":"behaviourally exact, built on a brittle supervisor-protocol shortcut"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Completed result is discarded when the caller is delayed","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"An earlier overflow can become a timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests depend on an unguaranteed executable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"`{:os_pid, pid} = Port.info(port, :os_pid)`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`Cleanup.run/1` is invoked from the caller's `catch` (lib/bounded_tool.ex:187) with a ticket the runner has usually already used; it re-reads /proc and normally no-ops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Expiration masks resolution and admission errors","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Earlier overflow loses to delayed processing","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Caller-side `Cleanup.run` runs `System.cmd` in the caller","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Sends DynamicSupervisor's private `{:start_child, child_tuple}` request directly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Most kill, overflow and VM tests need `python3`, which SPEC's environment doesn't guarantee","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Admission bypasses the public supervisor API","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":303,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":14,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.8,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":14.44,"tokens":{"uncached_input":16230,"cache_read":507008,"cache_write":0,"output":22011,"reasoning":7862},"cost_usd":1.7699,"cost_breakdown":{"input":0.1623,"cache_read":0.507008,"cache_write":0.0,"output":1.10055},"cost_usd_as_run":1.7699,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":388,"code":307,"comment":18,"blank":63,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":423,"code":353,"comment":1,"blank":69,"role":"test"}],"impl_lines":388,"impl_code":307,"impl_comment":18,"test_lines":423,"comment_ratio":0.055,"tests_declared":22},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":469.639,"reference":444.156,"speedup":0.946,"baselines":{"oracle":444.156}},{"id":"throughput_64m","candidate":2774.354,"reference":2776.564,"speedup":0.999,"baselines":{"oracle":2776.564}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Useful coverage, but overflow ordering is wrong and the suite is flaky","deepseek-v4.1-flash-max":"Good implementation and a strong test suite undermined by a flaky test helper (reproduced 1 run in 6).","gpt-6.1-sol-max":"Strong safety handling, with two timeout-precedence differences.","opus-5.5-xhigh":"Matches the reference on every probe; thorough tests; only sparse comments on the runner's invariants","muse-wonder-full-xhigh":"cleanest entry: oracle-equivalent on every probe"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"An earlier overflow can become a timeout","status":"valid","defect":"BT-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Process-death polling can crash during normal reaping","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"`live?/1` matches only `{:error, :enoent}`","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`receive do after 5 -> kill_group(saved, kill, until) end` is used as a sleep","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Expired calls hide a full pool’s `:busy` result","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Delayed overflow is reclassified as timeout","status":"valid","defect":"BT-016","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":265,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.1,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":365.41,"tokens":{"uncached_input":555582,"cache_read":5441444,"cache_write":0,"output":81037,"reasoning":46893},"cost_usd":4.9314,"cost_breakdown":{"input":2.222328,"cache_read":1.088289,"cache_write":0.0,"output":1.62074},"cost_usd_as_run":4.9314,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":926,"code":765,"comment":16,"blank":145,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":516,"code":424,"comment":0,"blank":92,"role":"test"}],"impl_lines":926,"impl_code":765,"impl_comment":16,"test_lines":516,"comment_ratio":0.02,"tests_declared":26},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":456.012,"reference":427.416,"speedup":0.937,"baselines":{"oracle":427.416}},{"id":"throughput_64m","candidate":2773.151,"reference":2758.512,"speedup":1.005,"baselines":{"oracle":2758.512}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"The runner works on the main paths but rejects valid options and pool names.","deepseek-v4.1-flash-max":"correct on mainstream paths; two small validation divergences and under-documented subtleties","gpt-6.1-sol-max":"Reliable execution, with unnecessary restrictions on valid API inputs.","opus-5.5-xhigh":"robust, but over-engineered and stricter validation than the contract","muse-wonder-full-xhigh":"strong execution core but validation is stricter than the oracle and the state machine is over-built"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid duplicate options are rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid atom names are rejected","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The large-output test stops at one million bytes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"lib/bounded_tool.ex:325","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"lib/bounded_tool.ex:358-360","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"lib/bounded_tool.ex:422,17,432","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"lib/bounded_tool.ex:168-250","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid duplicate options are rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid atom pool names are excluded","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Duplicate option keys are rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil/true/false` gives `:invalid_argument` instead of `:unavailable`","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Defensive padding everywhere","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"The state machine is undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate valid options are rejected as `:invalid_argument`","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil`/`true` rejected as `:invalid_argument`","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`start_link/1` with an unknown pool option returns `{:error, :invalid_argument}`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Validation tests encode the oracle-divergent behaviour","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Two-event terminal state machine far exceeds the problem","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":607,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":55,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.0,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":4.4,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":28.2,"tokens":{"uncached_input":68771,"cache_read":18682624,"cache_write":0,"output":255792,"reasoning":217909},"cost_usd":0.2198,"cost_breakdown":{"input":0.010316,"cache_read":0.056048,"cache_write":0.0,"output":0.153475},"cost_usd_as_run":0.2198,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2198,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":798,"code":635,"comment":26,"blank":137,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":587,"code":443,"comment":12,"blank":132,"role":"test"}],"impl_lines":798,"impl_code":635,"impl_comment":26,"test_lines":587,"comment_ratio":0.039,"tests_declared":23},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":452.567,"reference":439.332,"speedup":0.971,"baselines":{"oracle":439.332}},{"id":"throughput_64m","candidate":2757.887,"reference":2747.02,"speedup":1.004,"baselines":{"oracle":2747.02}}]},"scores":{"gpt-6.1-terra-max":6.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Path normalization can execute the wrong tool.","deepseek-v4.1-flash-max":"solid and correct with an uncommon amount of machinery; one avoidable per-call round trip","gpt-6.1-sol-max":"Strong lifecycle handling, but path normalization can execute the wrong tool.","opus-5.5-xhigh":"correct, with a narrow `Port.info` window and an extra sys call per run","muse-wonder-full-xhigh":"behaviour matches the oracle almost everywhere; only pool-verification and PATH quirks"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Collapsing `..` changes symlink path resolution","status":"valid","defect":"BT-010","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"lib/bounded_tool.ex:158,174","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"lib/bounded_tool.ex:446","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"lib/bounded_tool.ex:162","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"lib/bounded_tool.ex:294-330","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Slash paths change meaning","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Tool exiting before `Port.info` gives `:unavailable`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Synchronous `:sys.get_state` on the pool every run","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Over-built fallback cleanup","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Unset `PATH` falls back to hardcoded bin directories","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Every `run/3` synchronously interrogates the pool supervisor","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`start_link/1` raises on unknown pool options","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":523,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":111,"flags":2,"audited":"2026-11-02","reviewed":true},"score":7.0,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":2.9,"score_cross_vendor":6.875,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1-b2","task":"bounded-tool","lead":"gpt-6.1-terra-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":13.91,"tokens":{"cache_read":3995345,"cache_write":172929,"output":65991,"reasoning":45818,"uncached_input":66},"cost_usd":3.5026,"cost_breakdown":{"cache_read":0.799069,"cache_write":1.383432,"input":0.000264,"output":1.31982},"cost_usd_as_run":3.5026,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":11.33,"tokens":{"cache_read":3334784,"cache_write":0,"output":28513,"reasoning":17858,"uncached_input":71437},"cost_usd":0.3807,"cost_breakdown":{"cache_read":0.166739,"cache_write":0.0,"input":0.071437,"output":0.142565},"cost_usd_as_run":0.3807,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":13.28,"tokens":{"cache_read":1945728,"cache_write":0,"output":28902,"reasoning":18078,"uncached_input":87101},"cost_usd":0.6578,"cost_breakdown":{"cache_read":0.194573,"cache_write":0.0,"input":0.174202,"output":0.28902},"cost_usd_as_run":0.6578,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":15.08,"tokens":{"cache_read":3100200,"cache_write":0,"output":35360,"reasoning":20947,"uncached_input":174808},"cost_usd":2.0265,"cost_breakdown":{"cache_read":0.62004,"cache_write":0.0,"input":0.699232,"output":0.7072},"cost_usd_as_run":2.0265,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":15.13,"tokens":{"cache_read":10850944,"cache_write":0,"output":130903,"reasoning":100826,"uncached_input":100202},"cost_usd":0.1261,"cost_breakdown":{"cache_read":0.032553,"cache_write":0.0,"input":0.01503,"output":0.078542},"cost_usd_as_run":0.1261,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"C","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"D","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":4.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"C","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"D","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":5.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"C","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"D","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":5.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"C","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"D","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"C","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"D","score":9.0}],"panel":["opus-5.5-xhigh","gpt-6.1-terra-max","gpt-6.1-sol-max","muse-wonder-full-xhigh","deepseek-v4.1-flash-max"]},"entries":[{"letter":"M","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":15.33,"tokens":{"uncached_input":153317,"cache_read":2183625,"cache_write":0,"output":79954,"reasoning":50790},"cost_usd":1.3245,"cost_breakdown":{"input":0.306634,"cache_read":0.218362,"cache_write":0.0,"output":0.79954},"cost_usd_as_run":1.3245,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":785,"code":649,"comment":12,"blank":124,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":567,"code":417,"comment":26,"blank":124,"role":"test"}],"impl_lines":785,"impl_code":649,"impl_comment":12,"test_lines":567,"comment_ratio":0.018,"tests_declared":12},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":498.939,"reference":424.215,"speedup":0.85,"baselines":{"oracle":424.215}},{"id":"throughput_64m","candidate":2703.945,"reference":2777.518,"speedup":0.974,"baselines":{"oracle":2777.518}}]},"isolation":{"tool_calls":38,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"opus-5.5-xhigh":5.5,"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":5.5,"muse-wonder-full-xhigh":6.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"opus-5.5-xhigh":"passes the core contract, but its hand-written permission check is wrong, it can leak a pool slot, and it carries a lot of dead and verbose code","gpt-6.1-terra-max":"Broad coverage, with several executable-resolution and naming defects","gpt-6.1-sol-max":"Strong execution coverage, but a runner leak and several contract violations need fixes.","muse-wonder-full-xhigh":"sound core, but PATH search and atom-name handling diverge from the reference","deepseek-v4.1-flash-max":"strongest tests and defensive structure; one PATH edge, dead code and ~25 duplicated lines."},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"The permission check ignores supplementary groups","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"The caller monitor is dropped when a run finishes before `:await` arrives","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Pool names `nil`, `true` and `false` are rejected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Empty PATH entries are ignored","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Two clauses can never fire","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Extra work on every run","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"785 lines with catch-all `rescue`/`catch` blocks in many places","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The relay script and the identity/backstop design have no explanation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The BEAM-SIGKILL test failed once in 9 runs at its 2 s wait","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Boolean pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Empty PATH components are discarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Supplementary-group execute permission is ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"The watchdog can kill before a valid long deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Port handling is duplicated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Completed runners can permanently consume capacity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Empty PATH components are discarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Backstop can kill before the requested deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Port-exit handling duplicates result processing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Empty PATH components dropped","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"`nil`/boolean pool names rejected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Hand-rolled uid/gid executability check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Avoidable per-run hot-path work","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Duplicated port-exit drain","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable EXIT-from-pid clause","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"empty PATH components are dropped","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"`handle_port_exit_signal/1` re-implements the data/exit-status/cap logic inside a `receive` loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`handle_info({:EXIT, pid, _}, state) when is_pid(pid)` cannot fire","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"`name: nil`, `true` and `false` are rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"the execute check approximates `access(X_OK)` from the file's uid/gid only","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the relay heredoc and the group-leader wait are uncommented","status":"valid","detail":"","location":""}],"score":6.5,"slop":{"score":1.3,"per_seat":{"opus-5.5-xhigh":2.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.5,"deepseek-v4.1-flash-max":1.5}},"defect_points":7.7,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","task":"bounded-tool","lead":"gpt-6.1-terra-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":35.42,"tokens":{"cache_read":15580160,"cache_write":0,"output":87504,"reasoning":45866,"uncached_input":360219},"cost_usd":1.5768,"cost_breakdown":{"cache_read":0.779008,"cache_write":0.0,"input":0.360219,"output":0.43752},"cost_usd_as_run":1.5768,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":73.2,"tokens":{"cache_read":13105340,"cache_write":0,"output":156290,"reasoning":96458,"uncached_input":682201},"cost_usd":8.4756,"cost_breakdown":{"cache_read":2.621067,"cache_write":0.0,"input":2.728804,"output":3.1258},"cost_usd_as_run":8.4756,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":64.72,"tokens":{"cache_read":19540510,"cache_write":755798,"output":240378,"reasoning":168482,"uncached_input":234},"cost_usd":14.7629,"cost_breakdown":{"cache_read":3.908102,"cache_write":6.046384,"input":0.000936,"output":4.80756},"cost_usd_as_run":14.7629,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":55.83,"tokens":{"cache_read":10083328,"cache_write":0,"output":121153,"reasoning":78976,"uncached_input":435315},"cost_usd":3.0905,"cost_breakdown":{"cache_read":1.008333,"cache_write":0.0,"input":0.87063,"output":1.21153},"cost_usd_as_run":3.0905,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":67.24,"tokens":{"cache_read":41356160,"cache_write":0,"output":399986,"reasoning":314993,"uncached_input":444070},"cost_usd":0.4306,"cost_breakdown":{"cache_read":0.124069,"cache_write":0.0,"input":0.066611,"output":0.239991},"cost_usd_as_run":0.4306,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":4.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":5.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":3.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":4.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"A","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-gpt-6.1-terra-max-r1","letter":"B","score":7.0}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","opus-5.5-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"A","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.04,"tokens":{"uncached_input":9660,"cache_read":496512,"cache_write":0,"output":10908,"reasoning":3637},"cost_usd":0.089,"cost_breakdown":{"input":0.00966,"cache_read":0.024826,"cache_write":0.0,"output":0.05454},"cost_usd_as_run":0.089,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":294,"code":244,"comment":5,"blank":45,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":216,"code":182,"comment":0,"blank":34,"role":"test"}],"impl_lines":294,"impl_code":244,"impl_comment":5,"test_lines":216,"comment_ratio":0.02,"tests_declared":13},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":413.962,"reference":459.293,"speedup":1.11,"baselines":{"oracle":459.293}},{"id":"throughput_64m","candidate":2596.691,"reference":2538.644,"speedup":1.023,"baselines":{"oracle":2538.644}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":207,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.0,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":3.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Path normalization can silently select a different executable.","muse-wonder-full-xhigh":"core correct; tilde expansion, duplicate-validation hole, thin docs/tests","opus-5.5-xhigh":"minimal and works, but several differences from the reference and thin docs and tests","gpt-6.1-sol-max":"Path rewriting can run the wrong tool, and validation can execute invalid calls.","deepseek-v4.1-flash-max":"Compact and correct, but hardcodes helper paths and shares the duplicate-value validation hole."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Expanding a path changes filesystem resolution","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later duplicate options are not validated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public docs omit option names and defaults","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Slash paths get `~` expansion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Duplicate options after the first are unvalidated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Hardcoded helper locations","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Missing interface documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinnest assertion density in the cohort","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Later duplicate options are never validated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`Path.expand` on slash paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Hard-coded helper paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Thin docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The deadline test never checks that the group dies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes which executable runs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Invalid repeated options are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Run limits and defaults are undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"Helper paths are hardcoded","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Invalid duplicate option values are accepted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Sparse public docs","status":"valid","detail":"","location":""}],"score":5.9,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.3,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.25,"tokens":{"uncached_input":17182,"cache_read":120576,"cache_write":0,"output":8502,"reasoning":1879},"cost_usd":0.7175,"cost_breakdown":{"input":0.17182,"cache_read":0.120576,"cache_write":0.0,"output":0.4251},"cost_usd_as_run":0.7175,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":326,"code":265,"comment":6,"blank":55,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":240,"code":203,"comment":0,"blank":37,"role":"test"}],"impl_lines":326,"impl_code":265,"impl_comment":6,"test_lines":240,"comment_ratio":0.022,"tests_declared":14},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":522.75,"reference":476.088,"speedup":0.911,"baselines":{"oracle":476.088}},{"id":"throughput_64m","candidate":2463.254,"reference":2426.949,"speedup":1.015,"baselines":{"oracle":2426.949}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":226,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":5.5,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":4.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Path handling can run the wrong executable, with two further availability mismatches.","muse-wonder-full-xhigh":"clean validation, but tilde expansion and runner-deferred resolution","opus-5.5-xhigh":"compact and correct at the core; resolution and slash paths differ from the reference; no docs","gpt-6.1-sol-max":"Correct lifecycle mechanics are outweighed by executing the wrong path.","deepseek-v4.1-flash-max":"Clean, correct and well tested, but the public interface is essentially undocumented and resolution order differs on a full pool."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Expanding a path changes filesystem resolution","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A full pool masks a missing executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A shell must be on `PATH`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Slash paths get `~` expansion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`:busy` beats `:unavailable` for a missing tool","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Relay `$0` changes shell error output","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unguarded `terminate_child` in `after` can discard a result","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"`child_spec/1`/`start_link/1` undocumented and unspecced","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A missing command is reported as `:busy` when the pool is full","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Slash paths are rewritten by `Path.expand`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public API undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Identity read twice in a row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Lexical expansion selects the wrong executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Pool admission masks missing commands","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public functions lack API documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"No docs or specs for the public interface","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"A missing command on a full pool reports `:busy`, not `:unavailable`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`:infinity` await","status":"valid","detail":"","location":""}],"score":6.1,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.4,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":12.83,"tokens":{"uncached_input":82362,"cache_read":2442496,"cache_write":0,"output":46209,"reasoning":28015},"cost_usd":0.4355,"cost_breakdown":{"input":0.082362,"cache_read":0.122125,"cache_write":0.0,"output":0.231045},"cost_usd_as_run":0.4355,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":567,"code":459,"comment":24,"blank":84,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":408,"code":340,"comment":0,"blank":68,"role":"test"}],"impl_lines":567,"impl_code":459,"impl_comment":24,"test_lines":408,"comment_ratio":0.05,"tests_declared":18},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":449.836,"reference":480.62,"speedup":1.068,"baselines":{"oracle":480.62}},{"id":"throughput_64m","candidate":2317.374,"reference":2399.161,"speedup":0.966,"baselines":{"oracle":2399.161}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":382,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":49,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"The pool is bounded carefully, but resolution differs from the oracle.","muse-wonder-full-xhigh":"inventive async design buys suspended-pool safety at real fragility cost","opus-5.5-xhigh":"clever but over-engineered; leans on DynamicSupervisor internals; resolution order differs","gpt-6.1-sol-max":"Careful deadline handling, but admission order and relative invocation differ from the reference.","deepseek-v4.1-flash-max":"Sophisticated, bounded everywhere and excellently tested, but depends on DynamicSupervisor internals."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A full pool masks an unavailable executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Relative executable `$0` differs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Raw-byte `PATH` lookup differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"A test asserts the wrong full-pool result","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Run options and defaults are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"abi","title":"Bypasses the public supervisor API with internal formats","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`:busy` beats `:unavailable` for a missing tool","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Shell `-x` resolution deviates from oracle on subset exec modes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`nil`/`:undefined` pools actually work, oracle says unavailable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"`BoundedTool.Runner.child_spec/1` is never called","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A missing command returns `:busy` when the pool is full, and processes are spawned to find out","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Talks to DynamicSupervisor through private messages","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`BoundedTool.Runner.child_spec/1` is never used","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Thin public docs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Busy masks executable unavailability","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Relative executable names remain observable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public documentation omits the option contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"Hand-built internal child record sent as a raw `:start_child` request","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"A missing command on a full pool reports `:busy`, not `:unavailable`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`valid_pool_options?/1` re-checks what the enclosing `with` already matched","status":"valid","detail":"","location":""}],"score":7.0,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":4.9,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":17.09,"tokens":{"uncached_input":61311,"cache_read":828416,"cache_write":0,"output":35854,"reasoning":20051},"cost_usd":0.564,"cost_breakdown":{"input":0.122622,"cache_read":0.082842,"cache_write":0.0,"output":0.35854},"cost_usd_as_run":0.564,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":395,"code":315,"comment":15,"blank":65,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":477,"code":411,"comment":1,"blank":65,"role":"test"}],"impl_lines":395,"impl_code":315,"impl_comment":15,"test_lines":477,"comment_ratio":0.045,"tests_declared":18},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":424.358,"reference":463.934,"speedup":1.093,"baselines":{"oracle":463.934}},{"id":"throughput_64m","candidate":2535.521,"reference":2579.923,"speedup":0.983,"baselines":{"oracle":2579.923}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":260,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":19,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":9.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong cleanup and tests, with two reference-resolution differences.","muse-wonder-full-xhigh":"closest to oracle on every probe; only a deliberate nil-name superset","opus-5.5-xhigh":"correct and well tested; helpers are pinned to /usr/bin and /bin","gpt-6.1-sol-max":"Compact, well-tested implementation with a narrow observable mismatch and incomplete API documentation.","deepseek-v4.1-flash-max":"Correct, fast and very well tested; only an unbounded await and thin public docs hold it back."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Relative executable `$0` differs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Raw-byte `PATH` lookup differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Pool functions lack public documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`nil`/`:undefined` pools actually work, oracle says unavailable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"`child_spec/1` and `start_link/1` have specs but no `@doc`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Helpers are looked up only in /usr/bin and /bin","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"`child_spec/1` and `start_link/1` have no `@doc` and `reason` has no `@typedoc`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Relative invocation differs from the reference","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public options and defaults are undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`run/3` can hang if the runner stops answering","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"The public pool functions and the reason type are undocumented","status":"valid","detail":"","location":""}],"score":8.3,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.5,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":32.8,"tokens":{"uncached_input":128,"cache_read":16253443,"cache_write":370973,"output":322548,"reasoning":276152},"cost_usd":1.7015,"cost_breakdown":{"input":6e-05,"cache_read":0.807162,"cache_write":0.344674,"output":0.549566},"cost_usd_as_run":1.7015,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":625,"code":455,"comment":63,"blank":107,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":553,"code":436,"comment":5,"blank":112,"role":"test"}],"impl_lines":625,"impl_code":455,"impl_comment":63,"test_lines":553,"comment_ratio":0.122,"tests_declared":32},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":502.167,"reference":467.976,"speedup":0.932,"baselines":{"oracle":467.976}},{"id":"throughput_64m","candidate":2471.301,"reference":2474.887,"speedup":0.999,"baselines":{"oracle":2474.887}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":395,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":63,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.0,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Several valid calls are rejected or produce different reference output.","muse-wonder-full-xhigh":"best test count and docs, undermined by rejecting all duplicate options","opus-5.5-xhigh":"thorough and well documented; rejects duplicate keys that are valid","gpt-6.1-sol-max":"Broad tests, but valid inputs are rejected and relative invocation differs.","deepseek-v4.1-flash-max":"Richest docs and largest suite; two over-strict validation choices diverge from the reference."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid duplicate options are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Relative executable `$0` differs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Raw-byte `PATH` lookup differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"A test enshrines duplicate rejection","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Any duplicate option key is `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Atom pool names `nil`/`true`/`false` rejected as invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Per-credential exec check deviates from oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Invented default PATH when unset","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unguarded `terminate_child` in `after` can discard a result","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Valid duplicate option keys are rejected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"`name: nil | true | false` returns `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The third `handle_call(:await…)` clause is unreachable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid repeated options are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Boolean atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Relative invocation changes observable output","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Test title describes a different scenario","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys are rejected outright","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` is rejected as `:invalid_argument`","status":"valid","detail":"","location":""}],"score":7.1,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.1,"score_cross_vendor":6.875,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.28,"tokens":{"uncached_input":7341,"cache_read":206976,"cache_write":0,"output":5764,"reasoning":669},"cost_usd":0.093,"cost_breakdown":{"input":0.014682,"cache_read":0.020698,"cache_write":0.0,"output":0.05764},"cost_usd_as_run":0.093,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":260,"code":209,"comment":4,"blank":47,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":166,"code":144,"comment":0,"blank":22,"role":"test"}],"impl_lines":260,"impl_code":209,"impl_comment":4,"test_lines":166,"comment_ratio":0.019,"tests_declared":9},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":504.377,"reference":461.572,"speedup":0.915,"baselines":{"oracle":461.572}},{"id":"throughput_64m","candidate":2496.892,"reference":2569.254,"speedup":0.972,"baselines":{"oracle":2569.254}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":174,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.0,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":4.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Executable resolution can run the wrong file.","muse-wonder-full-xhigh":"compact and core-correct, but no public docs at all plus tilde expansion","opus-5.5-xhigh":"terse and works, but no docs or contract type, slash paths rewritten, thin tests","gpt-6.1-sol-max":"Compact implementation, but path expansion can execute a different tool.","deepseek-v4.1-flash-max":"Correct and compact, but the public interface is undocumented and the suite is the thinnest of the twelve."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Expanding a path changes symlink semantics","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A shell must be on `PATH`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation and the public reason type are missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Slash paths get `~` expansion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Entire fixed-interface documentation missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Relay `$0` changes shell error output","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Runner `init` raises instead of failing cleanly","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unguarded `terminate_child` in `after` can discard a result","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Fewest tests, with gaps at its own weak spots","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No docs, specs or `reason` type","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`Path.expand` on slash paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes executable selection","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Published API documentation and types are missing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Public API has no docs, specs or `reason/0` type","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`:infinity` await","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Nine tests leave clear gaps","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Parenthesised `else:` block reads poorly","status":"valid","detail":"","location":""}],"score":5.7,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.2,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":23.44,"tokens":{"uncached_input":54,"cache_read":3663044,"cache_write":174989,"output":130710,"reasoning":100022},"cost_usd":4.7469,"cost_breakdown":{"input":0.000216,"cache_read":0.732609,"cache_write":1.399912,"output":2.6142},"cost_usd_as_run":4.7469,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":634,"code":464,"comment":68,"blank":102,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":673,"code":525,"comment":27,"blank":121,"role":"test"}],"impl_lines":634,"impl_code":464,"impl_comment":68,"test_lines":673,"comment_ratio":0.128,"tests_declared":28},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":458.512,"reference":471.88,"speedup":1.029,"baselines":{"oracle":471.88}},{"id":"throughput_64m","candidate":2563.536,"reference":2503.19,"speedup":1.024,"baselines":{"oracle":2503.19}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":400,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":31,"flags":2,"audited":"2026-11-02","reviewed":true},"conduct":{"severity":"serious","summary":"While checking for leftover processes it ran a broad `pkill -KILL -f '^sleep 30$'`, which killed three processes of an unrelated job on the host, outside its work folder. It noticed within seconds and reported it. It gained nothing, so its scores stand (owner decision, 2026-11-02)."},"scores":{"gpt-6.1-terra-max":6.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Resetting inherited signals changes ordinary command output.","muse-wonder-full-xhigh":"best test suite, but three oracle deviations from added signal/PATH machinery","opus-5.5-xhigh":"excellent tests and docs; resets SIGPIPE, so some output differs from the reference","gpt-6.1-sol-max":"Extensive tests and sound bounds, but launch changes alter tool behavior.","deepseek-v4.1-flash-max":"The most complete submission: careful docs, an extra real fix (SIGPIPE) and the strongest suite."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Signal reset changes captured output","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Relative executable `$0` differs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Raw-byte `PATH` lookup differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"A test requires the incompatible signal behavior","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Per-credential exec check deviates from oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Invented default PATH when unset","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"SIGPIPE reset changes observable output","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Host/env probe apparatus for the non-spec SIGPIPE behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"The tool's signal dispositions differ from the reference","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"an immediately closed stdin\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Resetting signals changes reference results","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Relative invocation changes `$0`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Launch-chain comment incorrectly describes exec behavior","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"The one-time host probe waits without a timeout","status":"valid","detail":"","location":""}],"score":7.5,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.6,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":89.16,"tokens":{"uncached_input":363720,"cache_read":4532228,"cache_write":0,"output":62796,"reasoning":36011},"cost_usd":3.6172,"cost_breakdown":{"input":1.45488,"cache_read":0.906446,"cache_write":0.0,"output":1.25592},"cost_usd_as_run":3.6172,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":752,"code":572,"comment":57,"blank":123,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":455,"code":343,"comment":25,"blank":87,"role":"test"}],"impl_lines":752,"impl_code":572,"impl_comment":57,"test_lines":455,"comment_ratio":0.091,"tests_declared":25},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":497.704,"reference":445.563,"speedup":0.895,"baselines":{"oracle":445.563}},{"id":"throughput_64m","candidate":2550.771,"reference":2548.097,"speedup":1.001,"baselines":{"oracle":2548.097}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":43,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":6.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Several accepted inputs differ from the reference.","muse-wonder-full-xhigh":"solid core with three small validation/resolution deviations","opus-5.5-xhigh":"sound runner; the option validator ignores later duplicates; some redundant plumbing","gpt-6.1-sol-max":"Sound lifecycle handling, but invalid repeated options can execute commands.","deepseek-v4.1-flash-max":"Good implementation and suite, with three spec edges and a missing 64 MiB test."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Duplicate invalid values are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Relative executable names reach the tool unchanged","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The large-output test stops at 1 MiB","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Duplicate options after the first are unvalidated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Atom pool names `nil`/`true`/`false` rejected as invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Per-credential exec check deviates from oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Unreachable `start_child` result clauses","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Later duplicate options are never validated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"`name: nil | true | false` is rejected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicated exception handling and unreachable branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"No 64 MiB test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Repeated options bypass validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Relative invocation changes observable output","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Invalid duplicate option values are accepted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` is rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`await_runner/2` has no timeout","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No 64 MiB case","status":"valid","detail":"","location":""}],"score":6.9,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.8,"score_cross_vendor":6.875,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":62.98,"tokens":{"uncached_input":172875,"cache_read":4986880,"cache_write":0,"output":253176,"reasoning":114567},"cost_usd":4.3582,"cost_breakdown":{"input":0.34575,"cache_read":2.49344,"cache_write":0.0,"output":1.519056},"cost_usd_as_run":4.3582,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":801,"code":625,"comment":38,"blank":138,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":502,"code":399,"comment":3,"blank":100,"role":"test"}],"impl_lines":801,"impl_code":625,"impl_comment":38,"test_lines":502,"comment_ratio":0.057,"tests_declared":13},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":404.027,"reference":456.184,"speedup":1.129,"baselines":{"oracle":456.184}},{"id":"throughput_64m","candidate":2647.816,"reference":2531.015,"speedup":1.046,"baselines":{"oracle":2531.015}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":48,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":4.0,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":3.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"An unknown startup identity can authorize signalling an unrelated group.","muse-wonder-full-xhigh":"wrong duplicate-option semantics and a fail-open identity check inside an over-complex design","opus-5.5-xhigh":"works, but last-wins options, an unguarded kill path and a lot of dead code","gpt-6.1-sol-max":"An unsafe identity fallback undermines otherwise working execution and cleanup.","deepseek-v4.1-flash-max":"Very thorough and careful, but over-built at 801 lines, with dead clauses and a `name: nil` divergence."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"A missing identity is treated as permission to signal","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"The last duplicate option wins","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Relative executable `$0` differs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Raw-byte `PATH` lookup differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A shell must be on `PATH` on the first run","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate options are last-wins, reference is first-wins","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Group kill adopts an unverified starttime","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Atom pool names `nil`/`true`/`false` rejected as invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Per-credential exec check deviates from oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Largest, most indirect design in the cohort","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Duplicate options: the last one wins","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Group kill without an identity when the start time is unknown","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"`name: nil | true | false` returns `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The group is signalled again from `terminate` after `fail`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`relay/0` and `mono_ms/0` are public functions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Missing identity can authorize an unrelated group kill","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Last repeated option wins","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Boolean atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Relative invocation changes `$0`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` is rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable clause in `format_backstop/1`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable clause in `take_queued_data/2`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"A failed run re-reads identity and re-kills on terminate","status":"valid","detail":"","location":""}],"score":5.6,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.5}},"defect_points":7.2,"score_cross_vendor":5.6,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":8.57,"tokens":{"uncached_input":24,"cache_read":785941,"cache_write":77337,"output":47049,"reasoning":26562},"cost_usd":1.717,"cost_breakdown":{"input":9.6e-05,"cache_read":0.157188,"cache_write":0.618696,"output":0.94098},"cost_usd_as_run":1.717,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":542,"code":401,"comment":48,"blank":93,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":474,"code":357,"comment":19,"blank":98,"role":"test"}],"impl_lines":542,"impl_code":401,"impl_comment":48,"test_lines":474,"comment_ratio":0.107,"tests_declared":25},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":427.562,"reference":466.236,"speedup":1.09,"baselines":{"oracle":466.236}},{"id":"throughput_64m","candidate":2554.52,"reference":2516.626,"speedup":1.015,"baselines":{"oracle":2516.626}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":349,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":11,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Well-tested runner with two lookup mismatches.","muse-wonder-full-xhigh":"oracle-faithful except credential-based exec check; strong docs and tests","opus-5.5-xhigh":"every probe matches the reference, docs are accurate, tests are thorough","gpt-6.1-sol-max":"Strong execution and cleanup, with a narrow path-validation defect.","deepseek-v4.1-flash-max":"Clean, correct and well documented; only one test-coverage gap."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash is stripped before the file check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Raw-byte `PATH` lookup differs from the oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Per-credential exec-bit check deviates from oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Credential machinery is needless complexity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Absolutizing before validation accepts trailing slashes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The runner-crash path is untested","status":"valid","detail":"","location":""}],"score":8.3,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.7,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":27.08,"tokens":{"uncached_input":28749,"cache_read":1202816,"cache_write":0,"output":51767,"reasoning":33914},"cost_usd":0.6955,"cost_breakdown":{"input":0.057498,"cache_read":0.120282,"cache_write":0.0,"output":0.51767},"cost_usd_as_run":0.6955,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":502,"code":390,"comment":36,"blank":76,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":621,"code":519,"comment":2,"blank":100,"role":"test"}],"impl_lines":502,"impl_code":390,"impl_comment":36,"test_lines":621,"comment_ratio":0.085,"tests_declared":28},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":449.845,"reference":474.585,"speedup":1.055,"baselines":{"oracle":474.585}},{"id":"throughput_64m","candidate":2454.381,"reference":2508.447,"speedup":0.978,"baselines":{"oracle":2508.447}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":338,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":8.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Thoroughly tested, but admission and reference name behavior differ","muse-wonder-full-xhigh":"Most thorough resolution and test coverage; admission happens before executable resolution","gpt-6.1-sol-max":"Strong byte handling and lifecycle tests, with admission and precedence mismatches.","opus-5.5-xhigh":"robust and thoroughly tested; resolving the executable inside the pool changes one error","deepseek-v4.1-flash-max":"full docs, the most complete tests, careful relay protocol; two nits"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A full pool masks an unavailable executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Deadline is checked before full-pool admission","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Reserved-name remapping changes reference behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The registration comment incorrectly includes boolean atoms","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Missing executable on a full pool returns `:busy`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Admission occurs before executable availability is known","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Global remapping changes nil-name behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline check masks an earlier overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Boolean names are incorrectly described as reserved","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A missing executable reports `:busy` when the pool is full","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Helpers are looked up only in `/usr/bin` and `/bin`, never on PATH","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Pools named `nil`/`true`/`false`/`:undefined` are mapped to cluster-wide `:global` registration","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"the executable is resolved in the relay, after admission","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"helpers are looked up only in `/usr/bin`, `/bin`","status":"valid","detail":"","location":""}],"score":8.0,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.2,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":28.58,"tokens":{"uncached_input":378,"cache_read":7473952,"cache_write":172413,"output":135423,"reasoning":0},"cost_usd":0.2178,"cost_breakdown":{"input":5.7e-05,"cache_read":0.119583,"cache_write":0.034483,"output":0.063649},"cost_usd_as_run":0.2178,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2178,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":694,"code":513,"comment":67,"blank":114,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":534,"code":382,"comment":28,"blank":124,"role":"test"}],"impl_lines":694,"impl_code":513,"impl_comment":67,"test_lines":534,"comment_ratio":0.116,"tests_declared":29},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":498.913,"reference":450.857,"speedup":0.904,"baselines":{"oracle":450.857}},{"id":"throughput_64m","candidate":2501.73,"reference":2545.296,"speedup":0.983,"baselines":{"oracle":2545.296}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":63,"flags":2,"audited":"2026-11-02","reviewed":true},"conduct":{"severity":"minor","summary":"While working it ran `pkill -f \"sleep 4\"`, a host-wide pattern that also matches other jobs' processes (for example any `sleep 40`), and inspected other processes on the host. No other process is known to have been hit; scores stand (owner decision, 2026-11-02)."},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Capable lifecycle handling, with path and pool-name collisions","muse-wonder-full-xhigh":"Sound semaphore design and strong tests; slash paths are expanded","gpt-6.1-sol-max":"Extra pool machinery introduces capacity failures, permanent atom growth and name collisions.","opus-5.5-xhigh":"well tested and documented; replaces DynamicSupervisor's cap with a hand-rolled semaphore","deepseek-v4.1-flash-max":"full docs and excellent tests; an unreachable clause, runtime atom creation and duplicate capacity machinery"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Slash paths are normalized before execution","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Internal registrations collide with valid pool names","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"An unnamed `nil` pool becomes addressable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Slash paths are lexically expanded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable permissive branch in option validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Relative path expansion selects the wrong executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Semaphore restart loses the concurrency limit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Auxiliary registrations collide with valid pool names","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Missing-pool lookups permanently allocate atoms","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"An unnamed nil pool becomes addressable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` in a slash path is expanded","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Each `run/3` creates two atoms from the pool name, even for pools that don't exist","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"If the semaphore crashes, it restarts with zero tokens while runs are in flight, so admissions can exceed the cap","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable `{_other, _value} -> true`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"derived atoms from caller-supplied names","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"a 90-line semaphore duplicates `DynamicSupervisor.max_children`","status":"valid","detail":"","location":""}],"score":6.8,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":4.6,"score_cross_vendor":6.8,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.68,"tokens":{"uncached_input":8795,"cache_read":95104,"cache_write":0,"output":2846,"reasoning":78},"cost_usd":0.0278,"cost_breakdown":{"input":0.008795,"cache_read":0.004755,"cache_write":0.0,"output":0.01423},"cost_usd_as_run":0.0278,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":255,"code":214,"comment":1,"blank":40,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":25,"code":20,"comment":0,"blank":5,"role":"test"}],"impl_lines":255,"impl_code":214,"impl_comment":1,"test_lines":25,"comment_ratio":0.005,"tests_declared":2},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":474.858,"reference":450.11,"speedup":0.948,"baselines":{"oracle":450.11}},{"id":"throughput_64m","candidate":2566.256,"reference":2644.868,"speedup":0.97,"baselines":{"oracle":2644.868}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":175,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":5.0,"muse-wonder-full-xhigh":5.5,"gpt-6.1-sol-max":4.5,"opus-5.5-xhigh":5.0,"deepseek-v4.1-flash-max":5.0},"council_summaries":{"gpt-6.1-terra-max":"Pool shutdown leaves a command running, and tests cover little of the contract","muse-wonder-full-xhigh":"Minimal implementation passes core cases but lacks exit trapping, tests and docs","gpt-6.1-sol-max":"Functional core, but wrong path resolution, thin tests and cleanup duplication reduce confidence.","opus-5.5-xhigh":"core behaviour works, but no pool-shutdown cleanup, almost no tests, no docs","deepseek-v4.1-flash-max":"correct core but two tests, no docs, and the tool outlives a pool shutdown"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Pool shutdown bypasses cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Slash paths are normalized through symlinks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Two tests omit the major lifecycle guarantees","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation and the declared reason type are missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Slash paths are lexically expanded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Pool shutdown leaves the tool group alive","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Test suite is two tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No public documentation or types","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Group-kill failures leak to host stderr","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline check masks an earlier overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Two tests leave major guarantees unprotected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public types and lifecycle documentation are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Caller death performs group termination twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"The runner doesn't trap exits, so stopping the pool leaves the tool running","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Two tests in total","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"The catch-all `rescue` maps internal raises to `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` in a slash path is expanded","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc`, `@spec` or `@type reason`, and `BoundedTool.Runner` has no `@moduledoc false`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`kill` runs without `stderr_to_stdout`, so \"kill: … No such process\" lands on the BEAM's stderr; the caller-death path also kills twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"runner never traps exits, so `terminate/2` is skipped when the pool stops it","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"2 tests, 25 lines","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no public documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"internal exceptions are reported as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`kill` writes its failure to the VM's stderr","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"the group is killed twice on caller death","status":"valid","detail":"","location":""}],"score":5.0,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":8.1,"score_cross_vendor":5.167,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":6.31,"tokens":{"uncached_input":22,"cache_read":643609,"cache_write":63420,"output":32959,"reasoning":18219},"cost_usd":1.2954,"cost_breakdown":{"input":8.8e-05,"cache_read":0.128722,"cache_write":0.50736,"output":0.65918},"cost_usd_as_run":1.2954,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":473,"code":356,"comment":39,"blank":78,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":375,"code":310,"comment":1,"blank":64,"role":"test"}],"impl_lines":473,"impl_code":356,"impl_comment":39,"test_lines":375,"comment_ratio":0.099,"tests_declared":24},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":410.123,"reference":461.318,"speedup":1.125,"baselines":{"oracle":461.318}},{"id":"throughput_64m","candidate":2542.351,"reference":2506.747,"speedup":1.014,"baselines":{"oracle":2506.747}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":312,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Good lifecycle coverage, with option precedence and atom-name errors","muse-wonder-full-xhigh":"Fast and well tested, but option semantics can run on the wrong pool","gpt-6.1-sol-max":"Sound execution core, but option handling and cached helper paths diverge from the reference.","opus-5.5-xhigh":"closest to the reference, with excellent tests and docs","deepseek-v4.1-flash-max":"complete, well-documented, thoroughly tested implementation; one pool-selection edge case and two nits"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"False and nil names are treated as defaults","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate options are last-wins","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"`name: nil`/`name: false` silently use the default pool","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Helper paths are cached forever in `:persistent_term`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"False and nil pool names select the default pool","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Helper cache survives PATH changes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A repeated option key takes the last value, not the first","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` / `name: false` silently run on the default pool","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"pool-identity invariant is undocumented","status":"valid","detail":"","location":""}],"score":7.7,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.3,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.75,"tokens":{"uncached_input":21691,"cache_read":110336,"cache_write":0,"output":4479,"reasoning":173},"cost_usd":0.5512,"cost_breakdown":{"input":0.21691,"cache_read":0.110336,"cache_write":0.0,"output":0.22395},"cost_usd_as_run":0.5512,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":264,"code":221,"comment":2,"blank":41,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":144,"code":121,"comment":1,"blank":22,"role":"test"}],"impl_lines":264,"impl_code":221,"impl_comment":2,"test_lines":144,"comment_ratio":0.009,"tests_declared":7},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":480.461,"reference":456.736,"speedup":0.951,"baselines":{"oracle":456.736}},{"id":"throughput_64m","candidate":2613.376,"reference":2624.136,"speedup":0.996,"baselines":{"oracle":2624.136}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":186,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":6.0,"gpt-6.1-sol-max":4.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Core execution works, but validation and path handling differ","muse-wonder-full-xhigh":"Core runner is sound, but option validation and slash-path resolution diverge from the oracle","gpt-6.1-sol-max":"Invalid options can launch commands, and slash paths can resolve to the wrong file.","opus-5.5-xhigh":"works, but the interface is undocumented and the tests are thin","deepseek-v4.1-flash-max":"correct clean core, but no public docs, thin tests and a lax duplicate-option check"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Shadowed invalid options are accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Slash paths are normalized through symlinks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"The public functions and reason type lack the specified documentation and specs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate invalid options are accepted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Slash paths are lexically expanded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No public documentation or types","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Invalid shadowed options still execute commands","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline check masks an earlier overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Fixed public type and API documentation are absent","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` in a slash path is expanded","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@type reason`, no `@doc`, no `@spec` on any public function","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"An invalid repeated key is accepted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"7 compressed tests, and the \"non-executable file\" assertion is hollow","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no public documentation at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"duplicate option keys bypass value validation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"hard `{:os_pid, pid} = Port.info(port, :os_pid)`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"7 tests and one vacuous assertion","status":"valid","detail":"","location":""}],"score":6.0,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.5,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":63.11,"tokens":{"uncached_input":283304,"cache_read":4330536,"cache_write":0,"output":57723,"reasoning":31400},"cost_usd":3.1538,"cost_breakdown":{"input":1.133216,"cache_read":0.866107,"cache_write":0.0,"output":1.15446},"cost_usd_as_run":3.1538,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":793,"code":626,"comment":39,"blank":128,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":453,"code":359,"comment":4,"blank":90,"role":"test"}],"impl_lines":793,"impl_code":626,"impl_comment":39,"test_lines":453,"comment_ratio":0.059,"tests_declared":21},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":421.271,"reference":441.514,"speedup":1.048,"baselines":{"oracle":441.514}},{"id":"throughput_64m","candidate":2552.449,"reference":2653.488,"speedup":0.962,"baselines":{"oracle":2653.488}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":40,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":8.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong cleanup and tests, but rejects valid atom names","muse-wonder-full-xhigh":"Oracle-matching core with an extra verified-kill channel; one validation edge","gpt-6.1-sol-max":"Careful cleanup, with restricted atom names and unnecessary caller-side protocol work.","opus-5.5-xhigh":"robust (it even cleans up after a brutally killed runner) but very verbose","deepseek-v4.1-flash-max":"full docs and strong tests; a nil-name rejection and a possible over-run of the return bound"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Boolean atom names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Deadline is checked before full-pool admission","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"A valid atom name is asserted invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Boolean/`nil` pool names are `:invalid_argument` instead of `:unavailable`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Boolean atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Extra monitor duplicates existing termination guarantees","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Relay comment identifies the wrong group leader","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil`, `true` or `false` are rejected as invalid","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"793 lines with nested `try`/`rescue`/`catch` around calls that cannot raise (`File.stat`, `File.read`), plus an extra `:group_info` call on every run","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The comment calls the group leader \"`timeout`'s child, this `sh`\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The VM-kill test allows the tool to live 4 s after the kill, looser than deadline + 2 s","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` is rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`@leader_wait_ms 800` plus `@death_wait_ms 500` can exceed the contract's `timeout + 1_000` return bound","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"two synchronous GenServer calls per run plus the `verified_kill_group`/`wait_dead` machinery","status":"valid","detail":"","location":""}],"score":7.9,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":1.5,"deepseek-v4.1-flash-max":0.5}},"defect_points":2.8,"score_cross_vendor":7.75,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":21.9,"tokens":{"uncached_input":40799,"cache_read":11002880,"cache_write":0,"output":190891,"reasoning":161104},"cost_usd":0.1537,"cost_breakdown":{"input":0.00612,"cache_read":0.033009,"cache_write":0.0,"output":0.114535},"cost_usd_as_run":0.1537,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1537,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":717,"code":524,"comment":79,"blank":114,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":509,"code":370,"comment":26,"blank":113,"role":"test"}],"impl_lines":717,"impl_code":524,"impl_comment":79,"test_lines":509,"comment_ratio":0.131,"tests_declared":20},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":409.893,"reference":449.514,"speedup":1.097,"baselines":{"oracle":449.514}},{"id":"throughput_64m","candidate":2614.465,"reference":2602.876,"speedup":1.004,"baselines":{"oracle":2602.876}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":459,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":72,"flags":2,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Well-structured runner, with two option/deadline edge cases","muse-wonder-full-xhigh":"Best-documented entry; duplicate options change its answers","gpt-6.1-sol-max":"Robust lifecycle handling, but duplicate-option and overflow precedence differ.","opus-5.5-xhigh":"thorough docs and tests; one option-precedence deviation","deepseek-v4.1-flash-max":"full docs, excellent tests, careful implementation; two nits only"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Long calls time out before their requested deadline","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate options are last-wins","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate options overwrite earlier values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline check masks an earlier overflow","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A repeated option key takes the last value","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable or redundant code: re-checking `is_atom(name)` after validation, `format_duration(ms) when ms < 0`, `resolve_executable(_name)`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable negative-duration clause","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"an optional `kill` turns group kills into silent no-ops","status":"valid","detail":"","location":""}],"score":7.9,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":2.3,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":20.06,"tokens":{"uncached_input":14540,"cache_read":847744,"cache_write":0,"output":33993,"reasoning":15299},"cost_usd":2.6928,"cost_breakdown":{"input":0.1454,"cache_read":0.847744,"cache_write":0.0,"output":1.69965},"cost_usd_as_run":2.6928,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":427,"code":334,"comment":21,"blank":72,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":534,"code":448,"comment":2,"blank":84,"role":"test"}],"impl_lines":427,"impl_code":334,"impl_comment":21,"test_lines":534,"comment_ratio":0.059,"tests_declared":27},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":438.071,"reference":453.963,"speedup":1.036,"baselines":{"oracle":453.963}},{"id":"throughput_64m","candidate":2510.557,"reference":2592.989,"speedup":0.968,"baselines":{"oracle":2592.989}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":290,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":21,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.0,"muse-wonder-full-xhigh":9.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Thorough tests and careful cleanup, with reserved-name divergence","muse-wonder-full-xhigh":"Matches the oracle across the battery, with the strongest misuse tests","gpt-6.1-sol-max":"Strong safety tests and clear execution structure, with observable edge-case differences.","opus-5.5-xhigh":"solid, with the strongest test suite; public docs incomplete","deepseek-v4.1-flash-max":"excellent tests and solid implementation; partial docs and a test-only public module"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Reserved-name remapping changes reference behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Pool functions lack public documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Pool functions lack `@doc`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Reserved-name remapping changes pool behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline check masks an earlier overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Pool APIs lack public documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Mailbox isolation is overstated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`child_spec/1` and `start_link/1` have no `@doc` and `reason` has no `@typedoc`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Executability is checked twice, once in Elixir (`resolve`) and again in the relay (`[ -x ]`)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Helpers are found only in fixed `/usr/bin` and `/bin` locations","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"partial interface docs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`BoundedTool.ProcessGroup` is public API only for tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"helpers only from `/usr/bin`, `/bin`","status":"valid","detail":"","location":""}],"score":8.3,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.4,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.04,"tokens":{"uncached_input":7364,"cache_read":251904,"cache_write":0,"output":5634,"reasoning":652},"cost_usd":0.0481,"cost_breakdown":{"input":0.007364,"cache_read":0.012595,"cache_write":0.0,"output":0.02817},"cost_usd_as_run":0.0481,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":243,"code":202,"comment":1,"blank":40,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":117,"code":98,"comment":0,"blank":19,"role":"test"}],"impl_lines":243,"impl_code":202,"impl_comment":1,"test_lines":117,"comment_ratio":0.005,"tests_declared":6},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":476.615,"reference":455.761,"speedup":0.956,"baselines":{"oracle":455.761}},{"id":"throughput_64m","candidate":2542.439,"reference":2566.166,"speedup":0.991,"baselines":{"oracle":2566.166}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":170,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":5.5,"muse-wonder-full-xhigh":6.0,"gpt-6.1-sol-max":5.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":5.5},"council_summaries":{"gpt-6.1-terra-max":"Pool shutdown leaks a running command and path normalization changes execution","muse-wonder-full-xhigh":"Same shutdown and expansion defects as entry-I, with somewhat better tests","gpt-6.1-sol-max":"Ordinary execution works, but path resolution can execute the wrong file.","opus-5.5-xhigh":"functional, but no pool-shutdown cleanup and light tests and docs","deepseek-v4.1-flash-max":"correct core but no docs, six thin tests, and a pool shutdown leaks the running tool"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Pool shutdown bypasses cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Slash paths are normalized through symlinks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Caller-death test checks only slot release","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack documentation and specs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Slash paths are lexically expanded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Pool shutdown leaves the tool group alive","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Caller-death test never checks the group died","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No public documentation or specs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline check masks an earlier overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Cleanup assertions do not establish group death","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public interface lacks documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Same missing trap-exit as B, so stopping the pool leaves the tool running","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`~` in a slash path is expanded","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc` or `@spec` on public functions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The caller-death test checks only that the slot is freed, not that the tool was killed, and synchronises with `Process.sleep`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"runner does not trap exits","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no public documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"6 tests, 117 lines","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`File.stat!` after `File.regular?`","status":"valid","detail":"","location":""}],"score":5.7,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.1,"score_cross_vendor":5.833,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":135.53,"tokens":{"uncached_input":214027,"cache_read":9336960,"cache_write":0,"output":108722,"reasoning":70114},"cost_usd":4.8979,"cost_breakdown":{"input":0.856108,"cache_read":1.867392,"cache_write":0.0,"output":2.17444},"cost_usd_as_run":4.8979,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":965,"code":778,"comment":20,"blank":167,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":597,"code":464,"comment":3,"blank":130,"role":"test"}],"impl_lines":965,"impl_code":778,"impl_comment":20,"test_lines":597,"comment_ratio":0.025,"tests_declared":20},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":529.779,"reference":456.004,"speedup":0.861,"baselines":{"oracle":456.004}},{"id":"throughput_64m","candidate":2521.089,"reference":2593.596,"speedup":0.972,"baselines":{"oracle":2593.596}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":63,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Broad coverage, with incorrect duplicate-option and pool-name handling","muse-wonder-full-xhigh":"Behaviourally strong on all hard paths, but over-strict validation and a very heavy implementation","gpt-6.1-sol-max":"Thorough implementation and tests, with overly restrictive option validation.","opus-5.5-xhigh":"correct core buried in defensive padding","deepseek-v4.1-flash-max":"full docs and thorough tests; stricter-than-oracle validation and some dead padding"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid duplicate options are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"`nil` is rejected as a pool name","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests enforce those incorrect results","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"The BEAM group check is repeated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Duplicate valid options are rejected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil` is `:invalid_argument` instead of `:unavailable`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Redundant identity machinery run twice per call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid duplicate options are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Nil pool option gets the wrong error","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Valid inputs are rejected: duplicate keys and `name: nil`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"965 lines, with a hand-rolled await (send, monitor, receive loop, flush) that re-implements `GenServer.call`, unreachable `handle_call`/`handle_cast`, and `is_map(state)` guards in `terminate`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`max_restarts: 100_000` does nothing for temporary children; each run also reads `/proc/<beam>/status` twice and does four PATH lookups","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"duplicate option keys are rejected outright","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` is rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`handle_info({port, :eof}, state)` is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`max_restarts: 100_000` on the pool","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"defensive `is_map(state)`/`Map.get(state, ...)` in `terminate/2`","status":"valid","detail":"","location":""}],"score":7.5,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.5}},"defect_points":3.5,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.48,"tokens":{"uncached_input":13915,"cache_read":297600,"cache_write":0,"output":10384,"reasoning":2845},"cost_usd":0.1614,"cost_breakdown":{"input":0.02783,"cache_read":0.02976,"cache_write":0.0,"output":0.10384},"cost_usd_as_run":0.1614,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":329,"code":270,"comment":5,"blank":54,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":257,"code":214,"comment":2,"blank":41,"role":"test"}],"impl_lines":329,"impl_code":270,"impl_comment":5,"test_lines":257,"comment_ratio":0.018,"tests_declared":14},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":494.547,"reference":461.749,"speedup":0.934,"baselines":{"oracle":461.749}},{"id":"throughput_64m","candidate":2637.668,"reference":2650.052,"speedup":0.995,"baselines":{"oracle":2650.052}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":224,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":7.5,"gpt-6.1-terra-max":6.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":4.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"muse-wonder-full-xhigh":"Core behaviour correct, several small contract divergences","gpt-6.1-terra-max":"Cleanup and option-order defects need correction","opus-5.5-xhigh":"Correct runs, but pool shutdown leaves tools alive, and public API docs are missing","gpt-6.1-sol-max":"Graceful shutdown leaks tools, and duplicate options change required results.","deepseek-v4.1-flash-max":"good normal-path behaviour and tests, but a run interrupted by pool shutdown leaves the tool alive"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Duplicate option keys use last value, oracle uses first","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`:busy` beats `:unavailable` for a missing tool","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"`start_link/1` can return bare `:error`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No `@doc` on any public function","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Pool shutdown leaves the tool running","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A full pool masks an unavailable executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack their option documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Runner doesn't trap exits, so pool shutdown never kills the group","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A missing executable on a full pool returns `:busy`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc` or `@spec` on `child_spec/1`/`start_link/1`, no `@typedoc`, no `@doc` on `run/3`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"Duplicate keys: last one wins","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"`start_link` can return a bare `:error`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`rescue _` together with `catch _, _` (37-41)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Pool shutdown bypasses cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"correctness","title":"Capacity rejection masks unavailable executables","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is missing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"robustness","title":"Pool shutdown leaks the tool's process group","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Public API undocumented","status":"valid","detail":"","location":""}],"score":6.1,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.7,"score_cross_vendor":6.833,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":101.83,"tokens":{"uncached_input":421366,"cache_read":13889344,"cache_write":0,"output":151084,"reasoning":121720},"cost_usd":0.5554,"cost_breakdown":{"input":0.063205,"cache_read":0.41668,"cache_write":0.0,"output":0.075542},"cost_usd_as_run":0.5554,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":809,"code":601,"comment":75,"blank":133,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":549,"code":387,"comment":44,"blank":118,"role":"test"}],"impl_lines":809,"impl_code":601,"impl_comment":75,"test_lines":549,"comment_ratio":0.111,"tests_declared":22},"hidden":{"passed":14,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":426.412,"reference":426.327,"speedup":1.0,"baselines":{"oracle":426.327}},{"id":"throughput_64m","candidate":2826.428,"reference":2732.524,"speedup":1.034,"baselines":{"oracle":2732.524}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":111,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"muse-wonder-full-xhigh":4.0,"gpt-6.1-terra-max":3.0,"opus-5.5-xhigh":3.5,"gpt-6.1-sol-max":3.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"muse-wonder-full-xhigh":"Kills unverified process groups and fails valid huge timeouts, despite extensive tests/docs","gpt-6.1-terra-max":"PID reuse can kill an unrelated group, and large valid timeouts leak work","opus-5.5-xhigh":"The PID-reuse guard doesn't work, results get lost under load, and huge timeouts break hygiene","gpt-6.1-sol-max":"PID ownership is unsafe, and large valid timeouts break mailbox hygiene.","deepseek-v4.1-flash-max":"thorough and well-documented, with a real timer-overflow bug and a heavier, bare-GenServer pool design"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"robustness","title":"Group kill has no recorded identity, so a recycled PID's group is SIGKILLed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Timeouts ≥ ~4_294_937_295 ms return `:invalid_argument` and pollute the mailbox","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"Comment falsely claims PID-reuse protection","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Any duplicated option key is rejected as unknown","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Helpers resolve off-PATH via absolute fallbacks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Hand-rolled PATH search and double kill spawn","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Custom `Pool` reimplements `DynamicSupervisor` max-children","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `Port.info` clause","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"critical","category":"robustness","title":"Cleanup establishes identity too late","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Large valid timeouts return early and leave a runner behind","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Valid duplicate keys are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Pool shutdown waits on a circular call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Deadline tests do not create the stated edge cases","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Group-kill identity check compares the process with itself","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Result lost when the tool finishes before the caller's await message arrives","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"A timeout above ~4.29e9 ms returns `:invalid_argument` and leaks messages","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Pool shutdown blocks for about 1 s","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Fallback sends a second group SIGKILL with no fresh identity check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Over-built plumbing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"Valid duplicate option keys rejected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"truncated by `:max_bytes`\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Cleanup can kill a recycled, unrelated PID","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Large timeouts fail after spawning and leave messages","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Allowed duplicate keys are classified as unknown","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"VM cleanup timing assertions permit contract overruns","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"A valid large `:timeout` is rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"The pool is a plain GenServer with no catch-all call clause","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"Pool shutdown stalls ~1 s","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"\"output … truncated by `:max_bytes`\"","status":"valid","detail":"","location":""}],"score":4.3,"slop":{"score":1.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":4.5,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":10.2,"score_cross_vendor":4.3,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1-b2","task":"bounded-tool","lead":"muse-wonder-full-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":12.0,"tokens":{"cache_read":3859456,"cache_write":0,"output":31577,"reasoning":20408,"uncached_input":75777},"cost_usd":0.4266,"cost_breakdown":{"cache_read":0.192973,"cache_write":0.0,"input":0.075777,"output":0.157885},"cost_usd_as_run":0.4266,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":12.59,"tokens":{"cache_read":3857328,"cache_write":0,"output":50885,"reasoning":34974,"uncached_input":191171},"cost_usd":2.5539,"cost_breakdown":{"cache_read":0.771466,"cache_write":0.0,"input":0.764684,"output":1.0177},"cost_usd_as_run":2.5539,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":13.9,"tokens":{"cache_read":3883970,"cache_write":164353,"output":58354,"reasoning":40418,"uncached_input":64},"cost_usd":3.259,"cost_breakdown":{"cache_read":0.776794,"cache_write":1.314824,"input":0.000256,"output":1.16708},"cost_usd_as_run":3.259,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":19.97,"tokens":{"cache_read":2104704,"cache_write":0,"output":36666,"reasoning":24193,"uncached_input":115420},"cost_usd":0.808,"cost_breakdown":{"cache_read":0.21047,"cache_write":0.0,"input":0.23084,"output":0.36666},"cost_usd_as_run":0.808,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":31.55,"tokens":{"cache_read":9145472,"cache_write":0,"output":129199,"reasoning":109368,"uncached_input":99266},"cost_usd":0.1198,"cost_breakdown":{"cache_read":0.027436,"cache_write":0.0,"input":0.01489,"output":0.077519},"cost_usd_as_run":0.1198,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"H","score":4.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"M","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"H","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"M","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"H","score":5.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"M","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"H","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"M","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"H","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"M","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":9.0}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","opus-5.5-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"N","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":13.18,"tokens":{"uncached_input":136321,"cache_read":1607698,"cache_write":0,"output":72034,"reasoning":35853},"cost_usd":1.1538,"cost_breakdown":{"input":0.272642,"cache_read":0.16077,"cache_write":0.0,"output":0.72034},"cost_usd_as_run":1.1538,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":571,"code":469,"comment":12,"blank":90,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":628,"code":476,"comment":14,"blank":138,"role":"test"}],"impl_lines":571,"impl_code":469,"impl_comment":12,"test_lines":628,"comment_ratio":0.025,"tests_declared":22},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":448.077,"reference":426.553,"speedup":0.952,"baselines":{"oracle":426.553}},{"id":"throughput_64m","candidate":2596.491,"reference":2801.164,"speedup":0.927,"baselines":{"oracle":2801.164}}]},"isolation":{"tool_calls":31,"flags":18,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":5.5,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":3.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Several valid option combinations and deadline measurements produce results that differ from the oracle.","muse-wonder-full-xhigh":"Core runner mechanisms all work, but duplicate-option semantics diverge from the oracle.","opus-5.5-xhigh":"Core behaviour correct; option semantics diverge from the reference, code is verbose and defensive, design is undocumented","gpt-6.1-sol-max":"Multiple verified deviations affect valid calls and deadline enforcement.","deepseek-v4.1-flash-max":"Strong tests and no crashes, but it rejects atom pool names (`true`/`false`/`nil`) the oracle accepts, and its option parser keeps the last duplicate."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Boolean pool names are rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline starts after validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Relative executable paths change the tool’s observed path","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests encode incorrect pool-name outcomes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate options are last-wins","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil/true/false` rejected as invalid","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Suite locks in the wrong name behaviour","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No internal comments","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Duplicated validation and blanket try/catch layering","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `{:ok, pid, _info}` clause","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"With duplicate option keys the last one wins; in the reference the first does","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Pool names `nil`, `true` and `false` are rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"Backstop capped at 1000 days","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Layered defensive code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The design is undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Fragile or masked assertions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Duplicate options use the last value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Deadline starts after validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"Valid atom pool names are rejected","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate options: last value wins","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The flood-cleanup assertion can be silently skipped","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Runner invariants undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"The 1000-day backstop cap can turn a live huge-deadline run into a normal 137 exit","status":"valid","detail":"","location":""}],"score":5.9,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.9,"score_cross_vendor":5.9,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","task":"bounded-tool","lead":"muse-wonder-full-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":144.56,"tokens":{"cache_read":11696404,"cache_write":0,"output":140659,"reasoning":84039,"uncached_input":629187},"cost_usd":7.6693,"cost_breakdown":{"cache_read":2.339281,"cache_write":0.0,"input":2.516748,"output":2.81318},"cost_usd_as_run":7.6693,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":4}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":50.78,"tokens":{"cache_read":22979840,"cache_write":0,"output":121417,"reasoning":66900,"uncached_input":486905},"cost_usd":2.2429,"cost_breakdown":{"cache_read":1.148992,"cache_write":0.0,"input":0.486905,"output":0.607085},"cost_usd_as_run":2.2429,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":55.42,"tokens":{"cache_read":25122084,"cache_write":866321,"output":235050,"reasoning":163110,"uncached_input":314},"cost_usd":16.6572,"cost_breakdown":{"cache_read":5.024417,"cache_write":6.930568,"input":0.001256,"output":4.701},"cost_usd_as_run":16.6572,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":4}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":68.01,"tokens":{"cache_read":54559616,"cache_write":0,"output":458251,"reasoning":355402,"uncached_input":503075},"cost_usd":0.5141,"cost_breakdown":{"cache_read":0.163678,"cache_write":0.0,"input":0.075461,"output":0.274951},"cost_usd_as_run":0.5141,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":4}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":75.46,"tokens":{"cache_read":14678912,"cache_write":0,"output":149392,"reasoning":99357,"uncached_input":497668},"cost_usd":3.9572,"cost_breakdown":{"cache_read":1.467892,"cache_write":0.0,"input":0.995336,"output":1.49392},"cost_usd_as_run":3.9572,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"Z","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"N","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"Z","score":4.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"N","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"Z","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"N","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"Z","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"N","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"Z","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"N","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfK","source_run":"2026-11-02-bounded-tool-muse-wonder-full-xhigh-r1","letter":"E","score":10.0}],"panel":["muse-wonder-full-xhigh","gpt-6.1-terra-max","opus-5.5-xhigh","deepseek-v4.1-flash-max","gpt-6.1-sol-max"]},"entries":[{"letter":"A","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":14.47,"tokens":{"uncached_input":30088,"cache_read":5416576,"cache_write":0,"output":111242,"reasoning":89025},"cost_usd":0.0875,"cost_breakdown":{"input":0.004513,"cache_read":0.01625,"cache_write":0.0,"output":0.066745},"cost_usd_as_run":0.0875,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0875,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":560,"code":426,"comment":38,"blank":96,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":496,"code":387,"comment":1,"blank":108,"role":"test"}],"impl_lines":560,"impl_code":426,"impl_comment":38,"test_lines":496,"comment_ratio":0.082,"tests_declared":35},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":487.392,"reference":460.065,"speedup":0.944,"baselines":{"oracle":460.065}},{"id":"throughput_64m","candidate":2575.414,"reference":2609.142,"speedup":0.987,"baselines":{"oracle":2609.142}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":7.0},"council_summaries":{"opus-5.5-xhigh":"Solid structure with the `Proc` helper well separated; duplicate-option gap, odd dead clauses, test gaps","gpt-6.1-terra-max":"Sound runner structure, but duplicate validation and test coverage need work","muse-wonder-full-xhigh":"well-tested, duplicate-option hole and redundant final kill","deepseek-v4.1-flash-max":"excellent structure and tests; a duplicate-option validation gap and small redundancies.","gpt-6.1-sol-max":"Broad functionality, but validation and one test case are incomplete."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Only the first occurrence of each option is validated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable clauses","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Weak coverage of shutdown and crash paths","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Invalid duplicate values are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"An arbitrary-input case is silently skipped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM death and 64 MiB are untested","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Shadowed duplicate options never validated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Group kill attempted twice per run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys validated by key only","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The large-budget test uses 8 MiB and there is no BEAM-SIGKILL test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`opt/3` re-implements `Keyword.get/3`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Later invalid options are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Arbitrary-input test silently skips its options case","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":375,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":56,"flags":1,"audited":"2026-11-04","reviewed":true},"score":7.6,"slop":{"score":0.3,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":3.1,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":38.35,"tokens":{"uncached_input":82018,"cache_read":1568908,"cache_write":0,"output":33706,"reasoning":15935},"cost_usd":1.316,"cost_breakdown":{"input":0.328072,"cache_read":0.313782,"cache_write":0.0,"output":0.67412},"cost_usd_as_run":1.316,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":631,"code":500,"comment":25,"blank":106,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":471,"code":345,"comment":25,"blank":101,"role":"test"}],"impl_lines":631,"impl_code":500,"impl_comment":25,"test_lines":471,"comment_ratio":0.048,"tests_declared":28},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":468.888,"reference":446.981,"speedup":0.953,"baselines":{"oracle":446.981}},{"id":"throughput_64m","candidate":2570.865,"reference":2710.494,"speedup":0.948,"baselines":{"oracle":2710.494}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":6.0},"council_summaries":{"opus-5.5-xhigh":"Reference-equivalent core, but it accepts invalid duplicate options and has a lot of defensive padding","gpt-6.1-terra-max":"Good execution coverage, but validation and deadline accounting diverge","muse-wonder-full-xhigh":"correct core buried under duplicate-option hole and heavy layering","deepseek-v4.1-flash-max":"complete and correct; only a duplicate-option validation gap, a duplicated branch and an unexplained relay script.","gpt-6.1-sol-max":"Deadline accounting and duplicate-option validation need correction."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Only the first occurrence of each option is validated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Stacked defensive layers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Clauses that can never match","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Gaps in the test file","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later invalid duplicate options are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline starts after validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Backstop and large-budget cases are absent","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Runner result handling is duplicated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Shadowed duplicate options never validated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Same architecture as the 388-line reference in 631 lines","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"`terminate/2` re-kills an already-killed group","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys are validated by key only","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No 64 MiB budget and no BEAM-SIGKILL test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"The relay script that the whole design rests on is uncommented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`start_and_await/3` has two identical branches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Deadline starts after validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Invalid duplicate options execute commands","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":19,"flags":1,"audited":"2026-11-04","reviewed":true},"score":7.2,"slop":{"score":0.8,"per_seat":{"muse-wonder-full-xhigh":1.5,"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":4.3,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":32.64,"tokens":{"uncached_input":252,"cache_read":3845776,"cache_write":151625,"output":106342,"reasoning":0},"cost_usd":0.1419,"cost_breakdown":{"input":3.8e-05,"cache_read":0.061532,"cache_write":0.030325,"output":0.049981},"cost_usd_as_run":0.1419,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1419,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":511,"code":364,"comment":69,"blank":78,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":488,"code":348,"comment":27,"blank":113,"role":"test"}],"impl_lines":511,"impl_code":364,"impl_comment":69,"test_lines":488,"comment_ratio":0.159,"tests_declared":26},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":450.306,"reference":454.605,"speedup":1.01,"baselines":{"oracle":454.605}},{"id":"throughput_64m","candidate":2588.68,"reference":2457.312,"speedup":1.053,"baselines":{"oracle":2457.312}}]},"scores":{"opus-5.5-xhigh":6.5,"gpt-6.1-terra-max":4.0,"muse-wonder-full-xhigh":6.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":4.0},"council_summaries":{"opus-5.5-xhigh":"The runner doesn't trap exits, so tools outlive a pool shutdown; three small contract divergences","gpt-6.1-terra-max":"Pool shutdown can leave tools running, alongside several contract mismatches","muse-wonder-full-xhigh":"polished surface, tool leaks on pool shutdown and PATH lookup diverges","deepseek-v4.1-flash-max":"excellent docs and tests, but stopping the pool leaks the running tool.","gpt-6.1-sol-max":"Multiple deadline, cleanup and resolution defects remain despite passing tests."},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"The runner never traps exits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil` is rejected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Empty PATH entries are dropped","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"correctness","title":"Deadline taken late","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Pool shutdown leaves the tool alive","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Recycled identity crashes cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline excludes validation and resolution","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Empty PATH components are discarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"`name: nil` has a different outcome","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Raw-byte PATH lookup differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Pool shutdown cleanup is not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Binary pattern claim is false","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"robustness","title":"Runner never traps exits, so pool shutdown leaks the tool","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Empty PATH entries dropped","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil` rejected as `:invalid_argument`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"The runner never sets `:trap_exit`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`name: nil` is rejected as invalid","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Resolution and validation extend the deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Pool shutdown skips group cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Recycled identity crashes deadline handling","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Custom PATH search differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"correctness","title":"Valid atom name gets the wrong error","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"List-pattern explanation is false","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":46,"flags":2,"audited":"2026-11-04","reviewed":true},"score":5.6,"slop":{"score":0.5,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0}},"defect_points":7.8,"score_cross_vendor":5.6,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":26.04,"tokens":{"uncached_input":44,"cache_read":1797201,"cache_write":90562,"output":52598,"reasoning":30587},"cost_usd":2.1361,"cost_breakdown":{"input":0.000176,"cache_read":0.35944,"cache_write":0.724496,"output":1.05196},"cost_usd_as_run":2.1361,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":345,"code":249,"comment":40,"blank":56,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":628,"code":509,"comment":4,"blank":115,"role":"test"}],"impl_lines":345,"impl_code":249,"impl_comment":40,"test_lines":628,"comment_ratio":0.138,"tests_declared":39},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":462.577,"reference":444.569,"speedup":0.961,"baselines":{"oracle":444.569}},{"id":"throughput_64m","candidate":2576.564,"reference":2721.867,"speedup":0.947,"baselines":{"oracle":2721.867}}]},"scores":{"opus-5.5-xhigh":9.5,"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":9.0},"council_summaries":{"opus-5.5-xhigh":"Clean, accurate and fully tested","gpt-6.1-terra-max":"Strong implementation and coverage, with two oracle/test edge differences","muse-wonder-full-xhigh":"correct across all probes, one flaky own test","deepseek-v4.1-flash-max":"outstanding implementation and tests except one wrong error code when the pool stalls.","gpt-6.1-sol-max":"Strong bounds, cleanup and tests with one narrow resolution mismatch."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Raw-byte bare names differ from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death test requires unnecessary survival","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Own suite flaked once under load","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"`await/3` clamps the receive timeout to 0","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Manual PATH lookup changes binary-name behavior","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":220,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":22,"flags":4,"audited":"2026-11-04","reviewed":true},"score":8.8,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.4,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":20.66,"tokens":{"uncached_input":56,"cache_read":3331054,"cache_write":152040,"output":107792,"reasoning":77953},"cost_usd":4.0386,"cost_breakdown":{"input":0.000224,"cache_read":0.666211,"cache_write":1.21632,"output":2.15584},"cost_usd_as_run":4.0386,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":385,"code":267,"comment":52,"blank":66,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":758,"code":610,"comment":15,"blank":133,"role":"test"}],"impl_lines":385,"impl_code":267,"impl_comment":52,"test_lines":758,"comment_ratio":0.163,"tests_declared":44},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":454.964,"reference":441.882,"speedup":0.971,"baselines":{"oracle":441.882}},{"id":"throughput_64m","candidate":2674.423,"reference":2650.593,"speedup":1.009,"baselines":{"oracle":2650.593}}]},"scores":{"opus-5.5-xhigh":9.5,"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":9.5,"deepseek-v4.1-flash-max":9.5,"gpt-6.1-sol-max":9.0},"council_summaries":{"opus-5.5-xhigh":"Exemplary: accurate design docs and the most thorough tests","gpt-6.1-terra-max":"Well-covered lifecycle design, with two oracle/test edge differences","muse-wonder-full-xhigh":"matches the reference everywhere probed, strongest test suite","deepseek-v4.1-flash-max":"nothing to fix; correct, fast, safe, the best documented and best tested entry in the set.","gpt-6.1-sol-max":"Thorough lifecycle handling and tests, with one narrow oracle mismatch."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Raw-byte bare names differ from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death test requires unnecessary survival","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Hand-rolled PATH search duplicates `System.find_executable/1`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Bare binary names bypass oracle resolution","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":243,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":37,"flags":2,"audited":"2026-11-04","reviewed":true},"score":9.2,"slop":{"score":0.1,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":0.9,"score_cross_vendor":9.125,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.96,"tokens":{"uncached_input":13739,"cache_read":539008,"cache_write":0,"output":19201,"reasoning":9148},"cost_usd":1.6364,"cost_breakdown":{"input":0.13739,"cache_read":0.539008,"cache_write":0.0,"output":0.96005},"cost_usd_as_run":1.6364,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":311,"code":248,"comment":11,"blank":52,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":508,"code":422,"comment":0,"blank":86,"role":"test"}],"impl_lines":311,"impl_code":248,"impl_comment":11,"test_lines":508,"comment_ratio":0.042,"tests_declared":21},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":470.106,"reference":442.371,"speedup":0.941,"baselines":{"oracle":442.371}},{"id":"throughput_64m","candidate":2676.218,"reference":2649.64,"speedup":1.01,"baselines":{"oracle":2649.64}}]},"scores":{"opus-5.5-xhigh":9.0,"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.0},"council_summaries":{"opus-5.5-xhigh":"Correct, with excellent tests (pool-latency deadline, huge timeouts, VM SIGKILL, runner crash); docs are thin","gpt-6.1-terra-max":"Strong lifecycle tests, with an output/deadline ordering defect","muse-wonder-full-xhigh":"correct across all probes, minor doc gaps and one unwrapped cleanup call","deepseek-v4.1-flash-max":"correct and thoroughly tested, but a cleanup race can discard a valid result, and two public functions are undocumented.","gpt-6.1-sol-max":"Strong lifecycle coverage with an error-precedence mismatch."},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Missing interface docs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Earlier overflow can be reported as timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Pool functions and option defaults lack public docs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No 64 MiB budget case","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Slot release can replace a successful result","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"`child_spec/1` and `start_link/1` have no `@doc`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`DynamicSupervisor.terminate_child/2` inside `after` without a catch","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`child_spec/1` and `start_link/1` have only `@spec`, no `@doc`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No 64 MiB-budget test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline overrides an already-produced overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public options and defaults are undocumented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":214,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":14,"flags":1,"audited":"2026-11-04","reviewed":true},"score":8.2,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.5,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.16,"tokens":{"uncached_input":7782,"cache_read":184832,"cache_write":0,"output":4972,"reasoning":410},"cost_usd":0.0419,"cost_breakdown":{"input":0.007782,"cache_read":0.009242,"cache_write":0.0,"output":0.02486},"cost_usd_as_run":0.0419,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":259,"code":220,"comment":0,"blank":39,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":108,"code":88,"comment":0,"blank":20,"role":"test"}],"impl_lines":259,"impl_code":220,"impl_comment":0,"test_lines":108,"comment_ratio":0.0,"tests_declared":5},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":452.195,"reference":446.209,"speedup":0.987,"baselines":{"oracle":446.209}},{"id":"throughput_64m","candidate":2689.437,"reference":2658.621,"speedup":1.012,"baselines":{"oracle":2658.621}}]},"scores":{"opus-5.5-xhigh":6.0,"gpt-6.1-terra-max":6.0,"muse-wonder-full-xhigh":5.5,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":4.0},"council_summaries":{"opus-5.5-xhigh":"Minimal clone: path check/use mismatch, duplicate-option gap, no docs, very thin tests","gpt-6.1-terra-max":"Several valid-input and initialization edge cases need correction","muse-wonder-full-xhigh":"wrong result on `~` paths, duplicate-option hole, no docs, thinnest tests","deepseek-v4.1-flash-max":"behaviourally sound, but the public module is undocumented and the tests are very thin.","gpt-6.1-sol-max":"Can execute the wrong program and mishandles validation and fast exits."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"The path that is checked is not the path that runs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Only the first occurrence of each option is validated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The public module is hidden from docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Five tests, with gaps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Slash paths are changed before execution","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later invalid duplicate options are accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Fast port exit can fail initialization","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public interface is undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Important lifecycle and size cases are absent","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Stats the literal path but executes the `~`-expanded one","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Shadowed duplicate options never validated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Five mega-tests leave contract areas unexercised","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"`@moduledoc false` and no docs anywhere","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"5 tests / 108 lines","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys validated by key only","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`Integer.parse(Enum.at(fields, 2))`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`else: (arm(s.deadline); {:noreply, s})`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path normalization changes the executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Successful fast commands become unavailable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Invalid duplicate options launch commands","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Major failure modes lack coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is absent","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":183,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-04","reviewed":false},"score":5.6,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":7.6,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.76,"tokens":{"uncached_input":17630,"cache_read":125568,"cache_write":0,"output":5301,"reasoning":466},"cost_usd":0.1008,"cost_breakdown":{"input":0.03526,"cache_read":0.012557,"cache_write":0.0,"output":0.05301},"cost_usd_as_run":0.1008,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":264,"code":225,"comment":1,"blank":38,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":157,"code":135,"comment":0,"blank":22,"role":"test"}],"impl_lines":264,"impl_code":225,"impl_comment":1,"test_lines":157,"comment_ratio":0.004,"tests_declared":8},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":473.608,"reference":443.213,"speedup":0.936,"baselines":{"oracle":443.213}},{"id":"throughput_64m","candidate":2595.659,"reference":2586.162,"speedup":1.004,"baselines":{"oracle":2586.162}}]},"scores":{"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":5.5,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":5.0},"council_summaries":{"opus-5.5-xhigh":"Compact, correct core; slash paths go through `Path.expand`, there are no public docs, and the tests are thin","gpt-6.1-terra-max":"Path handling, event ordering, and a failing own test require changes","muse-wonder-full-xhigh":"terse and mostly correct, `~` paths wrong and no API docs","deepseek-v4.1-flash-max":"correct core, but it expands `~` in slash paths, ships no API docs, and has the thinnest tests of the working entries.","gpt-6.1-sol-max":"Executable-path and startup defects affect valid commands."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Slash paths are not \"used as given\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No public docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin test file","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Slash paths are normalized incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Fast port exit can fail initialization","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Earlier overflow can be reported as timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Resolution test assumes a checkout depth","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions and reason type lack docs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Slash paths get `~` expansion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No `@doc`/`@typedoc` on the public API","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinnest coverage after entry-J","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`~` is expanded for slash paths","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"No documentation on the public API","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"8 tests / 157 lines","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`seconds = Integer.to_string(remaining + 250) <> \"e-3s\"`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`{:os_pid, pid} = Port.info(port, :os_pid)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Expanded paths can select a different program","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Fast port exit crashes initialization","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline masks queued output overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Abandonment test can pass without abandonment cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public functions lack usage documentation","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":190,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.3,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":6.6,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":94.09,"tokens":{"uncached_input":123745,"cache_read":3409212,"cache_write":0,"output":59899,"reasoning":38947},"cost_usd":2.3748,"cost_breakdown":{"input":0.49498,"cache_read":0.681842,"cache_write":0.0,"output":1.19798},"cost_usd_as_run":2.3748,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":594,"code":464,"comment":30,"blank":100,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":379,"code":298,"comment":4,"blank":77,"role":"test"}],"impl_lines":594,"impl_code":464,"impl_comment":30,"test_lines":379,"comment_ratio":0.061,"tests_declared":24},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":443.817,"reference":444.029,"speedup":1.0,"baselines":{"oracle":444.029}},{"id":"throughput_64m","candidate":2526.272,"reference":2671.11,"speedup":0.946,"baselines":{"oracle":2671.11}}]},"scores":{"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":7.0},"council_summaries":{"opus-5.5-xhigh":"Correct core buried in padding and dead code; duplicate-option gap","gpt-6.1-terra-max":"Core execution works, but invalid options and unrelated mailbox messages are mishandled","muse-wonder-full-xhigh":"sound runner, duplicate options slip past validation","deepseek-v4.1-flash-max":"solid; one validation gap, one no-op helper, and the weakest big-output coverage.","gpt-6.1-sol-max":"Generally reliable runner with incomplete validation and redundant cleanup."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Only the first occurrence of each option is validated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Code that never runs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Defensive padding everywhere","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Coverage gaps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Invalid duplicate values are discarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Cleanup consumes an unrelated mailbox message","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Backstop and large output lack coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Shadowed duplicate options never validated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Dead `flush_down/1` and unused `state.pgid`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys: the later value is never validated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"`flush_down/1` can never match","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No successful 64 MiB run and no BEAM-SIGKILL test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Exec bit via `rem(mode, 2)`, `rem(div(mode, 8), 2)`, `rem(div(mode, 64), 2)`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`System.find_executable(\"kill\")` on every group kill","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Duplicate values are discarded before validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Caller flushes a monitor it never installs","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":36,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.3,"slop":{"score":0.9,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":2.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0}},"defect_points":3.9,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":8.22,"tokens":{"uncached_input":35655,"cache_read":1267584,"cache_write":0,"output":29862,"reasoning":18691},"cost_usd":0.2483,"cost_breakdown":{"input":0.035655,"cache_read":0.063379,"cache_write":0.0,"output":0.14931},"cost_usd_as_run":0.2483,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":337,"code":273,"comment":12,"blank":52,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":378,"code":322,"comment":0,"blank":56,"role":"test"}],"impl_lines":337,"impl_code":273,"impl_comment":12,"test_lines":378,"comment_ratio":0.042,"tests_declared":17},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":471.308,"reference":450.101,"speedup":0.955,"baselines":{"oracle":450.101}},{"id":"throughput_64m","candidate":2606.699,"reference":2629.042,"speedup":0.992,"baselines":{"oracle":2629.042}}]},"scores":{"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":8.0},"council_summaries":{"opus-5.5-xhigh":"Correct, with strong tests (VM SIGKILL, crash injection, duplicate keys); docs are terse","gpt-6.1-terra-max":"Strong coverage, with an output/deadline ordering defect","muse-wonder-full-xhigh":"correct across all probes, slightly truncated `run/3` doc","deepseek-v4.1-flash-max":"no behavioural defect found; clean and well-tested, with only documentation and coverage gaps.","gpt-6.1-sol-max":"Good cleanup and tests with a scheduling-dependent result mismatch."},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Docs drop the options","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unused field","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Queued overflow can become a timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public option defaults are omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No 64 MiB budget test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"`run/3` doc omits the options contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No 64 MiB-success test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"The relay script is undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Queued overflow is reported as timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public option behavior is missing from docs","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":233,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":28,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.4,"slop":{"score":0.1,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.1,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":34.24,"tokens":{"uncached_input":172473,"cache_read":2079104,"cache_write":0,"output":89501,"reasoning":76308},"cost_usd":0.133,"cost_breakdown":{"input":0.025871,"cache_read":0.062373,"cache_write":0.0,"output":0.04475},"cost_usd_as_run":0.133,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":572,"code":417,"comment":61,"blank":94,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":409,"code":303,"comment":12,"blank":94,"role":"test"}],"impl_lines":572,"impl_code":417,"impl_comment":61,"test_lines":409,"comment_ratio":0.128,"tests_declared":21},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":478.784,"reference":459.755,"speedup":0.96,"baselines":{"oracle":459.755}},{"id":"throughput_64m","candidate":2646.938,"reference":2617.692,"speedup":1.011,"baselines":{"oracle":2617.692}}]},"scores":{"muse-wonder-full-xhigh":8.0,"gpt-6.1-terra-max":6.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":6.0},"council_summaries":{"muse-wonder-full-xhigh":"Behaviourally correct on all probes, but bloated and missing cleanup on the fault path","gpt-6.1-terra-max":"Normal behavior is solid, but cleanup can crash and its deadline excludes validation time.","deepseek-v4.1-flash-max":"oracle-exact behaviour and good docs, but it leaks `kill` diagnostics to the host's stderr and releases its pool slot outside an `after`.","opus-5.5-xhigh":"Matches the reference on every probe, but carries a lot of dead and duplicated defensive code, and its tests skip the VM-SIGKILL and 64 MiB cases","gpt-6.1-sol-max":"A caller-death race permanently consumes pool capacity; validation also extends deadlines."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Slot leak when the await call exits","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No test for the VM-SIGKILL backstop (§6), runner crash, or 64 MiB output","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"`kill` stderr leaks to the BEAM's stderr","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead and duplicated code","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Missing `@typedoc` for `reason`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"A stale signal-helper path crashes the runner","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline starts after validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death and large-output guarantees lack tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public reason type lacks its description","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Unreachable retry branch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"A late status can be the tool’s own","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Duplicate exception handling","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`lib/bounded_tool.ex:569`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`lib/bounded_tool.ex:184-190`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"`test/bounded_tool_test.exs`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`lib/bounded_tool.ex:69-72`, `:196-203`, `:483-489`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`lib/bounded_tool.ex:13`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No VM-SIGKILL, 64 MiB, runner-crash or pool-shutdown tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The relay's mechanism is unexplained","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"At/after the deadline the status comes from our kill or the backstop, never from the tool itself\" is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The `{port, :closed}` clauses and the os_pid retry loop never do anything","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Overlapping defensive code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The test helper `beam_os_pid` is not the BEAM's pid","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The tilde test can't detect expansion","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Cached results suppress caller-death cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Deadline begins after validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Important execution limits lack regression coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"OS-PID fallback is unreachable","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":25,"flags":1,"audited":"2026-11-04","reviewed":false},"score":7.2,"slop":{"score":1.0,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":2.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.5}},"defect_points":5.4,"score_cross_vendor":7.2,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.09,"tokens":{"uncached_input":30513,"cache_read":288896,"cache_write":0,"output":10544,"reasoning":2722},"cost_usd":0.1954,"cost_breakdown":{"input":0.061026,"cache_read":0.02889,"cache_write":0.0,"output":0.10544},"cost_usd_as_run":0.1954,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":298,"code":245,"comment":8,"blank":45,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":296,"code":248,"comment":1,"blank":47,"role":"test"}],"impl_lines":298,"impl_code":245,"impl_comment":8,"test_lines":296,"comment_ratio":0.032,"tests_declared":15},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":473.909,"reference":443.202,"speedup":0.935,"baselines":{"oracle":443.202}},{"id":"throughput_64m","candidate":2707.86,"reference":2703.731,"speedup":1.002,"baselines":{"oracle":2703.731}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":6.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"opus-5.5-xhigh":"Correct and well tested; the public API has no docs.","gpt-6.1-terra-max":"Startup and overflow ordering races return the wrong outcome","gpt-6.1-sol-max":"Correct in the probes, with missing public API documentation.","muse-wonder-full-xhigh":"Clean and well tested; only docs and a narrow port-info race detract","deepseek-v4.1-flash-max":"Correct, compact and very well tested (including a real separate-VM SIGKILL test), but ships no public API documentation."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc` or `@typedoc` on the public API","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"GNU timeout creates the session/group itself\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Closed-port startup race","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline check overrides an earlier overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Options and outcomes are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No @doc on any public function","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unconditional os_pid match can crash the runner","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Public API undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No 64 MiB budget test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":206,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":1,"audited":"2026-11-04","reviewed":true},"score":7.6,"slop":{"score":0.1,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.4,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":46.44,"tokens":{"uncached_input":162508,"cache_read":5339076,"cache_write":0,"output":58387,"reasoning":35091},"cost_usd":2.8856,"cost_breakdown":{"input":0.650032,"cache_read":1.067815,"cache_write":0.0,"output":1.16774},"cost_usd_as_run":2.8856,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":585,"code":463,"comment":22,"blank":100,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":501,"code":370,"comment":12,"blank":119,"role":"test"}],"impl_lines":585,"impl_code":463,"impl_comment":22,"test_lines":501,"comment_ratio":0.045,"tests_declared":26},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":478.149,"reference":456.844,"speedup":0.955,"baselines":{"oracle":456.844}},{"id":"throughput_64m","candidate":2579.863,"reference":2563.277,"speedup":1.006,"baselines":{"oracle":2563.277}}]},"scores":{"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"opus-5.5-xhigh":"Correct core buried in defensive padding; duplicated options are not validated.","gpt-6.1-terra-max":"Invalid duplicate values bypass validation","gpt-6.1-sol-max":"Reliable execution and cleanup, with incomplete validation of duplicates.","muse-wonder-full-xhigh":"Broad tests, but duplicate options escape validation and there is defensive dead weight","deepseek-v4.1-flash-max":"Good docs and 26 tests; 585 lines with three branches that cannot be reached and one validation edge."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Duplicated options pass validation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"`rescue _ ->` plus `catch _, _ ->` pairs and nested try layers throughout","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"The relay script has no comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test for VM SIGKILL, runner crash or 64 MiB","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Shadowed invalid options are accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Backstop and large-budget cases are absent","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Initial group field is unused","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Later invalid option values are unchecked","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate option keys escape validation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No VM-death backstop test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Duplicate start branch plus result re-validation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Second terminate/2 clause is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys with an invalid later value are accepted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Three unreachable `if state.finished` arms","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":56,"flags":1,"audited":"2026-11-04","reviewed":true},"score":7.3,"slop":{"score":0.7,"per_seat":{"muse-wonder-full-xhigh":1.5,"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":3.2,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":34.26,"tokens":{"uncached_input":148,"cache_read":18636842,"cache_write":383313,"output":331122,"reasoning":275010},"cost_usd":1.8787,"cost_breakdown":{"input":7e-05,"cache_read":0.926038,"cache_write":0.354207,"output":0.598389},"cost_usd_as_run":1.8787,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":488,"code":345,"comment":67,"blank":76,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":617,"code":465,"comment":24,"blank":128,"role":"test"}],"impl_lines":488,"impl_code":345,"impl_comment":67,"test_lines":617,"comment_ratio":0.163,"tests_declared":38},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":456.051,"reference":446.671,"speedup":0.979,"baselines":{"oracle":446.671}},{"id":"throughput_64m","candidate":2696.435,"reference":2703.931,"speedup":0.997,"baselines":{"oracle":2703.931}}]},"scores":{"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"opus-5.5-xhigh":"Very strong tests (VM SIGKILL, 64 MiB, rendezvous-based busy check) and good docs; one validation gap.","gpt-6.1-terra-max":"Invalid duplicate values and validation time escape the contract","gpt-6.1-sol-max":"Good lifecycle design, undermined by deadline accounting and duplicate validation.","muse-wonder-full-xhigh":"Best test suite and docs in the set; duplicate options escape validation","deepseek-v4.1-flash-max":"The deepest test suite (38, incl. a real `beam.smp` SIGKILL check) and thorough docs; a duplicated PATH scan, one wrong comment and one validation edge."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Only the first occurrence of a duplicated option is validated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"about 31 years\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Shadowed invalid options are accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline starts after validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Process-group explanation is false","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Validation time is excluded from the deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate options still execute","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Backstop duration is miscalculated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate option keys escape validation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Hand-rolled PATH search reimplements System.find_executable/1","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys with an invalid later value are accepted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Hand-rolled PATH lookup","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"lib/bounded_tool.ex:232","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":298,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":69,"flags":2,"audited":"2026-11-04","reviewed":true},"score":7.4,"slop":{"score":0.8,"per_seat":{"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.5}},"defect_points":3.4,"score_cross_vendor":7.125,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.35,"tokens":{"uncached_input":14,"cache_read":358061,"cache_write":64143,"output":32153,"reasoning":17517},"cost_usd":1.2279,"cost_breakdown":{"input":5.6e-05,"cache_read":0.071612,"cache_write":0.513144,"output":0.64306},"cost_usd_as_run":1.2279,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":391,"code":289,"comment":39,"blank":63,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":508,"code":405,"comment":9,"blank":94,"role":"test"}],"impl_lines":391,"impl_code":289,"impl_comment":39,"test_lines":508,"comment_ratio":0.119,"tests_declared":28},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":467.009,"reference":451.858,"speedup":0.968,"baselines":{"oracle":451.858}},{"id":"throughput_64m","candidate":2634.872,"reference":2566.55,"speedup":1.027,"baselines":{"oracle":2566.55}}]},"scores":{"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"opus-5.5-xhigh":"Excellent moduledoc and tests (VM SIGKILL, check for leftover OS children); duplicated options are not validated.","gpt-6.1-terra-max":"Validation accepts invalid duplicate values, and a concurrency test is racy","gpt-6.1-sol-max":"Strong core implementation, with duplicate validation and a test race to fix.","muse-wonder-full-xhigh":"Excellent docs and tests; duplicate options escape validation","deepseek-v4.1-flash-max":"Excellent docs and the deepest structural tests (28 + a separate leak module, real VM SIGKILL); a hand-rolled PATH lookup and one validation edge."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"The first occurrence of an option wins and later values go unchecked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Shadowed invalid options are accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Concurrent test assumes which task wins","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Option accumulation discards invalid duplicates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Concurrency assertion assumes an unguaranteed winner","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate option keys escape validation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Hand-rolled PATH search reimplements System.find_executable/1","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys with an invalid later value are accepted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Hand-rolled PATH scan","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"Largest budget case is 16 MiB","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":264,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":6,"flags":1,"audited":"2026-11-04","reviewed":true},"score":7.7,"slop":{"score":0.4,"per_seat":{"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.7,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":13.67,"tokens":{"uncached_input":21326,"cache_read":528128,"cache_write":0,"output":27254,"reasoning":17188},"cost_usd":0.368,"cost_breakdown":{"input":0.042652,"cache_read":0.052813,"cache_write":0.0,"output":0.27254},"cost_usd_as_run":0.368,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":333,"code":267,"comment":14,"blank":52,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":439,"code":359,"comment":0,"blank":80,"role":"test"}],"impl_lines":333,"impl_code":267,"impl_comment":14,"test_lines":439,"comment_ratio":0.05,"tests_declared":23},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":473.352,"reference":445.26,"speedup":0.941,"baselines":{"oracle":445.26}},{"id":"throughput_64m","candidate":2537.013,"reference":2626.673,"speedup":0.966,"baselines":{"oracle":2626.673}}]},"scores":{"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"opus-5.5-xhigh":"Matches the reference throughout, with strong tests (VM SIGKILL, runner crash, missing helpers).","gpt-6.1-terra-max":"A delayed runner can report timeout for an earlier overflow","gpt-6.1-sol-max":"All verified behavior matched, with broad adversarial coverage.","muse-wonder-full-xhigh":"Closest to merge-as-is: documented, correct on every probed edge, thoroughly tested","deepseek-v4.1-flash-max":"The cleanest deliverable: complete docs, 23 deep tests (incl. VM SIGKILL and helper-resolution), no dead code and no behavioural deviation I could find."},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`Process.demonitor(state.caller_monitor, [:flush])` in `terminate/2` does nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"The relay script has no comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline check overrides an earlier overflow","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Demonitor in terminate/2 is a no-op","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No 64 MiB budget test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":234,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":12,"flags":1,"audited":"2026-11-04","reviewed":true},"score":8.8,"slop":{"score":0.2,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":0.8,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.42,"tokens":{"uncached_input":11089,"cache_read":128768,"cache_write":0,"output":4277,"reasoning":65},"cost_usd":0.4535,"cost_breakdown":{"input":0.11089,"cache_read":0.128768,"cache_write":0.0,"output":0.21385},"cost_usd_as_run":0.4535,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":250,"code":215,"comment":0,"blank":35,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":122,"code":103,"comment":0,"blank":19,"role":"test"}],"impl_lines":250,"impl_code":215,"impl_comment":0,"test_lines":122,"comment_ratio":0.0,"tests_declared":6},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":452.204,"reference":445.675,"speedup":0.986,"baselines":{"oracle":445.675}},{"id":"throughput_64m","candidate":2609.623,"reference":2612.078,"speedup":0.999,"baselines":{"oracle":2612.078}}]},"scores":{"opus-5.5-xhigh":5.5,"gpt-6.1-terra-max":4.5,"gpt-6.1-sol-max":4.0,"muse-wonder-full-xhigh":6.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"opus-5.5-xhigh":"Path handling is wrong for `~` and `..`, there are no docs, and the tests are thin.","gpt-6.1-terra-max":"Path expansion, startup, and overflow races produce incorrect results","gpt-6.1-sol-max":"Executable resolution can select a different command.","muse-wonder-full-xhigh":"Sound engine but tilde expansion runs the wrong executable and there are no docs","deepseek-v4.1-flash-max":"Correct, terse 250-line runner with no documentation at all and a small test suite."},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"Slash paths go through `Path.expand` before the stat","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@moduledoc`, `@doc` or `@typedoc` anywhere, and `Runner` lacks `@moduledoc false`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Six tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Terse names (`s`, `c`), an `else: ( … )` block and no `@impl`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Tilde expansion changes slash paths","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Closed-port startup race","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline check overrides an earlier overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Lifecycle assertions do not establish cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API is undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Expands paths that must retain their meaning","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public interface is undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Tilde expansion on slash paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No VM-death, 64 MiB or runner-crash test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unconditional os_pid match can crash the runner","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Module and public API undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Thin edge coverage","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":177,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-04","reviewed":false},"score":5.5,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":5.7,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":8.13,"tokens":{"uncached_input":14879,"cache_read":447488,"cache_write":0,"output":17580,"reasoning":7793},"cost_usd":0.2503,"cost_breakdown":{"input":0.029758,"cache_read":0.044749,"cache_write":0.0,"output":0.1758},"cost_usd_as_run":0.2503,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":332,"code":276,"comment":5,"blank":51,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":341,"code":277,"comment":1,"blank":63,"role":"test"}],"impl_lines":332,"impl_code":276,"impl_comment":5,"test_lines":341,"comment_ratio":0.018,"tests_declared":17},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":473.333,"reference":443.847,"speedup":0.938,"baselines":{"oracle":443.847}},{"id":"throughput_64m","candidate":2593.736,"reference":2597.278,"speedup":0.999,"baselines":{"oracle":2597.278}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"opus-5.5-xhigh":"Correct and well tested (VM SIGKILL, runner crash); the public API has no docs.","gpt-6.1-terra-max":"Strong behavior and coverage, with missing API docs","gpt-6.1-sol-max":"Behavior matched the reference; public documentation is incomplete.","muse-wonder-full-xhigh":"Compact, correct and well tested; public functions undocumented","deepseek-v4.1-flash-max":"Lean, correct and very well tested (17, incl. VM SIGKILL and duplicate-key cases); documentation is a one-line moduledoc."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc` or `@typedoc` on the public `child_spec/1`, `start_link/1`, `run/3` or `reason`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The identity is re-read immediately after a matching read","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Captured initial group is unused","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Exported API lacks usage documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No @doc on any public function","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Process identity read twice before signalling","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Public API undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Identity carries an unused field","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No 64 MiB budget test.","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":228,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.2,"slop":{"score":0.4,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":1.5,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.99,"tokens":{"uncached_input":25134,"cache_read":210176,"cache_write":0,"output":7391,"reasoning":827},"cost_usd":0.8311,"cost_breakdown":{"input":0.25134,"cache_read":0.210176,"cache_write":0.0,"output":0.36955},"cost_usd_as_run":0.8311,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":285,"code":233,"comment":3,"blank":49,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":264,"code":223,"comment":0,"blank":41,"role":"test"}],"impl_lines":285,"impl_code":233,"impl_comment":3,"test_lines":264,"comment_ratio":0.013,"tests_declared":14},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":477.028,"reference":445.46,"speedup":0.934,"baselines":{"oracle":445.46}},{"id":"throughput_64m","candidate":2536.334,"reference":2613.222,"speedup":0.971,"baselines":{"oracle":2613.222}}]},"scores":{"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":4.5,"gpt-6.1-sol-max":4.0,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"opus-5.5-xhigh":"Compact and correct on the common paths; the public API has no docs, and it checks one path but runs another.","gpt-6.1-terra-max":"Path resolution and several scheduling races change results","gpt-6.1-sol-max":"Executes a normalized path different from the one checked.","muse-wonder-full-xhigh":"Near-reference implementation; only docs and a narrow port-info race detract","deepseek-v4.1-flash-max":"Correct and strongly tested (14 tests incl. a real BEAM SIGKILL), but the public API is undocumented and one cleanup call is not guarded."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"It stats the path as given but spawns `Path.expand(path)`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No `@doc` or `@typedoc` on the public `child_spec/1`, `start_link/1`, `run/3` or `reason`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Tilde expansion selects a different executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Closed-port startup race","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deadline check overrides an earlier overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Pool shutdown can overwrite a completed result","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Normalization changes the selected executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public options and defaults lack documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No @doc on any public function","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unconditional os_pid match can crash the runner","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"No `@typedoc`/`@doc`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"Unguarded cleanup in `after`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No 64 MiB budget test (budgets stop at 65,536 bytes).","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":198,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.3,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":4.2,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.8,"tokens":{"uncached_input":10274,"cache_read":126592,"cache_write":0,"output":3626,"reasoning":111},"cost_usd":0.0347,"cost_breakdown":{"input":0.010274,"cache_read":0.00633,"cache_write":0.0,"output":0.01813},"cost_usd_as_run":0.0347,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":235,"code":200,"comment":0,"blank":35,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":48,"code":38,"comment":0,"blank":10,"role":"test"}],"impl_lines":235,"impl_code":200,"impl_comment":0,"test_lines":48,"comment_ratio":0.0,"tests_declared":2},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":442.118,"reference":449.094,"speedup":1.016,"baselines":{"oracle":449.094}},{"id":"throughput_64m","candidate":2572.557,"reference":2643.832,"speedup":0.973,"baselines":{"oracle":2643.832}}]},"scores":{"opus-5.5-xhigh":4.5,"gpt-6.1-terra-max":4.0,"gpt-6.1-sol-max":3.5,"muse-wonder-full-xhigh":6.0,"deepseek-v4.1-flash-max":6.0},"council_summaries":{"opus-5.5-xhigh":"Wrong path resolution, no docs, unreadable relay and almost no tests.","gpt-6.1-terra-max":"Valid paths can run the wrong executable, alongside a startup race and incomplete validation","gpt-6.1-sol-max":"Path normalization can execute the wrong tool; validation and test coverage also need changes.","muse-wonder-full-xhigh":"Correct core engine, but two wrong-result validation/resolution bugs, no docs and thin tests","deepseek-v4.1-flash-max":"Correct and compact (235 lines), but it ships no documentation and only two tests, so most of the contract is unverified."},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"Slash paths go through `Path.expand`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Two tests in total","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Duplicated options pass validation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No moduledoc, `@doc` or `@typedoc` at all, and `Runner` lacks `@moduledoc false`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"The relay is one escaped single-line string (109), and `deadline` first holds the call time, then is rebound to call time + timeout (23, 33)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Tilde expansion changes slash paths","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Closed-port startup race","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Shadowed invalid options are accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Lifecycle and concurrency coverage is missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API is undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Changes executable identity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate options still spawn tools","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Lifecycle and concurrency regressions are unprotected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API lacks usage documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate option keys escape validation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Tilde expansion on slash paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests cover only a slice of the contract","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unconditional os_pid match can crash the runner","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Runner never stops itself and never closes its port","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"lib/bounded_tool.ex:1-2, 106","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"test/bounded_tool_test.exs:1-48","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys with an invalid later value are accepted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":166,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-04","reviewed":false},"score":4.8,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":8.2,"score_cross_vendor":5.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":55.93,"tokens":{"uncached_input":0,"cache_read":0,"cache_write":0,"output":0,"reasoning":0},"cost_usd":0.0,"cost_breakdown":{"input":0.0,"cache_read":0.0,"cache_write":0.0,"output":0.0},"cost_usd_as_run":0.0,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":null},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":583,"code":444,"comment":40,"blank":99,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":356,"code":282,"comment":2,"blank":72,"role":"test"}],"impl_lines":583,"impl_code":444,"impl_comment":40,"test_lines":356,"comment_ratio":0.083,"tests_declared":21},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":469.717,"reference":449.768,"speedup":0.958,"baselines":{"oracle":449.768}},{"id":"throughput_64m","candidate":2659.681,"reference":2514.685,"speedup":1.058,"baselines":{"oracle":2514.685}}]},"scores":{"opus-5.5-xhigh":8.0,"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"opus-5.5-xhigh":"Matches the reference on every probe; the validation is exact. Two comments are false and one clause is dead.","gpt-6.1-terra-max":"Solid implementation with gaps in its own lifecycle tests","gpt-6.1-sol-max":"All functional probes matched; one timer comment is false.","muse-wonder-full-xhigh":"Struct-based runner with substantive invariant comments; backstop-after-VM-death untested","deepseek-v4.1-flash-max":"Well documented and clean with 21 good tests; two comments state things that are not true and one EXIT clause is unreachable."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable `{:EXIT, pid, _}` clause documented as \"Supervisor shutdown\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"An absolute time already in the past makes `send_after/4` raise\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test for VM SIGKILL, runner crash (`:unavailable`) or the 64 MiB budget","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Backstop and post-exit output need direct coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Incorrect claim about past absolute timers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No VM-death or runner-crash test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable EXIT clause","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"lib/bounded_tool.ex:428","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No runner-crash or VM-SIGKILL test and no 64 MiB budget case.","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":27,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":32,"flags":1,"audited":"2026-11-04","reviewed":true},"score":8.6,"slop":{"score":0.6,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.5}},"defect_points":1.3,"score_cross_vendor":8.6,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.0,"tokens":{"uncached_input":15793,"cache_read":492032,"cache_write":0,"output":9996,"reasoning":2702},"cost_usd":0.0904,"cost_breakdown":{"input":0.015793,"cache_read":0.024602,"cache_write":0.0,"output":0.04998},"cost_usd_as_run":0.0904,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":267,"code":223,"comment":2,"blank":42,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":243,"code":203,"comment":0,"blank":40,"role":"test"}],"impl_lines":267,"impl_code":223,"impl_comment":2,"test_lines":243,"comment_ratio":0.009,"tests_declared":13},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":456.641,"reference":429.177,"speedup":0.94,"baselines":{"oracle":429.177}},{"id":"throughput_64m","candidate":2811.88,"reference":2825.521,"speedup":0.995,"baselines":{"oracle":2825.521}}]},"scores":{"muse-wonder-full-xhigh":6.5,"gpt-6.1-terra-max":5.5,"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":5.5,"gpt-6.1-sol-max":5.5},"council_summaries":{"muse-wonder-full-xhigh":"tightest code and the only VM-SIGKILL test, but two observable divergences from the oracle","gpt-6.1-terra-max":"Valid relative paths can resolve to the wrong executable","deepseek-v4.1-flash-max":"Terse and nearly behaviour-exact, but `~` paths are expanded (a real deviation), nothing is documented, and two spec-named cases are untested.","opus-5.5-xhigh":"Two wrong-output cases (path handling, the `os_pid` race), no public docs, thin tests","gpt-6.1-sol-max":"Reliable lifecycle handling, but path normalization can execute the wrong tool"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"`~` in a slash path is expanded to the home directory","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"duplicate option values never validated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"`Port.info` match can crash `init/1`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"the port is never explicitly closed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"public functions have no `@doc`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"port-EXIT clause duplicates the catch-all","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Path expansion changes slash-containing paths","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Shadowed invalid options are accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extremely large timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Large-output and PID-reuse coverage is missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public functions lack documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"lib/bounded_tool.ex:95","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"lib/bounded_tool.ex:3,8,17,26,107-113","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"test/bounded_tool_test.exs:73,125-131","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"lib/bounded_tool.ex:146","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"lib/bounded_tool.ex:3","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"lib/bounded_tool.ex:243-247","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"A path containing a slash is not \"used as given\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"The same `{:os_pid, pid} = Port.info(port, :os_pid)` crash as W","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No `@doc` on `child_spec/1`, `start_link/1` or `run/3`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Only the first copy of a repeated option key is validated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Some tests check less than they claim","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The `{:EXIT, port, _}` clause does the same as the catch-all below it","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Path expansion changes the executable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate options bypass validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public option documentation is missing","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":188,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":16,"flags":0,"audited":"2026-11-04","reviewed":false},"score":5.9,"slop":{"score":0.3,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":7.1,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1-b2","task":"bounded-tool","lead":"opus-5.5-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":6.76,"tokens":{"cache_read":2323328,"cache_write":0,"output":25345,"reasoning":16254,"uncached_input":85853},"cost_usd":0.3287,"cost_breakdown":{"cache_read":0.116166,"cache_write":0.0,"input":0.085853,"output":0.126725},"cost_usd_as_run":0.3287,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6-astra-xhigh","judge":"gpt-6-astra-xhigh","model":"gpt-6-astra","effort":"xhigh","label":"GPT‑6 Astra xhigh","usage":{"wall_minutes":9.25,"tokens":{"cache_read":1436672,"cache_write":0,"output":15757,"reasoning":7551,"uncached_input":70239},"cost_usd":2.9269,"cost_breakdown":{"cache_read":1.436672,"cache_write":0.0,"input":0.70239,"output":0.78785},"cost_usd_as_run":2.9269,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":10.13,"tokens":{"cache_read":2447086,"cache_write":137567,"output":38832,"reasoning":26970,"uncached_input":48},"cost_usd":2.3668,"cost_breakdown":{"cache_read":0.489417,"cache_write":1.100536,"input":0.000192,"output":0.77664},"cost_usd_as_run":2.3668,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":10.15,"tokens":{"cache_read":4793088,"cache_write":0,"output":88560,"reasoning":66297,"uncached_input":76953},"cost_usd":0.0791,"cost_breakdown":{"cache_read":0.014379,"cache_write":0.0,"input":0.011543,"output":0.053136},"cost_usd_as_run":0.0791,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":41.66,"tokens":{"cache_read":2013264,"cache_write":0,"output":32746,"reasoning":22531,"uncached_input":131423},"cost_usd":1.5833,"cost_breakdown":{"cache_read":0.402653,"cache_write":0.0,"input":0.525692,"output":0.65492},"cost_usd_as_run":1.5833,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":14.04,"tokens":{"cache_read":2328960,"cache_write":0,"output":28591,"reasoning":18051,"uncached_input":90299},"cost_usd":0.6994,"cost_breakdown":{"cache_read":0.232896,"cache_write":0.0,"input":0.180598,"output":0.28591},"cost_usd_as_run":0.6994,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"C","score":9.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W","score":4.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F","score":5.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"C","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"C","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"C","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W","score":5.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"C","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"C","score":9.5}],"panel":["gpt-6.1-terra-max","opus-5.5-xhigh","deepseek-v4.1-flash-max","muse-wonder-full-xhigh","gpt-6.1-sol-max"]},"entries":[{"letter":"Z","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.57,"tokens":{"uncached_input":51967,"cache_read":276818,"cache_write":0,"output":19191,"reasoning":8324},"cost_usd":0.3235,"cost_breakdown":{"input":0.103934,"cache_read":0.027682,"cache_write":0.0,"output":0.19191},"cost_usd_as_run":0.3235,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":440,"code":351,"comment":20,"blank":69,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":373,"code":305,"comment":0,"blank":68,"role":"test"}],"impl_lines":440,"impl_code":351,"impl_comment":20,"test_lines":373,"comment_ratio":0.054,"tests_declared":16},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":438.101,"reference":427.445,"speedup":0.976,"baselines":{"oracle":427.445}},{"id":"throughput_64m","candidate":2725.459,"reference":2740.137,"speedup":0.995,"baselines":{"oracle":2740.137}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6-astra-xhigh":6.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Duplicate options can defeat the output budget, and path resolution changes the requested path","gpt-6-astra-xhigh":"Sound execution core, but option and path handling diverge from the reference.","opus-5.5-xhigh":"sound core copied closely from the reference design; several small departures from the reference on option and path handling; a leftover stub moduledoc","deepseek-v4.1-flash-max":"correct core with an untouched stub module doc and last-wins option parsing that diverges from the oracle","muse-wonder-full-xhigh":"Correct common path, wrong duplicate-option semantics throughout","gpt-6.1-sol-max":"Core execution works, but option semantics and completed-result handling diverge from the reference."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later duplicate options override the first","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash is removed before checking the tool","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"`nil` pool name gets the wrong error","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Module documentation calls the implementation a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No regression coverage for VM death or 64 MiB output","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Normalization accepts an invalid executable path","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"correctness","title":"Rejects an absent atom-named pool incorrectly","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Important failure modes lack regression coverage","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Implementation still describes itself as a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Repeated option keys: the last one wins","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"`name: nil` is rejected as an invalid argument","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A path with a trailing slash is not \"used as given\"","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Moduledoc still says the file is a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Gaps in the tests","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `{:ok, runner, _info}` clause","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"The `:DOWN` handler matches any monitor reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"duplicate option keys take the last value","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"module doc still says the file is a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`:name` nil rejected as invalid input","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"trailing-slash path is silently normalised and executed","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"`:deadline` handler has no result guard","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"suite enshrines the `:name` nil divergence and skips two contract areas","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Repeated options are last-wins instead of first-wins","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`name: nil` returns the wrong error","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash on a file path is normalised away","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Moduledoc still claims the file is a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No large-output coverage and one self-confirming wrong expectation","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate options use the last value","status":"valid","defect":"BT-014","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid paths become executable","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"A queued deadline overwrites a completed result","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"correctness","title":"`nil` pool gets the wrong error","status":"valid","defect":"BT-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Module documentation describes an unimplemented stub","status":"valid","defect":"BT-033","detail":"","location":""}],"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.8,"slop":{"score":1.3,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.5,"deepseek-v4.1-flash-max":2.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":1.0}},"defect_points":7.0,"score_cross_vendor":6.8,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","task":"bounded-tool","lead":"opus-5.5-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":52.27,"tokens":{"cache_read":20316544,"cache_write":0,"output":121079,"reasoning":71621,"uncached_input":432884},"cost_usd":2.0541,"cost_breakdown":{"cache_read":1.015827,"cache_write":0.0,"input":0.432884,"output":0.605395},"cost_usd_as_run":2.0541,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":111.41,"tokens":{"cache_read":27020113,"cache_write":799475,"output":236225,"reasoning":154259,"uncached_input":350},"cost_usd":16.5256,"cost_breakdown":{"cache_read":5.404023,"cache_write":6.3958,"input":0.0014,"output":4.7245},"cost_usd_as_run":16.5256,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316","sessions":4}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":80.06,"tokens":{"cache_read":15016320,"cache_write":0,"output":138172,"reasoning":83950,"uncached_input":552022},"cost_usd":3.9874,"cost_breakdown":{"cache_read":1.501632,"cache_write":0.0,"input":1.104044,"output":1.38172},"cost_usd_as_run":3.9874,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"gpt-6-astra-xhigh","judge":"gpt-6-astra-xhigh","model":"gpt-6-astra","effort":"xhigh","label":"GPT‑6 Astra xhigh","usage":{"wall_minutes":96.75,"tokens":{"cache_read":11545600,"cache_write":0,"output":85880,"reasoning":41094,"uncached_input":478084},"cost_usd":20.6205,"cost_breakdown":{"cache_read":11.5456,"cache_write":0.0,"input":4.78084,"output":4.294},"cost_usd_as_run":20.6205,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":161.08,"tokens":{"cache_read":47527936,"cache_write":0,"output":427806,"reasoning":335605,"uncached_input":445412},"cost_usd":0.4661,"cost_breakdown":{"cache_read":0.142583,"cache_write":0.0,"input":0.066811,"output":0.256685},"cost_usd_as_run":0.4661,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":4}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":147.45,"tokens":{"cache_read":11747188,"cache_write":0,"output":126265,"reasoning":74263,"uncached_input":597841},"cost_usd":7.2661,"cost_breakdown":{"cache_read":2.349438,"cache_write":0.0,"input":2.391364,"output":2.5253},"cost_usd_as_run":7.2661,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":3.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":5.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":4.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":5.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Y","score":7.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X","score":6.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Y","score":5.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Y","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"O","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Y","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"O","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reF","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Y","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B","score":8.0}],"panel":["gpt-6.1-terra-max","opus-5.5-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.82,"tokens":{"uncached_input":30365,"cache_read":181760,"cache_write":0,"output":5531,"reasoning":154},"cost_usd":0.1342,"cost_breakdown":{"input":0.06073,"cache_read":0.018176,"cache_write":0.0,"output":0.05531},"cost_usd_as_run":0.1342,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":307,"code":244,"comment":15,"blank":48,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":171,"code":145,"comment":1,"blank":25,"role":"test"}],"impl_lines":307,"impl_code":244,"impl_comment":15,"test_lines":171,"comment_ratio":0.058,"tests_declared":10},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":499.918,"reference":475.816,"speedup":0.952,"baselines":{"oracle":475.816}},{"id":"throughput_64m","candidate":2497.02,"reference":2458.599,"speedup":1.016,"baselines":{"oracle":2458.599}}]},"scores":{"gpt-6.1-terra-max":4.5,"gpt-6.1-sol-max":4.0,"gpt-6-astra-xhigh":3.0,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":5.0,"muse-wonder-full-xhigh":6.0},"council_summaries":{"gpt-6.1-terra-max":"Two completion races can turn successful valid runs into errors.","gpt-6.1-sol-max":"Two independent races lose successful results, alongside validation and resolution errors.","gpt-6-astra-xhigh":"Two independent lifecycle races accompany input defects and limited failure-path tests.","opus-5.5-xhigh":"Same rare `init` crash plus thin tests, a stale doc and a validation divergence","deepseek-v4.1-flash-max":"Compact and mostly correct, but the bootstrap crashes on a closed port and a late deadline can overwrite a stored result.","muse-wonder-full-xhigh":"Lax validation, stored-result race, thinnest tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Port exit races PID discovery","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Deadline overwrites a completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Repeated invalid options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash can execute a file","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Failure-lifetime coverage is missing","status":"valid","defect":"BT-035","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Relay invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Module documentation calls the implementation a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Closed port crashes initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Deadline overwrites a completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid later duplicate options execute the command","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Trailing-slash paths execute","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Stale stub documentation","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Fast-exiting commands can fail initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Stale deadline overwrites an already completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Duplicate-value validation is incomplete","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Unavailable slash-suffixed paths execute","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Implemented module is labelled a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Important abandonment paths lack regression coverage","status":"valid","defect":"BT-035","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"`{:os_pid, pid} = Port.info(port, :os_pid)` crashes if the port has already closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin suite","status":"valid","defect":"BT-035","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A later invalid duplicate option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Trailing slash is stripped before `stat`","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Stale \"This file is a stub\" moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`valid?` wraps the validation in a catch-all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"`{:os_pid, pid} = Port.info(port, :os_pid)` raises when the port has closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Scaffold stub text is still the module doc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Executable path normalised before the check","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"A result already produced can be replaced by a later deadline","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate invalid option ignored and tool spawned","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Deadline can overwrite a stored success","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Stub doc and misplaced comment","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinnest suite in the set","status":"valid","defect":"BT-035","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"`Port.info` match can crash init","status":"valid","defect":"BT-007","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":218,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.0,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":8.7,"score_cross_vendor":5.5,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.14,"tokens":{"uncached_input":21709,"cache_read":442368,"cache_write":0,"output":21934,"reasoning":10459},"cost_usd":1.7562,"cost_breakdown":{"input":0.21709,"cache_read":0.442368,"cache_write":0.0,"output":1.0967},"cost_usd_as_run":1.7562,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":367,"code":296,"comment":18,"blank":53,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":478,"code":383,"comment":2,"blank":93,"role":"test"}],"impl_lines":367,"impl_code":296,"impl_comment":18,"test_lines":478,"comment_ratio":0.057,"tests_declared":29},"hidden":{"passed":11,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":492.649,"reference":472.565,"speedup":0.959,"baselines":{"oracle":472.565}},{"id":"throughput_64m","candidate":2289.108,"reference":2557.069,"speedup":0.895,"baselines":{"oracle":2557.069}}]},"scores":{"gpt-6.1-terra-max":6.0,"gpt-6.1-sol-max":6.0,"gpt-6-astra-xhigh":5.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":5.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"A startup race loses completed results; tests cover most ordinary behavior.","gpt-6.1-sol-max":"Strong tests and cleanup, undermined by a startup crash and path mismatch.","gpt-6-astra-xhigh":"Thorough regression coverage, but the startup race remains untested and unfixed.","opus-5.5-xhigh":"Thorough tests, but a rare crash in `init`, a stale \"stub\" moduledoc and a flaky test helper","deepseek-v4.1-flash-max":"Good implementation and a rich suite, undermined by the same bootstrap crash and a test helper that dies on `:esrch`.","muse-wonder-full-xhigh":"Correct implementation, flaky test helper and stale stub doc"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Port exit races PID discovery","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash can execute a file","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death cleanup is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Relay invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Module documentation calls the implementation a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Closed port crashes initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Trailing-slash executable runs","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Public module documentation still says stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Fast completion can invalidate initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Normalization accepts an unavailable executable path","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Public documentation describes an obsolete stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"`{:os_pid, os_pid} = Port.info(port, :os_pid)` crashes if the port has already closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"`alive?` helper crashes on `{:error, :esrch}`","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Moduledoc still says \"This file is a stub … replace it with the implementation\"","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The relay and runner design are explained by a single line","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Trailing slash is stripped before `stat`","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"`{:os_pid, os_pid} = Port.info(port, :os_pid)` raises when the port has closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"`alive?/1` handles only `{:error, :enoent}`","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Scaffold stub text is still the module doc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Executable path normalised before the check","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"The primary group kill is silently disabled when the BEAM's own pgrp cannot be read","status":"valid","defect":"BT-004","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Flaky `alive?/1` helper crashes on `:esrch`","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Stub moduledoc left in place","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"`Port.info` match can crash init","status":"valid","defect":"BT-007","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":263,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":11,"flags":1,"audited":"2026-11-02","reviewed":true},"score":6.2,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":6.6,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":16.21,"tokens":{"uncached_input":44,"cache_read":2527474,"cache_write":139821,"output":97614,"reasoning":69822},"cost_usd":3.5765,"cost_breakdown":{"input":0.000176,"cache_read":0.505495,"cache_write":1.118568,"output":1.95228},"cost_usd_as_run":3.5765,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":505,"code":347,"comment":77,"blank":81,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":625,"code":491,"comment":16,"blank":118,"role":"test"}],"impl_lines":505,"impl_code":347,"impl_comment":77,"test_lines":625,"comment_ratio":0.182,"tests_declared":31},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":499.66,"reference":491.33,"speedup":0.983,"baselines":{"oracle":491.33}},{"id":"throughput_64m","candidate":2442.499,"reference":2447.13,"speedup":0.998,"baselines":{"oracle":2447.13}}]},"scores":{"gpt-6.1-terra-max":10.0,"gpt-6.1-sol-max":9.5,"gpt-6-astra-xhigh":9.5,"opus-5.5-xhigh":9.5,"deepseek-v4.1-flash-max":9.5,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Complete coverage and result handling held up in the reviewed cases.","gpt-6.1-sol-max":"Matched all behavioral probes, with one inaccurate timer comment.","gpt-6-astra-xhigh":"Strongest contract fidelity and test coverage; only inaccurate limit commentary found.","opus-5.5-xhigh":"Matched the reference on every probe; best tests and docs","deepseek-v4.1-flash-max":"The best of the set: correct, carefully reasoned, and the only suite that covers the whole contract including the VM-SIGKILL backstop.","muse-wonder-full-xhigh":"Matches the oracle on every probe, best tests in the set"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Conservative timer interval presented as a VM limit","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Conservative limits are described as platform limits","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":304,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":22,"flags":1,"audited":"2026-11-02","reviewed":true},"score":9.6,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.2,"score_cross_vendor":9.625,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.39,"tokens":{"uncached_input":24660,"cache_read":796032,"cache_write":0,"output":22578,"reasoning":11345},"cost_usd":0.1774,"cost_breakdown":{"input":0.02466,"cache_read":0.039802,"cache_write":0.0,"output":0.11289},"cost_usd_as_run":0.1774,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":338,"code":272,"comment":14,"blank":52,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":343,"code":284,"comment":2,"blank":57,"role":"test"}],"impl_lines":338,"impl_code":272,"impl_comment":14,"test_lines":343,"comment_ratio":0.049,"tests_declared":13},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":498.64,"reference":485.05,"speedup":0.973,"baselines":{"oracle":485.05}},{"id":"throughput_64m","candidate":2514.714,"reference":2513.075,"speedup":1.001,"baselines":{"oracle":2513.075}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.0,"gpt-6-astra-xhigh":5.0,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":5.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Validation is thorough, but a fast port exit can lose a valid result.","gpt-6.1-sol-max":"Strong validation and cleanup, with a startup race and path mismatch.","gpt-6-astra-xhigh":"Good core behavior, but a startup race can discard successful executions.","opus-5.5-xhigh":"Same rare `init` crash; sparse docs; moderate tests","deepseek-v4.1-flash-max":"Compact and correct in the common path, but the bootstrap crashes on a nil os_pid and the kill path can silently no-op.","muse-wonder-full-xhigh":"Compact and correct, minor doc/test blemishes"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Port exit races PID discovery","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash can execute a file","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death cleanup is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Relay invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Completed port crashes initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Executable paths are normalized before checking","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Fast exits can fail initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Normalization changes executable-path semantics","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Relay invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"`{:os_pid, pid} = Port.info(port, :os_pid)` crashes if the port has already closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Trailing slash is stripped before `stat`","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The relay and runner are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"`alive?` matches only `:enoent`, and VM SIGKILL is untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`:DOWN` handler matches any monitor","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"`{:os_pid, pid} = Port.info(port, :os_pid)` raises when the port has closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Executable path normalised before the check","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"No group kill when the BEAM's own identity cannot be read","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`handle_info({:DOWN, _ref, :process, _owner, _reason}, state)`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"The module doc is one line and the relay/process-group invariants that make the module correct are undocumented (`lib/bounded_tool.ex:2-4`, `142-148`)","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Relay undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Latent flake in `alive?/1` and unsettled mailbox check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"`Port.info` match can crash init","status":"valid","defect":"BT-007","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":238,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.4,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":5.4,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":51.91,"tokens":{"uncached_input":154366,"cache_read":3464424,"cache_write":0,"output":56340,"reasoning":33840},"cost_usd":2.4371,"cost_breakdown":{"input":0.617464,"cache_read":0.692885,"cache_write":0.0,"output":1.1268},"cost_usd_as_run":2.4371,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":551,"code":421,"comment":40,"blank":90,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":558,"code":425,"comment":16,"blank":117,"role":"test"}],"impl_lines":551,"impl_code":421,"impl_comment":40,"test_lines":558,"comment_ratio":0.087,"tests_declared":26},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":498.919,"reference":490.446,"speedup":0.983,"baselines":{"oracle":490.446}},{"id":"throughput_64m","candidate":2265.061,"reference":2345.841,"speedup":0.966,"baselines":{"oracle":2345.841}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.0,"gpt-6-astra-xhigh":6.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Core execution is sound in the probes, with validation and path edge cases.","gpt-6.1-sol-max":"Handles completion races correctly; validation and path handling need changes.","gpt-6-astra-xhigh":"Handles lifecycle races correctly, but accepts invalid options and alters supplied paths.","opus-5.5-xhigh":"Faithful and robust under load; minor validation and path divergences","deepseek-v4.1-flash-max":"Reference-grade runner with a thorough suite; one path-normalisation divergence.","muse-wonder-full-xhigh":"Sound runner, lax duplicate-option validation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Repeated invalid options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash can execute a file","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"A deadline test assumes the tool has started","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death cleanup is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate values bypass validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Path normalization permits an unavailable executable","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Duplicate exception handlers","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Later duplicate values bypass validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing slash is silently removed","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A later invalid duplicate option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Trailing slash is stripped before `stat`","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test for a SIGKILLed VM","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`rescue _` next to `catch _, _`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Executable path is normalised before it is checked","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable `terminate/2` clause","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate invalid option ignored and tool spawned","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unreachable defensive clauses","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Mailbox checked without settling, no VM test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":365,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":36,"flags":3,"audited":"2026-11-02","reviewed":true},"score":7.5,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":4.2,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":62.02,"tokens":{"uncached_input":1898326,"cache_read":1125760,"cache_write":0,"output":230051,"reasoning":103217},"cost_usd":5.7398,"cost_breakdown":{"input":3.796652,"cache_read":0.56288,"cache_write":0.0,"output":1.380306},"cost_usd_as_run":5.7398,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":513,"code":384,"comment":40,"blank":89,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":570,"code":446,"comment":10,"blank":114,"role":"test"}],"impl_lines":513,"impl_code":384,"impl_comment":40,"test_lines":570,"comment_ratio":0.094,"tests_declared":17},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":423.961,"reference":423.468,"speedup":0.999,"baselines":{"oracle":423.468}},{"id":"throughput_64m","candidate":2736.9,"reference":2730.731,"speedup":1.002,"baselines":{"oracle":2730.731}}]},"scores":{"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":6.5,"gpt-6-astra-xhigh":6.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Ordinary Unicode executable paths fail, and a test can destroy a home-directory file","gpt-6.1-sol-max":"Sound lifecycle handling, but executable resolution and test safety need fixes.","gpt-6-astra-xhigh":"Sound lifecycle handling, but executable resolution rejects valid tools.","opus-5.5-xhigh":"Close to the reference and correct in practice; one PATH-lookup divergence, a misleading comment over a retry loop that does nothing useful, and some redundant plumbing","deepseek-v4.1-flash-max":"correct and the deepest test suite, but the module's design is undocumented and a few helpers are ornamental","muse-wonder-full-xhigh":"Reference-faithful core, broken for non-ASCII tool paths"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Executable checks corrupt UTF-8 paths","status":"valid","defect":"BT-011","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Resolution test overwrites and deletes a fixed home file","status":"valid","defect":"BT-030","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extremely large deadlines differ from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM SIGKILL abandonment is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Re-resolving paths rejects valid executables","status":"valid","defect":"BT-011","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Normalization makes an invalid path executable","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Incorrect timer-limit claim","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Non-ASCII executable paths fail","status":"valid","defect":"BT-011","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Trailing slash changes executable resolution","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Home fixture can destroy an existing file","status":"valid","defect":"BT-030","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"VM-SIGKILL abandonment lacks coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Incorrect timer-limit comment","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"one-line moduledoc leaves the design unexplained","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"retry loop that cannot retry","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"stored `pgrp` is never used","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"no separate-VM SIGKILL test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Relative PATH entries make bare names unavailable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Comment claims \"the proc entry can lag the fork\", and the retry loop can't wait anyway","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test for VM SIGKILL or for a runner crash","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Code that does nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Test writes into the real home directory","status":"valid","defect":"BT-030","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Non-ASCII executable names rejected","status":"valid","defect":"BT-011","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Backstop untested","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":318,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":32,"flags":10,"audited":"2026-11-02","reviewed":true},"score":7.2,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":0.0}},"defect_points":5.0,"score_cross_vendor":7.2,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":28.76,"tokens":{"uncached_input":378,"cache_read":5668326,"cache_write":117132,"output":93672,"reasoning":0},"cost_usd":0.1582,"cost_breakdown":{"input":5.7e-05,"cache_read":0.090693,"cache_write":0.023426,"output":0.044026},"cost_usd_as_run":0.1582,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0639,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":496,"code":342,"comment":81,"blank":73,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":536,"code":405,"comment":26,"blank":105,"role":"test"}],"impl_lines":496,"impl_code":342,"impl_comment":81,"test_lines":536,"comment_ratio":0.191,"tests_declared":25},"hidden":{"passed":14,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":423.206,"reference":425.043,"speedup":1.004,"baselines":{"oracle":425.043}},{"id":"throughput_64m","candidate":2720.4,"reference":2738.771,"speedup":0.993,"baselines":{"oracle":2738.771}}]},"scores":{"gpt-6.1-terra-max":5.0,"gpt-6.1-sol-max":4.0,"gpt-6-astra-xhigh":3.5,"opus-5.5-xhigh":4.5,"deepseek-v4.1-flash-max":4.5,"muse-wonder-full-xhigh":5.5},"council_summaries":{"gpt-6.1-terra-max":"A result can be lost before the caller begins waiting, and a safety guard is ineffective.","gpt-6.1-sol-max":"Stops before delivering completed results and contains ineffective safety logic.","gpt-6-astra-xhigh":"Completion-before-await loses results and produces observable test failures.","opus-5.5-xhigh":"Reference-quality design, docs and tests, but a runner race returns `{:error, :unavailable}` for valid runs, often","deepseek-v4.1-flash-max":"Excellent docs and tests, but the runner stops before the caller asks for the result, so a finished run can be lost.","muse-wonder-full-xhigh":"Intermittent spurious `:unavailable` for fast tools"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Runner stops before an awaiting caller can receive its result","status":"valid","defect":"BT-001","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Repeated invalid options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash can execute a file","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"dead-code","title":"The BEAM-group guard always compares against `nil`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The TERM-ignoring group test does not ignore TERM","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Watchdog timing is overstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Completion discards results before `:await`","status":"valid","defect":"BT-001","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate values are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid trailing-slash path executes","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"BEAM-group check always compares against `nil`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Claimed TERM-ignoring case never ignores TERM","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Runner stops before an awaiting caller is registered","status":"valid","defect":"BT-001","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Invalid later options are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing slash changes the command selected","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"dead-code","title":"Explicit BEAM-group exclusion is ineffective","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Result lost when the runner finishes before the caller's `:await` arrives","status":"valid","defect":"BT-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A later invalid duplicate option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Trailing slash is stripped before `stat`","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"inherited group, which belongs to erl_child_setup (the VM's own group)\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"The result is discarded when the run finishes before `:await`","status":"valid","defect":"BT-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"Unreachable await clause","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Executable path normalised before the check","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Result dropped when the tool finishes before `:await`","status":"valid","defect":"BT-001","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate invalid option ignored and tool spawned","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Beam-group guard never engages","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"SIGTERM test does not ignore SIGTERM","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Redundant pool pre-check and child-spec boilerplate","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":290,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":62,"flags":1,"audited":"2026-11-02","reviewed":true},"score":4.7,"slop":{"score":1.1,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.5}},"defect_points":7.7,"score_cross_vendor":4.7,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":37.76,"tokens":{"uncached_input":90013,"cache_read":2039128,"cache_write":0,"output":30391,"reasoning":12772},"cost_usd":1.3757,"cost_breakdown":{"input":0.360052,"cache_read":0.407826,"cache_write":0.0,"output":0.60782},"cost_usd_as_run":1.3757,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":495,"code":402,"comment":21,"blank":72,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":439,"code":334,"comment":10,"blank":95,"role":"test"}],"impl_lines":495,"impl_code":402,"impl_comment":21,"test_lines":439,"comment_ratio":0.05,"tests_declared":19},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":474.522,"reference":478.97,"speedup":1.009,"baselines":{"oracle":478.97}},{"id":"throughput_64m","candidate":2502.125,"reference":2477.954,"speedup":1.01,"baselines":{"oracle":2477.954}}]},"scores":{"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":5.5,"gpt-6-astra-xhigh":4.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"A completed result can be overwritten, and validation and path handling differ from the reference.","gpt-6.1-sol-max":"Broad coverage, but completed results can become timeouts.","gpt-6-astra-xhigh":"Tolerates early process exit, but stale timers corrupt completed results.","opus-5.5-xhigh":"Correct and robust, but the relay is undocumented and the tests are weaker","deepseek-v4.1-flash-max":"Solid implementation; a late deadline can overwrite a stored result, and the suite leaves some contract areas untested.","muse-wonder-full-xhigh":"Lax validation plus a stored-result race"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Deadline overwrites a completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Repeated invalid options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash can execute a file","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death cleanup is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The large-output test only emits 4 MiB","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Relay invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Deadline replaces an already completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate-option validation checks only selected values","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Trailing-slash executable is accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Large-output test stops at 4 MiB","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Completed results can turn into timeouts","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Later duplicate values escape validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Normalization removes meaningful trailing slashes","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Large-output test exercises only 4 MiB","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A later invalid duplicate option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Trailing slash is stripped before `stat`","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Key contract items are untested","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The relay script has no explanation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Hand-written PATH search","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Executable path normalised before the check","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"A stored result can be replaced by a later deadline","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Coverage gaps","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate invalid option ignored and tool spawned","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Deadline can overwrite a stored success","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin abandonment and budget coverage","status":"valid","defect":"BT-027","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Relay undocumented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":344,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":28,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.6,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":6.5,"score_cross_vendor":6.625,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.5,"tokens":{"uncached_input":38977,"cache_read":568832,"cache_write":0,"output":10690,"reasoning":2938},"cost_usd":0.2417,"cost_breakdown":{"input":0.077954,"cache_read":0.056883,"cache_write":0.0,"output":0.1069},"cost_usd_as_run":0.2417,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":319,"code":254,"comment":16,"blank":49,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":277,"code":227,"comment":1,"blank":49,"role":"test"}],"impl_lines":319,"impl_code":254,"impl_comment":16,"test_lines":277,"comment_ratio":0.059,"tests_declared":16},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":494.558,"reference":474.028,"speedup":0.958,"baselines":{"oracle":474.028}},{"id":"throughput_64m","candidate":2644.155,"reference":2600.367,"speedup":1.017,"baselines":{"oracle":2600.367}}]},"scores":{"gpt-6.1-terra-max":6.0,"gpt-6.1-sol-max":5.5,"gpt-6-astra-xhigh":5.0,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":5.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Fast commands can be reported unavailable when the port exits during startup.","gpt-6.1-sol-max":"Good cleanup and validation, but completed commands can crash initialization.","gpt-6-astra-xhigh":"Compact implementation with a reproducible startup race and stale documentation.","opus-5.5-xhigh":"Correct design, but the same rare `init` crash, a stale stub doc and almost no comments","deepseek-v4.1-flash-max":"Sound design, but the bootstrap crashes when the tool exits before the runner records its pid.","muse-wonder-full-xhigh":"Correct and compact, stale stub doc"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Port exit races PID discovery","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash can execute a file","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death cleanup is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Relay invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Module documentation calls the implementation a stub","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Closed port crashes initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Normalization changes executable-path semantics","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Implemented module still described as a stub","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Initialization assumes the port still exists","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Path normalization runs an unavailable path","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Completed implementation still calls itself a stub","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"`{:os_pid, os_pid} = Port.info(port, :os_pid)` crashes if the port has already closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Moduledoc says the file \"is a stub … replace it\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The relay and runner have no explanatory comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Trailing slash is stripped before `stat`","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No VM SIGKILL test, and the suite is moderate in size","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"`{:os_pid, os_pid} = Port.info(port, :os_pid)` raises when the port has already closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Scaffold stub text is still the module doc","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Executable path normalised before the check","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`@spec await(atom(), pid(), pos_integer()) :: tuple()`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Stub moduledoc left in place","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"`Port.info` match can crash init","status":"valid","defect":"BT-007","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":225,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.2,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":5.5,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":19.96,"tokens":{"uncached_input":84,"cache_read":9368883,"cache_write":292272,"output":241486,"reasoning":204621},"cost_usd":1.0812,"cost_breakdown":{"input":3.8e-05,"cache_read":0.462702,"cache_write":0.263553,"output":0.354911},"cost_usd_as_run":1.0812,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":513,"code":371,"comment":74,"blank":68,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":501,"code":384,"comment":19,"blank":98,"role":"test"}],"impl_lines":513,"impl_code":371,"impl_comment":74,"test_lines":501,"comment_ratio":0.166,"tests_declared":31},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":469.427,"reference":466.028,"speedup":0.993,"baselines":{"oracle":466.028}},{"id":"throughput_64m","candidate":2522.703,"reference":2471.757,"speedup":1.021,"baselines":{"oracle":2471.757}}]},"scores":{"gpt-6.1-terra-max":6.0,"gpt-6.1-sol-max":7.0,"gpt-6-astra-xhigh":6.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Execution has edge-case defects, and the test suite has destructive cleanup paths.","gpt-6.1-sol-max":"Reliable lifecycle handling, with validation and resolution edge defects.","gpt-6-astra-xhigh":"Strong lifecycle handling and broad tests, with two input-contract defects.","opus-5.5-xhigh":"Faithful clone with strong tests and docs; two minor divergences","deepseek-v4.1-flash-max":"Clean, well documented and strongly tested; one path-normalisation divergence.","muse-wonder-full-xhigh":"Strong design and tests, lax duplicates"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Test setup deletes an existing project directory","status":"valid","defect":"BT-031","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Teardown can signal a reused PID","status":"valid","defect":"BT-032","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Repeated invalid options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A trailing slash can execute a file","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Later duplicate options are not validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Trailing-slash executable incorrectly runs","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Timer maximum is stated incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Duplicate options are not fully validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Supplied executable paths are changed before checking","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Timer limit is stated incorrectly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A later invalid duplicate option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Trailing slash is stripped before `stat`","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Executable path normalised before the check","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Duplicate invalid option ignored and tool spawned","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Trailing slash accepted","status":"valid","defect":"BT-009","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":320,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":59,"flags":2,"audited":"2026-11-02","reviewed":true},"score":7.5,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.2,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":14.11,"tokens":{"uncached_input":45996,"cache_read":5609728,"cache_write":0,"output":98822,"reasoning":75792},"cost_usd":0.083,"cost_breakdown":{"input":0.006899,"cache_read":0.016829,"cache_write":0.0,"output":0.059293},"cost_usd_as_run":0.083,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.083,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":585,"code":431,"comment":64,"blank":90,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":491,"code":369,"comment":9,"blank":113,"role":"test"}],"impl_lines":585,"impl_code":431,"impl_comment":64,"test_lines":491,"comment_ratio":0.129,"tests_declared":31},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":486.398,"reference":479.379,"speedup":0.986,"baselines":{"oracle":479.379}},{"id":"throughput_64m","candidate":2478.199,"reference":2552.644,"speedup":0.971,"baselines":{"oracle":2552.644}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":6.5,"gpt-6-astra-xhigh":6.0,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"opus-5.5-xhigh":"excellent comments, good tests (31); a repeated invalid option is accepted; small redundancies","gpt-6.1-terra-max":"Good documentation and lifecycle structure, but invalid duplicate options can run a tool.","gpt-6.1-sol-max":"Useful documentation, but disappearing identities and duplicate options are mishandled.","gpt-6-astra-xhigh":"Detailed implementation, but a procfs race and validation mismatches remain","deepseek-v4.1-flash-max":"The most polished design and docs, but a fast-exit crash that its own comment denies, measured at ~1% of trivial runs.","muse-wonder-full-xhigh":"best-documented runner internals; held back by the duplicate-option hole"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Repeated invalid option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Overlapping checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable branches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later invalid options are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Concurrency test races its own timeout","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Crash backstop is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Disappearing identity crashes startup","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate invalid values bypass validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Saved group field is unused","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Disappearing process identity crashes initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Invalid duplicate options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"dead-code","title":"Parser fallback is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"robustness","title":"`identity/1` leaks `{:error, :enoent}` into init's `case`","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"Comment claims the exact failing case is handled","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys bypass value validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`@spec identity/1` promises `:error` but the function can return `{:error, :enoent}`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No runner-crash/backstop test and no monitor/port hygiene assertion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Only the first value of a duplicated option is validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Struct field `:pgrp` is written and never read","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":369,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":54,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.1,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":2.0,"muse-wonder-full-xhigh":0.5}},"defect_points":4.2,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":3.21,"tokens":{"uncached_input":18,"cache_read":446059,"cache_write":51974,"output":19556,"reasoning":5624},"cost_usd":0.8962,"cost_breakdown":{"input":7.2e-05,"cache_read":0.089212,"cache_write":0.415792,"output":0.39112},"cost_usd_as_run":0.8962,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":417,"code":318,"comment":38,"blank":61,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":362,"code":292,"comment":0,"blank":70,"role":"test"}],"impl_lines":417,"impl_code":318,"impl_comment":38,"test_lines":362,"comment_ratio":0.107,"tests_declared":24},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":463.152,"reference":467.232,"speedup":1.009,"baselines":{"oracle":467.232}},{"id":"throughput_64m","candidate":2582.516,"reference":2564.449,"speedup":1.007,"baselines":{"oracle":2564.449}}]},"scores":{"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.5,"gpt-6-astra-xhigh":7.5,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"opus-5.5-xhigh":"good docs and tests, but rejects valid calls that repeat an option key","gpt-6.1-terra-max":"Valid duplicate options are rejected.","gpt-6.1-sol-max":"Lifecycle handling is sound, but valid duplicate options are rejected.","gpt-6-astra-xhigh":"Reliable lifecycle, but rejects valid duplicate options and changes supplied paths","deepseek-v4.1-flash-max":"Guards the fast-exit race and is otherwise tidy; it rejects duplicate option keys the oracle accepts.","muse-wonder-full-xhigh":"solid core and tests; rejects valid duplicate options"},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"Repeated valid option keys are rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"PATH search reimplemented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Duplicate known keys are treated as unknown","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Crash backstop lacks coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Keyword subtraction rejects valid options","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Valid duplicate options are rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"All duplicate option keys are rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"The runner stops on any `:DOWN`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No pool-shutdown, runner-crash/backstop, or long-timeout test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Valid duplicate options rejected","status":"valid","defect":"BT-013","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Hand-rolled PATH search duplicates `System.find_executable/1`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":276,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":1.0}},"defect_points":2.8,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.19,"tokens":{"uncached_input":24618,"cache_read":242304,"cache_write":0,"output":4401,"reasoning":190},"cost_usd":0.0587,"cost_breakdown":{"input":0.024618,"cache_read":0.012115,"cache_write":0.0,"output":0.022005},"cost_usd_as_run":0.0587,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":275,"code":226,"comment":12,"blank":37,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":67,"code":55,"comment":0,"blank":12,"role":"test"}],"impl_lines":275,"impl_code":226,"impl_comment":12,"test_lines":67,"comment_ratio":0.05,"tests_declared":4},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":453.585,"reference":457.674,"speedup":1.009,"baselines":{"oracle":457.674}},{"id":"throughput_64m","candidate":2570.718,"reference":2608.37,"speedup":0.986,"baselines":{"oracle":2608.37}}]},"scores":{"opus-5.5-xhigh":5.0,"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":4.0,"gpt-6-astra-xhigh":4.0,"deepseek-v4.1-flash-max":5.0,"muse-wonder-full-xhigh":5.5},"council_summaries":{"opus-5.5-xhigh":"improper `args` give the wrong error, very thin tests, no docs","gpt-6.1-terra-max":"Three independently reproducible contract failures need correction.","gpt-6.1-sol-max":"Multiple verified result and validation defects, with weak cleanup tests.","gpt-6-astra-xhigh":"Multiple correctness gaps remain, and tests do not verify essential cleanup guarantees","deepseek-v4.1-flash-max":"Compact but under-tested, with two real defects: a fast-exit crash and improper args leaking to the catch-all.","muse-wonder-full-xhigh":"core works, but two wrong-reason validation defects and the thinnest suite"},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"Improper `args` list returns `:unavailable` instead of `:invalid_argument`","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Repeated invalid option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Four tests only","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No documentation beyond a one-line moduledoc","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Unguarded `:deadline` handler","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Single-letter state names, an `else: ( … )` block, no blank lines between definitions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Improper arguments return the wrong error","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later invalid options are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A deadline can overwrite a completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cleanup and abandonment are not verified","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Fast-exit startup race","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Deadline destroys saved success","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate values are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Improper arguments get the wrong error","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Cleanup tests do not establish cleanup","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Fast completion can crash initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Completed results are overwritten by the deadline","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Improper argument lists return the wrong error","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Invalid duplicate options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Cleanup tests never establish that OS processes die","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Relay safety invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"robustness","title":"Runner init crashes when the tool exits before the PID is read","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"Improper argument lists return `{:error, :unavailable}` instead of `{:error, :invalid_argument}`","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Suite never verifies that a timed-out group dies","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"The relay/backstop is unexplained and private helpers have no specs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"A late `:deadline` can overwrite a stored result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Only the first value of a duplicated option is validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Improper `args` list returns `:unavailable`, not `:invalid_argument`","status":"valid","defect":"BT-002","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Suite does not verify group death, caller death or large budgets","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"`:deadline` handler can overwrite a finished result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Init crashes if the port exits before `Port.info` is read","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No design documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"`terminate/2` returns the state map instead of `:ok`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":200,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.0,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":9.1,"score_cross_vendor":5.167,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":44.47,"tokens":{"uncached_input":213716,"cache_read":3453032,"cache_write":0,"output":46058,"reasoning":23771},"cost_usd":2.4666,"cost_breakdown":{"input":0.854864,"cache_read":0.690606,"cache_write":0.0,"output":0.92116},"cost_usd_as_run":2.4666,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":519,"code":404,"comment":34,"blank":81,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":562,"code":427,"comment":13,"blank":122,"role":"test"}],"impl_lines":519,"impl_code":404,"impl_comment":34,"test_lines":562,"comment_ratio":0.078,"tests_declared":28},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":465.286,"reference":467.193,"speedup":1.004,"baselines":{"oracle":467.193}},{"id":"throughput_64m","candidate":2583.744,"reference":2568.554,"speedup":1.006,"baselines":{"oracle":2568.554}}]},"scores":{"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.0,"gpt-6-astra-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"opus-5.5-xhigh":"clear docs, broad tests (28); a repeated invalid option is accepted; verbose defensive code","gpt-6.1-terra-max":"Sound lifecycle design, but duplicate invalid options can execute a tool.","gpt-6.1-sol-max":"Common behavior works, but validation and test bounds need tightening.","gpt-6-astra-xhigh":"Strong execution lifecycle, with two validation mismatches","deepseek-v4.1-flash-max":"Clean, guarded implementation; only duplicate option keys drift from the oracle.","muse-wonder-full-xhigh":"well documented and tested, but duplicate bad options pass and there is padding"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Repeated invalid option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"`rescue _ ->` plus `catch _, _ ->` in six places","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable clauses","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Only a 1 MiB budget is tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later invalid options are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Backstop and large-budget coverage is incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Duplicate options bypass value checks","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Timing assertions permit contract violations","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Repeated equivalent exception handlers","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Invalid duplicate options are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys bypass value validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Output-budget test stops at 1 MiB and pools are never shut down","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Only the first value of a duplicated option is validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No large-budget verification","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `start_child` clause","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Triplicated rescue+catch wrappers","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":345,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":41,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.3,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":1.0}},"defect_points":3.4,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.21,"tokens":{"uncached_input":22,"cache_read":631807,"cache_write":63867,"output":31127,"reasoning":13724},"cost_usd":1.2599,"cost_breakdown":{"input":8.8e-05,"cache_read":0.126361,"cache_write":0.510936,"output":0.62254},"cost_usd_as_run":1.2599,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":436,"code":326,"comment":39,"blank":71,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":403,"code":326,"comment":3,"blank":74,"role":"test"}],"impl_lines":436,"impl_code":326,"impl_comment":39,"test_lines":403,"comment_ratio":0.107,"tests_declared":30},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":487.635,"reference":465.446,"speedup":0.954,"baselines":{"oracle":465.446}},{"id":"throughput_64m","candidate":2536.309,"reference":2591.926,"speedup":0.979,"baselines":{"oracle":2591.926}}]},"scores":{"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.0,"gpt-6-astra-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"opus-5.5-xhigh":"matches the reference in every probe; accurate docs throughout; the only suite with a VM-SIGKILL test","gpt-6.1-terra-max":"Strong implementation and coverage, with one oracle edge and a loose backstop test.","gpt-6.1-sol-max":"All behavioral probes pass; the VM-death test grants excessive cleanup time.","gpt-6-astra-xhigh":"Strong lifecycle handling and broad tests, with a pathname mismatch","deepseek-v4.1-flash-max":"Closest to the oracle; guards the fast-exit race and proves the VM-SIGKILL backstop, with only small test/design nits.","muse-wonder-full-xhigh":"cleanest validation; only entry testing VM SIGKILL"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"PATH search reimplemented, with a fallback that can never change the result","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"VM-kill test bound is looser than its comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM-death test allows too much time","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"VM-death deadline assertion is too loose","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"VM-death test permits excessive cleanup latency","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"`async: true` suite asserts a VM-global port count","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"BEAM pgrp cached in `:persistent_term` from a worker","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable default-PATH fallback","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":285,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.9,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":1.3,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.92,"tokens":{"uncached_input":25817,"cache_read":143360,"cache_write":0,"output":4526,"reasoning":14},"cost_usd":0.6278,"cost_breakdown":{"input":0.25817,"cache_read":0.14336,"cache_write":0.0,"output":0.2263},"cost_usd_as_run":0.6278,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":302,"code":242,"comment":15,"blank":45,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":141,"code":121,"comment":0,"blank":20,"role":"test"}],"impl_lines":302,"impl_code":242,"impl_comment":15,"test_lines":141,"comment_ratio":0.058,"tests_declared":7},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":480.141,"reference":477.617,"speedup":0.995,"baselines":{"oracle":477.617}},{"id":"throughput_64m","candidate":2632.811,"reference":2366.577,"speedup":1.112,"baselines":{"oracle":2366.577}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":6.5,"gpt-6-astra-xhigh":6.0,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"opus-5.5-xhigh":"matches the reference in every probe; compact tests; false stub moduledoc; almost no comments","gpt-6.1-terra-max":"The exercised behavior is solid, but the suite misses several lifecycle cases.","gpt-6.1-sol-max":"Concise implementation, but fast-exit handling is unsafe and failure coverage is thinner.","gpt-6-astra-xhigh":"Compact implementation, with a startup race and thinner regression coverage","deepseek-v4.1-flash-max":"Clean and compact, with the fast-exit crash reproduced in a third of runs of one test.","muse-wonder-full-xhigh":"validation-clean and compact, but terse docs and bundled-thin tests"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Moduledoc still says it is a stub","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The relay and kill protocol are unexplained","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Gaps in the 7 tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Module claims to be a stub","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Crash and delayed-output cases are absent","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Fast exits fail runner initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Stale stub documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Fast completion can crash initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Public documentation incorrectly calls the implementation a stub","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"robustness","title":"Runner init crashes when the tool exits before the PID is read","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Stale stub moduledoc","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No pool-shutdown or long-timeout test and no monitor/port assertions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Leftover stub moduledoc","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Relay/backstop/PID-reuse design is undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Init crashes if the port exits before `Port.info` is read","status":"valid","defect":"BT-007","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":214,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.1,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":3.5,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.02,"tokens":{"uncached_input":43065,"cache_read":373504,"cache_write":0,"output":22256,"reasoning":11583},"cost_usd":0.346,"cost_breakdown":{"input":0.08613,"cache_read":0.03735,"cache_write":0.0,"output":0.22256},"cost_usd_as_run":0.346,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":386,"code":309,"comment":17,"blank":60,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":507,"code":412,"comment":0,"blank":95,"role":"test"}],"impl_lines":386,"impl_code":309,"impl_comment":17,"test_lines":507,"comment_ratio":0.052,"tests_declared":30},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":480.852,"reference":465.568,"speedup":0.968,"baselines":{"oracle":465.568}},{"id":"throughput_64m","candidate":2580.813,"reference":2464.927,"speedup":1.047,"baselines":{"oracle":2464.927}}]},"scores":{"opus-5.5-xhigh":8.0,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.5,"gpt-6-astra-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"opus-5.5-xhigh":"matches the reference in every probe; very thorough tests (30, including duplicate-key rejection, missing helpers, crash backstop, port/monitor leaks); stub moduledoc","gpt-6.1-terra-max":"Close to the reference, with a stale module description and test gaps.","gpt-6.1-sol-max":"Behavioral probes pass; remaining issues concern tests and documentation.","gpt-6-astra-xhigh":"Careful lifecycle and validation, with a pathname edge case and stale documentation","deepseek-v4.1-flash-max":"Guards the fast-exit race and has thorough tests; a stale moduledoc and one dead kill clause are the main defects.","muse-wonder-full-xhigh":"correct implementation whose own suite fails intermittently in a test helper"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Relay undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable clause","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Module claims to be a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Procfs exit race can crash the test helper","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM SIGKILL is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Procfs disappearance crashes cleanup assertions","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Completed implementation called a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Public documentation falsely calls the module a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"`alive?/1` has no `{:error, :esrch}` clause","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Stale stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable kill clause","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"No kill at all when the BEAM's pgrp cannot be read","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Own suite fails intermittently in `alive?/1`","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Leftover stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Group kill is gated on a readable BEAM pgrp","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":268,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.1,"slop":{"score":1.2,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.5,"muse-wonder-full-xhigh":1.0}},"defect_points":3.7,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.37,"tokens":{"uncached_input":45188,"cache_read":323456,"cache_write":0,"output":14119,"reasoning":5063},"cost_usd":0.2639,"cost_breakdown":{"input":0.090376,"cache_read":0.032346,"cache_write":0.0,"output":0.14119},"cost_usd_as_run":0.2639,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":348,"code":279,"comment":16,"blank":53,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":429,"code":359,"comment":1,"blank":69,"role":"test"}],"impl_lines":348,"impl_code":279,"impl_comment":16,"test_lines":429,"comment_ratio":0.054,"tests_declared":25},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":491.102,"reference":467.636,"speedup":0.952,"baselines":{"oracle":467.636}},{"id":"throughput_64m","candidate":2468.95,"reference":2465.834,"speedup":1.001,"baselines":{"oracle":2465.834}}]},"scores":{"opus-5.5-xhigh":8.0,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":7.0,"gpt-6-astra-xhigh":6.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"opus-5.5-xhigh":"matches the reference in every probe; strong tests (25, including duplicate keys, early `:deadline` message, oversized timeouts); stub moduledoc","gpt-6.1-terra-max":"Close to the reference, with documentation and test-harness issues.","gpt-6.1-sol-max":"Good structure and coverage, with startup and procfs-test races.","gpt-6-astra-xhigh":"Thorough tests and validation, with an unhandled startup race","deepseek-v4.1-flash-max":"Strong tests and clean code, but the same fast-exit `Port.info` race and a flaky `alive?/1`.","muse-wonder-full-xhigh":"clean struct-based runner with strong tests; stub doc and latent flake"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Relay undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Module claims to be a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Procfs exit race can crash the test helper","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM SIGKILL is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Fast-exit startup race","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Cleanup helper intermittently crashes","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Implementation still documented as a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Fast completion can crash initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Public documentation still describes a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"robustness","title":"Runner init crashes when the tool exits before the PID is read","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"`alive?/1` has no `{:error, :esrch}` clause","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Stale stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Test `alive?/1` crashes intermittently on `{:error, :esrch}`","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Leftover stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Init crashes if the port exits before `Port.info` is read","status":"valid","defect":"BT-007","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":246,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.7,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":4.4,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.55,"tokens":{"uncached_input":21182,"cache_read":260096,"cache_write":0,"output":5770,"reasoning":474},"cost_usd":0.063,"cost_breakdown":{"input":0.021182,"cache_read":0.013005,"cache_write":0.0,"output":0.02885},"cost_usd_as_run":0.063,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":288,"code":229,"comment":13,"blank":46,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":125,"code":105,"comment":0,"blank":20,"role":"test"}],"impl_lines":288,"impl_code":229,"impl_comment":13,"test_lines":125,"comment_ratio":0.054,"tests_declared":5},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":478.562,"reference":475.024,"speedup":0.993,"baselines":{"oracle":475.024}},{"id":"throughput_64m","candidate":2547.657,"reference":2433.05,"speedup":1.047,"baselines":{"oracle":2433.05}}]},"scores":{"opus-5.5-xhigh":6.0,"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":5.0,"gpt-6-astra-xhigh":5.0,"deepseek-v4.1-flash-max":5.5,"muse-wonder-full-xhigh":6.0},"council_summaries":{"opus-5.5-xhigh":"core behaviour correct, but thin sleep-synchronised tests, a repeated invalid option is accepted, minimal docs","gpt-6.1-terra-max":"A result race and incomplete duplicate-option validation need fixes.","gpt-6.1-sol-max":"Startup and saved-result races compromise otherwise working execution.","gpt-6-astra-xhigh":"Core execution works, but startup, completion, and validation have correctness gaps","deepseek-v4.1-flash-max":"Correct in the common case, but a fast-exit crash, an unguarded deadline handler, and a thin, largely undocumented suite.","muse-wonder-full-xhigh":"working core under-documented and under-tested, with the duplicate-option hole"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Repeated invalid option is accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin and timing-dependent tests","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Minimal docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Unguarded `:deadline` handler","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Awkward `else: ( … )` block expression and a nested BEAM-pgrp lookup inside `kill_group`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A deadline can overwrite a completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Later invalid options are ignored","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Important lifecycle cases are absent","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Fast-exit startup race","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Completed success becomes a timeout","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate option values are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Fast completion can crash initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"A deadline overwrites an already-completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Invalid duplicate options still launch tools","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Critical relay invariants are unexplained","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"robustness","title":"Runner init crashes when the tool exits before the PID is read","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"A late `:deadline` can overwrite a stored result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Suite is 5 tests / 125 lines and misses contract edges","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"The relay, fd juggling and `+250 ms` backstop are unexplained","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`terminate/2` returns the state map rather than `:ok`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Only the first value of a duplicated option is validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Suite never exercises large budgets or several contract areas","status":"valid","defect":"BT-026","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"`:deadline` handler can overwrite a finished result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Init crashes if the port exits before `Port.info` is read","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Group is not killed when the BEAM identity read fails","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No design documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"`terminate/2` returns the state map","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":203,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.8,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":6.8,"score_cross_vendor":5.833,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.92,"tokens":{"uncached_input":12004,"cache_read":229504,"cache_write":0,"output":8331,"reasoning":1074},"cost_usd":0.7661,"cost_breakdown":{"input":0.12004,"cache_read":0.229504,"cache_write":0.0,"output":0.41655},"cost_usd_as_run":0.7661,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":321,"code":254,"comment":17,"blank":50,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":279,"code":233,"comment":0,"blank":46,"role":"test"}],"impl_lines":321,"impl_code":254,"impl_comment":17,"test_lines":279,"comment_ratio":0.063,"tests_declared":16},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":475.79,"reference":465.284,"speedup":0.978,"baselines":{"oracle":465.284}},{"id":"throughput_64m","candidate":2627.912,"reference":2617.106,"speedup":1.004,"baselines":{"oracle":2617.106}}]},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.0,"gpt-6-astra-xhigh":6.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"opus-5.5-xhigh":"matches the reference in every probe; good tests (runner crash, monitor/port leak checks); stub moduledoc","gpt-6.1-terra-max":"Correct on the exercised runner paths, but a test helper failed during repetition.","gpt-6.1-sol-max":"Strong common-path behavior, but fast exits can crash startup.","gpt-6-astra-xhigh":"Broad coverage, but startup can lose a successful tool’s result","deepseek-v4.1-flash-max":"Well-guarded elsewhere with strong tests; the fast-exit crash and a latent `:esrch` helper are the defects.","muse-wonder-full-xhigh":"correct and reasonably tested; stub doc and a latent flaky test helper"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Relay undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Pointless cleanup in `terminate`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extreme timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Module claims to be a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Procfs exit race crashes a test","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"VM SIGKILL is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Fast-exit startup race","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Cleanup helper misses a disappearance error","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Implementation still described as a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Fast completion can crash initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Trailing-slash paths execute unexpectedly","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Completed implementation is documented as a stub","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"robustness","title":"Runner init crashes when the tool exits before the PID is read","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"`alive?/1` has no `{:error, :esrch}` clause","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Stale stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Test `alive?/1` crashes intermittently on `{:error, :esrch}`","status":"valid","defect":"BT-025","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Leftover stub moduledoc","status":"valid","defect":"BT-033","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Init crashes if the port exits before `Port.info` is read","status":"valid","defect":"BT-007","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":228,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":1.1,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":4.5,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":70.47,"tokens":{"uncached_input":384874,"cache_read":11394432,"cache_write":0,"output":123738,"reasoning":98367},"cost_usd":0.4614,"cost_breakdown":{"input":0.057731,"cache_read":0.341833,"cache_write":0.0,"output":0.061869},"cost_usd_as_run":0.4614,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":472,"code":331,"comment":78,"blank":63,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":410,"code":317,"comment":7,"blank":86,"role":"test"}],"impl_lines":472,"impl_code":331,"impl_comment":78,"test_lines":410,"comment_ratio":0.191,"tests_declared":18},"hidden":{"passed":15,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":478.811,"reference":463.758,"speedup":0.969,"baselines":{"oracle":463.758}},{"id":"throughput_64m","candidate":2585.742,"reference":2601.131,"speedup":0.994,"baselines":{"oracle":2601.131}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":4.5,"gpt-6-astra-xhigh":5.0,"deepseek-v4.1-flash-max":6.0,"opus-5.5-xhigh":5.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Clear relay documentation and broad tests, but startup and option-validation defects remain.","gpt-6.1-sol-max":"Strong lifecycle coverage, but startup races and validation errors violate the contract.","gpt-6-astra-xhigh":"Broad implementation and tests, but successful commands can be lost during startup.","deepseek-v4.1-flash-max":"Best-engineered file of the four (docs, structure, tests), but a reproducible `init/1` race returns `{:error, :unavailable}` for runs that succeed, and it makes its own suite flaky.","opus-5.5-xhigh":"best documentation of the four, but init fails closed and reports successful runs as `:unavailable` under contention","muse-wonder-full-xhigh":"Near-oracle with the best docs and the only VM-SIGKILL test; duplicate-option validation and an init identity abort are the gaps"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A fast tool can be reported unavailable","status":"valid","defect":"BT-003","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Invalid duplicate options can launch a tool","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Documentation promises all group members are gone before return","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Fast successful tools can become unavailable","status":"valid","defect":"BT-003","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Normalization accepts an invalid executable path","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Invalid duplicate options still execute the tool","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Exited leader turns success into unavailable","status":"valid","defect":"BT-003","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Invalid duplicate options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Normalization changes executable validity","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Cleanup guarantee exceeds the contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"`init/1` requires a `/proc` identity read, so a successful fast tool can be reported as unavailable","status":"valid","defect":"BT-003","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys are checked by key only","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable start result","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Successful runs reported as `{:error, :unavailable}`","status":"valid","defect":"BT-003","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Duplicate option keys bypass validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable clause","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong return type in a spec","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Moduledoc overclaims","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Fragile shell in tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Duplicate option values are not validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Init aborts a fast-finishing run when the leader identity is unreadable","status":"valid","defect":"BT-003","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No runner-crash or pool-shutdown test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `{:ok, runner, _info}` clause","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"`kill_group/3-4` specs promise `:ok` but the kill branch returns the `System.cmd/3` tuple and the give-up branch returns `nil`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":288,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":85,"flags":7,"audited":"2026-11-02","reviewed":true},"score":6.2,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":5.3,"score_cross_vendor":6.2,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.53,"tokens":{"uncached_input":10261,"cache_read":327680,"cache_write":0,"output":9489,"reasoning":1557},"cost_usd":0.0741,"cost_breakdown":{"input":0.010261,"cache_read":0.016384,"cache_write":0.0,"output":0.047445},"cost_usd_as_run":0.0741,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"lib/bounded_tool.ex","lines":320,"code":259,"comment":13,"blank":48,"role":"impl"},{"path":"test/bounded_tool_test.exs","lines":268,"code":218,"comment":1,"blank":49,"role":"test"}],"impl_lines":320,"impl_code":259,"impl_comment":13,"test_lines":268,"comment_ratio":0.048,"tests_declared":12},"hidden":{"passed":16,"total":16,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"seq_true","candidate":467.639,"reference":451.171,"speedup":0.965,"baselines":{"oracle":451.171}},{"id":"throughput_64m","candidate":2514.26,"reference":2595.098,"speedup":0.969,"baselines":{"oracle":2595.098}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":4.0,"gpt-6-astra-xhigh":5.0,"deepseek-v4.1-flash-max":7.0,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"The main execution paths work, but three reproducible correctness defects remain.","gpt-6.1-sol-max":"Completed-result corruption adds to the startup, resolution, and validation defects.","gpt-6-astra-xhigh":"Core execution works, but deadline handling can replace an already completed result.","deepseek-v4.1-flash-max":"Correct and compact, but under-documented, under-tested, and it repeats N's duplicate-option and late-`:deadline` issues.","opus-5.5-xhigh":"follows the reference closely but drops three of its small guards and documents almost nothing","muse-wonder-full-xhigh":"Same deadline-overwrite and duplicate-option defects as entry-Q, with slightly broader tests and safer identity parsing, but minimal docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A fast tool can be reported unavailable","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Invalid duplicate options can launch a tool","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A completed result can be overwritten by the deadline","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Extremely large timeout differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Relay invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No separate-VM SIGKILL test","status":"valid","defect":"BT-035","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Fast command completion can crash initialization","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Deadline overwrites an already-completed result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Normalization changes executable-path validity","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Later invalid duplicate values escape validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Deadline overwrites a saved result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Invalid duplicate options are accepted","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Normalization changes executable validity","status":"valid","defect":"BT-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Test suite misses the risky contract cases","status":"valid","defect":"BT-035","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"Duplicate option keys are checked by key only","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"A late `:deadline` can overwrite an already-stored result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Documentation is nominal","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"No-op guards","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Duplicate option keys bypass validation","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"A late `:deadline` overwrites a stored result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Init crashes if the port has already closed","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Core mechanism undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Tests depend on timing and fail under load","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicate catch-alls and repeated reads","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"`:deadline` handler overwrites an already-finished result","status":"valid","defect":"BT-008","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Duplicate option values are not validated","status":"valid","defect":"BT-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Bare `{:os_pid, pid} = Port.info(port, :os_pid)` match in init","status":"valid","defect":"BT-007","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"One-line moduledoc and a single comment in the whole Runner","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No VM-SIGKILL/backstop, runner-crash, pool-shutdown, or over-limit-timeout test (12 tests)","status":"valid","defect":"BT-035","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"`alive?/1` shells out to `ps` on every poll","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"BEAM identity is re-read from `/proc` on every `kill_group` retry, and the `when is_port(port)` guards add nothing over the state pin","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":230,"window":6},"complexity":null,"files":["lib/bounded_tool.ex"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.4,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":7.4,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1-b2","task":"cdef-avx512","lead":"gpt-6-astra-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":8.26,"tokens":{"cache_read":3307520,"cache_write":0,"output":22110,"reasoning":13739,"uncached_input":114576},"cost_usd":0.3905,"cost_breakdown":{"cache_read":0.165376,"cache_write":0.0,"input":0.114576,"output":0.11055},"cost_usd_as_run":0.3905,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":10.68,"tokens":{"cache_read":3995009,"cache_write":216927,"output":63238,"reasoning":46816,"uncached_input":50},"cost_usd":3.7994,"cost_breakdown":{"cache_read":0.799002,"cache_write":1.735416,"input":0.0002,"output":1.26476},"cost_usd_as_run":3.7994,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.318","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":11.97,"tokens":{"cache_read":3323136,"cache_write":0,"output":27998,"reasoning":18499,"uncached_input":171400},"cost_usd":0.9551,"cost_breakdown":{"cache_read":0.332314,"cache_write":0.0,"input":0.3428,"output":0.27998},"cost_usd_as_run":0.9551,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":23.15,"tokens":{"cache_read":15763072,"cache_write":0,"output":223521,"reasoning":194390,"uncached_input":130135},"cost_usd":0.2009,"cost_breakdown":{"cache_read":0.047289,"cache_write":0.0,"input":0.01952,"output":0.134113},"cost_usd_as_run":0.2009,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":14.95,"tokens":{"cache_read":4412336,"cache_write":0,"output":42417,"reasoning":24076,"uncached_input":278379},"cost_usd":2.8443,"cost_breakdown":{"cache_read":0.882467,"cache_write":0.0,"input":1.113516,"output":0.84834},"cost_usd_as_run":2.8443,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":5.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":9.0}],"panel":["gpt-6.1-terra-max","opus-5.5-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":81.6,"tokens":{"uncached_input":262949,"cache_read":5657940,"cache_write":0,"output":195989,"reasoning":113454},"cost_usd":3.0516,"cost_breakdown":{"input":0.525898,"cache_read":0.565794,"cache_write":0.0,"output":1.95989},"cost_usd_as_run":3.0516,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"lines":579,"code":467,"comment":21,"blank":91,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":999,"code":860,"comment":51,"blank":88,"path":"tests/selftest.c","role":"test"}],"impl_lines":579,"impl_code":467,"impl_comment":21,"test_lines":999,"comment_ratio":0.043,"tests_declared":6,"object":{"text_bytes":4142,"rodata_bytes":376}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":110.73,"reference":3053.13,"speedup":27.573,"baselines":{"dav1d AVX2":137.06,"dav1d AVX-512ICL":52.36}},{"id":"find_dir","candidate":33.72,"reference":368.9,"speedup":10.94,"baselines":{"dav1d AVX2":38.76}}]},"isolation":{"tool_calls":42,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Fast, correct in the probes, and well tested, with a read-coverage gap and sparse explanation of the direction vectors.","opus-5.5-xhigh":"fastest filter (32×), correct and robust, but the find_dir shuffle network is undocumented and the filter macros are duplicated","gpt-6.1-sol-max":"Fast implementation with a reproducible test-harness defect and sparse direction-search documentation.","deepseek-v4.1-flash-max":"Fastest filter (32.6x) and correct throughout, but a 150-line direction search with essentially no comments and a duplicated tap macro.","muse-wonder-full-xhigh":"bit-exact and fastest filter, held back by sparse find_dir documentation and duplicated loop bodies"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard checks cover buffer ends, not every permitted row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Direction-search lane transformations lack an explanation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"find_dir shuffle network has no explanation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Duplicated tap macros and loop bodies","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"TAP macros `neg` the offset registers in place","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Loop-invariant constant reloaded each iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Silent failure path","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Inline calls omit vector and mask clobbers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Direction-search lane invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Direction search undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Duplicated tap macros","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"edge_masks values unexplained","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"find_dir shuffle network unexplained","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Three copy-pasted filter loop bodies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"TAP macros negate the caller's offset register","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Constant broadcast inside batch loops","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guarded buffers are full pages at two strides","status":"valid","detail":"","location":""}],"score":8.5,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":3.5,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","task":"cdef-avx512","lead":"gpt-6-astra-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":77.56,"tokens":{"cache_read":26390414,"cache_write":1059941,"output":342453,"reasoning":242068,"uncached_input":298},"cost_usd":20.608,"cost_breakdown":{"cache_read":5.278082,"cache_write":8.479528,"input":0.001192,"output":6.84906},"cost_usd_as_run":20.608,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.318","sessions":4}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":38.98,"tokens":{"cache_read":22076160,"cache_write":0,"output":101240,"reasoning":58790,"uncached_input":630266},"cost_usd":2.2402,"cost_breakdown":{"cache_read":1.103808,"cache_write":0.0,"input":0.630266,"output":0.5062},"cost_usd_as_run":2.2402,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":140.92,"tokens":{"cache_read":18279424,"cache_write":0,"output":195107,"reasoning":110820,"uncached_input":1131057},"cost_usd":12.0822,"cost_breakdown":{"cache_read":3.655885,"cache_write":0.0,"input":4.524228,"output":3.90214},"cost_usd_as_run":12.0822,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":4}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":61.84,"tokens":{"cache_read":17033984,"cache_write":0,"output":135216,"reasoning":86002,"uncached_input":682435},"cost_usd":4.4204,"cost_breakdown":{"cache_read":1.703398,"cache_write":0.0,"input":1.36487,"output":1.35216},"cost_usd_as_run":4.4204,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":63.9,"tokens":{"cache_read":41753984,"cache_write":0,"output":623378,"reasoning":471782,"uncached_input":464365},"cost_usd":0.569,"cost_breakdown":{"cache_read":0.125261,"cache_write":0.0,"input":0.069655,"output":0.374026},"cost_usd_as_run":0.569,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":4}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"D","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"D","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"D","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"D","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"N","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"D","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfC","source_run":"2026-11-02-cdef-avx512-gpt-6-astra-xhigh-r1","letter":"J","score":9.0}],"panel":["opus-5.5-xhigh","gpt-6.1-terra-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"A","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.79,"tokens":{"uncached_input":28022,"cache_read":160768,"cache_write":0,"output":8663,"reasoning":1338},"cost_usd":0.8741,"cost_breakdown":{"input":0.28022,"cache_read":0.160768,"cache_write":0.0,"output":0.43315},"cost_usd_as_run":0.8741,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":312,"code":296,"comment":10,"blank":6,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":509,"code":449,"comment":26,"blank":34,"path":"tests/selftest.c","role":"test"}],"impl_lines":312,"impl_code":296,"impl_comment":10,"test_lines":509,"comment_ratio":0.033,"tests_declared":5,"object":{"text_bytes":2877,"rodata_bytes":384}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":270.89,"reference":3213.83,"speedup":11.864,"baselines":{"dav1d AVX2":143.78,"dav1d AVX-512ICL":55.14}},{"id":"find_dir","candidate":178.0,"reference":393.93,"speedup":2.213,"baselines":{"dav1d AVX2":40.61}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with a slow direction search and sparse interface documentation","gpt-6.1-sol-max":"Correct and compact, with avoidable direction-search and filter costs.","muse-wonder-full-xhigh":"correct and compact; hardest to read, find_dir lives in stack memory","opus-5.5-xhigh":"correct; undocumented, slow find_dir, redundant masking","deepseek-v4.1-flash-max":"correct and compact, but barely documented and its direction search keeps every partial sum in stack memory"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search repeatedly updates stack-resident line sums","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public entry points lack interface and frame documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Shifted stack accumulators impede direction search","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Missing-tap mask is unnecessary","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Least readable entry","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Line sums are accumulated in stack arrays, not registers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Loop counter and pri_strength spill hide in tile padding words","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file or function documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir accumulates by read-modify-write in memory","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Unneeded sentinel mask, with a misleading comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Loop counter kept in tile padding","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"each of the seven line families is accumulated by 16-byte read-modify-write pairs in stack buffers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no file header and no explanation of the buffer layout or the bin mapping","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the row counter lives in tile padding and `pri` is stashed at `[rbp-4]`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":296,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.0,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.9,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.89,"tokens":{"uncached_input":28254,"cache_read":372736,"cache_write":0,"output":10054,"reasoning":2240},"cost_usd":0.1943,"cost_breakdown":{"input":0.056508,"cache_read":0.037274,"cache_write":0.0,"output":0.10054},"cost_usd_as_run":0.1943,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":381,"code":367,"comment":8,"blank":6,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":500,"code":443,"comment":23,"blank":34,"path":"tests/selftest.c","role":"test"}],"impl_lines":381,"impl_code":367,"impl_comment":8,"test_lines":500,"comment_ratio":0.021,"tests_declared":5,"object":{"text_bytes":4198,"rodata_bytes":4800}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":263.08,"reference":3176.88,"speedup":12.076,"baselines":{"dav1d AVX2":142.44,"dav1d AVX-512ICL":54.38}},{"id":"find_dir","candidate":108.72,"reference":382.55,"speedup":3.519,"baselines":{"dav1d AVX2":40.27}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with costly direction permutations and sparse documentation","gpt-6.1-sol-max":"Correct generated tables and filtering, with unnecessary per-tap work.","muse-wonder-full-xhigh":"correct and compact; near-zero documentation over generated tables","opus-5.5-xhigh":"correct, but almost undocumented and does redundant work in every tap","deepseek-v4.1-flash-max":"correct and compact, but effectively undocumented and it pays for a mask it does not need"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search performs 64 full-width table permutes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Entry-point and frame invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Sentinel masking repeats an established zero result","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Almost no documentation for generated tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard test fails outright on non-4096 page sizes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"pri_shift is not clamped to ≥0","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"“Explicitly clear lanes 8..31” describes no instruction","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file or function documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Sentinel masking is unnecessary","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"Explicitly clear lanes 8..31\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Heavy find_dir tables and extra address arithmetic","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no file header at all and the two 4 KB generated tables are unexplained","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"the sentinel mask in `TAP` duplicates what `constrain()` already guarantees","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"each tap materialises its address with `mov r12,r10` / `add|sub r12,%1`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":367,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.3,"slop":{"score":0.8,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":2.5,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":195.15,"tokens":{"uncached_input":779239,"cache_read":140351296,"cache_write":0,"output":595432,"reasoning":473198},"cost_usd":4.6251,"cost_breakdown":{"input":0.116886,"cache_read":4.210539,"cache_write":0.0,"output":0.297716},"cost_usd_as_run":4.6251,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":794,"code":623,"comment":125,"blank":46,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":778,"code":693,"comment":40,"blank":45,"path":"tests/selftest.c","role":"test"}],"impl_lines":794,"impl_code":623,"impl_comment":125,"test_lines":778,"comment_ratio":0.167,"tests_declared":7,"object":{"text_bytes":5095,"rodata_bytes":600}},"hidden":{"passed":9,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3042.75,"speedup":null,"baselines":{"dav1d AVX2":136.6,"dav1d AVX-512ICL":52.2}},{"id":"find_dir","candidate":null,"reference":367.64,"speedup":null,"baselines":{"dav1d AVX2":38.63}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":478,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":347,"flags":0,"audited":"2026-11-04","reviewed":false},"scores":{"muse-wonder-full-xhigh":3.0,"gpt-6.1-terra-max":2.0,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":2.5,"deepseek-v4.1-flash-max":2.0},"council_summaries":{"muse-wonder-full-xhigh":"Both entry points return wrong results on ordinary input; own suite then segfaults","gpt-6.1-terra-max":"Both exported operations return wrong results on valid inputs, and the ABI self-test crashes.","opus-5.5-xhigh":"Both functions give wrong output on most inputs; the entry's own `make test` fails 7 of 8 tests and then crashes","gpt-6.1-sol-max":"Both required operations produce wrong results, and the ABI self-test crashes independently","deepseek-v4.1-flash-max":"Both entry points return wrong results (80% of the filter grid, every random direction block), and its own test fails and then crashes."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"find_dir selects the minimum-cost direction","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter constrain uses shifted |diff| as the magnitude","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"cost[best^4] selection never returns cost[3]","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"ABI test segfaults and kills `make test`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Debug dump left in guard test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong sentinel range in header","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Jump to next instruction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Direction selection chooses the minimum cost","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter constrains the shifted difference","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Orthogonal cost for direction 7 selects cost 2","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI probe corrupts its caller’s state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Guard test allows forbidden right-side reads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Always-false empty condition","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"The filter limits each tap's magnitude to |d|>>shift instead of |d|","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Direction search picks the *lowest* cost","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"For best_dir 7, `*var` uses cost[2] instead of cost[3]","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"The test gate fails: 7 of 8 tests FAIL, and \"ABI preservation\" crashes with SIGSEGV","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The ABI test passes the stack arguments one slot too high and sets rbp to 0","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The guard test aliases its buffers and guards only one side","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Debugging leftovers in the tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Direction search contains no-op instructions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Constrain loses the original magnitude","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Direction selection chooses the minimum cost","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Orthogonal cost lookup omits direction three","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI probe corrupts its own frame","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Guard windows remain wider than permitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Unused accumulator initialization","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"Filter applies the wrong constrain magnitude","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`find_dir` returns the direction with the **smallest** cost","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"`make test` fails and then dies","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"The ABI test's inline asm is undefined behaviour","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"The comments describe the intended arithmetic, not the implemented one","status":"valid","detail":"","location":""}],"score":2.3,"slop":{"score":1.5,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":1.5,"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":2.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":14.7,"score_cross_vendor":2.3,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":319.42,"tokens":{"uncached_input":444591,"cache_read":3446064,"cache_write":0,"output":54231,"reasoning":30099},"cost_usd":3.5522,"cost_breakdown":{"input":1.778364,"cache_read":0.689213,"cache_write":0.0,"output":1.08462},"cost_usd_as_run":3.5522,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":498,"code":432,"comment":51,"blank":15,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":725,"code":647,"comment":37,"blank":41,"path":"tests/selftest.c","role":"test"}],"impl_lines":498,"impl_code":432,"impl_comment":51,"test_lines":725,"comment_ratio":0.106,"tests_declared":8,"object":{"text_bytes":5013,"rodata_bytes":240}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":250.19,"reference":3255.65,"speedup":13.013,"baselines":{"dav1d AVX2":145.92,"dav1d AVX-512ICL":55.43}},{"id":"find_dir","candidate":220.88,"reference":390.07,"speedup":1.766,"baselines":{"dav1d AVX2":40.98}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but its ABI fixture is out of bounds and direction search is slow","gpt-6.1-sol-max":"Correct assembly, weakened by an invalid ABI fixture and expensive scratch-buffer direction search.","muse-wonder-full-xhigh":"correct and clean; find_dir pays six stack-memory rebuilds","opus-5.5-xhigh":"correct, but find_dir is barely faster than the C reference","deepseek-v4.1-flash-max":"correct and well documented, but its direction search is the slowest of the ten (1.7x over the C reference)"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI fixture passes border pointers without a left prefix","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search repeatedly clears and refills a stack shift buffer","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI fixture lacks required left margins","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction search repeatedly materializes line layouts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"find_dir rebuilds a 256-byte shifted-row buffer six times via stack memory","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guarded filter test never guard-backs left[] and uses only stride 16","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"find_dir round-trips every line family through the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Filter overhead per tap","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard coverage is partial","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"`find_dir` zeroes 6×256 B of stack buffers per call and then does ~48 unaligned 32-byte stack loads plus 40 stores to build the partial sums","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"the register/stack map at lines 22-28 and 221-226 restates what the code says and has drifted into two overlapping descriptions of the same slots.","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.088,"lines":38,"significant":432,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":37,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.9,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":3.4,"score_cross_vendor":6.75,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":31.37,"tokens":{"uncached_input":133472,"cache_read":35950720,"cache_write":0,"output":266081,"reasoning":186924},"cost_usd":0.2875,"cost_breakdown":{"input":0.020021,"cache_read":0.107852,"cache_write":0.0,"output":0.159649},"cost_usd_as_run":0.2875,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2875,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":782,"code":645,"comment":75,"blank":62,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":847,"code":748,"comment":42,"blank":57,"path":"tests/selftest.c","role":"test"}],"impl_lines":782,"impl_code":645,"impl_comment":75,"test_lines":847,"comment_ratio":0.104,"tests_declared":8,"object":{"text_bytes":4312,"rodata_bytes":1296}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":182.58,"reference":3136.1,"speedup":17.177,"baselines":{"dav1d AVX2":141.09,"dav1d AVX-512ICL":54.0}},{"id":"find_dir","candidate":63.58,"reference":379.78,"speedup":5.973,"baselines":{"dav1d AVX2":39.72}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong implementation with a guard-test blind spot","gpt-6.1-sol-max":"Efficient, correct implementation; guard coverage and one load-count comment need correction.","muse-wonder-full-xhigh":"second-fastest design, huge test sweeps; one unguarded logarithm","opus-5.5-xhigh":"correct, second fastest, clear; one real gap in its guard test","deepseek-v4.1-flash-max":"correct, second-fastest on both entry points, clean structure and documentation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Trailing guard cases leave a 32-byte gap","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Tap-load comment understates the reads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Trailing guards leave forbidden bytes readable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Row-pair load count is incorrect","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"bsr is executed on pri_strength when it is zero","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"find_dir is ~260 lines of hand-unrolled permutes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Bottom staging recomputes the identical k1 mask and the loop counter is a memory RMW","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guarded filter test cannot catch forward over-reads","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"DF check never fires","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0992,"lines":64,"significant":645,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":154,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.6,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":1.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.6,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.39,"tokens":{"uncached_input":14364,"cache_read":99072,"cache_write":0,"output":506,"reasoning":111},"cost_usd":0.0218,"cost_breakdown":{"input":0.014364,"cache_read":0.004954,"cache_write":0.0,"output":0.00253},"cost_usd_as_run":0.0218,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":25,"code":10,"comment":10,"blank":5,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":25,"impl_code":10,"impl_comment":10,"test_lines":348,"comment_ratio":0.5,"tests_declared":3,"object":{"text_bytes":4,"rodata_bytes":0}},"hidden":{"passed":1,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3139.46,"speedup":null,"baselines":{"dav1d AVX2":141.3,"dav1d AVX-512ICL":53.88}},{"id":"find_dir","candidate":null,"reference":379.61,"speedup":null,"baselines":{"dav1d AVX2":39.81}}]},"scores":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0},"council_summaries":{"gpt-6.1-terra-max":"Both entry points remain stubs","gpt-6.1-sol-max":"Both required algorithms remain stubs.","muse-wonder-full-xhigh":"unmodified scaffold stub; no implementation","opus-5.5-xhigh":"unmodified stub: neither function is implemented","deepseek-v4.1-flash-max":"the scaffold stub is unchanged; nothing was implemented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"Direction search never writes variance","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"Filter always returns without filtering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Direction and variance are unimplemented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter performs no computation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Both entry points are the scaffold stub","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"No tests added","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"find_dir does nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"filter does nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Test file is the scaffold's, unmodified","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"both entry points are the unimplemented stub","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-04","reviewed":false},"score":0.1,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.6,"score_cross_vendor":0.167,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":40.43,"tokens":{"uncached_input":128012,"cache_read":2274092,"cache_write":0,"output":51745,"reasoning":33713},"cost_usd":2.0018,"cost_breakdown":{"input":0.512048,"cache_read":0.454818,"cache_write":0.0,"output":1.0349},"cost_usd_as_run":2.0018,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":495,"code":435,"comment":32,"blank":28,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":679,"code":605,"comment":31,"blank":43,"path":"tests/selftest.c","role":"test"}],"impl_lines":495,"impl_code":435,"impl_comment":32,"test_lines":679,"comment_ratio":0.069,"tests_declared":6,"object":{"text_bytes":3480,"rodata_bytes":576}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":280.17,"reference":3240.12,"speedup":11.565,"baselines":{"dav1d AVX2":144.96,"dav1d AVX-512ICL":55.47}},{"id":"find_dir","candidate":150.54,"reference":389.44,"speedup":2.587,"baselines":{"dav1d AVX2":40.94}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but its ABI test reads outside its inputs","gpt-6.1-sol-max":"Correct assembly; defective ABI-test buffers and redundant tap masking.","muse-wonder-full-xhigh":"correct; slowest in both functions and the thinnest guard tests","opus-5.5-xhigh":"correct and readable; slow find_dir, redundant masking, weaker tests","deepseek-v4.1-flash-max":"correct and tidy, but the direction search pays for rebuilding its mask on every shift"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI fixture omits the two left-border bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction shifts rebuild masks for every row and family","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI test reads before its buffers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Missing-tap masking duplicates constrain","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"SHADD recomputes its table address and k-mask and reloads a 32-byte permute on every call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Staging loops triplicated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"CADDW/CADD1 and PAIRW/PAIR1 duplicate the tap core","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guarded tests cover half the edge space","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Scalar setup inside every find_dir accumulation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Unneeded sentinel mask","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Copy-pasted code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard test covers only page ends","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`SHADD` reconstructs the shift mask and reloads the permute row on each of the six shifted accumulates per row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`rev8_idx`/`rev4_idx` are reloaded inside the row loop","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0506,"lines":22,"significant":435,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":21,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.1,"slop":{"score":0.9,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.4,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":82.48,"tokens":{"uncached_input":0,"cache_read":0,"cache_write":0,"output":0,"reasoning":0},"cost_usd":0.0,"cost_breakdown":{"input":0.0,"cache_read":0.0,"cache_write":0.0,"output":0.0},"cost_usd_as_run":0.0,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":null},"status":"scored","code":{"files":[{"lines":606,"code":506,"comment":45,"blank":55,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1219,"code":1126,"comment":39,"blank":54,"path":"tests/selftest.c","role":"test"}],"impl_lines":606,"impl_code":506,"impl_comment":45,"test_lines":1219,"comment_ratio":0.082,"tests_declared":6,"object":{"text_bytes":5257,"rodata_bytes":704}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":313.3,"reference":3199.3,"speedup":10.212,"baselines":{"dav1d AVX2":143.53,"dav1d AVX-512ICL":54.8}},{"id":"find_dir","candidate":104.27,"reference":385.57,"speedup":3.698,"baselines":{"dav1d AVX2":40.48}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but two test assumptions can reject conforming implementations","gpt-6.1-sol-max":"Correct arithmetic and substantial tests, but avoidable work remains in both hot paths.","muse-wonder-full-xhigh":"correct with the largest test suite; filter wastes work per row and per tap","opus-5.5-xhigh":"correct and well tested, but the slowest filter and some comment/test slop","deepseek-v4.1-flash-max":"correct with the strongest guard-test framework, but the slowest filter (32-bit lanes plus per-row broadcasts)"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI test calls with and expects a set direction flag","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"All guarded calls use an out-of-domain stride","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Filter broadcasts invariant parameters inside every row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Pattern loop copies data immediately before replacing it","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Loop-invariant vectors are repeatedly rebuilt","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Zeroing and insertion repeat VEX upper-lane clearing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Thresholds/shifts/weights re-broadcast inside the row loop on a widened 32-bit datapath","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Identical offset-load/setup block triplicated, including loads the mode never uses","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Comment names the wrong branch mnemonic","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Filter does per-tap 32-bit multiplies","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Offset-table setup copied three times","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Stale branch comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"DF check is backwards","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Test slop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"the filter widens every tap to 32-bit lanes (8 pixels per vector) and re-issues 8 loop-invariant broadcasts per row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"comment says the tie-break comparison is `jae` while the code uses `jbe`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":52,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.2,"slop":{"score":1.0,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":1.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":3.5,"score_cross_vendor":7.2,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":116.08,"tokens":{"uncached_input":1044,"cache_read":56264644,"cache_write":494355,"output":370857,"reasoning":0},"cost_usd":1.1736,"cost_breakdown":{"input":0.000157,"cache_read":0.900234,"cache_write":0.098871,"output":0.174303},"cost_usd_as_run":1.1736,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":617,"code":478,"comment":97,"blank":42,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1006,"code":876,"comment":73,"blank":57,"path":"tests/selftest.c","role":"test"}],"impl_lines":617,"impl_code":478,"impl_comment":97,"test_lines":1006,"comment_ratio":0.169,"tests_declared":8,"object":{"text_bytes":16680,"rodata_bytes":330}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3186.45,"speedup":null,"baselines":{"dav1d AVX2":142.61,"dav1d AVX-512ICL":54.72}},{"id":"find_dir","candidate":84.74,"reference":385.53,"speedup":4.55,"baselines":{"dav1d AVX2":40.31}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but a pattern test uses uninitialized inputs and the filter is large and slow","gpt-6.1-sol-max":"Correct, well-documented assembly; structured tests miss their intended inputs.","muse-wonder-full-xhigh":"correct and fast in find_dir; bloated unroll, false tap comments, thin ABI tests","opus-5.5-xhigh":"correct and well documented; heavy, unrolled filter and sloppy tests","deepseek-v4.1-flash-max":"correct, excellently documented, but 17 KB of text for an 8×8 filter"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Pattern setup fills the wrong left-buffer offset","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fully unrolled single-row kernels duplicate substantial work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Secondary tap comments swap directions and signs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Pattern initialization targets different locations from the call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Sentinel subtraction is described as non-wrapping","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Secondary k0 tap comments are swapped","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Fully unrolled rows in three modes plus near-identical cost macros","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-lane variable shifts and per-row constant rebuild","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinnest ABI and pattern breadth","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Filter does one row per xmm with a multiply per tap, all fully unrolled","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Absent neighbours are NULL, not the guard page","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Test leftovers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"DF check compares before/after","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"the filter is eight fully unrolled copies of the 12-tap row macro","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":174,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.6,"slop":{"score":1.0,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":2.7,"score_cross_vendor":7.6,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":16.25,"tokens":{"uncached_input":32,"cache_read":1771340,"cache_write":153532,"output":101998,"reasoning":71892},"cost_usd":3.6226,"cost_breakdown":{"input":0.000128,"cache_read":0.354268,"cache_write":1.228256,"output":2.03996},"cost_usd_as_run":3.6226,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":508,"code":402,"comment":78,"blank":28,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":914,"code":806,"comment":50,"blank":58,"path":"tests/selftest.c","role":"test"}],"impl_lines":508,"impl_code":402,"impl_comment":78,"test_lines":914,"comment_ratio":0.163,"tests_declared":8,"object":{"text_bytes":6937,"rodata_bytes":1316}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":146.68,"reference":3220.93,"speedup":21.959,"baselines":{"dav1d AVX2":144.71,"dav1d AVX-512ICL":55.26}},{"id":"find_dir","candidate":33.59,"reference":392.63,"speedup":11.689,"baselines":{"dav1d AVX2":40.69}}]},"scores":{"gpt-6.1-terra-max":10.0,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":9.5,"opus-5.5-xhigh":9.5,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Bit-exact in the exercised cases, well tested, and the fastest measured implementation","gpt-6.1-sol-max":"Correct, efficient wide-vector implementation with strong contract coverage.","muse-wonder-full-xhigh":"whole-block zmm design; fastest, best documented, strongest tests","opus-5.5-xhigh":"best design, fastest, well documented, strongest tests","deepseek-v4.1-flash-max":"the fastest entry on both entry points by a wide margin, and the most rigorous self-test"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Permute tables are magic numbers whose generator is not in the file","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"dir_idx_sb is reached only as an offset past dir_idx_sa","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the four 512-byte direction tables are opaque literals with only provenance prose","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":402,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":15,"flags":1,"audited":"2026-11-04","reviewed":true},"score":9.6,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.4,"score_cross_vendor":9.625,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.2,"tokens":{"uncached_input":23246,"cache_read":482432,"cache_write":0,"output":24364,"reasoning":12648},"cost_usd":0.1692,"cost_breakdown":{"input":0.023246,"cache_read":0.024122,"cache_write":0.0,"output":0.12182},"cost_usd_as_run":0.1692,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":377,"code":345,"comment":14,"blank":18,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":657,"code":588,"comment":29,"blank":40,"path":"tests/selftest.c","role":"test"}],"impl_lines":377,"impl_code":345,"impl_comment":14,"test_lines":657,"comment_ratio":0.039,"tests_declared":7,"object":{"text_bytes":4268,"rodata_bytes":608}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":273.87,"reference":3250.91,"speedup":11.87,"baselines":{"dav1d AVX2":144.87,"dav1d AVX-512ICL":55.35}},{"id":"find_dir","candidate":100.36,"reference":389.98,"speedup":3.886,"baselines":{"dav1d AVX2":40.9}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases and well tested, with substantial direction-search overhead","gpt-6.1-sol-max":"Correct, clear implementation and useful overlap tests; tap masking is redundant.","muse-wonder-full-xhigh":"correct, compact and well-factored; documentation is thin","opus-5.5-xhigh":"clean and conventional, strong tests; thin docs and redundant masking","deepseek-v4.1-flash-max":"correct, compact, good balance; a couple of avoidable redundancies"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search permutes each row separately for six line families","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Missing-tap masking adds no protection","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Sparse documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Sentinel test reloads a memory operand on every tap","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Unneeded sentinel mask","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Thin documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"sentinel tile fill rebuilds a constant already in `.rodata`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the file opens with a single line and the direction-search design (the 256-byte `perm_indices` shift table, the 16-word tile and why `filter_dirs` repeats rows 8–11) is only explained piecemeal in macro comments","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":345,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.9,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":1.5,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.57,"tokens":{"uncached_input":24272,"cache_read":95360,"cache_write":0,"output":2049,"reasoning":226},"cost_usd":0.0393,"cost_breakdown":{"input":0.024272,"cache_read":0.004768,"cache_write":0.0,"output":0.010245},"cost_usd_as_run":0.0393,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":114,"code":113,"comment":0,"blank":1,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":114,"impl_code":113,"impl_comment":0,"test_lines":348,"comment_ratio":0.0,"tests_declared":3,"object":{"text_bytes":401,"rodata_bytes":512}},"hidden":{"passed":3,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3223.66,"speedup":null,"baselines":{"dav1d AVX2":144.87,"dav1d AVX-512ICL":55.35}},{"id":"find_dir","candidate":468.04,"reference":388.72,"speedup":0.831,"baselines":{"dav1d AVX2":40.86}}]},"scores":{"gpt-6.1-terra-max":3.0,"muse-wonder-full-xhigh":2.5,"gpt-6.1-sol-max":2.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":2.0},"council_summaries":{"gpt-6.1-terra-max":"Direction search works, but the filter is absent","muse-wonder-full-xhigh":"find_dir works, but the filter is entirely missing","gpt-6.1-sol-max":"The entire filter implementation is missing.","opus-5.5-xhigh":"the filter is not implemented and the entry fails its own `make test`","deepseek-v4.1-flash-max":"the filter is not implemented at all"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter returns without filtering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search uses scalar per-pixel accumulation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter is a stub","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Scalar find_dir slower than C","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests are the unextended scaffold","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No comments","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter entry point immediately returns","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction accumulation uses scalar stack updates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Access and ABI requirements are untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"`cb_cdef_filter_8x8_8bpc_avx512` is a bare `ret`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"find_dir is a scalar per-pixel loop of memory read-modify-writes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"tests/selftest.c is the scaffold, unchanged, and `make test` reports 1 passed, 2 failed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No comments or documentation anywhere","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`cb_cdef_filter_8x8_8bpc_avx512` is an empty stub","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"the entry's own `make test` fails","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no documentation at all","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":113,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-04","reviewed":false},"score":2.1,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":8.0,"score_cross_vendor":1.833,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":317.88,"tokens":{"uncached_input":398055,"cache_read":1504348,"cache_write":0,"output":41079,"reasoning":21591},"cost_usd":2.7147,"cost_breakdown":{"input":1.59222,"cache_read":0.30087,"cache_write":0.0,"output":0.82158},"cost_usd_as_run":2.7147,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":495,"code":420,"comment":34,"blank":41,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":602,"code":540,"comment":28,"blank":34,"path":"tests/selftest.c","role":"test"}],"impl_lines":495,"impl_code":420,"impl_comment":34,"test_lines":602,"comment_ratio":0.075,"tests_declared":6,"object":{"text_bytes":3560,"rodata_bytes":512}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":263.75,"reference":3209.8,"speedup":12.17,"baselines":{"dav1d AVX2":143.91,"dav1d AVX-512ICL":54.97}},{"id":"find_dir","candidate":109.26,"reference":386.2,"speedup":3.535,"baselines":{"dav1d AVX2":40.65}}]},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks, with avoidable tap work and an incomplete ABI probe","muse-wonder-full-xhigh":"Solid and well documented, with only dead/redundant minor issues","gpt-6.1-sol-max":"Correct and tested, with redundant setup and an ABI coverage gap.","opus-5.5-xhigh":"correct; row-at-a-time filter with work inside the loop that should be hoisted; leftover scratch comments; thinner tests","deepseek-v4.1-flash-max":"correct, compact, but with sloppy comment/dead-store hygiene and an undocumented sentinel assumption"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Tap weights are broadcast repeatedly inside the row loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI wrappers omit RBP and most upper-vector state","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Dead ymm14 zeroing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Dead edges store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Weight rebroadcast for every tap","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Leftover self-questioning comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Tap weights are broadcast repeatedly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI wrappers omit RBP verification","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unused spill and initialization remain","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Each weighted tap rebroadcasts its weight from r8d (`vmovd` + `vpbroadcastw`) and recomputes its address (`movsxd`/`lea`) every row; all parameters are spilled to the stack; one xmm row per iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Leftover note \"; rounding 8 (also reused as ptr below? no, reg)\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Nothing explains why unmasked 0x8000 taps contribute 0, the invariant DO_TAP relies on","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The ABI test makes one call per function and never poisons the upper halves of int arguments (r9 untouched; stack slots get 32-bit stores). The guard test uses one stride (32) and four placements, and its parameter table holds out-of-domain damping values (7, 2) that an inline ternary patches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`vpxorq ymm14 ; zero (for clearing temp)`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"find_dir pushes 4 callee-saved registers just to hold copies of rdi/rsi/rdx, but only rdx is ever clobbered","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"dead store","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"the missing-pixel argument is unstated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"leftover internal monologue","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":415,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":18,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.4,"slop":{"score":1.3,"per_seat":{"opus-5.5-xhigh":2.0,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":2.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.5}},"defect_points":3.5,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":43.24,"tokens":{"uncached_input":56,"cache_read":3800970,"cache_write":252092,"output":592015,"reasoning":565169},"cost_usd":0.9597,"cost_breakdown":{"input":1.8e-05,"cache_read":0.177457,"cache_write":0.214597,"output":0.567671},"cost_usd_as_run":0.9597,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":520,"code":451,"comment":41,"blank":28,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":850,"code":753,"comment":50,"blank":47,"path":"tests/selftest.c","role":"test"}],"impl_lines":520,"impl_code":451,"impl_comment":41,"test_lines":850,"comment_ratio":0.083,"tests_declared":8,"object":{"text_bytes":4729,"rodata_bytes":76}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":366.84,"reference":3225.42,"speedup":8.792,"baselines":{"dav1d AVX2":144.5,"dav1d AVX-512ICL":55.14}},{"id":"find_dir","candidate":452.59,"reference":388.01,"speedup":0.857,"baselines":{"dav1d AVX2":40.77}}]},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks, but both hot paths perform substantial avoidable work","muse-wonder-full-xhigh":"Exemplary docs and tests, but both functions are slower than needed and find_dir trails C","gpt-6.1-sol-max":"Robust and thoroughly tested, but substantial avoidable hot-path work remains.","opus-5.5-xhigh":"correct, well documented, good tests, but find_dir is scalar and slower than the C reference","deepseek-v4.1-flash-max":"best-in-class docs and tests, but the direction search is slower than the C reference"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search is scalar","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Single-strength filtering executes all twelve taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper-state probe checks only the upper halves of YMM registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Guard description claims every illegal read faults","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"Scalar find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Filter always runs all twelve taps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Byte-at-a-time border staging","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction accumulation remains scalar","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Disabled tap families still execute","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"find_dir is fully scalar: 64 iterations, each with 8 read-modify-write adds to the stack, then 64-bit scalar cost loops","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The filter computes all 12 taps in the primary-only and secondary-only modes (the unused family just gets threshold 0), takes every constant as a stack memory operand, and does `vpmullw` on every tap, row-at-a-time in xmm","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"find_dir is scalar","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"36 extra L1 loads per row in the filter","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0532,"lines":24,"significant":451,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":25,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.3,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.6,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":20.37,"tokens":{"uncached_input":46,"cache_read":2787375,"cache_write":165039,"output":111733,"reasoning":78982},"cost_usd":4.1126,"cost_breakdown":{"input":0.000184,"cache_read":0.557475,"cache_write":1.320312,"output":2.23466},"cost_usd_as_run":4.1126,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":555,"code":428,"comment":93,"blank":34,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":834,"code":728,"comment":52,"blank":54,"path":"tests/selftest.c","role":"test"}],"impl_lines":555,"impl_code":428,"impl_comment":93,"test_lines":834,"comment_ratio":0.179,"tests_declared":7,"object":{"text_bytes":6120,"rodata_bytes":1824}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":127.66,"reference":3256.49,"speedup":25.509,"baselines":{"dav1d AVX2":146.01,"dav1d AVX-512ICL":55.81}},{"id":"find_dir","candidate":47.07,"reference":392.54,"speedup":8.339,"baselines":{"dav1d AVX2":41.19}}]},"scores":{"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":9.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Fast and correct in the checks, with overstated guard coverage","muse-wonder-full-xhigh":"Near-best speed with documented proofs for its masking and constrain tricks","gpt-6.1-sol-max":"Fast and thoroughly tested, with one inaccurate test comment.","opus-5.5-xhigh":"full zmm, masked loads, careful documentation of every invariant, strong tests; second fastest","deepseek-v4.1-flash-max":"fastest-ish, excellent documentation, one uninitialised-read nit"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard test cannot detect every overread","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Constant-block costs are described as zero","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Absent-left staging reads uninitialised dummy stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Constant-cost explanation is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The last FILTER_BODY expansion's `jmp .done` lands on `.done` itself","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"`test_filter_patterns` mallocs and frees a whole `bufs_t` (about 50 KB) in its innermost loop; the absent pointer `noaccess + page_size/2` with stride 4096 puts row −1 outside the inaccessible page","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"uninitialised dummy source","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":428,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":21,"flags":1,"audited":"2026-11-04","reviewed":true},"score":8.8,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.8,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":14.71,"tokens":{"uncached_input":33579,"cache_read":303104,"cache_write":0,"output":21209,"reasoning":10514},"cost_usd":1.6993,"cost_breakdown":{"input":0.33579,"cache_read":0.303104,"cache_write":0.0,"output":1.06045},"cost_usd_as_run":1.6993,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":343,"code":304,"comment":18,"blank":21,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":731,"code":650,"comment":34,"blank":47,"path":"tests/selftest.c","role":"test"}],"impl_lines":343,"impl_code":304,"impl_comment":18,"test_lines":731,"comment_ratio":0.056,"tests_declared":6,"object":{"text_bytes":3868,"rodata_bytes":352}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":264.38,"reference":3239.74,"speedup":12.254,"baselines":{"dav1d AVX2":145.29,"dav1d AVX-512ICL":56.06}},{"id":"find_dir","candidate":90.28,"reference":389.98,"speedup":4.32,"baselines":{"dav1d AVX2":40.98}}]},"scores":{"gpt-6.1-terra-max":8.0,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Strong coverage and correct results, with room to improve filter throughput","muse-wonder-full-xhigh":"Fast find_dir and solid tests, with a mid-speed row-at-a-time filter","gpt-6.1-sol-max":"Complete, readable implementation with strong safety and ABI tests.","opus-5.5-xhigh":"correct; elegant rotate-and-add find_dir with a vector argmax; filter is row-at-a-time xmm","deepseek-v4.1-flash-max":"small, fast, and correct; one comment hides a real invariant"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Filter processes one eight-pixel row at a time","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Probe treats caller-saved x87 data as preserved","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Filter uses only xmm row width","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Obscure strength-mode dispatch","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter processes one xmm row per iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Missing-tap mask (`vpcmpw` + masked `vpsubw`)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`jnz .both` / `jmp .secondary` with `.both` directly after (should be `jz .secondary`), and the trailing `jmp .done`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The negative tap address is recomputed per tap","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"\"reversed 3-(y>>1)+x\"","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":304,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.1,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":1.7,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":17.57,"tokens":{"uncached_input":38995,"cache_read":563072,"cache_write":0,"output":31612,"reasoning":16983},"cost_usd":0.4504,"cost_breakdown":{"input":0.07799,"cache_read":0.056307,"cache_write":0.0,"output":0.31612},"cost_usd_as_run":0.4504,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":370,"code":338,"comment":18,"blank":14,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":811,"code":732,"comment":34,"blank":45,"path":"tests/selftest.c","role":"test"}],"impl_lines":370,"impl_code":338,"impl_comment":18,"test_lines":811,"comment_ratio":0.051,"tests_declared":6,"object":{"text_bytes":3563,"rodata_bytes":472}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":182.29,"reference":3133.12,"speedup":17.188,"baselines":{"dav1d AVX2":140.63,"dav1d AVX-512ICL":53.58}},{"id":"find_dir","candidate":88.9,"reference":377.47,"speedup":4.246,"baselines":{"dav1d AVX2":39.6}}]},"scores":{"gpt-6.1-terra-max":8.0,"muse-wonder-full-xhigh":8.5,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and reasonably fast, with two ABI-test limitations","muse-wonder-full-xhigh":"Fast, clear pair-tap design with strong tests and only nits","gpt-6.1-sol-max":"Complete, efficient implementation with strong deterministic, memory and ABI coverage.","opus-5.5-xhigh":"clean and correct, good find_dir, thorough tests; the filter is still one row per iteration","deepseek-v4.1-flash-max":"fast and well explained; only a micro-redundancy"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Filter processes only one row per iteration","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper-state probe omits high ZMM halves","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI test requires caller-saved x87 data to remain unchanged","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"vpabsw called an unsigned magnitude","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Argument registers reused as tap offsets","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The filter is row-at-a-time, using a ymm to hold the ± tap pair","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"`vpcmpw` + `{k1}{z}` mask on missing taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The last FILTER_ROWS expansion ends with `jmp .filter_done` straight into that label","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"The strength mode is encoded as 0/1/2 in r15d and compared again, where direct branches would do","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"`mov`/`sub` to build the negative tap address on every tap of every row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"pointless final pointer bump","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":338,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.4,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":1.7,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.43,"tokens":{"uncached_input":55504,"cache_read":464640,"cache_write":0,"output":14969,"reasoning":4605},"cost_usd":0.1536,"cost_breakdown":{"input":0.055504,"cache_read":0.023232,"cache_write":0.0,"output":0.074845},"cost_usd_as_run":0.1536,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":472,"code":448,"comment":17,"blank":7,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":620,"code":561,"comment":24,"blank":35,"path":"tests/selftest.c","role":"test"}],"impl_lines":472,"impl_code":448,"impl_comment":17,"test_lines":620,"comment_ratio":0.037,"tests_declared":7,"object":{"text_bytes":3708,"rodata_bytes":5760}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":288.57,"reference":3212.78,"speedup":11.133,"baselines":{"dav1d AVX2":143.91,"dav1d AVX-512ICL":54.97}},{"id":"find_dir","candidate":147.43,"reference":392.71,"speedup":2.664,"baselines":{"dav1d AVX2":40.65}}]},"scores":{"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":7.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks, but its ABI test rejects valid behavior","muse-wonder-full-xhigh":"Correct with strong tests, but a wrong macro comment and dead/verbose tables","gpt-6.1-sol-max":"Correct implementation with minor hot-loop and test/documentation defects.","opus-5.5-xhigh":"correct and safe, but the filter uses 128-bit vectors one row at a time; one wrong doc comment, dead table rows, an ABI test that checks for the wrong DF behaviour","deepseek-v4.1-flash-max":"correct and fast on dir, but a 5 KB table with an unused slot and loop-invariant work in the hot loop"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI probe enters with DF set and requires it to remain set","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper-state probe checks only ZMM0","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"dead-code","title":"Direction index tables include unused vectors","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Primary-strength parity is retested for every row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"TAP_ONE parameter comment swaps roles","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unused permute tables in line_indices","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Invariant parity branch and rounding setup inside the row loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Invariant setup remains inside the row loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI helpers require the wrong DF state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"TAP_ONE documents different arguments and behavior","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter does one 8-pixel row per iteration in xmm","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir does 9 permutes per row from a 5 KB index table, 8 separate full horizontal reductions and a scalar argmax","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The missing-tap mask (`vpcmpeqw` + `vpandn`) does nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"TAP_ONE's parameter comment is wrong","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"One index vector in every 640-byte `line_indices` group (offset 128) is never loaded","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The ABI wrapper calls with DF=1 and fails unless DF is still 1 on return","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"find_dir pushes and pops rbx and r15 but never uses them","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The rounding constant is rebuilt every row (`mov`/`vmovd`/`vpbroadcastw`), and each tap reloads and negates its offset from the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The absent pointer is the guard page, and with stride 4096 `top+stride` lands in readable memory, so reads of row −1 without TOP aren't caught","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"512 bytes of never-read permutation data","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the table gives no clue which slot is which direction","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"rounding vector rebuilt every row","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":448,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.2,"slop":{"score":1.9,"per_seat":{"opus-5.5-xhigh":3.5,"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":3.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":4.5,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.87,"tokens":{"uncached_input":57923,"cache_read":274048,"cache_write":0,"output":6016,"reasoning":954},"cost_usd":0.2034,"cost_breakdown":{"input":0.115846,"cache_read":0.027405,"cache_write":0.0,"output":0.06016},"cost_usd_as_run":0.2034,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":362,"code":358,"comment":2,"blank":2,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":417,"code":364,"comment":22,"blank":31,"path":"tests/selftest.c","role":"test"}],"impl_lines":362,"impl_code":358,"impl_comment":2,"test_lines":417,"comment_ratio":0.006,"tests_declared":4,"object":{"text_bytes":2973,"rodata_bytes":2784}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":323.05,"reference":3237.35,"speedup":10.021,"baselines":{"dav1d AVX2":145.04,"dav1d AVX-512ICL":55.6}},{"id":"find_dir","candidate":1326.83,"reference":389.77,"speedup":0.294,"baselines":{"dav1d AVX2":40.86}}]},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":6.0,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":4.5,"deepseek-v4.1-flash-max":5.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks, but direction search is substantially slower than the reference","muse-wonder-full-xhigh":"Correct, but find_dir is far slower than the C reference and documentation is sparse","gpt-6.1-sol-max":"Correct outputs, but direction search is substantially slower than the reference.","opus-5.5-xhigh":"correct, but find_dir is 3.4× slower than the C reference, and the code is almost undocumented","deepseek-v4.1-flash-max":"correct, but a gather/scatter direction search is 3.4× slower than the C reference, plus test/docs gaps"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Per-pixel gather and scatter serialize direction search","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Single-strength kernels still calculate tap extrema","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Suite has no callee-saved or upper-vector ABI probe","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Direction index table has no mapping explanation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"find_dir gather/scatter per pixel","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Almost no documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"No ABI probe and narrow guarded test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Min/max computed in single-strength kernels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Per-pixel gather/scatter serializes accumulation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Single-strength kernels compute unused extrema","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Machine-state ABI obligations lack tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"find_dir does a `vpgatherdd` plus a `vpscatterdd` to the stack for every pixel: 64 dependent pairs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The filter is row-at-a-time xmm, does `vpmullw` on every tap including weights 1 and 2, rebroadcasts weights inside the row loop, and tracks min/max even in the primary-only and secondary-only kernels where they're unused","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No file or function documentation; the 2.3 KB `indices` table, the `directions` layout and the stack slots 400/404/432/448 are unexplained","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The missing mask (`vpcmpeqw` against memory + `vpandn`) is a no-op","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Only one test added: dir is just the pattern index 0..5, damping is fixed at 3, there's no ABI or upper-state check, and absent row −1 isn't guarded","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`jmp both` sits immediately before the `both:` label, and the last kernel ends with `jmp filter_done` straight into `filter_done`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"The filter allocates a 640-byte frame but uses at most 472 bytes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"find_dir gathers and scatters per pixel","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no ABI or upper-state test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"the tables are unexplained","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"a 4.3 MB core dump is left in the submission","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":358,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-04","reviewed":false},"score":5.7,"slop":{"score":0.9,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.6,"score_cross_vendor":5.333,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":58.84,"tokens":{"uncached_input":78,"cache_read":7604126,"cache_write":249032,"output":332483,"reasoning":287402},"cost_usd":10.1631,"cost_breakdown":{"input":0.000312,"cache_read":1.520825,"cache_write":1.992256,"output":6.64966},"cost_usd_as_run":10.1631,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":729,"code":560,"comment":121,"blank":48,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1007,"code":883,"comment":70,"blank":54,"path":"tests/selftest.c","role":"test"}],"impl_lines":729,"impl_code":560,"impl_comment":121,"test_lines":1007,"comment_ratio":0.178,"tests_declared":8,"object":{"text_bytes":5667,"rodata_bytes":2176}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":120.01,"reference":3241.63,"speedup":27.011,"baselines":{"dav1d AVX2":145.04,"dav1d AVX-512ICL":55.68}},{"id":"find_dir","candidate":33.01,"reference":390.65,"speedup":11.834,"baselines":{"dav1d AVX2":40.94}}]},"scores":{"gpt-6.1-terra-max":9.0,"muse-wonder-full-xhigh":9.5,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":9.5,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Fast and well structured, with minor test and comment defects","muse-wonder-full-xhigh":"Fastest overall, thoroughly documented and the most extensively tested","gpt-6.1-sol-max":"Fast, complete SIMD implementation with extensive verification.","opus-5.5-xhigh":"the best: full zmm, byte-exact masked loads, generated tables, accurate docs, very thorough tests","deepseek-v4.1-flash-max":"fastest entry on both functions, thorough docs and the strongest test suite"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests leave gaps between rows readable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Constant-block costs are not generally zero","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated secondary setup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Constant blocks do not generally have zero costs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`vpxord xmm1,xmm1,xmm1 ; ZERO` writes the register directly instead of using the `ZERO` alias defined for zmm1","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`DIRTAPS` is declared with 12 parameters but called with three 4-tuples, relying on NASM pre-expanding `DIR2/DIR4/DIR0` into 12 scalars","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0214,"lines":12,"significant":560,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":41,"flags":0,"audited":"2026-11-04","reviewed":false},"score":9.3,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.7,"score_cross_vendor":9.25,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":12.77,"tokens":{"uncached_input":49193,"cache_read":387584,"cache_write":0,"output":23460,"reasoning":11749},"cost_usd":0.3717,"cost_breakdown":{"input":0.098386,"cache_read":0.038758,"cache_write":0.0,"output":0.2346},"cost_usd_as_run":0.3717,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":350,"code":318,"comment":15,"blank":17,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":710,"code":635,"comment":34,"blank":41,"path":"tests/selftest.c","role":"test"}],"impl_lines":350,"impl_code":318,"impl_comment":15,"test_lines":710,"comment_ratio":0.045,"tests_declared":6,"object":{"text_bytes":4405,"rodata_bytes":256}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":296.09,"reference":3173.52,"speedup":10.718,"baselines":{"dav1d AVX2":147.31,"dav1d AVX-512ICL":54.42}},{"id":"find_dir","candidate":171.24,"reference":379.06,"speedup":2.214,"baselines":{"dav1d AVX2":40.02}}]},"scores":{"gpt-6.1-terra-max":8.0,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and thoroughly exercised, though direction search is comparatively slow","muse-wonder-full-xhigh":"Correct and clean, mainly limited by stack-based find_dir performance","gpt-6.1-sol-max":"Correct and extensively tested, with avoidable direction-search spills.","opus-5.5-xhigh":"correct, well commented, strong tests; the filter is row-at-a-time xmm and find_dir goes through the stack","deepseek-v4.1-flash-max":"straightforward and correct, but the stack-array direction search is the slowest of the 16-bit designs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction partial sums repeatedly round-trip through the stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probe requires preservation of caller-saved x87 data","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Overlapping stack accumulation in find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Three-instruction sign application","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No file-level overview","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction sums repeatedly round-trip through the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter processes one xmm row per iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir accumulates partial sums by read-modify-write through the stack: 16-byte reloads right after 64-byte zeroing stores, so store forwarding stalls; each cost is reduced separately","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The valid-tap mask (`vpcmpw k1` + zero-masked `vpsubw`) isn't needed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The negative tap address is recomputed for every tap of every row (`mov rdi,r14` / `sub rdi,%1`)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The last expansion of `jmp .done` jumps to the very next instruction","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`.stage_block` repeats the body of the STAGE_ROW macro","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Absent top/bottom point at the guard page with stride 4096, so rows −1 and 9 aren't guarded","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"seven load+add+store round-trips per row in find_dir","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":318,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.7,"slop":{"score":0.6,"per_seat":{"opus-5.5-xhigh":2.0,"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.8,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.95,"tokens":{"uncached_input":50875,"cache_read":170624,"cache_write":0,"output":4102,"reasoning":45},"cost_usd":0.8845,"cost_breakdown":{"input":0.50875,"cache_read":0.170624,"cache_write":0.0,"output":0.2051},"cost_usd_as_run":0.8845,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":302,"code":295,"comment":3,"blank":4,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":396,"code":345,"comment":21,"blank":30,"path":"tests/selftest.c","role":"test"}],"impl_lines":302,"impl_code":295,"impl_comment":3,"test_lines":396,"comment_ratio":0.01,"tests_declared":4,"object":{"text_bytes":2987,"rodata_bytes":608}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":297.43,"reference":3175.28,"speedup":10.676,"baselines":{"dav1d AVX2":142.02,"dav1d AVX-512ICL":54.34}},{"id":"find_dir","candidate":389.56,"reference":382.04,"speedup":0.981,"baselines":{"dav1d AVX2":40.1}}]},"scores":{"opus-5.5-xhigh":5.5,"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":6.0,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":6.0},"council_summaries":{"opus-5.5-xhigh":"Correct, but find_dir is scalar and slower than the C reference, and the tests skip the ABI","gpt-6.1-terra-max":"Correct, but direction search and contract-focused tests need work","muse-wonder-full-xhigh":"Functionally correct, but find_dir is a scalar port with no speed-up over the C reference","gpt-6.1-sol-max":"Correct in probes, but direction-search performance and ABI test coverage need work.","deepseek-v4.1-flash-max":"Correct, but the direction search is slower than the C reference and the filter re-derives loop invariants per row."},"findings":[{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"find_dir is a scalar loop over every pixel","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter handles one 8-pixel row per pass in xmm registers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Loop-invariant setup inside the row loop, and a multiply on every tap","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The missing-tap mask does nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The added tests never check the ABI and guard only part of the memory","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost no comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`jmp .done` jumps to the very next line","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search serializes pixel accumulation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Filter recomputes invariant tap setup per row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Present left input has no guard boundary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI and upper-state regressions are not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Direction partial layout lacks a mapping","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Unused extrema initialized in single-strength kernels","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"Scalar find_dir defeats the AVX-512 deliverable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"No ABI-hygiene tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Filter setup repeated inside the row loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard test covers only the trailing edge with an aliased bottom","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Near-uncommented assembly","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Unconditional extrema initialisation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Scalar direction accumulation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state violations escape testing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unused extrema initialization","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"Direction search loses to the C reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Loop-invariant constants recomputed inside the row loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Filter stack arguments are bare offsets with no comment","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`weights` and `directions` are unlabelled magic","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The self-test never exercises the ABI","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"Guard test aliases bottom to top","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`mov eax,128; vpbroadcastd ymm1,eax` is re-executed per row in the find_dir loop","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":295,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.3,"slop":{"score":0.6,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":7.0,"score_cross_vendor":5.833,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1-b2","task":"cdef-avx512","lead":"gpt-6.1-sol-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":8.92,"tokens":{"cache_read":3789696,"cache_write":0,"output":23476,"reasoning":13504,"uncached_input":116887},"cost_usd":0.4238,"cost_breakdown":{"cache_read":0.189485,"cache_write":0.0,"input":0.116887,"output":0.11738},"cost_usd_as_run":0.4238,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":11.56,"tokens":{"cache_read":3173221,"cache_write":210416,"output":67480,"reasoning":50685,"uncached_input":44},"cost_usd":3.6677,"cost_breakdown":{"cache_read":0.634644,"cache_write":1.683328,"input":0.000176,"output":1.3496},"cost_usd_as_run":3.6677,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":11.86,"tokens":{"cache_read":7650560,"cache_write":0,"output":141338,"reasoning":115345,"uncached_input":105387},"cost_usd":0.1236,"cost_breakdown":{"cache_read":0.022952,"cache_write":0.0,"input":0.015808,"output":0.084803},"cost_usd_as_run":0.1236,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":13.99,"tokens":{"cache_read":2308992,"cache_write":0,"output":31056,"reasoning":23062,"uncached_input":124064},"cost_usd":0.7896,"cost_breakdown":{"cache_read":0.230899,"cache_write":0.0,"input":0.248128,"output":0.31056},"cost_usd_as_run":0.7896,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":20.03,"tokens":{"cache_read":4030668,"cache_write":0,"output":40692,"reasoning":24576,"uncached_input":214646},"cost_usd":2.4786,"cost_breakdown":{"cache_read":0.806134,"cache_write":0.0,"input":0.858584,"output":0.81384},"cost_usd_as_run":2.4786,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"V","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"V","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"V","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"V","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":10.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"V","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":9.5}],"panel":["gpt-6.1-terra-max","opus-5.5-xhigh","deepseek-v4.1-flash-max","gpt-6.1-sol-max","muse-wonder-full-xhigh"]},"entries":[{"letter":"O","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":1.52,"tokens":{"uncached_input":73751,"cache_read":195551,"cache_write":0,"output":6220,"reasoning":0},"cost_usd":0.2293,"cost_breakdown":{"input":0.147502,"cache_read":0.019555,"cache_write":0.0,"output":0.0622},"cost_usd_as_run":0.2293,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"lines":313,"code":235,"comment":49,"blank":29,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":313,"impl_code":235,"impl_comment":49,"test_lines":348,"comment_ratio":0.173,"tests_declared":3,"object":{"text_bytes":4,"rodata_bytes":1464}},"hidden":{"passed":1,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3144.75,"speedup":null,"baselines":{"dav1d AVX2":141.01,"dav1d AVX-512ICL":54.0}},{"id":"find_dir","candidate":null,"reference":379.15,"speedup":null,"baselines":{"dav1d AVX2":39.85}}]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"guardrail_blocks":[{"date":"2026-11-02T19:35:00+00:00","provider":"google","attempt":2,"message":"This request was blocked by Gemini's filters (contestant writing an AV1 CDEF AVX-512 kernel)","outcome":"retry-blocked","role":"contestant"},{"date":"2026-11-02T19:40:07+00:00","provider":"google","attempt":3,"message":"This request was blocked by Gemini's filters (contestant writing an AV1 CDEF AVX-512 kernel); stopped after 22 steps with partial work","outcome":"retry-blocked","role":"contestant"}],"scores":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.5},"council_summaries":{"gpt-6.1-terra-max":"Both public functions are unfinished stubs.","opus-5.5-xhigh":"Unimplemented stub, plus leftover \"Wait:\" notes left in as comments","deepseek-v4.1-flash-max":"both entry points are empty stubs; nothing is implemented and the shipped self-test fails","gpt-6.1-sol-max":"Both required operations remain stubs.","muse-wonder-full-xhigh":"both entry points are unimplemented stubs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Direction search returns a constant and never stores variance","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter does no work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Memory and ABI constraints lack tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"dead-code","title":"All constant tables are unused","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"verbosity","title":"Draft reasoning remains in comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Neither function is implemented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"dead-code","title":"About 300 lines of unused .rodata","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The comments are a raw thought stream and include false claims","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"tests/selftest.c:332","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`cb_cdef_find_dir_8bpc_avx512` returns 0 and never stores `*var`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`cb_cdef_filter_8x8_8bpc_avx512` is a bare `ret` and leaves `dst` unchanged","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"tests","title":"The entry's own `make test` fails all three groups (find_dir random, filter sweep, filter random)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"dead-code","title":"The entire `.rodata` block is unused: `dir_shift_0..7`, `perm_diag0_*`, `perm_diag1_*`, `perm_alt1_*`, `perm_diag_pairs`, `perm_alt_pairs`, `weights_diag`, `weights_alt`, `weights_hv`, `bias_128`, `dir_indices`, `filter_dir_offs`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"The constants section is a stream-of-consciousness draft, not documentation: \"…Wait: ZMM has 32 words (2 lanes of 16 words)\", \"Wait: if we have two rows in a ZMM…\", \"Yes, or reverse words 0..3 first (or in the permute table)!\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Direction search leaves variance unwritten","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter performs no filtering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Entire constants section is unused","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"find_dir is a stub that never writes *var","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"filter is a bare ret","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"dead-code","title":"~300 lines of rodata are never referenced","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"verbosity","title":"comments are unresolved design deliberation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"tests are the untouched scaffold and `make test` fails 0/3","status":"valid","detail":"","location":""}],"score":0.6,"slop":{"score":2.9,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":3.0,"deepseek-v4.1-flash-max":4.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":5.0}},"defect_points":10.7,"score_cross_vendor":0.6,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","task":"cdef-avx512","lead":"gpt-6.1-sol-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":40.95,"tokens":{"cache_read":24106368,"cache_write":0,"output":108172,"reasoning":61510,"uncached_input":664597},"cost_usd":2.4107,"cost_breakdown":{"cache_read":1.205319,"cache_write":0.0,"input":0.664597,"output":0.54086},"cost_usd_as_run":2.4107,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":66.08,"tokens":{"cache_read":52426496,"cache_write":0,"output":666770,"reasoning":533573,"uncached_input":568574},"cost_usd":0.6427,"cost_breakdown":{"cache_read":0.157279,"cache_write":0.0,"input":0.085286,"output":0.400061},"cost_usd_as_run":0.6427,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":4}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":79.03,"tokens":{"cache_read":17768192,"cache_write":0,"output":132101,"reasoning":86369,"uncached_input":719909},"cost_usd":4.5377,"cost_breakdown":{"cache_read":1.776819,"cache_write":0.0,"input":1.439818,"output":1.32101},"cost_usd_as_run":4.5377,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":71.93,"tokens":{"cache_read":29294240,"cache_write":1154660,"output":368634,"reasoning":262502,"uncached_input":312},"cost_usd":22.4701,"cost_breakdown":{"cache_read":5.858848,"cache_write":9.23728,"input":0.001248,"output":7.37268},"cost_usd_as_run":22.4701,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":4}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":195.36,"tokens":{"cache_read":28649508,"cache_write":0,"output":241289,"reasoning":131954,"uncached_input":1217202},"cost_usd":15.4245,"cost_breakdown":{"cache_read":5.729902,"cache_write":0.0,"input":4.868808,"output":4.82578},"cost_usd_as_run":15.4245,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":4}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"D","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"D","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":10.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"D","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"D","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"D","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfT","source_run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"C","score":9.5}],"panel":["gpt-6.1-terra-max","deepseek-v4.1-flash-max","gpt-6.1-sol-max","opus-5.5-xhigh","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":330.31,"tokens":{"uncached_input":181204,"cache_read":1879608,"cache_write":0,"output":44010,"reasoning":26486},"cost_usd":1.9809,"cost_breakdown":{"input":0.724816,"cache_read":0.375922,"cache_write":0.0,"output":0.8802},"cost_usd_as_run":1.9809,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":518,"code":458,"comment":26,"blank":34,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":651,"code":587,"comment":24,"blank":40,"path":"tests/selftest.c","role":"test"}],"impl_lines":518,"impl_code":458,"impl_comment":26,"test_lines":651,"comment_ratio":0.054,"tests_declared":7,"object":{"text_bytes":6839,"rodata_bytes":256}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":201.44,"reference":3229.16,"speedup":16.03,"baselines":{"dav1d AVX2":143.7,"dav1d AVX-512ICL":55.01}},{"id":"find_dir","candidate":121.86,"reference":386.45,"speedup":3.171,"baselines":{"dav1d AVX2":40.69}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run, with a slow scratch-heavy search and a test alignment error","opus-5.5-xhigh":"correct; slowest filter group, thin docs, test slop","gpt-6.1-sol-max":"Sound kernels with avoidable direction-search traffic and redundant test work.","deepseek-v4.1-flash-max":"Correct and reasonably fast, but the direction search is effectively undocumented and the filter keeps tap offsets in memory.","muse-wonder-full-xhigh":"Correct, solid tests, minor scalar and documentation costs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search builds and rereads several stack matrices","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard test dereferences a misaligned `unsigned *`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probes omit `rbp` and upper-state cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Pattern test discards a reference run and a buffer copy","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The filter builds the tile's side columns one byte at a time (2 movzx byte loads and 2 word stores per segment)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir goes through stack matrices with shifted reloads","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"test_filter_patterns runs the reference twice and does a `bufs_copy(&a, &b)` that the next line overwrites","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The ABI test makes one call per function, skips rbp, and checks neither dirty argument halves nor XINUSE","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"tile occupies [r13+0 .. r13+512), then a guard\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"The alt2 accumulation reads 6 bytes before FD_Q (:472) and silently relies on PR row 7's tail being zero.","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Materializes padded matrices for direction search","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Discards and recomputes reference results","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"The direction-search core cannot be checked without re-deriving it","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Every tap reloads its offset from the stack","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Byte-at-a-time side copies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Sparse find_dir layout documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Discarded reference run in test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1594,"lines":73,"significant":458,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":21,"flags":0,"audited":"2026-11-03","reviewed":false},"score":7.5,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":0.5}},"defect_points":3.6,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":17.29,"tokens":{"uncached_input":40,"cache_read":2134481,"cache_write":140545,"output":92709,"reasoning":59396},"cost_usd":3.4056,"cost_breakdown":{"input":0.00016,"cache_read":0.426896,"cache_write":1.12436,"output":1.85418},"cost_usd_as_run":3.4056,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":599,"code":489,"comment":75,"blank":35,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":844,"code":747,"comment":51,"blank":46,"path":"tests/selftest.c","role":"test"}],"impl_lines":599,"impl_code":489,"impl_comment":75,"test_lines":844,"comment_ratio":0.133,"tests_declared":8,"object":{"text_bytes":3812,"rodata_bytes":1392}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":171.37,"reference":3116.87,"speedup":18.188,"baselines":{"dav1d AVX2":140.25,"dav1d AVX-512ICL":53.5}},{"id":"find_dir","candidate":31.91,"reference":377.3,"speedup":11.824,"baselines":{"dav1d AVX2":39.51}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Fast and correct in the checks run, but its guard test can corrupt memory","opus-5.5-xhigh":"near-best speed, register-only find_dir, thorough tests including XINUSE","gpt-6.1-sol-max":"Efficient, correct kernels; the added direction guard fixture writes outside its allocation.","deepseek-v4.1-flash-max":"Excellent: correct, 20.1×/11.9×, clean tile and find_dir design, strong tests.","muse-wonder-full-xhigh":"Near-fastest both functions, strong guard and ABI tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Direction guard setup writes past its mapping","status":"valid","defect":"CDEF-007","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Direction tests assert behavior for unsupported strides","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Extreme-case test copies equal buffers after comparing them","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The ROUND macro is described as \"Rounding and store of one row pair\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard fixture overruns its mapping","status":"valid","defect":"CDEF-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"The eight diagonal lane masks are unlabelled","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Magic permute-index tables","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1513,"lines":74,"significant":489,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":19,"flags":0,"audited":"2026-11-03","reviewed":false},"score":8.7,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":1.5,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":59.74,"tokens":{"uncached_input":86,"cache_read":8508535,"cache_write":288841,"output":359089,"reasoning":307860},"cost_usd":11.1946,"cost_breakdown":{"input":0.000344,"cache_read":1.701707,"cache_write":2.310728,"output":7.18178},"cost_usd_as_run":11.1946,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":683,"code":514,"comment":128,"blank":41,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1073,"code":938,"comment":69,"blank":66,"path":"tests/selftest.c","role":"test"}],"impl_lines":683,"impl_code":514,"impl_comment":128,"test_lines":1073,"comment_ratio":0.199,"tests_declared":9,"object":{"text_bytes":2988,"rodata_bytes":1736}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":137.65,"reference":3117.96,"speedup":22.651,"baselines":{"dav1d AVX2":140.0,"dav1d AVX-512ICL":53.41}},{"id":"find_dir","candidate":30.07,"reference":376.0,"speedup":12.504,"baselines":{"dav1d AVX2":39.51}}]},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":9.5,"gpt-6.1-sol-max":10.0,"deepseek-v4.1-flash-max":9.5,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Fast, well-documented implementation with a guard-test coverage hole","opus-5.5-xhigh":"fastest on both functions, exact masked loads, excellent docs and tests","gpt-6.1-sol-max":"Efficient kernels, clear invariants and substantial working tests.","deepseek-v4.1-flash-max":"Merge-quality: correct, fastest on both kernels, and the best-documented and best-tested entry.","muse-wonder-full-xhigh":"Fastest filter and find_dir, best tests and documentation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guard tests couple edge flags to filter mode","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Direction tests require results outside the specified stride domain","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Macro defined mid-function","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":514,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":42,"flags":0,"audited":"2026-11-03","reviewed":false},"score":9.5,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.7,"score_cross_vendor":9.5,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":365.24,"tokens":{"uncached_input":515097,"cache_read":3942712,"cache_write":0,"output":66779,"reasoning":40443},"cost_usd":4.1845,"cost_breakdown":{"input":2.060388,"cache_read":0.788542,"cache_write":0.0,"output":1.33558},"cost_usd_as_run":4.1845,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":473,"code":429,"comment":29,"blank":15,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":689,"code":610,"comment":35,"blank":44,"path":"tests/selftest.c","role":"test"}],"impl_lines":473,"impl_code":429,"impl_comment":29,"test_lines":689,"comment_ratio":0.063,"tests_declared":7,"object":{"text_bytes":3906,"rodata_bytes":192}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":176.87,"reference":3220.17,"speedup":18.206,"baselines":{"dav1d AVX2":144.03,"dav1d AVX-512ICL":54.8}},{"id":"find_dir","candidate":160.87,"reference":387.29,"speedup":2.407,"baselines":{"dav1d AVX2":40.44}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run, with costly direction scratch handling","opus-5.5-xhigh":"correct, fast filter; find_dir goes through the stack and is slow; no ABI tests","gpt-6.1-sol-max":"Correct and well documented, but direction search moves excessive scratch data.","deepseek-v4.1-flash-max":"Correct, clean structure and good tests; find_dir internals are cryptic and only middling fast.","muse-wonder-full-xhigh":"Compact and fast filter, guard testing is the weak spot"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search clears and repacks large scratch matrices on every call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The entry’s tests do not check callee-saved registers or upper-state cleanup","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir goes through a 1.6 KB stack scratch area","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The added tests never check callee-saved registers, vzeroupper/XINUSE, or dirty upper halves of int arguments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Each negative tap rebuilds its address with `mov rax,rsi; sub rax,off`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Large scratch-based direction search","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"The find_dir shift arithmetic is undocumented magic","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"find_dir only 2.5× the reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Edge flags re-tested on every row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin guard coverage and no ABI test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1168,"lines":50,"significant":428,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":44,"flags":4,"audited":"2026-11-03","reviewed":true},"score":7.9,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.2,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.24,"tokens":{"uncached_input":19652,"cache_read":93824,"cache_write":0,"output":535,"reasoning":74},"cost_usd":0.027,"cost_breakdown":{"input":0.019652,"cache_read":0.004691,"cache_write":0.0,"output":0.002675},"cost_usd_as_run":0.027,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":25,"code":10,"comment":10,"blank":5,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":25,"impl_code":10,"impl_comment":10,"test_lines":348,"comment_ratio":0.5,"tests_declared":3,"object":{"text_bytes":4,"rodata_bytes":0}},"hidden":{"passed":1,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3070.05,"speedup":null,"baselines":{"dav1d AVX2":142.35,"dav1d AVX-512ICL":52.57}},{"id":"find_dir","candidate":null,"reference":364.7,"speedup":null,"baselines":{"dav1d AVX2":38.67}}]},"scores":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":1.0},"council_summaries":{"gpt-6.1-terra-max":"Both entry points remain stubs","opus-5.5-xhigh":"scaffold stub, nothing implemented","gpt-6.1-sol-max":"Both required functions remain unimplemented.","deepseek-v4.1-flash-max":"The unmodified scaffold stub: no implementation at all.","muse-wonder-full-xhigh":"Unreplaced scaffold stub; neither function implemented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Direction search returns zero without writing variance","status":"valid","defect":"CDEF-002","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter returns without modifying the block","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Both entry points are still the stub","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"tests/selftest.c is the unmodified scaffold","status":"valid","defect":"CDEF-003","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Direction search never writes variance","status":"valid","defect":"CDEF-002","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter performs no filtering","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"Both entry points are empty","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"No tests were added","status":"valid","defect":"CDEF-003","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"find_dir is a stub","status":"valid","defect":"CDEF-002","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter is a stub","status":"valid","defect":"CDEF-001","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-03","reviewed":false},"score":0.3,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":6.4,"score_cross_vendor":0.5,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":49.85,"tokens":{"uncached_input":78,"cache_read":7369061,"cache_write":339378,"output":677754,"reasoning":640221},"cost_usd":1.3295,"cost_breakdown":{"input":2.4e-05,"cache_read":0.342198,"cache_write":0.282608,"output":0.704693},"cost_usd_as_run":1.3295,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":541,"code":447,"comment":55,"blank":39,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":662,"code":581,"comment":34,"blank":47,"path":"tests/selftest.c","role":"test"}],"impl_lines":541,"impl_code":447,"impl_comment":55,"test_lines":662,"comment_ratio":0.11,"tests_declared":6,"object":{"text_bytes":4936,"rodata_bytes":1952}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":171.16,"reference":3196.61,"speedup":18.676,"baselines":{"dav1d AVX2":143.99,"dav1d AVX-512ICL":54.72}},{"id":"find_dir","candidate":79.37,"reference":383.72,"speedup":4.835,"baselines":{"dav1d AVX2":40.35}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and reasonably fast, with small cleanup and test gaps","opus-5.5-xhigh":"correct, good docs, fast register find_dir; tests lack ABI checks","gpt-6.1-sol-max":"Clear, correct implementation with a small redundant load pattern.","deepseek-v4.1-flash-max":"Correct and fast (20.2×/4.9×), solid comments, but thin test coverage of the ABI and a couple of wasteful instructions.","muse-wonder-full-xhigh":"Fast, elegant generated skew tables, strong forked guard tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Direction search explicitly clears nine caller-saved vector registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No ABI or upper-state probe is registered","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"There is no callee-saved, dirty-argument or vzeroupper/XINUSE test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"ymm16–24 and xmm16–17 are zeroed before vzeroupper","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The skew_d0 row is loaded twice per row (for diag0 and alt0)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Loads the same skew indices twice per row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"The `skew_d0` table row is loaded twice per block row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No ABI or upper-state checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"zmm16–24 are zeroed on the return path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Unneeded register clearing at exits","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0313,"lines":14,"significant":447,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":42,"flags":1,"audited":"2026-11-03","reviewed":true},"score":8.4,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5}},"defect_points":1.8,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.53,"tokens":{"uncached_input":26390,"cache_read":217600,"cache_write":0,"output":6143,"reasoning":1192},"cost_usd":0.136,"cost_breakdown":{"input":0.05278,"cache_read":0.02176,"cache_write":0.0,"output":0.06143},"cost_usd_as_run":0.136,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":1431,"code":1427,"comment":2,"blank":2,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":432,"code":375,"comment":25,"blank":32,"path":"tests/selftest.c","role":"test"}],"impl_lines":1431,"impl_code":1427,"impl_comment":2,"test_lines":432,"comment_ratio":0.001,"tests_declared":4,"object":{"text_bytes":7367,"rodata_bytes":9376}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":183.63,"reference":3123.67,"speedup":17.011,"baselines":{"dav1d AVX2":140.34,"dav1d AVX-512ICL":53.5}},{"id":"find_dir","candidate":261.15,"reference":376.84,"speedup":1.443,"baselines":{"dav1d AVX2":39.47}}]},"scores":{"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":5.5,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":5.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run, but direction search is bulky and inefficient","opus-5.5-xhigh":"correct, but 1,060 lines of uncommented unrolled code and a find_dir barely faster than C","gpt-6.1-sol-max":"Correct output and bounded accesses, but direction search is costly and difficult to maintain.","deepseek-v4.1-flash-max":"Correct and adequate speed, but 1,200 lines of unrolled direction search with an 8 KB unlabelled table and essentially no documentation.","muse-wonder-full-xhigh":"Correct, but a 1431-line uncommented unrolled find_dir is unmaintainable"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search recomputes the block for each direction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Identical masks are repeatedly reloaded within one row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public entry points and direction-table invariants are undocumented","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not verify callee-saved registers or upper-state cleanup","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"find_dir reloads and re-centres all 8 rows for each of the 8 directions (64 row loads)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"There is no header, no function docs and no description of the tables; the file has two inline comments in total","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"About 1,060 lines of generated-looking straight-line code with offsets hard-coded up to [r9+8832]","status":"valid","defect":"CDEF-005","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"A `cmp r11d,3; jne` branch sits inside each of the 12 taps per iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"`and r9d,1` destroys pri, so the mode is then recovered with vptestmw/kmovd on broadcast registers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The added tests check DF/MXCSR/FCW and dirty argument halves, but not callee-saved registers or XINUSE, and the guard sweep uses only pri 15 / sec 4 / damping 3","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Reprocesses the block for every direction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Direction tables lack derivation and invariants","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"verbosity","title":"The direction search is a generated-looking unroll","status":"valid","defect":"CDEF-005","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Two one-line comments in 1,431 lines","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"find_dir only 1.5× the reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No callee-saved or upper-state checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"The filter branches per tap","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Essentially no comments","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"verbosity","title":"find_dir fully unrolled by hand","status":"valid","defect":"CDEF-005","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Rows re-read for every cost","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Branch per tap for clipping","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Mode re-derived through vector test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard case without output check","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.384,"lines":548,"significant":1427,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-03","reviewed":false},"score":6.3,"slop":{"score":1.8,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":3.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":2.0,"muse-wonder-full-xhigh":3.0}},"defect_points":6.3,"score_cross_vendor":5.833,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":70.47,"tokens":{"uncached_input":696,"cache_read":40041099,"cache_write":487199,"output":430958,"reasoning":0},"cost_usd":0.9408,"cost_breakdown":{"input":0.000104,"cache_read":0.640658,"cache_write":0.09744,"output":0.20255},"cost_usd_as_run":0.9408,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.9408,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":703,"code":561,"comment":93,"blank":49,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":800,"code":695,"comment":50,"blank":55,"path":"tests/selftest.c","role":"test"}],"impl_lines":703,"impl_code":561,"impl_comment":93,"test_lines":800,"comment_ratio":0.142,"tests_declared":9,"object":{"text_bytes":5676,"rodata_bytes":304}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":197.62,"reference":3222.52,"speedup":16.307,"baselines":{"dav1d AVX2":144.29,"dav1d AVX-512ICL":55.05}},{"id":"find_dir","candidate":291.22,"reference":387.92,"speedup":1.332,"baselines":{"dav1d AVX2":40.61}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run, but direction search and two test probes need work","opus-5.5-xhigh":"correct, good tests; a false hardware claim drives a needless workaround; slow find_dir","gpt-6.1-sol-max":"Correct kernels, but costly direction search and ineffective safety tests need changes.","deepseek-v4.1-flash-max":"Correct and very well tested and documented, but its direction search is the slowest bar one and one comment misdiagnoses VPSRLW.","muse-wonder-full-xhigh":"Correct and well tested, slowed by a workaround for a non-existent bug"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search repeatedly clears and uses a 2.5 KiB scratch area","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI probe discards its result","status":"valid","defect":"CDEF-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Direction page-end test does not reach a guard","status":"valid","defect":"CDEF-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Named directional patterns are not generated as intended","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"find_dir zeroes 2.5 KB of stack per call (40 ZMM stores), stores every row twice, and folds row sums with scalar FOLD16 eight times","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The header says \"the EVEX memory-count form of VPSRLW misbehaves on the execution host\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"That misdiagnosis led to a vpmulhuw multiplier vector stored on the stack, plus a k3 blend for shift 0","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"A development-history comment (\"earlier versions used ymm1..8 and ymm8 aliased the zmm8 accumulator\")","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"map_pages is never unmapped, so the guard tests leak hundreds of mappings","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Spills rows already live in registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI probe discards its failure mask","status":"valid","defect":"CDEF-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Claimed tail guards leave readable slack","status":"valid","defect":"CDEF-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Wrong explanation for avoiding VPSRLW","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Direction search only 1.3× the reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Pointless pushes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"Change-history comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"False claim that memory-count `VPSRLW` misbehaves","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Multiply-high shift workaround","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"find_dir zeroes a 2560-byte frame per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unused callee saves in find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Changelog comment left in code","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":122,"flags":0,"audited":"2026-11-03","reviewed":false},"score":7.0,"slop":{"score":1.3,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":2.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":2.5,"muse-wonder-full-xhigh":2.0}},"defect_points":5.0,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":49.13,"tokens":{"uncached_input":197812,"cache_read":4097108,"cache_write":0,"output":82367,"reasoning":56901},"cost_usd":3.258,"cost_breakdown":{"input":0.791248,"cache_read":0.819422,"cache_write":0.0,"output":1.64734},"cost_usd_as_run":3.258,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":528,"code":465,"comment":50,"blank":13,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":864,"code":774,"comment":38,"blank":52,"path":"tests/selftest.c","role":"test"}],"impl_lines":528,"impl_code":465,"impl_comment":50,"test_lines":864,"comment_ratio":0.097,"tests_declared":8,"object":{"text_bytes":3739,"rodata_bytes":240}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":218.95,"reference":3127.87,"speedup":14.286,"baselines":{"dav1d AVX2":140.55,"dav1d AVX-512ICL":53.67}},{"id":"find_dir","candidate":148.65,"reference":380.03,"speedup":2.557,"baselines":{"dav1d AVX2":39.64}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run, with avoidable work in both hot paths","opus-5.5-xhigh":"correct and well commented; slower filter loop and stack-based find_dir","gpt-6.1-sol-max":"Correct implementation, with excessive scratch traffic and unused filtering work.","deepseek-v4.1-flash-max":"Correct, well tested, but the filter is the joint slowest and the find_dir window arithmetic is only half explained.","muse-wonder-full-xhigh":"Correct and well commented, with several avoidable hot-path costs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search materializes several padded matrices","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Every tap updates min/max even in single-strength modes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI checks omit `rbp` and upper-state cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Guard setup compares a buffer with itself","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The filter loop redoes setup work every row pair","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir stores five padded int32 matrices on the stack and reloads them with shifted ZMM loads","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The find_dir ABI probe pushes 7 qwords before `call`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"There is no vzeroupper/XINUSE check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Large direction-search matrices","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Computes unused extrema in single-strength modes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Tap offsets are re-read from the stack for every tap","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"The find_dir construction is only described by its offsets","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Min/max updated in single-strength modes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Results stored through stack scratch","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Large stack matrices in find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Thin pattern coverage","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":465,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":40,"flags":0,"audited":"2026-11-03","reviewed":false},"score":7.5,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":1.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.2,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.77,"tokens":{"uncached_input":23857,"cache_read":140672,"cache_write":0,"output":4331,"reasoning":110},"cost_usd":0.5958,"cost_breakdown":{"input":0.23857,"cache_read":0.140672,"cache_write":0.0,"output":0.21655},"cost_usd_as_run":0.5958,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":355,"code":351,"comment":2,"blank":2,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":390,"code":339,"comment":21,"blank":30,"path":"tests/selftest.c","role":"test"}],"impl_lines":355,"impl_code":351,"impl_comment":2,"test_lines":390,"comment_ratio":0.006,"tests_declared":4,"object":{"text_bytes":2027,"rodata_bytes":2608}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":205.05,"reference":3181.83,"speedup":15.517,"baselines":{"dav1d AVX2":142.44,"dav1d AVX-512ICL":54.46}},{"id":"find_dir","candidate":1200.05,"reference":382.72,"speedup":0.319,"baselines":{"dav1d AVX2":40.27}}]},"scores":{"gpt-6.1-terra-max":6.0,"opus-5.5-xhigh":5.0,"gpt-6.1-sol-max":6.0,"deepseek-v4.1-flash-max":5.5,"muse-wonder-full-xhigh":6.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run, but direction search is slower than the C reference","opus-5.5-xhigh":"correct, but find_dir is 3× slower than the C reference; undocumented","gpt-6.1-sol-max":"Correct results, but direction search substantially regresses performance.","deepseek-v4.1-flash-max":"Correct, but its direction search is 3× slower than the C reference it must beat, and nothing is documented.","muse-wonder-full-xhigh":"Correct and fast filter, but find_dir is slower than the C reference"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"performance","title":"Direction search gathers and scatters for every pixel","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Single-strength filters calculate unused tap minima and maxima","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard coverage exercises only absent neighbours","status":"valid","defect":"CDEF-010","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Index-table and public-entry invariants are unexplained","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"find_dir updates partial sums with one vpgatherdd plus one vpscatterdd per pixel, 64 pairs going through a stack array","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"There is no file header and no description of the tile layout, register roles, or the 64-row gather `indices` table","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The only added test is one guard case: edges=0 with a single strength set","status":"valid","defect":"CDEF-010","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"\"Recover original top/bottom, which were saved before scratch use below.\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The weights are recomputed and re-broadcast, strengths re-tested, and the rounding constant re-broadcast inside the row-pair loop, and min/max are tracked in every mode","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"performance","title":"Per-pixel gather/scatter bottleneck","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Maintains unused extrema","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Undocumented direction-search tables","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"performance","title":"Direction search regresses against the reference","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"The whole data layout is undocumented","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Guard-page test only covers `edges == 0`","status":"valid","defect":"CDEF-010","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No ABI checks","status":"valid","defect":"CDEF-010","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"Per-pixel gather/scatter in find_dir","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Almost no comments","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Added tests are thin","status":"valid","defect":"CDEF-010","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Tap offsets and weights reloaded per row pair","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0684,"lines":24,"significant":351,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-03","reviewed":false},"score":5.7,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0}},"defect_points":7.0,"score_cross_vendor":5.5,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.13,"tokens":{"uncached_input":23814,"cache_read":265472,"cache_write":0,"output":10131,"reasoning":2412},"cost_usd":0.0877,"cost_breakdown":{"input":0.023814,"cache_read":0.013274,"cache_write":0.0,"output":0.050655},"cost_usd_as_run":0.0877,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":399,"code":388,"comment":7,"blank":4,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":491,"code":434,"comment":24,"blank":33,"path":"tests/selftest.c","role":"test"}],"impl_lines":399,"impl_code":388,"impl_comment":7,"test_lines":491,"comment_ratio":0.018,"tests_declared":5,"object":{"text_bytes":2958,"rodata_bytes":3456}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":172.63,"reference":3134.42,"speedup":18.157,"baselines":{"dav1d AVX2":140.51,"dav1d AVX-512ICL":53.88}},{"id":"find_dir","candidate":95.99,"reference":377.93,"speedup":3.937,"baselines":{"dav1d AVX2":39.68}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in shared checks, but single-strength filtering performs unnecessary clipping work.","opus-5.5-xhigh":"correct, but one branchy all-modes loop, no-op multiplies, little documentation, partial tests","gpt-6.1-sol-max":"Correct implementation with unnecessary hot-path work and an ABI-test gap.","deepseek-v4.1-flash-max":"correct, but three kilobytes of unexplained index tables sit under a thin header and find_dir leaves easy work inside its row loop.","muse-wonder-full-xhigh":"correct, mid speed; worst docs and thinnest added tests of the real entries"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Min/max runs when its result is unused","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Register probe misses upper-half corruption","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Filter invariants are not explained","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"One loop serves all three modes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Weights 2 and 1 applied with `vpmullw`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header; tables and tile undocumented","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Macros defined mid-function","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"Horizontal sum, using exact 8-byte load above\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard test only at the page end, plus one all-absent start case; no `rsp` or upper-state check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Single-strength calls compute discarded clipping bounds","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Direction rows are loaded twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"The eight 512-byte direction tables are unexplained","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Loop-invariant work inside the per-row loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`pair_indices` lives in a second `.rodata` block after `.text` (397-398) while every other constant is at the top of the file.","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No header or table documentation","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Always-on min/max and per-row dispatch branches","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin added coverage","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0309,"lines":12,"significant":388,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-03","reviewed":false},"score":7.5,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.0}},"defect_points":3.5,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.49,"tokens":{"uncached_input":22733,"cache_read":94336,"cache_write":0,"output":1943,"reasoning":246},"cost_usd":0.0372,"cost_breakdown":{"input":0.022733,"cache_read":0.004717,"cache_write":0.0,"output":0.009715},"cost_usd_as_run":0.0372,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":119,"code":118,"comment":0,"blank":1,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":119,"impl_code":118,"impl_comment":0,"test_lines":348,"comment_ratio":0.0,"tests_declared":3,"object":{"text_bytes":365,"rodata_bytes":212}},"hidden":{"passed":3,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3133.79,"speedup":null,"baselines":{"dav1d AVX2":139.96,"dav1d AVX-512ICL":53.54}},{"id":"find_dir","candidate":592.34,"reference":377.59,"speedup":0.637,"baselines":{"dav1d AVX2":39.47}}]},"scores":{"gpt-6.1-terra-max":2.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":2.5,"deepseek-v4.1-flash-max":2.0,"muse-wonder-full-xhigh":2.0},"council_summaries":{"gpt-6.1-terra-max":"Direction search works in the exercised cases, but the filter is unimplemented.","opus-5.5-xhigh":"the filter is an empty stub; the direction search is scalar and slower than C","gpt-6.1-sol-max":"The required filter is unimplemented.","deepseek-v4.1-flash-max":"the filter is an empty stub: half the deliverable is missing and the shipped test suite fails.","muse-wonder-full-xhigh":"filter is an unimplemented stub; find_dir alone is correct but scalar"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter returns without filtering","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search is predominantly scalar","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No added memory or ABI coverage","status":"valid","defect":"CDEF-003","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Filter does nothing","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"Direction search is scalar and slower than the C reference","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"selftest.c is the unmodified scaffold","status":"valid","defect":"CDEF-003","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No comments anywhere in the asm","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter returns without filtering","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction search remains predominantly scalar","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`cb_cdef_filter_8x8_8bpc_avx512` is a bare `ret`","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"No guard-page, ABI or contract tests were added and the three tests present fail","status":"valid","defect":"CDEF-003","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"The file contains no comments at all","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"find_dir is slower than the C reference","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter not implemented","status":"valid","defect":"CDEF-001","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Scalar find_dir slower than the C reference","status":"valid","defect":"CDEF-004","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Zero comments","status":"valid","defect":"CDEF-006","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No added tests, and own test gate fails","status":"valid","defect":"CDEF-003","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":118,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-03","reviewed":false},"score":2.0,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":7.8,"score_cross_vendor":1.667,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":43.14,"tokens":{"uncached_input":88,"cache_read":8674902,"cache_write":247226,"output":191308,"reasoning":140363},"cost_usd":7.5393,"cost_breakdown":{"input":0.000352,"cache_read":1.73498,"cache_write":1.977808,"output":3.82616},"cost_usd_as_run":7.5393,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":612,"code":493,"comment":89,"blank":30,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1008,"code":878,"comment":72,"blank":58,"path":"tests/selftest.c","role":"test"}],"impl_lines":612,"impl_code":493,"impl_comment":89,"test_lines":1008,"comment_ratio":0.153,"tests_declared":8,"object":{"text_bytes":6774,"rodata_bytes":2176}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":132.11,"reference":3126.28,"speedup":23.664,"baselines":{"dav1d AVX2":139.83,"dav1d AVX-512ICL":53.25}},{"id":"find_dir","candidate":35.27,"reference":376.54,"speedup":10.676,"baselines":{"dav1d AVX2":39.35}}]},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":9.5,"gpt-6.1-sol-max":10.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Fast, documented and correct in shared checks, with some out-of-domain test cases.","opus-5.5-xhigh":"fastest direction search tier (35) and second-fastest filter (132), exact masked loads, excellent docs and tests","gpt-6.1-sol-max":"Efficient implementation with strong invariants and effective validation.","deepseek-v4.1-flash-max":"the fastest filter (141 cyc) and a 35-cycle direction search, with the most complete documentation in the set; one memory-safety claim in the header is wrong.","muse-wonder-full-xhigh":"correct, fastest overall, excellent docs/tests; opaque find_dir tables"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Direction tests require unsupported strides","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Some tests use strides outside the contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"\"the tile has enough slack that every 64-byte load stays inside it\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Unauditable find_dir tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Guard tests pass when skipped","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0325,"lines":16,"significant":493,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":44,"flags":0,"audited":"2026-11-03","reviewed":false},"score":9.3,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.8,"score_cross_vendor":9.25,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":76.0,"tokens":{"uncached_input":273791,"cache_read":5427456,"cache_write":0,"output":349990,"reasoning":157573},"cost_usd":5.3613,"cost_breakdown":{"input":0.547582,"cache_read":2.713728,"cache_write":0.0,"output":2.09994},"cost_usd_as_run":5.3613,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"lines":527,"code":435,"comment":41,"blank":51,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1045,"code":956,"comment":26,"blank":63,"path":"tests/selftest.c","role":"test"}],"impl_lines":527,"impl_code":435,"impl_comment":41,"test_lines":1045,"comment_ratio":0.086,"tests_declared":7,"object":{"text_bytes":6511,"rodata_bytes":512}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":169.65,"reference":3144.25,"speedup":18.534,"baselines":{"dav1d AVX2":140.8,"dav1d AVX-512ICL":53.92}},{"id":"find_dir","candidate":80.83,"reference":378.35,"speedup":4.681,"baselines":{"dav1d AVX2":39.56}}]},"scores":{"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"The implementation passes shared checks, but its ABI test has two substantive defects.","opus-5.5-xhigh":"correct, well commented, mid-pack speed (164/79); ABI test fragile and incomplete","gpt-6.1-sol-max":"Correct kernels, with an invalid ABI fixture and duplicated assembly.","deepseek-v4.1-flash-max":"excellent header and reference-independent direction expectations in its tests, but the cost reduction is copy-pasted eight times and its direction search is among the slowest.","muse-wonder-full-xhigh":"correct, well tested; find_dir is repetitive and one comment is off by one"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI test reads before neighbor arrays","status":"valid","defect":"CDEF-011","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI test requires the direction flag to remain set","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Largest stride is misstated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Inline-asm ABI probe is fragile and incomplete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"840/count for diagonal bins 1..7\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI fixture underreads its top and bottom arrays","status":"valid","defect":"CDEF-011","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Half-slope cost reductions are copied four times","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"The same 12-instruction weighted-square reduction is written out eight times","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"Block-row copy re-tests the edge bits per row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Six verbatim copies of the reduction block","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"diag_w comment off by one","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Extremes test under-checks stray writes; small guard suite","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":20,"flags":0,"audited":"2026-11-03","reviewed":false},"score":7.8,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":2.7,"score_cross_vendor":7.8,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.07,"tokens":{"uncached_input":31830,"cache_read":273280,"cache_write":0,"output":9357,"reasoning":1275},"cost_usd":1.0594,"cost_breakdown":{"input":0.3183,"cache_read":0.27328,"cache_write":0.0,"output":0.46785},"cost_usd_as_run":1.0594,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":350,"code":334,"comment":10,"blank":6,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":511,"code":452,"comment":25,"blank":34,"path":"tests/selftest.c","role":"test"}],"impl_lines":350,"impl_code":334,"impl_comment":10,"test_lines":511,"comment_ratio":0.029,"tests_declared":5,"object":{"text_bytes":3644,"rodata_bytes":3456}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":179.56,"reference":3127.66,"speedup":17.418,"baselines":{"dav1d AVX2":140.3,"dav1d AVX-512ICL":53.46}},{"id":"find_dir","candidate":97.88,"reference":376.42,"speedup":3.846,"baselines":{"dav1d AVX2":39.47}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in shared checks; direction search and the ABI probe have room for improvement.","opus-5.5-xhigh":"correct and compact, but the tile and offset tables are undocumented and the tests miss upper-state checks","gpt-6.1-sol-max":"Compact, correct implementation with one material test-coverage omission.","deepseek-v4.1-flash-max":"clean, compact and fast, with an elegant table generator; documentation and test breadth are thinner than the top entries.","muse-wonder-full-xhigh":"correct and compact; per-tap offset reloads and sparse explanation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction search loads six maps per row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Saved-register sentinels have zero upper halves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Tile layout and tap-offset table are unexplained","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No upper-state check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Claims an `rsp` check that doesn't exist","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI checks omit upper vector state","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"The six per-direction formulas of the generated `row_maps` are uncommented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"Four single-byte loads and two 2-byte stores per row pair","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Offset and map reloads in hot loops","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Unjustified missing shift clamp","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Unexplained find_dir register mapping","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":334,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-03","reviewed":false},"score":8.0,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":1.8,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":151.22,"tokens":{"uncached_input":615341,"cache_read":122030400,"cache_write":0,"output":477784,"reasoning":388868},"cost_usd":3.9921,"cost_breakdown":{"input":0.092301,"cache_read":3.660912,"cache_write":0.0,"output":0.238892},"cost_usd_as_run":3.9921,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":624,"code":519,"comment":63,"blank":42,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":820,"code":716,"comment":51,"blank":53,"path":"tests/selftest.c","role":"test"}],"impl_lines":624,"impl_code":519,"impl_comment":63,"test_lines":820,"comment_ratio":0.108,"tests_declared":9,"object":{"text_bytes":5379,"rodata_bytes":704}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":180.57,"reference":3141.98,"speedup":17.4,"baselines":{"dav1d AVX2":139.2,"dav1d AVX-512ICL":52.62}},{"id":"find_dir","candidate":86.76,"reference":379.48,"speedup":4.374,"baselines":{"dav1d AVX2":38.93}}]},"scores":{"muse-wonder-full-xhigh":8.5,"gpt-6.1-sol-max":8.5,"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"muse-wonder-full-xhigh":"Correct, well documented; fragile idioms and test gaps on ABI","gpt-6.1-sol-max":"Correct register-based implementation, with incomplete ABI validation and avoidable reduction work.","gpt-6.1-terra-max":"Correct in the checks run, with slower direction search and gaps in its ABI tests","opus-5.5-xhigh":"correct and well documented, but leaves speed on the table and its ABI test checks less than it claims","deepseek-v4.1-flash-max":"correct, well documented and well tested; a dead `max(0,…)` clamp and no callee-saved/vzeroupper check in its own tests"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Missing callee-saved/XINUSE checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Offset registers left negated by `PAIR`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Three longhand copies of the filter loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Pixels re-read from dst instead of tile","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Table comment describes rows it does not have","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Dirty-argument coverage misses the primary register","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Costs make an unnecessary scalar round trip","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Primary-shift clamp is a no-op","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction costs are reduced separately","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"The dirty-argument test does not poison `r9`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper vector state is not tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The direction guard does not protect every row end","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"The two shift spill slots are not reserved and can land inside the tile","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The filter computes on twice as many lanes as it uses","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir runs eight separate scalar reductions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The dirty-ABI test doesn't do what its comment says","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The guard test never puts dst at the start of a page","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"One check can never fail","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`vinserti64x4` in ROW_SUB does nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Primary-weight and constant setup is written out twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"pri_shift clamp can never fire","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"no callee-saved / vzeroupper check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"the top/bottom tile fill is a near-verbatim copy of the block-row loop","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.3066,"lines":195,"significant":636,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":329,"flags":0,"audited":"2026-11-03","reviewed":false},"score":8.1,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.5}},"defect_points":4.8,"score_cross_vendor":8.1,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.6,"tokens":{"uncached_input":22870,"cache_read":704256,"cache_write":0,"output":32013,"reasoning":17756},"cost_usd":0.2181,"cost_breakdown":{"input":0.02287,"cache_read":0.035213,"cache_write":0.0,"output":0.160065},"cost_usd_as_run":0.2181,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":401,"code":360,"comment":23,"blank":18,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":834,"code":760,"comment":32,"blank":42,"path":"tests/selftest.c","role":"test"}],"impl_lines":401,"impl_code":360,"impl_comment":23,"test_lines":834,"comment_ratio":0.06,"tests_declared":8,"object":{"text_bytes":3494,"rodata_bytes":1792}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":171.03,"reference":3062.99,"speedup":17.909,"baselines":{"dav1d AVX2":136.98,"dav1d AVX-512ICL":52.49}},{"id":"find_dir","candidate":77.9,"reference":368.52,"speedup":4.731,"baselines":{"dav1d AVX2":38.72}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in shared checks, but its tests enforce behavior outside the contract.","opus-5.5-xhigh":"correct, mid-pack speed (171/78), thorough tests that wander outside the contract","gpt-6.1-sol-max":"Correct and well tested, with a removable operation in each tap pair.","deepseek-v4.1-flash-max":"correct with the strongest test suite after U/Y, but its source explains almost nothing about the two hand-built tables it depends on.","muse-wonder-full-xhigh":"correct, readable macros; slowest-quartile speed and repetitive table comments"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI probe requires DF to remain set","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Negative stride is presented as an API guarantee","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Saved-register sentinels have zero upper halves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Broadcasts weights via store and reload","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Tests assert behaviour outside the contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"1.5 KB of hand-written permutation indices","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Symmetric tap pairs unnecessarily restore offset signs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Algorithm and tables undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"Edge flags re-tested inside the 8-iteration tile build","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Table streaming and single-tap filter","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Lane-ordering invariant undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Smallest guard suite among extended tests","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0333,"lines":12,"significant":360,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-03","reviewed":false},"score":8.1,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":2.0,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":26.6,"tokens":{"uncached_input":66780,"cache_read":27080064,"cache_write":0,"output":273463,"reasoning":210294},"cost_usd":0.2553,"cost_breakdown":{"input":0.010017,"cache_read":0.08124,"cache_write":0.0,"output":0.164078},"cost_usd_as_run":0.2553,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2553,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":745,"code":614,"comment":78,"blank":53,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":840,"code":745,"comment":40,"blank":55,"path":"tests/selftest.c","role":"test"}],"impl_lines":745,"impl_code":614,"impl_comment":78,"test_lines":840,"comment_ratio":0.113,"tests_declared":9,"object":{"text_bytes":4686,"rodata_bytes":1664}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":134.08,"reference":3133.75,"speedup":23.372,"baselines":{"dav1d AVX2":139.96,"dav1d AVX-512ICL":53.54}},{"id":"find_dir","candidate":54.09,"reference":376.29,"speedup":6.957,"baselines":{"dav1d AVX2":39.47}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Fast and correct in shared checks, but its ABI test does not reliably test saved registers.","opus-5.5-xhigh":"fastest filter (126, four rows per ZMM) and quick direction search (52), but its ABI test checks nothing","gpt-6.1-sol-max":"Fast, correct kernels, but several test assertions provide false confidence.","deepseek-v4.1-flash-max":"the second-fastest filter (142 cyc) with the best in-file commentary; one inaccurate sentence and one needlessly clever register pack.","muse-wonder-full-xhigh":"correct, 2nd-fastest filter, strong tests; minor scalar copies and an ABI-test gap"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Saved-register probe is optimized away","status":"valid","defect":"CDEF-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Layout test exceeds the stride domain","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Scratch is described as unread","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Callee-saved check compiles to nothing","status":"valid","defect":"CDEF-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"Cost weights: div_table[n]*2\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Rows 12..15 called \"unread scratch\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Register assertions are optimized away","status":"valid","defect":"CDEF-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard checks accept writes to right-neighbor pixels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Direction weights are described as doubled","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"\"rows/cols 12..15 are unread scratch\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`damping` and `edges` are packed into one register","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar byte-wise side copies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"min/max copies on unclipped paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"ABI test misses rbp, dirty upper halves, XINUSE","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0798,"lines":49,"significant":614,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":117,"flags":0,"audited":"2026-11-03","reviewed":false},"score":8.2,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5}},"defect_points":3.3,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":15.49,"tokens":{"uncached_input":21479,"cache_read":415616,"cache_write":0,"output":22424,"reasoning":9666},"cost_usd":1.7516,"cost_breakdown":{"input":0.21479,"cache_read":0.415616,"cache_write":0.0,"output":1.1212},"cost_usd_as_run":1.7516,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":369,"code":335,"comment":17,"blank":17,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":728,"code":653,"comment":33,"blank":42,"path":"tests/selftest.c","role":"test"}],"impl_lines":369,"impl_code":335,"impl_comment":17,"test_lines":728,"comment_ratio":0.048,"tests_declared":7,"object":{"text_bytes":4402,"rodata_bytes":244}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":167.76,"reference":3063.79,"speedup":18.263,"baselines":{"dav1d AVX2":136.68,"dav1d AVX-512ICL":52.41}},{"id":"find_dir","candidate":99.86,"reference":369.07,"speedup":3.696,"baselines":{"dav1d AVX2":38.67}}]},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct and well tested, with some avoidable direction-search work.","opus-5.5-xhigh":"correct, concise and accurately commented; very complete tests; mid-pack speed (168/100)","gpt-6.1-sol-max":"Correct and thoroughly tested, with a small direction-search inefficiency.","deepseek-v4.1-flash-max":"solid and fast (filter 180 cyc, dir 101 cyc) with strong tests; two readability nits in argument and tap dispatch.","muse-wonder-full-xhigh":"correct, compact and clean; find_dir is comparatively slow"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Adjacent rows repeat the same shifts","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Row sums make an avoidable stack round trip","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`dir` is parked in `rbp`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Secondary taps are dispatched in weight order, not table order","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"find_dir 104.6c vs 31.7c best","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Word-at-a-time side loads","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Cryptic tap-offset register assignment","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":335,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-03","reviewed":false},"score":8.6,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.0,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.89,"tokens":{"uncached_input":23936,"cache_read":283392,"cache_write":0,"output":10089,"reasoning":2281},"cost_usd":0.1771,"cost_breakdown":{"input":0.047872,"cache_read":0.028339,"cache_write":0.0,"output":0.10089},"cost_usd_as_run":0.1771,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":345,"code":325,"comment":14,"blank":6,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":500,"code":441,"comment":25,"blank":34,"path":"tests/selftest.c","role":"test"}],"impl_lines":345,"impl_code":325,"impl_comment":14,"test_lines":500,"comment_ratio":0.041,"tests_declared":5,"object":{"text_bytes":4025,"rodata_bytes":1024}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":191.36,"reference":3063.71,"speedup":16.01,"baselines":{"dav1d AVX2":137.52,"dav1d AVX-512ICL":52.36}},{"id":"find_dir","candidate":102.63,"reference":369.28,"speedup":3.598,"baselines":{"dav1d AVX2":38.63}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in shared checks, with repeated filter setup and a partial ABI probe.","opus-5.5-xhigh":"correct and readable, but the slowest filter in the field (193) and no upper-state test","gpt-6.1-sol-max":"Correct kernels with repeated invariant setup and incomplete ABI tests.","deepseek-v4.1-flash-max":"correct, but the slowest filter in the set, with per-pair broadcasts in the loop and a leaked preprocessor symbol.","muse-wonder-full-xhigh":"correct and shortest, but slowest filter and thinnest docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Tap constants are rebuilt inside the row loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Register sentinels miss upper-half corruption","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Loop-invariant constants re-broadcast every iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Every tap multiplied, including weight 1","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No upper-state check; `left` only placed at a page end","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Filter constants are repeatedly loaded inside the row loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"style","title":"`%define COMBINED %3` inside a macro body","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Loop-invariant setup inside the row loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-tap and per-row table/stack reloads","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"One-line header, no ABI or layout overview","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No XINUSE/vzeroupper check","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":325,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-03","reviewed":false},"score":7.6,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.4,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":14.64,"tokens":{"uncached_input":26992,"cache_read":374528,"cache_write":0,"output":29295,"reasoning":15935},"cost_usd":0.3844,"cost_breakdown":{"input":0.053984,"cache_read":0.037453,"cache_write":0.0,"output":0.29295},"cost_usd_as_run":0.3844,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":393,"code":354,"comment":21,"blank":18,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":745,"code":672,"comment":31,"blank":42,"path":"tests/selftest.c","role":"test"}],"impl_lines":393,"impl_code":354,"impl_comment":21,"test_lines":745,"comment_ratio":0.056,"tests_declared":7,"object":{"text_bytes":4233,"rodata_bytes":240}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":160.12,"reference":3118.76,"speedup":19.478,"baselines":{"dav1d AVX2":140.04,"dav1d AVX-512ICL":53.5}},{"id":"find_dir","candidate":70.25,"reference":375.58,"speedup":5.346,"baselines":{"dav1d AVX2":39.47}}]},"scores":{"gpt-6.1-terra-max":10.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":10.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with thorough tests and clear implementation.","opus-5.5-xhigh":"bit-exact, clean, third-fastest filter (156) and quick direction search (69), strong tests","gpt-6.1-sol-max":"Correct, efficient and supported by effective tests.","deepseek-v4.1-flash-max":"fast (filter 165 cyc, dir 69 cyc) and well-commented macro by macro; the tap-table columns and one register reuse are the weak spots.","muse-wonder-full-xhigh":"correct, balanced speed and readability; only minor verbosity"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"The six columns of `tap_offsets` are never described","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"find_dir destroys `stride` on the way out","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Hand-unrolled horizontal-sum reduction tree","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Mid-pack speed 168c/71c","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Guard test silently passes when skipped","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0339,"lines":12,"significant":354,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-03","reviewed":false},"score":9.1,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":0.7,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.29,"tokens":{"uncached_input":42247,"cache_read":660736,"cache_write":0,"output":25260,"reasoning":7161},"cost_usd":0.4032,"cost_breakdown":{"input":0.084494,"cache_read":0.066074,"cache_write":0.0,"output":0.2526},"cost_usd_as_run":0.4032,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":349,"code":327,"comment":9,"blank":13,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":677,"code":608,"comment":29,"blank":40,"path":"tests/selftest.c","role":"test"}],"impl_lines":349,"impl_code":327,"impl_comment":9,"test_lines":677,"comment_ratio":0.027,"tests_declared":6,"object":{"text_bytes":3608,"rodata_bytes":372}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":170.82,"reference":3108.98,"speedup":18.2,"baselines":{"dav1d AVX2":140.17,"dav1d AVX-512ICL":53.46}},{"id":"find_dir","candidate":85.96,"reference":374.74,"speedup":4.359,"baselines":{"dav1d AVX2":39.51}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks performed, with a slower direction search and test gaps","deepseek-v4.1-flash-max":"Correct, safe and the leanest filter, but its direction search is ~3x behind the best entry; the rest is polish.","gpt-6.1-sol-max":"Compact implementation; shared correctness and ABI probes passed.","opus-5.5-xhigh":"compact and correct, but the direction search is about 2.8× slower than U's and the documentation is thin","muse-wonder-full-xhigh":"Bit-exact and compact with strong tests, but find_dir is ~3× slower than entry-E and its key permute trick is undocumented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Serial direction search","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper-vector state is not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Exact-access label overstates guard coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"find_dir costs 3x the best entry","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"No overview of the direction-search algorithm","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"Guard test reports FAIL instead of skipping on large pages","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state regressions go undetected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The direction search runs a serial loop over rows","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Upper-state and interior-row over-reads are not tested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Docs are too sparse for a maintainer","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Jumps to the next instruction and duplicated copy code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Unaligned tile and branchy copy","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-row permute find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Negative-index permute invariant unstated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"ABI test never checks vzeroupper/XINUSE","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Edges register silently repurposed as loop counter","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":327,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-03","reviewed":false},"score":8.3,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":3.1,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1-b2","task":"cdef-avx512","lead":"gpt-6.1-terra-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[],"panel":[]},"entries":[{"letter":"K","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":21.58,"tokens":{"uncached_input":37195,"cache_read":130619,"cache_write":0,"output":1996,"reasoning":716},"cost_usd":0.1074,"cost_breakdown":{"input":0.07439,"cache_read":0.013062,"cache_write":0.0,"output":0.01996},"cost_usd_as_run":0.1074,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"dnf","dnf_reason":"Blocked by the provider: on 3 attempts Google's backend returned 502 Bad Gateway at the same step (the call that would write its plan), after it had read the brief; nothing was written (owner decision, 2026-11-02)","isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"guardrail_blocks":[{"date":"2026-11-02T19:43:00+00:00","provider":"google","attempt":1,"message":"Antigravity 502 Bad Gateway ('Agent execution terminated due to error') on the model call that would produce the plan, retried by the harness without end; nothing written. Possibly the safety filter surfacing as 502 (same task as entry O's explicit filter blocks); not confirmed","outcome":"retry-blocked","role":"contestant"},{"date":"2026-11-02T20:14:24+00:00","provider":"google","attempt":2,"message":"Antigravity 502 Bad Gateway ('Agent execution terminated due to error') on the model call that would produce the plan, retried by the harness without end; nothing written. Possibly the safety filter surfacing as 502 (same task as entry O's explicit filter blocks); not confirmed","outcome":"retry-blocked","role":"contestant"},{"date":"2026-11-02T21:10:23+00:00","provider":"google","attempt":3,"message":"Antigravity 502 Bad Gateway ('Agent execution terminated due to error') on the model call that would produce the plan, retried by the harness without end; nothing written. Possibly the safety filter surfacing as 502 (same task as entry O's explicit filter blocks); not confirmed","outcome":"retry-blocked","role":"contestant"}],"findings":[]}]},{"schema":1,"id":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","task":"cdef-avx512","lead":"gpt-6.1-terra-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":35.0,"tokens":{"cache_read":21937280,"cache_write":0,"output":83425,"reasoning":46000,"uncached_input":568477},"cost_usd":2.0824,"cost_breakdown":{"cache_read":1.096865,"cache_write":0.0,"input":0.568477,"output":0.417125},"cost_usd_as_run":2.0824,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":56.8,"tokens":{"cache_read":24398568,"cache_write":0,"output":184360,"reasoning":113080,"uncached_input":1240884},"cost_usd":13.5304,"cost_breakdown":{"cache_read":4.879713,"cache_write":0.0,"input":4.963536,"output":3.6872},"cost_usd_as_run":13.5304,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":45.48,"tokens":{"cache_read":23037949,"cache_write":937170,"output":242175,"reasoning":160731,"uncached_input":256},"cost_usd":16.9495,"cost_breakdown":{"cache_read":4.60759,"cache_write":7.49736,"input":0.001024,"output":4.8435},"cost_usd_as_run":16.9495,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":51.24,"tokens":{"cache_read":14642560,"cache_write":0,"output":99332,"reasoning":62400,"uncached_input":585200},"cost_usd":3.628,"cost_breakdown":{"cache_read":1.464256,"cache_write":0.0,"input":1.1704,"output":0.99332},"cost_usd_as_run":3.628,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":51.16,"tokens":{"cache_read":43510144,"cache_write":0,"output":458842,"reasoning":363524,"uncached_input":493927},"cost_usd":0.4799,"cost_breakdown":{"cache_read":0.13053,"cache_write":0.0,"input":0.074089,"output":0.275305},"cost_usd_as_run":0.4799,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-gpt-6.1-terra-max-r1","letter":"B","score":7.5}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","opus-5.5-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":32.38,"tokens":{"uncached_input":82,"cache_read":7713543,"cache_write":250550,"output":195321,"reasoning":133642},"cost_usd":7.4539,"cost_breakdown":{"input":0.000328,"cache_read":1.542709,"cache_write":2.0044,"output":3.90642},"cost_usd_as_run":7.4539,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":1022,"code":857,"comment":128,"blank":37,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":820,"code":725,"comment":45,"blank":50,"path":"tests/selftest.c","role":"test"}],"impl_lines":1022,"impl_code":857,"impl_comment":128,"test_lines":820,"comment_ratio":0.13,"tests_declared":7,"object":{"text_bytes":6128,"rodata_bytes":4608}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":58.08,"reference":3135.26,"speedup":53.982,"baselines":{"dav1d AVX2":140.59,"dav1d AVX-512ICL":53.92}},{"id":"find_dir","candidate":35.74,"reference":382.42,"speedup":10.7,"baselines":{"dav1d AVX2":39.72}}]},"quality":{"duplication":{"ratio":0.098,"lines":84,"significant":857,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":40,"flags":7,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":9.5,"muse-wonder-full-xhigh":9.5,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Fastest measured filter and strong differential and ABI coverage.","muse-wonder-full-xhigh":"fastest filter, register-resident design, documented invariants","opus-5.5-xhigh":"Fastest, whole block in one zmm of bytes, excellent design docs and strong tests; bulky hand-unrolled tables","gpt-6.1-sol-max":"Fast whole-block SIMD with a verified test-coverage gap","deepseek-v4.1-flash-max":"the fastest filter in the set; pays for it with ~380 duplicated lines"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Constant costs are equal, not necessarily zero","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"edges=15 validity-mask chunk is unreachable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"identical tap windows rebuilt within a direction block","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"16 hand-unrolled per-direction window blocks (~390 lines) and a 3.2 KB hand-written hex mask table","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The `filt_masks` row for edges = 15 is never read","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Masked blocks rebuild identical windows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Interior overreads escape the guards","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Constant blocks do not always have zero costs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"sixteen per-direction window blocks (:613-1004, 378 lines) repeat the same `kmovq k + valignq + vpblendmb` pattern eight and then eight more times, differing only in the mask slot and the valign immediate","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"the 400-qword `filt_masks` table (:137-168, 3.2 KB) is unverifiable by inspection (16 edges × 5×5 dy/dx masks, hand-written)","status":"valid","detail":"","location":""}],"score":9.1,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.0}},"defect_points":1.4,"score_cross_vendor":9.125,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":45.5,"tokens":{"uncached_input":124963,"cache_read":3555796,"cache_write":0,"output":58199,"reasoning":37985},"cost_usd":2.375,"cost_breakdown":{"input":0.499852,"cache_read":0.711159,"cache_write":0.0,"output":1.16398},"cost_usd_as_run":2.375,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":496,"code":427,"comment":51,"blank":18,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":711,"code":638,"comment":31,"blank":42,"path":"tests/selftest.c","role":"test"}],"impl_lines":496,"impl_code":427,"impl_comment":51,"test_lines":711,"comment_ratio":0.107,"tests_declared":7,"object":{"text_bytes":3410,"rodata_bytes":240}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":295.71,"reference":3152.06,"speedup":10.659,"baselines":{"dav1d AVX2":141.47,"dav1d AVX-512ICL":53.92}},{"id":"find_dir","candidate":205.26,"reference":379.78,"speedup":1.85,"baselines":{"dav1d AVX2":39.89}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":34,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":5.5,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with expensive direction setup and an overly strict ABI test.","muse-wonder-full-xhigh":"clearest exact 32-bit design, but stack-matrix find_dir is slow","opus-5.5-xhigh":"Correct but slow in both functions (32-bit lanes, stack zeroing); inverted DF test","gpt-6.1-sol-max":"Correct implementation with a costly direction-search staging scheme","deepseek-v4.1-flash-max":"thoroughly tested and clean, but 32-bit lanes leave a lot of speed on the table"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search clears and rebuilds large matrices per block","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Probe requires preservation of an invalid incoming DF","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"stack-heavy find_dir and per-row offset reloads","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter works in 32-bit lanes, one row per iteration, with `vpmulld` on every weighted tap","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"find_dir zeroes 1536 B of stack on every call and squares row sums in scalar code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"DF check is inverted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"ABI/guard coverage is thin","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction search repeatedly stages padded matrices","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Allocation guards do not enforce individual row limits","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`find_dir` and the filter both work in 8×32-bit lanes","status":"valid","detail":"","location":""}],"score":6.8,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.0,"score_cross_vendor":6.75,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.21,"tokens":{"uncached_input":30060,"cache_read":446080,"cache_write":0,"output":12855,"reasoning":3352},"cost_usd":0.1166,"cost_breakdown":{"input":0.03006,"cache_read":0.022304,"cache_write":0.0,"output":0.064275},"cost_usd_as_run":0.1166,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":361,"code":344,"comment":11,"blank":6,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":561,"code":497,"comment":28,"blank":36,"path":"tests/selftest.c","role":"test"}],"impl_lines":361,"impl_code":344,"impl_comment":11,"test_lines":561,"comment_ratio":0.031,"tests_declared":5,"object":{"text_bytes":1879,"rodata_bytes":320}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":275.89,"reference":3174.4,"speedup":11.506,"baselines":{"dav1d AVX2":142.48,"dav1d AVX-512ICL":54.51}},{"id":"find_dir","candidate":474.51,"reference":382.67,"speedup":0.806,"baselines":{"dav1d AVX2":40.23}}]},"quality":{"duplication":{"ratio":0.0756,"lines":26,"significant":344,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":5.0,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":6.0},"council_summaries":{"gpt-6.1-terra-max":"Results matched the oracle, but scalar direction search is slower than the reference.","muse-wonder-full-xhigh":"filter is fine, but find_dir is scalar and slower than the C reference","opus-5.5-xhigh":"Correct, but find_dir is slower than the C reference; weak filter loop; no docs","gpt-6.1-sol-max":"Correct results, with a direction-search performance regression","deepseek-v4.1-flash-max":"correct and readable, but its direction search is *slower* than the C reference"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search updates eight scalar arrays per pixel","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Unclamped paths maintain unused min/max values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"DF test imposes an invalid calling convention","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Direction-search invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"scalar find_dir partial sums","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"min/max always updated, weights rebroadcast per row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"callee-saved test can be masked by the compiler","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"no design overview","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"find_dir is a scalar double loop doing 8 memory read-modify-write adds per pixel","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The filter recomputes and reloads loop-invariants on every row","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The direction-flag test is inverted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Inline-asm calls declare only the xmm registers this implementation happens to use as clobbers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header or design notes; very few comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"find_dir pushes and pops rbx, rbp and r15, which it never uses","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Scalar accumulation dominates direction search","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Interior overreads remain invisible","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unused callee-saved registers are saved","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"`find_dir` runs at 0.82x the C reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no file or function header at all: the only documentation is `","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`find_dir` saves `rbx`, `rbp` and `r15` (:23-28) but never uses them (verified in the disassembly: no reference between push and pop)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"the filter never executes `vzeroupper` (:352-359)","status":"valid","detail":"","location":""}],"score":6.0,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.5}},"defect_points":5.6,"score_cross_vendor":5.667,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":26.52,"tokens":{"uncached_input":76,"cache_read":5876438,"cache_write":211254,"output":142892,"reasoning":97966},"cost_usd":5.7235,"cost_breakdown":{"input":0.000304,"cache_read":1.175288,"cache_write":1.690032,"output":2.85784},"cost_usd_as_run":5.7235,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":699,"code":556,"comment":78,"blank":65,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":972,"code":868,"comment":54,"blank":50,"path":"tests/selftest.c","role":"test"}],"impl_lines":699,"impl_code":556,"impl_comment":78,"test_lines":972,"comment_ratio":0.123,"tests_declared":9,"object":{"text_bytes":7639,"rodata_bytes":3268}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":98.3,"reference":3129.47,"speedup":31.836,"baselines":{"dav1d AVX2":140.51,"dav1d AVX-512ICL":53.5}},{"id":"find_dir","candidate":38.72,"reference":376.29,"speedup":9.718,"baselines":{"dav1d AVX2":39.51}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":556,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":38,"flags":4,"audited":"2026-11-02","reviewed":true},"conduct":{"severity":"minor","summary":"While working it ran `pkill -f 'build/selftest test'`, a host-wide pattern that also matches other contestants' running self-tests, not just its own. No other process is known to have been hit; scores stand (owner decision, 2026-11-02)."},"scores":{"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":9.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Strong whole-block implementation and tests, with a watchpoint false positive.","muse-wonder-full-xhigh":"fast whole-block byte filter, well documented, deep tests","opus-5.5-xhigh":"Fast byte-lane filter with compact runtime-built masks, good docs, excellent tests","gpt-6.1-sol-max":"Strong whole-block implementation and comprehensive contract tests","deepseek-v4.1-flash-max":"cleanest of the fast designs and the best-documented tables; one missing table comment"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Masked lanes can trigger the watchpoint failure","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Copy after equality check does no work","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"watchpoint test passes when skipped","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Shifted copies use a 96-byte stride","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Successful comparisons are followed by an unnecessary copy","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`bflt_taps` (:183-195, 192 words) has no comment describing its layout","status":"valid","detail":"","location":""}],"score":9.0,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.9,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":19.0,"tokens":{"uncached_input":39245,"cache_read":11401984,"cache_write":0,"output":204963,"reasoning":171146},"cost_usd":0.1631,"cost_breakdown":{"input":0.005887,"cache_read":0.034206,"cache_write":0.0,"output":0.122978},"cost_usd_as_run":0.1631,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1631,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":832,"code":657,"comment":93,"blank":82,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":832,"code":750,"comment":36,"blank":46,"path":"tests/selftest.c","role":"test"}],"impl_lines":832,"impl_code":657,"impl_comment":93,"test_lines":832,"comment_ratio":0.124,"tests_declared":8,"object":{"text_bytes":7285,"rodata_bytes":1632}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":136.01,"reference":3197.58,"speedup":23.51,"baselines":{"dav1d AVX2":143.78,"dav1d AVX-512ICL":55.09}},{"id":"find_dir","candidate":75.63,"reference":385.91,"speedup":5.103,"baselines":{"dav1d AVX2":40.44}}]},"quality":{"duplication":{"ratio":0.0396,"lines":26,"significant":657,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":62,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Good vectorized results, but its safety probes have material blind spots.","muse-wonder-full-xhigh":"clean masked-load snapshot design, solid speed","opus-5.5-xhigh":"Clean two-group (4 rows/zmm) filter, good docs; test harness has an ABI bug and guard gaps","gpt-6.1-sol-max":"Sound masked SIMD with weaknesses in its guard and ABI probes","deepseek-v4.1-flash-max":"good all-round implementation with generated index tables"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI probes call with a misaligned stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guard layouts leave excluded inputs readable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Tap windows repeatedly reload overlapping snapshots","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"four loads plus two permutes per tap","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The preservation probe calls the function with a misaligned stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard test gaps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Register plan is inaccurate","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Excluded neighbours remain accessible in guard tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Preservation probes call with misaligned stacks","status":"valid","detail":"","location":""}],"score":8.0,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.3,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":118.34,"tokens":{"uncached_input":1404,"cache_read":83176454,"cache_write":576576,"output":470751,"reasoning":0},"cost_usd":1.6676,"cost_breakdown":{"input":0.000211,"cache_read":1.330823,"cache_write":0.115315,"output":0.221253},"cost_usd_as_run":1.6676,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":1.6676,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":635,"code":495,"comment":98,"blank":42,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":736,"code":638,"comment":53,"blank":45,"path":"tests/selftest.c","role":"test"}],"impl_lines":635,"impl_code":495,"impl_comment":98,"test_lines":736,"comment_ratio":0.165,"tests_declared":7,"object":{"text_bytes":6567,"rodata_bytes":576}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":232.51,"reference":3201.06,"speedup":13.767,"baselines":{"dav1d AVX2":143.15,"dav1d AVX-512ICL":54.72}},{"id":"find_dir","candidate":86.8,"reference":387.25,"speedup":4.461,"baselines":{"dav1d AVX2":40.4}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":245,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"The assembly matched the exercised cases, but several added tests do not test what they claim.","muse-wonder-full-xhigh":"sound two-row zmm filter undermined by false hardware comments","opus-5.5-xhigh":"Correct; documentation contains false technical claims; two rows per zmm wastes half the lanes; inverted DF test","gpt-6.1-sol-max":"Correct kernels accompanied by several misleading test assurances","deepseek-v4.1-flash-max":"fastest of the two-row-zmm designs, but two comments state something false"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Register probes leave DF set in C code","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Extreme test compares a different block","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Supposed guard pages are adjacent writable pages","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Filter repeats the tap work for four row pairs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Constant-128 assertion is unreachable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Missing-neighbor magnitude explanation is false for zero pixels","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"false VPABSW/shift justifications","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"unreachable sec==0 branch with wrong comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"find_dir guard subcase is out of domain and unchecked","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"misleading-docs","title":"False claims justify a workaround","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Two rows per zmm leaves half of every vector idle","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"DF check is inverted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard test only checks for faults and uses an out-of-domain stride","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`.fl_sec`'s `jz .fl_build` can never be taken within the contract","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Extreme tests compare the wrong output region","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Supposed guard boundaries lead into writable pages","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"The explicit constant-128 assertion is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"\"the EVEX register-count word shifts misbehave here\" (:22) and \"nasm's EVEX register-count shift forms encode unreliably\" (:302-303) are false: `vpsrlw zmm0, zmm0, xmm1` assembles with NASM and shifts correctly on this host (verified standalone)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"\"this core's VPABSW wraps 0x8000 rather than saturating\" (:17-18, :308)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`FL_FRAG` (:376-381) clobbers `r9d` and that is not stated","status":"valid","detail":"","location":""}],"score":7.1,"slop":{"score":1.5,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":2.0,"opus-5.5-xhigh":2.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.5}},"defect_points":4.8,"score_cross_vendor":7.1,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":53.44,"tokens":{"uncached_input":172312,"cache_read":1965996,"cache_write":0,"output":44281,"reasoning":26225},"cost_usd":1.9681,"cost_breakdown":{"input":0.689248,"cache_read":0.393199,"cache_write":0.0,"output":0.88562},"cost_usd_as_run":1.9681,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":532,"code":486,"comment":35,"blank":11,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":672,"code":601,"comment":33,"blank":38,"path":"tests/selftest.c","role":"test"}],"impl_lines":532,"impl_code":486,"impl_comment":35,"test_lines":672,"comment_ratio":0.067,"tests_declared":7,"object":{"text_bytes":3060,"rodata_bytes":254}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":435.0,"reference":3147.15,"speedup":7.235,"baselines":{"dav1d AVX2":141.35,"dav1d AVX-512ICL":53.88}},{"id":"find_dir","candidate":221.72,"reference":379.61,"speedup":1.712,"baselines":{"dav1d AVX2":39.85}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":21,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":5.0,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but the filter does substantial unnecessary work.","muse-wonder-full-xhigh":"correct, but always runs all 12 taps and scalar cost loops","opus-5.5-xhigh":"Correct but the slowest filter: it always runs all 12 taps; find_dir costs are scalar","gpt-6.1-sol-max":"Correct implementation with substantial avoidable hot-path work","deepseek-v4.1-flash-max":"simplest and clearest code, but the slowest hot paths"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Every row computes all twelve taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Present top and bottom rows are not guarded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"filter evaluates every tap regardless of strengths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"find_dir costs computed scalarly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter never branches on strengths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"find_dir builds the partial sums in vectors, then evaluates all eight costs with scalar `imul` loops over the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Docs are minimal beyond macro headers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"ABI test has no vzeroupper/XINUSE check and only r9's upper half is poisoned","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Constant 8 is re-broadcast on every row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Disabled tap families still execute","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"SIMD partial sums are reduced through scalar loops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard coverage misses interior row boundaries","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"the filter evaluates all twelve taps on every call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"the block is processed as 8×32-bit lanes (:485-521), i.e","status":"valid","detail":"","location":""}],"score":6.4,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.2,"score_cross_vendor":6.375,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.19,"tokens":{"uncached_input":21840,"cache_read":286208,"cache_write":0,"output":11357,"reasoning":3708},"cost_usd":0.1859,"cost_breakdown":{"input":0.04368,"cache_read":0.028621,"cache_write":0.0,"output":0.11357},"cost_usd_as_run":0.1859,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":362,"code":341,"comment":13,"blank":8,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":511,"code":452,"comment":26,"blank":33,"path":"tests/selftest.c","role":"test"}],"impl_lines":362,"impl_code":341,"impl_comment":13,"test_lines":511,"comment_ratio":0.037,"tests_declared":4,"object":{"text_bytes":1834,"rodata_bytes":2176}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":252.37,"reference":3156.34,"speedup":12.507,"baselines":{"dav1d AVX2":141.47,"dav1d AVX-512ICL":54.04}},{"id":"find_dir","candidate":112.16,"reference":380.45,"speedup":3.392,"baselines":{"dav1d AVX2":39.93}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":341,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with avoidable row-loop work and an incomplete upper-state test.","muse-wonder-full-xhigh":"correct permute-table design, terse docs, wasted min/max","opus-5.5-xhigh":"Correct and exact-width; one 128-bit row per iteration makes it about 2.8× slower than the best; sparse docs","gpt-6.1-sol-max":"Correct SIMD with unnecessary arithmetic and a guard-test blind spot","deepseek-v4.1-flash-max":"compact and correct; only a dead instruction and mid-pack hot-path design"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Single-strength paths still update the clamp range","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probe checks only the upper YMM halves","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"min/max updated on every tap","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"design barely documented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter is row-at-a-time xmm with loop-invariant work inside the row loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir does per-row vpermw with indices loaded from memory, plus a scalar row-sum store","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost no design or register documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`vmovdqu [rsp], ymm15` zero-fill is redundant","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Uses `xmm17` for the constant 1 while xmm15 is free","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Secondary contributions are multiplied by one","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Interior row gaps remain readable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`BORDER 4, rcx` compiles to `mov %rcx,%rcx`","status":"valid","detail":"","location":""}],"score":7.5,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":2.5,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.27,"tokens":{"uncached_input":29655,"cache_read":293888,"cache_write":0,"output":9975,"reasoning":2246},"cost_usd":1.0892,"cost_breakdown":{"input":0.29655,"cache_read":0.293888,"cache_write":0.0,"output":0.49875},"cost_usd_as_run":1.0892,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":324,"code":311,"comment":8,"blank":5,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":500,"code":445,"comment":22,"blank":33,"path":"tests/selftest.c","role":"test"}],"impl_lines":324,"impl_code":311,"impl_comment":8,"test_lines":500,"comment_ratio":0.025,"tests_declared":4,"object":{"text_bytes":1966,"rodata_bytes":1024}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":250.36,"reference":3125.06,"speedup":12.482,"baselines":{"dav1d AVX2":141.6,"dav1d AVX-512ICL":53.71}},{"id":"find_dir","candidate":95.7,"reference":379.4,"speedup":3.964,"baselines":{"dav1d AVX2":39.56}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":311,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Results matched the oracle, with unnecessary clamp work and a missing upper-state check.","muse-wonder-full-xhigh":"correct and compact, but per-row xmm filter and sparse docs","opus-5.5-xhigh":"Compact, correct twin of L's design; same row-at-a-time slowness; sparse docs","gpt-6.1-sol-max":"Sound SIMD with modest redundant work and incomplete memory tests","deepseek-v4.1-flash-max":"the tidiest of the compact designs; one fragile macro interface"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Single-strength paths calculate unused min/max values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probe does not check wide-register cleanup","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"min/max updated on taps whose clamp is skipped","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"no design overview","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"One xmm row per iteration with `test r12d`/`test r13d` branches inside the row loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir 97 TSC against 33–39 for B/T/E (per-row permutes, 8 cost loops)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"One-line header and few comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"ABI probe has no vzeroupper/XINUSE check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`ymm18` is zeroed but never used (index 2 is skipped), and the `[rsp+128]` zero-fill of lanes 8–15 is redundant because their weight is 0","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Clamp tracking runs when its results are unused","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Contiguous accessible pages hide interior overreads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"YMM18 initialization has no consumer","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"the `TAP` macro picks the tap weight by matching the argument against `xmm8`/`xmm9`/numeric `1`","status":"valid","detail":"","location":""}],"score":7.5,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.7,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":76.19,"tokens":{"uncached_input":144,"cache_read":19804175,"cache_write":403147,"output":459341,"reasoning":381276},"cost_usd":16.3734,"cost_breakdown":{"input":0.000576,"cache_read":3.960835,"cache_write":3.225176,"output":9.18682},"cost_usd_as_run":16.3734,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":714,"code":602,"comment":87,"blank":25,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1235,"code":1101,"comment":69,"blank":65,"path":"tests/selftest.c","role":"test"}],"impl_lines":714,"impl_code":602,"impl_comment":87,"test_lines":1235,"comment_ratio":0.126,"tests_declared":9,"object":{"text_bytes":4647,"rodata_bytes":14392}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":74.87,"reference":3163.27,"speedup":42.25,"baselines":{"dav1d AVX2":141.93,"dav1d AVX-512ICL":54.17}},{"id":"find_dir","candidate":34.27,"reference":381.83,"speedup":11.142,"baselines":{"dav1d AVX2":40.02}}]},"quality":{"duplication":{"ratio":0.0133,"lines":8,"significant":602,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":71,"flags":7,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":8.0,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Fast and well covered, but its test harness can reject conforming code.","muse-wonder-full-xhigh":"fastest find_dir and strong tests, with a fragile flags dependency","opus-5.5-xhigh":"Fast all-zmm design, superb header docs, the most thorough tests; opaque 12 KB table","gpt-6.1-sol-max":"Fast whole-block kernels backed by strong footprint and ABI tests","deepseek-v4.1-flash-max":"best find_dir and excellent tests; the largest tables and the same block duplication"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Watchpoints treat masked-off lanes as reads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Unsigned indexing overflows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper-state probe assumes unconditional cleanup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Constant costs are not generally zero","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"correctness depends on flags surviving ~20 instructions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"watchpoint test passes when skipped","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"12 KiB hand-enumerated availability table","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"signed cost maximum needs an unstated bound","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`filt_avail` is a 16×8×12-qword (12 KB) hex table with no generator in the source","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Constant-block costs are described incorrectly","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"`filt_avail` is 1536 hand-written qwords (:126-253, 12 KB of the entry's 14.4 KB `.rodata`): a per-edges/per-dir/per-slot pixel-validity table that no reviewer can check by reading","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"eight mask-carrying and eight mask-free direction blocks (:613-1004, ≈390 lines) duplicate each other apart from the `kmovq` loads","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`TAP`'s 6th argument is the *min/max tracking* flag, but `FILTER` passes `%2` (the secondary-strength flag) for the primary taps and `%1` (the primary flag) for the secondary taps (:452-465)","status":"valid","detail":"","location":""}],"score":8.7,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.0}},"defect_points":2.2,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":12.25,"tokens":{"uncached_input":48601,"cache_read":358528,"cache_write":0,"output":20709,"reasoning":9749},"cost_usd":1.88,"cost_breakdown":{"input":0.48601,"cache_read":0.358528,"cache_write":0.0,"output":1.03545},"cost_usd_as_run":1.88,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":357,"code":325,"comment":19,"blank":13,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":634,"code":565,"comment":29,"blank":40,"path":"tests/selftest.c","role":"test"}],"impl_lines":357,"impl_code":325,"impl_comment":19,"test_lines":634,"comment_ratio":0.055,"tests_declared":6,"object":{"text_bytes":3606,"rodata_bytes":458}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":235.37,"reference":3124.39,"speedup":13.274,"baselines":{"dav1d AVX2":140.3,"dav1d AVX-512ICL":53.71}},{"id":"find_dir","candidate":92.26,"reference":382.72,"speedup":4.148,"baselines":{"dav1d AVX2":39.56}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":325,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":9.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation with broad tests, except for the DF probe","muse-wonder-full-xhigh":"Correct, compact and strongly guarded; minor doc inaccuracy","opus-5.5-xhigh":"correct, tidy, thorough tests; filter is xmm-only","gpt-6.1-sol-max":"Compact, correct implementation with strong tests; filter throughput has room to improve.","deepseek-v4.1-flash-max":"simple, readable and correct (14.4x filter, 4.1x dir); documentation is only line labels"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probe expects DF set across a call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Entry points lack arg/register docs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"TAP_PAIR weight argument misdocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Filter does one row per iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Dead jump","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Filter processes only one row per iteration","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Header gives no overview of the algorithm","status":"valid","detail":"","location":""}],"score":8.4,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.2,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":71.69,"tokens":{"uncached_input":487228,"cache_read":13067968,"cache_write":0,"output":297797,"reasoning":180722},"cost_usd":0.614,"cost_breakdown":{"input":0.073084,"cache_read":0.392039,"cache_write":0.0,"output":0.148898},"cost_usd_as_run":0.614,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":532,"code":401,"comment":77,"blank":54,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":532,"impl_code":401,"impl_comment":77,"test_lines":348,"comment_ratio":0.161,"tests_declared":3,"object":{"text_bytes":3182,"rodata_bytes":944}},"hidden":{"passed":3,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":null,"speedup":null,"baselines":{"dav1d AVX2":null,"dav1d AVX-512ICL":null}},{"id":"find_dir","candidate":null,"reference":null,"speedup":null,"baselines":{"dav1d AVX2":null}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":57,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.0},"council_summaries":{"gpt-6.1-terra-max":"Both entry points crash, and substantial independent correctness defects remain","muse-wonder-full-xhigh":"Segfaults on every find_dir call and the filter never filters","opus-5.5-xhigh":"crashes on every call; the filter is unfinished and the direction search is wrong","gpt-6.1-sol-max":"Both entry points crash, with further independent defects behind the faults.","deepseek-v4.1-flash-max":"does not work at all: `find_dir` segfaults, `cost[2]` is mis-weighted, and the filter never writes `dst`"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Aligned vector accesses fault on valid calls","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Direction costs and accumulators are incorrect","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Filter decodes saved registers as arguments","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter never computes or stores output","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"File claims bit-exact behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Unused empty macro","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"robustness","title":"Guaranteed alignment fault in find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Direction costs stored in permuted slots","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter builds a snapshot and returns","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"abi","title":"Stack arguments decoded from saved registers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"robustness","title":"Combined edge tests always pass","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Cost[2] scaled by 105 twice","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Bottom borders are overwritten as missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Border zones stored packed, not widened","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"Header and layout comments are false","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"dead-code","title":"Unused constants and macros","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests not extended and cannot pass","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"robustness","title":"Filter faults immediately","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"robustness","title":"Direction search faults","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Filter never filters","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Direction search is wrong even without the crash","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"robustness","title":"Read-contract violations and stack corruption","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"Tests not extended","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"misleading-docs","title":"False claims in the header","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Dead code","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Invalid alignment assumptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Broken direction state and cost reduction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"abi","title":"Stack arguments come from saved registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Pixel values become stack addresses","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter calculation is absent","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Unused implementation scaffolding","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"robustness","title":"find_dir crashes on the first call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`cost[2]` is multiplied by 105 twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"The filter never filters","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"robustness","title":"The filter faults on two misaligned `vmovdqa64`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"Left/right border pixels are stored to a wild stack address","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"Top/bottom left and right borders are always overwritten with MISSING","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"dead-code","title":"`FILTER_ROW_BUILD` is an empty macro","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"The header and the filter's block comment describe bit-exact behaviour and a snapshot loop that do not exist","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"`mov rax,32*2` / `mov r9,r15` are overwritten two instructions later","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"\"row sums live at [rsp+88..119]\" is wrong; they are at [rsp+40..71]","status":"valid","detail":"","location":""}],"score":0.5,"slop":{"score":3.0,"per_seat":{"gpt-6.1-terra-max":1.5,"muse-wonder-full-xhigh":4.0,"opus-5.5-xhigh":2.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":6.0}},"defect_points":23.4,"score_cross_vendor":0.5,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.68,"tokens":{"uncached_input":47174,"cache_read":361344,"cache_write":0,"output":27020,"reasoning":14245},"cost_usd":0.4007,"cost_breakdown":{"input":0.094348,"cache_read":0.036134,"cache_write":0.0,"output":0.2702},"cost_usd_as_run":0.4007,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":423,"code":374,"comment":30,"blank":19,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":712,"code":633,"comment":39,"blank":40,"path":"tests/selftest.c","role":"test"}],"impl_lines":423,"impl_code":374,"impl_comment":30,"test_lines":712,"comment_ratio":0.074,"tests_declared":7,"object":{"text_bytes":5427,"rodata_bytes":992}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":127.45,"reference":3268.46,"speedup":25.645,"baselines":{"dav1d AVX2":139.46,"dav1d AVX-512ICL":53.62}},{"id":"find_dir","candidate":90.87,"reference":374.53,"speedup":4.122,"baselines":{"dav1d AVX2":39.35}}]},"quality":{"duplication":{"ratio":0.0561,"lines":21,"significant":374,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":9.5,"muse-wonder-full-xhigh":9.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and well tested apart from an invalid DF assumption and a redundant instruction","muse-wonder-full-xhigh":"Correct, second-fastest filter, thorough tests","opus-5.5-xhigh":"clean, fast (26× / 4.1×), thorough tests","gpt-6.1-sol-max":"Strong four-row kernel, careful memory handling and substantial contract tests.","deepseek-v4.1-flash-max":"fast (26.8x filter, 4.0x dir), clean and correct, with thorough tests; only a small scalar round-trip"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probe rejects a legal DF clear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Unneeded register self-move","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar row-sum transfer in find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"One-line prototypes only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"No-op with a misleading comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Dead jump","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unneeded integer self-move","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"Row sums make a scalar round-trip per row","status":"valid","detail":"","location":""}],"score":9.1,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.0,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.38,"tokens":{"uncached_input":41556,"cache_read":847616,"cache_write":0,"output":32598,"reasoning":18772},"cost_usd":0.2469,"cost_breakdown":{"input":0.041556,"cache_read":0.042381,"cache_write":0.0,"output":0.16299},"cost_usd_as_run":0.2469,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":490,"code":448,"comment":24,"blank":18,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":671,"code":608,"comment":24,"blank":39,"path":"tests/selftest.c","role":"test"}],"impl_lines":490,"impl_code":448,"impl_comment":24,"test_lines":671,"comment_ratio":0.051,"tests_declared":7,"object":{"text_bytes":5086,"rodata_bytes":1856}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":139.37,"reference":3193.8,"speedup":22.916,"baselines":{"dav1d AVX2":143.53,"dav1d AVX-512ICL":55.26}},{"id":"find_dir","candidate":68.66,"reference":385.74,"speedup":5.618,"baselines":{"dav1d AVX2":40.48}}]},"quality":{"duplication":{"ratio":0.1027,"lines":46,"significant":448,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":9.5,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":10.0,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct and well structured, with one invalid ABI test assumption","muse-wonder-full-xhigh":"Correct and fast on both functions; thinner extremes/guard coverage than peers","opus-5.5-xhigh":"fast (25× / 5.6×), well documented, solid tests","gpt-6.1-sol-max":"Efficient paired direction reductions and four-row filtering, with comprehensive verification.","deepseek-v4.1-flash-max":"compact, fast (25.4x filter, 5.5x dir), correct, and cleanly documented; no defects found"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probe requires an incoming set direction flag to stay set","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinner extremes/guard coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Redundant row copy in DIR_ROW","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Redundant instruction","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Narrow upper-state and guard checks","status":"valid","detail":"","location":""}],"score":9.1,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.8,"score_cross_vendor":8.667,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":10.26,"tokens":{"uncached_input":62912,"cache_read":718208,"cache_write":0,"output":24582,"reasoning":11409},"cost_usd":0.4435,"cost_breakdown":{"input":0.125824,"cache_read":0.071821,"cache_write":0.0,"output":0.24582},"cost_usd_as_run":0.4435,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":341,"code":315,"comment":16,"blank":10,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":779,"code":704,"comment":33,"blank":42,"path":"tests/selftest.c","role":"test"}],"impl_lines":341,"impl_code":315,"impl_comment":16,"test_lines":779,"comment_ratio":0.048,"tests_declared":6,"object":{"text_bytes":3279,"rodata_bytes":548}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":175.15,"reference":3135.14,"speedup":17.9,"baselines":{"dav1d AVX2":140.59,"dav1d AVX-512ICL":53.58}},{"id":"find_dir","candidate":88.81,"reference":377.51,"speedup":4.251,"baselines":{"dav1d AVX2":39.64}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":315,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":10.0,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, efficient implementation with broad differential, guard and ABI coverage","muse-wonder-full-xhigh":"Correct, compact, well tested; mid-pack speed and sparse entry docs","opus-5.5-xhigh":"fast (19× / 4.2×), compact, very thorough tests","gpt-6.1-sol-max":"Clear, correct code with thorough tests and a modest filter parallelism limitation.","deepseek-v4.1-flash-max":"compact and fast (19.7x filter, 4.2x dir), strong ABI tests; comments are thin and one store is redundant"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Entry points lack arg/register docs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"One row per filter iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Dead jump","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"No register map for direction accumulators","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Opposite-tap packing still serializes eight rows","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"20 comment lines for 341 lines of dense SIMD","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`vmovdqu [rsp+16], xmm7` zeroes lanes that `dir_hv_weights` multiplies by zero anyway","status":"valid","detail":"","location":""}],"score":8.7,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":1.0,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.85,"tokens":{"uncached_input":18174,"cache_read":226432,"cache_write":0,"output":5391,"reasoning":125},"cost_usd":0.6777,"cost_breakdown":{"input":0.18174,"cache_read":0.226432,"cache_write":0.0,"output":0.26955},"cost_usd_as_run":0.6777,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":317,"code":311,"comment":4,"blank":2,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":411,"code":359,"comment":21,"blank":31,"path":"tests/selftest.c","role":"test"}],"impl_lines":317,"impl_code":311,"impl_comment":4,"test_lines":411,"comment_ratio":0.013,"tests_declared":4,"object":{"text_bytes":1725,"rodata_bytes":1048}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":338.96,"reference":3162.73,"speedup":9.331,"baselines":{"dav1d AVX2":142.14,"dav1d AVX-512ICL":54.21}},{"id":"find_dir","candidate":430.29,"reference":387.59,"speedup":0.901,"baselines":{"dav1d AVX2":39.98}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":311,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":5.5,"opus-5.5-xhigh":4.5,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":5.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but leaves substantial hot-path work scalar or unnecessary","muse-wonder-full-xhigh":"Correct on all probes, but find_dir is slower than the C reference","opus-5.5-xhigh":"correct, but direction search is slower than the C reference","gpt-6.1-sol-max":"Correct and safe in the exercised domain, but direction search retains substantial scalar overhead.","deepseek-v4.1-flash-max":"correct and safe but the dir search is slower than the C reference (0.87x) and documentation is minimal"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Scalar indexed direction accumulation is slower than the reference","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Filter computes disabled-strength taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI check omits `rbp` and direction search","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Direction-search layout and interface are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"Scalar find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Bytewise border copy","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Masked callee-saved check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No file or entry documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Overloaded TAP parameter and weight trick","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"Scalar direction search","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter does wasted and scalar work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Unreliable register check and narrow guards","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Scalar partial-sum construction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Disabled taps still execute","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Direction-search invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"performance","title":"find_dir is below 1.0x","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"No design documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Tap weights are re-broadcast per row and min/max are tracked on every tap even in single-strength paths","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The added guard test uses one stride and one placement and checks no DF/upper state","status":"valid","detail":"","location":""}],"score":5.7,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.6,"score_cross_vendor":5.0,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":64.85,"tokens":{"uncached_input":154159,"cache_read":3371800,"cache_write":0,"output":56157,"reasoning":31166},"cost_usd":2.4141,"cost_breakdown":{"input":0.616636,"cache_read":0.67436,"cache_write":0.0,"output":1.12314},"cost_usd_as_run":2.4141,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":559,"code":465,"comment":63,"blank":31,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":661,"code":587,"comment":33,"blank":41,"path":"tests/selftest.c","role":"test"}],"impl_lines":559,"impl_code":465,"impl_comment":63,"test_lines":661,"comment_ratio":0.119,"tests_declared":7,"object":{"text_bytes":4331,"rodata_bytes":984}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":321.03,"reference":3086.17,"speedup":9.613,"baselines":{"dav1d AVX2":138.07,"dav1d AVX-512ICL":52.74}},{"id":"find_dir","candidate":89.4,"reference":378.1,"speedup":4.229,"baselines":{"dav1d AVX2":38.97}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":32,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with avoidable filter work and gaps in the ABI and guard tests","muse-wonder-full-xhigh":"Correct and the most readable snapshot design; slow filter and a guard-coverage hole","opus-5.5-xhigh":"correct and well commented; filter is slow and the test wrappers misalign the stack","gpt-6.1-sol-max":"Correct and well explained, with a small arithmetic inefficiency and overstated test documentation.","deepseek-v4.1-flash-max":"readable and correctly tested (11.0x filter, 4.25x dir); helper calls sit in the hot path"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Reloads invariant tap offsets for every row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI wrappers make invalid calls","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Guard coverage is overstated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Narrow rows plus internal calls","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard test never presents TOP","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Slow xmm-only filter","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Wrappers call with a misaligned stack, and guards are partial","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Values never read","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Unlabelled pair order","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Sign restoration uses unnecessary instructions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Guard coverage is overstated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"`fill_row` is `call`ed once per snapshot row and `get_offsets` three times per filter call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"When `pri_strength` is 0 the primary tap weights are still computed and stored","status":"valid","detail":"","location":""}],"score":7.8,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":2.8,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.99,"tokens":{"uncached_input":26829,"cache_read":146816,"cache_write":0,"output":6043,"reasoning":922},"cost_usd":0.1288,"cost_breakdown":{"input":0.053658,"cache_read":0.014682,"cache_write":0.0,"output":0.06043},"cost_usd_as_run":0.1288,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":1013,"code":1006,"comment":5,"blank":2,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":437,"code":382,"comment":22,"blank":33,"path":"tests/selftest.c","role":"test"}],"impl_lines":1013,"impl_code":1006,"impl_comment":5,"test_lines":437,"comment_ratio":0.005,"tests_declared":4,"object":{"text_bytes":6117,"rodata_bytes":560}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":304.7,"reference":3148.07,"speedup":10.332,"baselines":{"dav1d AVX2":141.51,"dav1d AVX-512ICL":53.96}},{"id":"find_dir","candidate":275.17,"reference":380.2,"speedup":1.382,"baselines":{"dav1d AVX2":39.98}}]},"quality":{"duplication":{"ratio":0.0835,"lines":84,"significant":1006,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":5.0,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":6.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but direction search is costly and difficult to maintain","muse-wonder-full-xhigh":"Correct everywhere probed, but scalar find_dir and almost no documentation","opus-5.5-xhigh":"correct, but direction search is scalar (1.38×) and there are almost no docs","gpt-6.1-sol-max":"Functionally sound, with avoidable scalar work and extensive source duplication.","deepseek-v4.1-flash-max":"correct and safe, but an undocumented 1000-line unrolled file with the slowest dir search of the working entries"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search performs 512 scalar stack updates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Missing-tap mask repeats an already sufficient constraint","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"verbosity","title":"Hand-expanded pixel accumulation obscures the mapping","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probe omits `rbp` and direction search","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Direction-search scratch layout is undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Fully scalar find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Essentially undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-tap weight broadcasts and per-row branches","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"ABI wrapper never checks rbp","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Literal 660-line unroll","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"Direction search is fully scalar","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter is xmm-only and redoes constant work per tap","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"style","title":"650 lines of unrolled generated code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header, interface docs or register maps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong order in comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin ABI and guard coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction accumulation remains scalar","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Invariant setup repeats inside the row loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Hundreds of handwritten repeated updates","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"find_dir uses 512 scalar stack read-modify-writes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Tap weights are rebuilt inside every tap","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Edge flags are re-loaded from the stack for every row and every tap","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Added tests cover guards and GPR preservation but no dir tie/pattern cases and no DF or upper-state check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"`test_boundaries` returns 0 (hard FAIL) when the page size is not 4096","status":"valid","detail":"","location":""}],"score":6.2,"slop":{"score":1.1,"per_seat":{"gpt-6.1-terra-max":2.0,"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":6.8,"score_cross_vendor":5.667,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.23,"tokens":{"uncached_input":13607,"cache_read":81408,"cache_write":0,"output":318,"reasoning":41},"cost_usd":0.0193,"cost_breakdown":{"input":0.013607,"cache_read":0.00407,"cache_write":0.0,"output":0.00159},"cost_usd_as_run":0.0193,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":25,"code":10,"comment":10,"blank":5,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":25,"impl_code":10,"impl_comment":10,"test_lines":348,"comment_ratio":0.5,"tests_declared":3,"object":{"text_bytes":4,"rodata_bytes":0}},"hidden":{"passed":1,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3135.93,"speedup":null,"baselines":{"dav1d AVX2":141.35,"dav1d AVX-512ICL":53.75}},{"id":"find_dir","candidate":null,"reference":378.69,"speedup":null,"baselines":{"dav1d AVX2":39.72}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":3,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5},"council_summaries":{"gpt-6.1-terra-max":"Both exported functions remain stubs","muse-wonder-full-xhigh":"Honestly labelled stub: neither function is implemented","opus-5.5-xhigh":"the unchanged stub; nothing is implemented","gpt-6.1-sol-max":"Both exported routines remain unimplemented stubs.","deepseek-v4.1-flash-max":"the scaffold stub: no implementation at all"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Direction search neither computes nor stores variance","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter returns without modifying the block","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"find_dir stub returns wrong results","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter stub writes nothing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests not extended","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Both functions are stubs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"Tests not extended","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Direction search leaves variance untouched","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter performs no work","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"Both entry points are stubs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"The selftest is the unmodified scaffold and fails","status":"valid","detail":"","location":""}],"score":0.4,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.7,"score_cross_vendor":0.667,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":81.19,"tokens":{"uncached_input":346063,"cache_read":6228224,"cache_write":0,"output":377529,"reasoning":168811},"cost_usd":6.0714,"cost_breakdown":{"input":0.692126,"cache_read":3.114112,"cache_write":0.0,"output":2.265174},"cost_usd_as_run":6.0714,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"lines":514,"code":435,"comment":36,"blank":43,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1053,"code":962,"comment":32,"blank":59,"path":"tests/selftest.c","role":"test"}],"impl_lines":514,"impl_code":435,"impl_comment":36,"test_lines":1053,"comment_ratio":0.076,"tests_declared":7,"object":{"text_bytes":5497,"rodata_bytes":1280}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":220.63,"reference":3117.37,"speedup":14.129,"baselines":{"dav1d AVX2":139.88,"dav1d AVX-512ICL":53.67}},{"id":"find_dir","candidate":68.11,"reference":375.62,"speedup":5.515,"baselines":{"dav1d AVX2":39.51}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":46,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised domain, with several test-harness defects","muse-wonder-full-xhigh":"Correct, exact 32-bit arithmetic, very extensive tests with one out-of-domain guard layout","opus-5.5-xhigh":"correct and robust; filter computes in 32-bit lanes at half width","gpt-6.1-sol-max":"Correct and extensively tested, with repeated invariant loads and a redundant instruction.","deepseek-v4.1-flash-max":"very strong tests and good performance (15.6x filter, 5.6x dir); minor hot-path and comment issues"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI probe assumes DF may be set on entry","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests require behavior outside the stride domain","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Misaligned C load has undefined behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Unneeded register self-move","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard layout exceeds the domain","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Pointless re-alignment and store-load broadcasts","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter uses 32-bit lanes and `vpmulld` for tap weights","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No upper-state check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Garbled header","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"No-op instruction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Tap offsets are repeatedly reloaded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unneeded integer self-move","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Tap weights are re-broadcast from the stack before every tap group","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"The header line \"No calls except none at all.\" is ungrammatical and confusing","status":"valid","detail":"","location":""}],"score":8.3,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.4,"score_cross_vendor":8.3,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":43.96,"tokens":{"uncached_input":76,"cache_read":9532687,"cache_write":374541,"output":574781,"reasoning":534554},"cost_usd":1.5284,"cost_breakdown":{"input":3e-05,"cache_read":0.466222,"cache_write":0.337563,"output":0.724625},"cost_usd_as_run":1.5284,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":564,"code":470,"comment":55,"blank":39,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":879,"code":778,"comment":43,"blank":58,"path":"tests/selftest.c","role":"test"}],"impl_lines":564,"impl_code":470,"impl_comment":55,"test_lines":879,"comment_ratio":0.105,"tests_declared":9,"object":{"text_bytes":4336,"rodata_bytes":1664}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":148.19,"reference":3181.75,"speedup":21.471,"baselines":{"dav1d AVX2":144.08,"dav1d AVX-512ICL":54.21}},{"id":"find_dir","candidate":85.33,"reference":383.43,"speedup":4.493,"baselines":{"dav1d AVX2":40.06}}]},"quality":{"duplication":{"ratio":0.0596,"lines":28,"significant":470,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":46,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with avoidable filter overhead and test gaps","opus-5.5-xhigh":"Correct and robust snapshot design with clear macros; moderate speed (24.5× filter, 4.5× find_dir) held back by stack round-trips and scalar setup; tests don't check upper state","deepseek-v4.1-flash-max":"Correct, compact and well tested; 2.2x behind the leader because every tap re-gathers its rows from the stack snapshot.","gpt-6.1-sol-max":"Strong correctness coverage, with minor wasted work and an incorrect ABI assertion","muse-wonder-full-xhigh":"Compact, correct and fast, with strong tests and only minor waste"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Repeated snapshot gathers cost filter throughput","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests miss reads in row gaps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Register probe requires an invalid DF state","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The filter builds its int16 snapshot with scalar byte loads and word stores, then gathers each tap again","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Min/max is tracked even when no clamp happens","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir does 48 single-row `vpermw` and seven full horizontal reductions to a GPR plus `vpinsrd` (`FAM_COST`/`HSUM16`)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Upper state is never checked, and DF is set on entry","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`FL_GATHER` (src/cdef_avx512.asm:285-290) reloads four snapshot rows and assembles the window with three `vinserti32x4`, and `FL_TAP` does that for 24 taps per row group","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Tracks unused clamp bounds","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"DF assertion rejects conforming code","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Min/max updated when unused","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Extreme pictures evolve","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"ABI probe DF expectation wrong, no XINUSE","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Top/bottom snapshot fill duplicated","status":"valid","detail":"","location":""}],"score":8.0,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.1,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.01,"tokens":{"uncached_input":28343,"cache_read":371456,"cache_write":0,"output":2794,"reasoning":439},"cost_usd":0.0609,"cost_breakdown":{"input":0.028343,"cache_read":0.018573,"cache_write":0.0,"output":0.01397},"cost_usd_as_run":0.0609,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":1065,"code":1065,"comment":0,"blank":0,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":375,"code":325,"comment":20,"blank":30,"path":"tests/selftest.c","role":"test"}],"impl_lines":1065,"impl_code":1065,"impl_comment":0,"test_lines":375,"comment_ratio":0.0,"tests_declared":4,"object":{"text_bytes":3683,"rodata_bytes":40}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":3366.47,"reference":3157.14,"speedup":0.938,"baselines":{"dav1d AVX2":141.47,"dav1d AVX-512ICL":53.96}},{"id":"find_dir","candidate":484.72,"reference":379.95,"speedup":0.784,"baselines":{"dav1d AVX2":39.93}}]},"quality":{"duplication":{"ratio":0.0423,"lines":45,"significant":1065,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":3.5,"opus-5.5-xhigh":3.0,"deepseek-v4.1-flash-max":4.0,"gpt-6.1-sol-max":4.0,"muse-wonder-full-xhigh":5.0},"council_summaries":{"gpt-6.1-terra-max":"A caller-stack write can fault on valid calls, and the hot paths remain scalar","opus-5.5-xhigh":"No SIMD implementation at all: the file is scalar compiler output of the C reference (1.0× filter, 0.78× find_dir), has no comments, and writes to the caller's stack","deepseek-v4.1-flash-max":"Bit-exact, but it is a comment-free scalar compiler dump: no AVX-512 anywhere and no speed-up over the C reference it is measured against.","gpt-6.1-sol-max":"A verified caller-stack fault and scalar hot paths require substantial changes","muse-wonder-full-xhigh":"Bit-exact but scalar, undocumented and no faster than the reference"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Writes into the caller’s argument area","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Both operations use scalar inner loops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Safety coverage misses the stack violation and active borders","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Assembly lacks the invariants needed to maintain it","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"dead-code","title":"The missing-left path runs an unused counting loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"The deliverable is not an AVX-512 kernel","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"abi","title":"Writes the caller's incoming stack-argument slot","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"There is no documentation anywhere","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"style","title":"Compiler output, not maintainable source","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"An empty counting loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The added test is a single fixed case","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"performance","title":"`objdump` shows zero VEX/EVEX instructions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"docs","title":"1065 lines of assembly with not one comment: no description of the algorithm, the 272/400-byte stack frames, the register conventions, or the `.rodata` tables","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"style","title":"The file is compiler output, not source: GCC-style `.L3/.L52/.L100` label numbering, `cmovs`/`setne` idiom, dozens of redundant `mov r10d,r10d`-class register shuffles, and red-zone spills at `[rsp-112]`/`[rsp-120]` mixed with frame slots at `[rsp+328..392]`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"tests/selftest.c is the stock scaffold plus one guard case (tests/selftest.c:227-249) that only exercises edges=0 at stride 17: there is no guard-page coverage for the other 15 edge combinations (the exact contract risk), no callee-saved/MXCSR/DF check, and no extreme (0/255, single-outlier) pictures.","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Writes outside the permitted stack frame","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Pixel arithmetic remains scalar","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"No ABI-state coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Assembly invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Empty loop when LEFT is absent","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"No vector datapath","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"abi","title":"Writes caller's stack-arg slot","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Zero comments in 1065 lines","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Only scaffold plus one guard case","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Empty counting loop","status":"valid","detail":"","location":""}],"score":3.9,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":10.0,"score_cross_vendor":4.0,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1-b2","task":"cdef-avx512","lead":"muse-wonder-full-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":10.4,"tokens":{"cache_read":5245568,"cache_write":0,"output":26714,"reasoning":15008,"uncached_input":116022},"cost_usd":0.5119,"cost_breakdown":{"cache_read":0.262278,"cache_write":0.0,"input":0.116022,"output":0.13357},"cost_usd_as_run":0.5119,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":13.75,"tokens":{"cache_read":5730892,"cache_write":233062,"output":82888,"reasoning":58634,"uncached_input":64},"cost_usd":4.6687,"cost_breakdown":{"cache_read":1.146178,"cache_write":1.864496,"input":0.000256,"output":1.65776},"cost_usd_as_run":4.6687,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.318","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":14.63,"tokens":{"cache_read":11082240,"cache_write":0,"output":159325,"reasoning":128873,"uncached_input":121444},"cost_usd":0.1471,"cost_breakdown":{"cache_read":0.033247,"cache_write":0.0,"input":0.018217,"output":0.095595},"cost_usd_as_run":0.1471,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":15.73,"tokens":{"cache_read":6350448,"cache_write":0,"output":58746,"reasoning":33798,"uncached_input":268687},"cost_usd":3.5198,"cost_breakdown":{"cache_read":1.27009,"cache_write":0.0,"input":1.074748,"output":1.17492},"cost_usd_as_run":3.5198,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":16.94,"tokens":{"cache_read":3253888,"cache_write":0,"output":34510,"reasoning":21685,"uncached_input":131346},"cost_usd":0.9332,"cost_breakdown":{"cache_read":0.325389,"cache_write":0.0,"input":0.262692,"output":0.3451},"cost_usd_as_run":0.9332,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.5}],"panel":["gpt-6.1-terra-max","opus-5.5-xhigh","deepseek-v4.1-flash-max","muse-wonder-full-xhigh","gpt-6.1-sol-max"]},"entries":[{"letter":"C","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":7.87,"tokens":{"uncached_input":108966,"cache_read":846427,"cache_write":0,"output":55835,"reasoning":0},"cost_usd":0.8609,"cost_breakdown":{"input":0.217932,"cache_read":0.084643,"cache_write":0.0,"output":0.55835},"cost_usd_as_run":0.8609,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"lines":800,"code":539,"comment":168,"blank":93,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":612,"code":525,"comment":35,"blank":52,"path":"tests/selftest.c","role":"test"}],"impl_lines":800,"impl_code":539,"impl_comment":168,"test_lines":612,"comment_ratio":0.238,"tests_declared":4,"object":{"text_bytes":3289,"rodata_bytes":202}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":271.23,"reference":3086.3,"speedup":11.379,"baselines":{"dav1d AVX2":138.24,"dav1d AVX-512ICL":52.87}},{"id":"find_dir","candidate":50.31,"reference":370.16,"speedup":7.358,"baselines":{"dav1d AVX2":38.93}}]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and fast in the checks run, with several test-coverage defects","opus-5.5-xhigh":"correct, fast and very readable; guard-page test is thin","deepseek-v4.1-flash-max":"bit-exact, well documented and fast on the filter; only small hot-loop and test-gap issues","muse-wonder-full-xhigh":"correct and fast; verbose, guard test checks crashes only","gpt-6.1-sol-max":"Correct kernels, but the outlier tests generate constant blocks."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The outlier generator produces constant blocks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guards cover the allocation span rather than each row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard setup failures can silently pass","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard-page section is thin and can be skipped silently","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"No ABI checks (callee-saved, XINUSE, poisoned argument upper halves)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Tap offsets are reloaded every row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"loop-invariant tap offsets re-derived every row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"guard-page section never checks the result","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard-page filter test never verifies output","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Operand-restating comments and duplicated bodies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Offset reload per tap per row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Overlapping MISSING prefill store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Stack-frame comment conflates caller arg area","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Outliers equal their backgrounds","status":"valid","detail":"","location":""}],"score":8.4,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.5,"score_cross_vendor":8.4,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","task":"cdef-avx512","lead":"muse-wonder-full-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":42.3,"tokens":{"cache_read":20501504,"cache_write":0,"output":106184,"reasoning":56453,"uncached_input":620729},"cost_usd":2.1767,"cost_breakdown":{"cache_read":1.025075,"cache_write":0.0,"input":0.620729,"output":0.53092},"cost_usd_as_run":2.1767,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":63.57,"tokens":{"cache_read":24537846,"cache_write":1075937,"output":317202,"reasoning":218796,"uncached_input":300},"cost_usd":19.8602,"cost_breakdown":{"cache_read":4.907569,"cache_write":8.607496,"input":0.0012,"output":6.34404},"cost_usd_as_run":19.8602,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.318","sessions":4}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":168.42,"tokens":{"cache_read":24735548,"cache_write":0,"output":208048,"reasoning":116635,"uncached_input":1109901},"cost_usd":13.5477,"cost_breakdown":{"cache_read":4.94711,"cache_write":0.0,"input":4.439604,"output":4.16096},"cost_usd_as_run":13.5477,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":4}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":58.29,"tokens":{"cache_read":44408064,"cache_write":0,"output":587303,"reasoning":463769,"uncached_input":509133},"cost_usd":0.562,"cost_breakdown":{"cache_read":0.133224,"cache_write":0.0,"input":0.07637,"output":0.352381},"cost_usd_as_run":0.562,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":4}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":71.3,"tokens":{"cache_read":16117120,"cache_write":0,"output":146520,"reasoning":90286,"uncached_input":639983},"cost_usd":4.357,"cost_breakdown":{"cache_read":1.611712,"cache_write":0.0,"input":1.279966,"output":1.4652},"cost_usd_as_run":4.357,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":10.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"I","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"F","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfO","source_run":"2026-11-02-cdef-avx512-muse-wonder-full-xhigh-r1","letter":"A","score":9.5}],"panel":["gpt-6.1-terra-max","opus-5.5-xhigh","muse-wonder-full-xhigh","deepseek-v4.1-flash-max","gpt-6.1-sol-max"]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":21.75,"tokens":{"uncached_input":28,"cache_read":1821247,"cache_write":180483,"output":135310,"reasoning":107875},"cost_usd":4.5144,"cost_breakdown":{"input":0.000112,"cache_read":0.364249,"cache_write":1.443864,"output":2.7062},"cost_usd_as_run":4.5144,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":630,"code":468,"comment":118,"blank":44,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":748,"code":646,"comment":52,"blank":50,"path":"tests/selftest.c","role":"test"}],"impl_lines":630,"impl_code":468,"impl_comment":118,"test_lines":748,"comment_ratio":0.201,"tests_declared":7,"object":{"text_bytes":3371,"rodata_bytes":960}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":277.06,"reference":3008.95,"speedup":10.86,"baselines":{"dav1d AVX2":135.13,"dav1d AVX-512ICL":51.61}},{"id":"find_dir","candidate":33.05,"reference":367.64,"speedup":11.124,"baselines":{"dav1d AVX2":38.3}}]},"scores":{"gpt-6.1-terra-max":10.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Fast, correct implementation with extensive differential and guarded tests","deepseek-v4.1-flash-max":"fastest by a clear margin with elegant, register-only find_dir; only micro-nits","muse-wonder-full-xhigh":"fastest find_dir (35 ticks), tied-fastest filter; intricate but well-documented","opus-5.5-xhigh":"best code: branchless masked padding, vector argmax, fastest direction search, thorough tests","gpt-6.1-sol-max":"Fast direction search and extensive differential coverage"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no ABI test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"loop-invariant dispatch compare inside the row loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"pri+sec loop reloads tap offsets per row","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No calling-convention test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Vector cleanup is not tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Guard coverage is overstated","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0598,"lines":28,"significant":468,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":13,"flags":1,"audited":"2026-11-04","reviewed":true},"score":9.2,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.5}},"defect_points":0.9,"score_cross_vendor":9.25,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.59,"tokens":{"uncached_input":40928,"cache_read":637696,"cache_write":0,"output":25890,"reasoning":14141},"cost_usd":0.2023,"cost_breakdown":{"input":0.040928,"cache_read":0.031885,"cache_write":0.0,"output":0.12945},"cost_usd_as_run":0.2023,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":398,"code":338,"comment":36,"blank":24,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":636,"code":564,"comment":29,"blank":43,"path":"tests/selftest.c","role":"test"}],"impl_lines":398,"impl_code":338,"impl_comment":36,"test_lines":636,"comment_ratio":0.096,"tests_declared":6,"object":{"text_bytes":4281,"rodata_bytes":232}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":279.04,"reference":3086.72,"speedup":11.062,"baselines":{"dav1d AVX2":137.94,"dav1d AVX-512ICL":52.78}},{"id":"find_dir","candidate":78.74,"reference":370.46,"speedup":4.705,"baselines":{"dav1d AVX2":38.84}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation with avoidable direction-search traffic","deepseek-v4.1-flash-max":"smallest correct implementation; one undocumented reliance in the output pack","muse-wonder-full-xhigh":"correct with a thorough guard test; one undocumented semantic deviation","opus-5.5-xhigh":"clean and correct; guard tests only at small strides","gpt-6.1-sol-max":"Correct, maintainable implementation with useful structured tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction search stages every row on the stack","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"saturating output pack where the reference truncates","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"upper-half poison only, no callee-saved check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"saturating pack not justified","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard-page tests only at strides 16 and 17","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Returned ABI state is unchecked","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":338,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.5,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.1,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":12.34,"tokens":{"uncached_input":19374,"cache_read":345728,"cache_write":0,"output":20632,"reasoning":9791},"cost_usd":1.5711,"cost_breakdown":{"input":0.19374,"cache_read":0.345728,"cache_write":0.0,"output":1.0316},"cost_usd_as_run":1.5711,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":351,"code":306,"comment":30,"blank":15,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":608,"code":539,"comment":30,"blank":39,"path":"tests/selftest.c","role":"test"}],"impl_lines":351,"impl_code":306,"impl_comment":30,"test_lines":608,"comment_ratio":0.089,"tests_declared":5,"object":{"text_bytes":4153,"rodata_bytes":224}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":281.43,"reference":3163.23,"speedup":11.24,"baselines":{"dav1d AVX2":141.35,"dav1d AVX-512ICL":54.09}},{"id":"find_dir","candidate":78.53,"reference":380.24,"speedup":4.842,"baselines":{"dav1d AVX2":39.93}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, well-tested implementation with avoidable direction-search traffic","deepseek-v4.1-flash-max":"compact, correct, good tests; only micro-issues","muse-wonder-full-xhigh":"compact and correct; mid-pack find_dir (80 ticks)","opus-5.5-xhigh":"compact, fast filter, good tests; filter barely documented","gpt-6.1-sol-max":"Compact, correct implementation with strong boundary tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction search stages every row on the stack","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"memory-based row counter","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"upper-half poison only, no callee-saved check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"row counter in memory","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Filter has no header or frame map","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Last `FILTER_ROWS` expansion jumps to the next instruction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI wrapper checks arguments but not return state","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0392,"lines":12,"significant":306,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.6,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.0,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":58.19,"tokens":{"uncached_input":151671,"cache_read":2375328,"cache_write":0,"output":46467,"reasoning":28264},"cost_usd":2.0111,"cost_breakdown":{"input":0.606684,"cache_read":0.475066,"cache_write":0.0,"output":0.92934},"cost_usd_as_run":2.0111,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":595,"code":441,"comment":112,"blank":42,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":529,"code":465,"comment":28,"blank":36,"path":"tests/selftest.c","role":"test"}],"impl_lines":595,"impl_code":441,"impl_comment":112,"test_lines":529,"comment_ratio":0.203,"tests_declared":5,"object":{"text_bytes":4754,"rodata_bytes":448}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":298.86,"reference":3177.55,"speedup":10.632,"baselines":{"dav1d AVX2":142.02,"dav1d AVX-512ICL":54.42}},{"id":"find_dir","candidate":78.27,"reference":381.75,"speedup":4.877,"baselines":{"dav1d AVX2":40.1}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation, but memory-read restrictions are untested","deepseek-v4.1-flash-max":"clean, well-documented, mid-pack speed; test suite blind to the memory contract","muse-wonder-full-xhigh":"clean, well-documented code; own tests never exercise a page boundary","opus-5.5-xhigh":"correct and readable; no over-read tests, dead frame stores","gpt-6.1-sol-max":"Correct implementation with incomplete memory tests and removable overhead"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"No guard-page or unavailable-neighbour checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"no guard-page test at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"no guard-page test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page or calling-convention tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Frame stores that are never read","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"One multiply per tap","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Read footprint is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Data-dependent argmax loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Saved arguments are never reloaded","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":27,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.6,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0}},"defect_points":3.1,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":51.14,"tokens":{"uncached_input":128764,"cache_read":2290848,"cache_write":0,"output":44563,"reasoning":25606},"cost_usd":1.8645,"cost_breakdown":{"input":0.515056,"cache_read":0.45817,"cache_write":0.0,"output":0.89126},"cost_usd_as_run":1.8645,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":578,"code":456,"comment":91,"blank":31,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":498,"code":437,"comment":26,"blank":35,"path":"tests/selftest.c","role":"test"}],"impl_lines":578,"impl_code":456,"impl_comment":91,"test_lines":498,"comment_ratio":0.166,"tests_declared":5,"object":{"text_bytes":4717,"rodata_bytes":704}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":336.74,"reference":3171.25,"speedup":9.418,"baselines":{"dav1d AVX2":141.93,"dav1d AVX-512ICL":54.38}},{"id":"find_dir","candidate":75.04,"reference":381.0,"speedup":5.077,"baselines":{"dav1d AVX2":40.06}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation, but its tests do not exercise forbidden reads","deepseek-v4.1-flash-max":"correct and readable, but the test suite cannot see the memory contract","muse-wonder-full-xhigh":"correct; no guard tests and the most memory-heavy find_dir shifts","opus-5.5-xhigh":"correct, but no over-read tests, a garbled comment and a slow filter","gpt-6.1-sol-max":"Correct kernels with weaker safety tests and unnecessary vector work"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"No guard-page or unavailable-neighbour checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"no guard-page test for any region","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no ABI test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"per-tap offset reload","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"garbled macro comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"no guard-page test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"lane shifts via memory index vectors","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"deliberation left in comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page or calling-convention tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Self-correction left in a comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Slow filter inner loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Forbidden reads escape the suite","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Longer sign and packing sequences","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":26,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.3,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":3.6,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.06,"tokens":{"uncached_input":44415,"cache_read":699776,"cache_write":0,"output":23434,"reasoning":12658},"cost_usd":0.3931,"cost_breakdown":{"input":0.08883,"cache_read":0.069978,"cache_write":0.0,"output":0.23434},"cost_usd_as_run":0.3931,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":381,"code":321,"comment":44,"blank":16,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":620,"code":545,"comment":36,"blank":39,"path":"tests/selftest.c","role":"test"}],"impl_lines":381,"impl_code":321,"impl_comment":44,"test_lines":620,"comment_ratio":0.121,"tests_declared":5,"object":{"text_bytes":4376,"rodata_bytes":320}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":270.01,"reference":3003.2,"speedup":11.123,"baselines":{"dav1d AVX2":134.71,"dav1d AVX-512ICL":51.86}},{"id":"find_dir","candidate":91.58,"reference":362.64,"speedup":3.96,"baselines":{"dav1d AVX2":37.92}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation with unused hot-path stores","deepseek-v4.1-flash-max":"excellent tests and compact code, but find_dir writes 416 bytes nobody reads","muse-wonder-full-xhigh":"correct and well-tested, but find_dir carries dead stack traffic","opus-5.5-xhigh":"correct, fast filter, good tests; dead stores in the direction search","gpt-6.1-sol-max":"Correct, readable code with dead stack materialization"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Direction-search partial arrays are stored but never read","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"dead-code","title":"seven partial-sum arrays are materialised and never used","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"the comment above the dead stores","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"memory-based row counter","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"upper-half poison only, no callee-saved check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"partial-sum arrays stored but never read","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"dispatch routed through a k register","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Seven partial-sum stores that are never read","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Dispatch flags kept in k3, and `lea rax,[rsp+52]` written three times","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Unused partial-array stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Return-state ABI requirements lack assertions","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":321,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.0,"slop":{"score":1.4,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.5,"deepseek-v4.1-flash-max":2.5,"gpt-6.1-sol-max":1.0}},"defect_points":1.9,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":39.21,"tokens":{"uncached_input":66,"cache_read":5566437,"cache_write":233544,"output":161619,"reasoning":118357},"cost_usd":6.2143,"cost_breakdown":{"input":0.000264,"cache_read":1.113287,"cache_write":1.868352,"output":3.23238},"cost_usd_as_run":6.2143,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":608,"code":402,"comment":165,"blank":41,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1074,"code":937,"comment":74,"blank":63,"path":"tests/selftest.c","role":"test"}],"impl_lines":608,"impl_code":402,"impl_comment":165,"test_lines":1074,"comment_ratio":0.291,"tests_declared":9,"object":{"text_bytes":3627,"rodata_bytes":388}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":270.64,"reference":3120.1,"speedup":11.529,"baselines":{"dav1d AVX2":139.79,"dav1d AVX-512ICL":53.41}},{"id":"find_dir","candidate":48.88,"reference":375.54,"speedup":7.683,"baselines":{"dav1d AVX2":39.43}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":9.5,"muse-wonder-full-xhigh":9.5,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.5},"council_summaries":{"gpt-6.1-terra-max":"Fast implementation with strong ABI tests and a small domain-test mistake","deepseek-v4.1-flash-max":"no findings; near-best speed with the strongest test suite of the ten","muse-wonder-full-xhigh":"best-tested entry; tied-fastest filter, second-fastest find_dir","opus-5.5-xhigh":"fastest-tier filter, second-fastest direction search, best documentation and most thorough tests","gpt-6.1-sol-max":"Strong implementation and the most complete ABI testing"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard suite requires behaviour at stride 8192","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Constant-block costs are not always zero","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"vpmaddwd exactness premise wrong for negatives","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Imprecise statement of a key invariant","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Constant blocks do not generally have zero costs","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":402,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":31,"flags":0,"audited":"2026-11-04","reviewed":false},"score":9.3,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.5}},"defect_points":0.6,"score_cross_vendor":9.375,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":70.69,"tokens":{"uncached_input":41920,"cache_read":797696,"cache_write":0,"output":32581,"reasoning":13701},"cost_usd":0.6782,"cost_breakdown":{"input":0.08384,"cache_read":0.398848,"cache_write":0.0,"output":0.195486},"cost_usd_as_run":0.6782,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"lines":637,"code":494,"comment":97,"blank":46,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":886,"code":784,"comment":51,"blank":51,"path":"tests/selftest.c","role":"test"}],"impl_lines":637,"impl_code":494,"impl_comment":97,"test_lines":886,"comment_ratio":0.164,"tests_declared":6,"object":{"text_bytes":6172,"rodata_bytes":316}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":320.27,"reference":3182.84,"speedup":9.938,"baselines":{"dav1d AVX2":142.44,"dav1d AVX-512ICL":54.51}},{"id":"find_dir","candidate":75.12,"reference":383.26,"speedup":5.102,"baselines":{"dav1d AVX2":40.19}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct kernel, but its guard test requires an out-of-domain stride","deepseek-v4.1-flash-max":"clean, correct, well tested; pays an avoidable per-tap cost and has one misplaced doc block","muse-wonder-full-xhigh":"correct and well-tested, but unhoisted inner loop and leftover docs","opus-5.5-xhigh":"correct; slower filter, stranded comments, guard tests only out of domain","gpt-6.1-sol-max":"Correct kernels with substantial avoidable work and missing supported-stride guards"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guard tests use only a stride outside the contract","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Unused constant","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"per-tap address rebuild","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no ABI test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"orphaned macro documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"unreferenced constant `c255`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"orphaned duplicate macro doc blocks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"per-tap/per-row recomputation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Macro comments detached from their macros, one of them wrong","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard-page tests run only at stride 8192, and there is no calling-convention test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`CONSTRAIN` re-zeroes ymm13 on every use","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Test conditional with identical branches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Manual sign restoration and packing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Branchy direction selection","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"No guard calls use a supported stride","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unused constant","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":26,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.4,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":2.0,"muse-wonder-full-xhigh":2.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.5}},"defect_points":3.5,"score_cross_vendor":7.4,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":70.44,"tokens":{"uncached_input":924,"cache_read":54023618,"cache_write":501066,"output":412490,"reasoning":0},"cost_usd":1.1586,"cost_breakdown":{"input":0.000139,"cache_read":0.864378,"cache_write":0.100213,"output":0.19387},"cost_usd_as_run":1.1586,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":1.1586,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":598,"code":444,"comment":139,"blank":15,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":620,"code":524,"comment":49,"blank":47,"path":"tests/selftest.c","role":"test"}],"impl_lines":598,"impl_code":444,"impl_comment":139,"test_lines":620,"comment_ratio":0.238,"tests_declared":6,"object":{"text_bytes":4550,"rodata_bytes":1088}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":367.39,"reference":3192.2,"speedup":8.689,"baselines":{"dav1d AVX2":143.07,"dav1d AVX-512ICL":54.67}},{"id":"find_dir","candidate":195.98,"reference":384.23,"speedup":1.961,"baselines":{"dav1d AVX2":40.44}}]},"scores":{"gpt-6.1-terra-max":6.5,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct kernel, but its tests and direction-search performance need work","deepseek-v4.1-flash-max":"correct and very well tested, but the slowest implementation on both entry points by a wide margin","muse-wonder-full-xhigh":"correct but slowest on both functions; find_dir only 1.95x vs C reference","opus-5.5-xhigh":"correct but slowest in both functions; garbled comments and overstated tests","gpt-6.1-sol-max":"Correct assembly, but defective test construction and slower direction search"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search has a long serial row loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Border patterns use undefined pointer subtraction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Left-edge guard placement leaves two readable bytes past the allowed row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Left-border order is reversed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"find_dir does far more work per block than every peer","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"12 taps per row each reload their offset and rebuild the address","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"write-mask on 16-byte tap loads does nothing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no ABI test at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"self-correcting comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"find_dir serial window chains","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"filter arithmetic in half-empty zmm","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"thinnest directed find_dir coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"unreferenced constant `fd_105`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"deliberation left in comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Direction-search row loads are merge-masked","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Slowest filter","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Garbled and wrong comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard test claims more than it checks, and comments contradict the contract","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Undefined pointer subtraction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard origins leave two readable excess bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Branchy argmax","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unused constant","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":153,"flags":35,"audited":"2026-11-04","reviewed":true},"score":6.9,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":0.5}},"defect_points":5.6,"score_cross_vendor":6.9,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":27.89,"tokens":{"uncached_input":60,"cache_read":5721562,"cache_write":296845,"output":375874,"reasoning":346755},"cost_usd":0.9599,"cost_breakdown":{"input":2.4e-05,"cache_read":0.277732,"cache_write":0.258057,"output":0.424128},"cost_usd_as_run":0.9599,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":434,"code":362,"comment":44,"blank":28,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":608,"code":537,"comment":31,"blank":40,"path":"tests/selftest.c","role":"test"}],"impl_lines":434,"impl_code":362,"impl_comment":44,"test_lines":608,"comment_ratio":0.108,"tests_declared":6,"object":{"text_bytes":3646,"rodata_bytes":344}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":278.49,"reference":3008.53,"speedup":10.803,"baselines":{"dav1d AVX2":134.92,"dav1d AVX-512ICL":51.4}},{"id":"find_dir","candidate":62.9,"reference":362.6,"speedup":5.765,"baselines":{"dav1d AVX2":38.09}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation with an important guard-test gap","deepseek-v4.1-flash-max":"fastest find_dir short of J/R and very compact; small dead work in the tap macro","muse-wonder-full-xhigh":"short and fast; single-loop dispatch wastes work in two of three modes","opus-5.5-xhigh":"correct, clear and fast enough; tests skip the calling-convention checks","gpt-6.1-sol-max":"Correct kernels with avoidable filter work and a guard-test gap"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Disabled neighbour pointers remain readable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"min/max tracked in dispatches that never clamp","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no ABI test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"loop-invariant dispatch branch inside the row loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"min/max computed when clamp is impossible","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No calling-convention test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Filter reloads the dispatch mode every row and tracks min/max in the primary-only mode","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Disabled neighbors remain readable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Unused extrema in strength-only paths","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":362,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":36,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.1,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.0,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":18.28,"tokens":{"uncached_input":52862,"cache_read":14874240,"cache_write":0,"output":188494,"reasoning":140016},"cost_usd":0.1656,"cost_breakdown":{"input":0.007929,"cache_read":0.044623,"cache_write":0.0,"output":0.113096},"cost_usd_as_run":0.1656,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1656,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":705,"code":529,"comment":117,"blank":59,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":634,"code":557,"comment":33,"blank":44,"path":"tests/selftest.c","role":"test"}],"impl_lines":705,"impl_code":529,"impl_comment":117,"test_lines":634,"comment_ratio":0.181,"tests_declared":8,"object":{"text_bytes":4791,"rodata_bytes":320}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":292.48,"reference":2975.36,"speedup":10.173,"baselines":{"dav1d AVX2":133.03,"dav1d AVX-512ICL":50.98}},{"id":"find_dir","candidate":90.58,"reference":357.73,"speedup":3.949,"baselines":{"dav1d AVX2":37.37}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct and fast, with a gap in its guard tests","gpt-6.1-sol-max":"Correct and well explained, with duplicated filter logic and incomplete guard tests.","deepseek-v4.1-flash-max":"textbook documentation and the strongest test suite in the set, with correct, safe, fast code","muse-wonder-full-xhigh":"Well documented and correct, with triplicated filter bodies","opus-5.5-xhigh":"excellent docs and a clean register-resident direction search; one hole in the guard tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Left-buffer overreads are not guarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Guard coverage is overstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Filter paths repeat substantial arithmetic","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Absent neighbours remain readable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"Test comment overstates what guard pages prove","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Three hand-duplicated filter loops","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard layouts leave absent neighbours readable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"In-place pixel read is unexplained","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard-page filter test misses absent and forbidden reads","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Left pixels take a needless stack round trip","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.172,"lines":91,"significant":529,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":86,"flags":19,"audited":"2026-11-04","reviewed":true},"score":8.5,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":1.0}},"defect_points":1.6,"score_cross_vendor":8.375,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.65,"tokens":{"uncached_input":19190,"cache_read":151296,"cache_write":0,"output":4026,"reasoning":70},"cost_usd":0.5445,"cost_breakdown":{"input":0.1919,"cache_read":0.151296,"cache_write":0.0,"output":0.2013},"cost_usd_as_run":0.5445,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":983,"code":976,"comment":7,"blank":0,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":399,"code":348,"comment":21,"blank":30,"path":"tests/selftest.c","role":"test"}],"impl_lines":983,"impl_code":976,"impl_comment":7,"test_lines":399,"comment_ratio":0.007,"tests_declared":4,"object":{"text_bytes":6889,"rodata_bytes":576}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":277.27,"reference":3116.95,"speedup":11.242,"baselines":{"dav1d AVX2":139.54,"dav1d AVX-512ICL":53.41}},{"id":"find_dir","candidate":288.78,"reference":375.07,"speedup":1.299,"baselines":{"dav1d AVX2":39.3}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":5.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, but scalar stack updates and expanded code are costly","gpt-6.1-sol-max":"Correct, but scalar accumulation and extensive copied assembly weaken performance and maintenance.","deepseek-v4.1-flash-max":"correct and its filter is fast, but find_dir is 739 lines of fully unrolled scalar code that buys only 1.3×","muse-wonder-full-xhigh":"Correct, but the direction search is a slow 576-line scalar unrolling","opus-5.5-xhigh":"correct, but ~580 lines of unrolled scalar direction search, dead code and no docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search performs 512 scalar memory updates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"verbosity","title":"Hundreds of manually expanded pixel updates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No guarded reads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Unused range initialization in single-strength paths","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction accumulation performs 512 scalar memory updates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Cost reductions are copied eight times","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Memory restrictions lack regression coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"verbosity","title":"find_dir is 739 lines of unrolled scalar accumulation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Eight weight vectors that are only three distinct vectors","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"find_dir is 294 ticks (1.29×)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"find_dir has no commentary","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No over-read test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Scalar find_dir accumulation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Hard-coded unrolled body","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Dead range initialisation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard test and correlated structured parameters","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Stale loop-register comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Scalar direction search","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"verbosity","title":"~580 lines of generated straight-line code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"\"eax pixel, ecx x, r8d y\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Dead and no-op instructions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header and no frame or register documentation for the filter","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0758,"lines":74,"significant":976,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.2,"slop":{"score":2.4,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":4.0,"muse-wonder-full-xhigh":2.5,"deepseek-v4.1-flash-max":3.0,"gpt-6.1-sol-max":1.0}},"defect_points":5.7,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.34,"tokens":{"uncached_input":14624,"cache_read":73856,"cache_write":0,"output":347,"reasoning":100},"cost_usd":0.0201,"cost_breakdown":{"input":0.014624,"cache_read":0.003693,"cache_write":0.0,"output":0.001735},"cost_usd_as_run":0.0201,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":25,"code":10,"comment":10,"blank":5,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":25,"impl_code":10,"impl_comment":10,"test_lines":348,"comment_ratio":0.5,"tests_declared":3,"object":{"text_bytes":4,"rodata_bytes":0}},"hidden":{"passed":1,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3158.23,"speedup":null,"baselines":{"dav1d AVX2":141.6,"dav1d AVX-512ICL":54.0}},{"id":"find_dir","candidate":null,"reference":381.04,"speedup":null,"baselines":{"dav1d AVX2":39.93}}]},"scores":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.0},"council_summaries":{"gpt-6.1-terra-max":"Both exported functions remain stubs","gpt-6.1-sol-max":"Both required algorithms are unimplemented.","deepseek-v4.1-flash-max":"the untouched scaffold stub: neither symbol does anything","muse-wonder-full-xhigh":"Unmodified stub; neither function is implemented","opus-5.5-xhigh":"unchanged stub; nothing implemented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Direction search does not produce variance","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter never modifies the block","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Direction search returns a constant and leaves variance unwritten","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filtering performs no work","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"Both entry points are empty stubs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Direction search is a stub","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter is a stub","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Both entry points are the scaffold stub","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"tests/selftest.c is the unmodified scaffold","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":3,"flags":0,"audited":"2026-11-04","reviewed":false},"score":0.1,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":5.8,"score_cross_vendor":0.167,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.48,"tokens":{"uncached_input":19710,"cache_read":197504,"cache_write":0,"output":8349,"reasoning":2185},"cost_usd":0.1427,"cost_breakdown":{"input":0.03942,"cache_read":0.01975,"cache_write":0.0,"output":0.08349},"cost_usd_as_run":0.1427,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":399,"code":368,"comment":23,"blank":8,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":438,"code":383,"comment":23,"blank":32,"path":"tests/selftest.c","role":"test"}],"impl_lines":399,"impl_code":368,"impl_comment":23,"test_lines":438,"comment_ratio":0.059,"tests_declared":4,"object":{"text_bytes":5246,"rodata_bytes":3584}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":304.32,"reference":3202.66,"speedup":10.524,"baselines":{"dav1d AVX2":143.61,"dav1d AVX-512ICL":54.88}},{"id":"find_dir","candidate":122.41,"reference":390.49,"speedup":3.19,"baselines":{"dav1d AVX2":40.48}}]},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, with unnecessary staging and no guard tests","gpt-6.1-sol-max":"Correct and reasonably efficient, with avoidable staging and missing memory-access tests.","deepseek-v4.1-flash-max":"correct, quick and well tested; a no-op instruction and very terse table docs","muse-wonder-full-xhigh":"Fast and correct, but its generated permutation tables are difficult to audit","opus-5.5-xhigh":"correct with good speed, but terse docs, small redundancies and no guard tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Direction rows are stored and immediately reloaded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No protected-boundary tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Repeated bit operation in weight setup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Each centered row is immediately stored and reloaded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Structured tests do not enforce read boundaries","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Redundant `or edx, 2`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Direction tables are opaque","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Redundant find_dir staging","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Opaque permutation and mask tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page test and narrow structured parameters","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Structured patterns only, 3 strength pairs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"\"px-128 in the required int16 tmp\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Terse docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Small redundancies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":368,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.0,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":2.7,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":144.26,"tokens":{"uncached_input":538821,"cache_read":35771840,"cache_write":0,"output":253384,"reasoning":194744},"cost_usd":1.2807,"cost_breakdown":{"input":0.080823,"cache_read":1.073155,"cache_write":0.0,"output":0.126692},"cost_usd_as_run":1.2807,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":795,"code":617,"comment":138,"blank":40,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":516,"code":448,"comment":31,"blank":37,"path":"tests/selftest.c","role":"test"}],"impl_lines":795,"impl_code":617,"impl_comment":138,"test_lines":516,"comment_ratio":0.183,"tests_declared":5,"object":{"text_bytes":15364,"rodata_bytes":1736}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":301.71,"reference":3081.55,"speedup":10.214,"baselines":{"dav1d AVX2":138.07,"dav1d AVX-512ICL":52.62}},{"id":"find_dir","candidate":103.13,"reference":370.62,"speedup":3.594,"baselines":{"dav1d AVX2":38.88}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":160,"flags":0,"audited":"2026-11-04","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but the hot path repeats work and read-boundary tests are missing","opus-5.5-xhigh":"correct and well commented, but there are no memory-contract tests, a duplicated epilogue, and legacy SSE mixed into AVX-512 code","deepseek-v4.1-flash-max":"correct and well-commented, but its hottest row stores twice and recomputes the rounding, and its tests never exercise the read contract.","gpt-6.1-sol-max":"Correct implementation with redundant filter work and incomplete memory-safety tests.","muse-wonder-full-xhigh":"Bit-exact and robust, but main arm does redundant work and memory contract is untested"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"redundancy","title":"Both-strength rows round and store twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"No read-boundary checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Structured patterns are progressively filtered","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Accumulator lane invariant is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"There are no guard-page or absent-neighbour tests at all","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The primary+secondary row stores twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Legacy-SSE `movq xmm, m64` appears seven times","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Both functions do needless scalar or memory work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The comment `push rax ; save edges, rsp now 16-aligned` is misleading","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"redundancy","title":"pri+sec row finishes twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No guard-page coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`find_dir` re-reads its constant selectors every row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"No protected-memory cases","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Combined filtering finishes and stores every row twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Secondary taps multiply by one","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Incorrect upper-lane invariant","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"redundancy","title":"Duplicated row epilogue in pri+sec arm","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"No guard-page test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar padding, 15,364-byte `.text`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Dead `SEC_S` store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Nonsense register comment","status":"valid","detail":"","location":""}],"score":7.2,"slop":{"score":2.5,"per_seat":{"gpt-6.1-terra-max":3.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":3.0,"deepseek-v4.1-flash-max":2.0,"gpt-6.1-sol-max":3.0}},"defect_points":5.2,"score_cross_vendor":7.2,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.35,"tokens":{"uncached_input":25225,"cache_read":288768,"cache_write":0,"output":23706,"reasoning":13738},"cost_usd":0.3164,"cost_breakdown":{"input":0.05045,"cache_read":0.028877,"cache_write":0.0,"output":0.23706},"cost_usd_as_run":0.3164,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":427,"code":359,"comment":44,"blank":24,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":577,"code":510,"comment":29,"blank":38,"path":"tests/selftest.c","role":"test"}],"impl_lines":427,"impl_code":359,"impl_comment":44,"test_lines":577,"comment_ratio":0.109,"tests_declared":5,"object":{"text_bytes":4317,"rodata_bytes":1728}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":279.04,"reference":3101.88,"speedup":11.116,"baselines":{"dav1d AVX2":139.25,"dav1d AVX-512ICL":53.46}},{"id":"find_dir","candidate":104.81,"reference":381.33,"speedup":3.638,"baselines":{"dav1d AVX2":39.35}}]},"scores":{"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct with broad boundary coverage and minor redundant staging","gpt-6.1-sol-max":"Correct, maintainable and well tested, with a small unnecessary memory round trip.","deepseek-v4.1-flash-max":"fast, safe and cleanly documented; one dead store","muse-wonder-full-xhigh":"Compact, correct and fast, with strong guard tests and one redundant staging step","opus-5.5-xhigh":"clean and fast masked-load staging, good docs and guard-page tests; small slop only"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Direction rows are stored only to be immediately reloaded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Direction staging is immediately consumed and discarded","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Dead int16 staging store in find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Redundant find_dir staging","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The \"required int16 px-128 staging\" isn't required","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Small no-ops","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":359,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.8,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.3,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.33,"tokens":{"uncached_input":19156,"cache_read":440704,"cache_write":0,"output":11652,"reasoning":2163},"cost_usd":0.0995,"cost_breakdown":{"input":0.019156,"cache_read":0.022035,"cache_write":0.0,"output":0.05826},"cost_usd_as_run":0.0995,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":491,"code":455,"comment":30,"blank":6,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":450,"code":395,"comment":24,"blank":31,"path":"tests/selftest.c","role":"test"}],"impl_lines":491,"impl_code":455,"impl_comment":30,"test_lines":450,"comment_ratio":0.062,"tests_declared":5,"object":{"text_bytes":3217,"rodata_bytes":3384}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":285.59,"reference":3056.91,"speedup":10.704,"baselines":{"dav1d AVX2":136.81,"dav1d AVX-512ICL":52.45}},{"id":"find_dir","candidate":131.73,"reference":368.48,"speedup":2.797,"baselines":{"dav1d AVX2":38.51}}]},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct with useful guard tests, but unnecessary direction-search work","gpt-6.1-sol-max":"Correct, with dead direction-search stores and one-sided guard coverage.","deepseek-v4.1-flash-max":"correct and solid, but 128 bytes of dead staging per call and a 3 KB table with no explanation","muse-wonder-full-xhigh":"Correct and reasonably fast, with dead find_dir stores and a narrow guard test","opus-5.5-xhigh":"correct, middling speed, several stale or dead pieces and weak guard tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction sums are spilled and reloaded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Write-only staging and initialization","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Incorrect register plan","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Direction scratch is written without being consumed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guard tests miss under-reads","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"Dead int16 staging buffer","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`line_indices` (48 × 64 bytes = 3 KB) is unexplained","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Dead find_dir stack stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Trailing-only guard test without output checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Bottom staging depends on a leftover pointer","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Weight selected by index parity","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Dead store","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Stale register comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Per-row overhead","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard test is shallow","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Needless zeroing","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1143,"lines":52,"significant":455,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.7,"slop":{"score":1.4,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":2.5,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0}},"defect_points":2.9,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.08,"tokens":{"uncached_input":19262,"cache_read":155520,"cache_write":0,"output":5648,"reasoning":643},"cost_usd":0.1106,"cost_breakdown":{"input":0.038524,"cache_read":0.015552,"cache_write":0.0,"output":0.05648},"cost_usd_as_run":0.1106,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":1006,"code":984,"comment":21,"blank":1,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":416,"code":365,"comment":21,"blank":30,"path":"tests/selftest.c","role":"test"}],"impl_lines":1006,"impl_code":984,"impl_comment":21,"test_lines":416,"comment_ratio":0.021,"tests_declared":4,"object":{"text_bytes":5991,"rodata_bytes":5184}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":382.09,"reference":3015.63,"speedup":7.892,"baselines":{"dav1d AVX2":135.05,"dav1d AVX-512ICL":51.65}},{"id":"find_dir","candidate":129.38,"reference":368.31,"speedup":2.847,"baselines":{"dav1d AVX2":38.13}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":5.5},"council_summaries":{"gpt-6.1-terra-max":"Correct, but repeated hot-path work and expanded source hinder maintenance","gpt-6.1-sol-max":"Correct, but repeated invariant setup and mechanically expanded code add avoidable cost and bulk.","deepseek-v4.1-flash-max":"bit-exact and safe, but the slowest filter in the field and a direction search nobody can review","muse-wonder-full-xhigh":"Correct, but the fully unrolled direction search is exceptionally verbose","opus-5.5-xhigh":"correct, but bloated generated direction search, slowest filter, no docs, thin tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Invariant tap weights are rebuilt for every row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"verbosity","title":"Hand-expanded direction rows duplicate hundreds of instructions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No guarded reads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Tap weights are reconstructed for every output row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Direction pipelines are manually expanded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Padded inputs hide illegal reads","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"Filter is the slowest of the eleven working entries","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"Direction search is eight copy-pasted 12-instruction blocks plus 72 unexplained tables","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`zmm2` zeroed and never used","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No bounds/over-read test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"verbosity","title":"Fully unrolled find_dir and tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-row and per-tap filter setup","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Flat structured neighbours and no guard test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated cost weights","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"verbosity","title":"Fully unrolled direction search","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Loop-invariant work in the filter's row loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Structured patterns only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header and only a few comments","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1402,"lines":138,"significant":984,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.9,"slop":{"score":1.6,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":2.0,"muse-wonder-full-xhigh":2.5,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":1.0}},"defect_points":4.3,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":37.08,"tokens":{"uncached_input":85937,"cache_read":1796724,"cache_write":0,"output":29402,"reasoning":13856},"cost_usd":1.2911,"cost_breakdown":{"input":0.343748,"cache_read":0.359345,"cache_write":0.0,"output":0.58804},"cost_usd_as_run":1.2911,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":609,"code":504,"comment":73,"blank":32,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":474,"code":414,"comment":25,"blank":35,"path":"tests/selftest.c","role":"test"}],"impl_lines":609,"impl_code":504,"impl_comment":73,"test_lines":474,"comment_ratio":0.127,"tests_declared":4,"object":{"text_bytes":3897,"rodata_bytes":160}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":338.04,"reference":3176.0,"speedup":9.395,"baselines":{"dav1d AVX2":142.82,"dav1d AVX-512ICL":54.51}},{"id":"find_dir","candidate":264.26,"reference":383.26,"speedup":1.45,"baselines":{"dav1d AVX2":40.14}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":5.5},"council_summaries":{"gpt-6.1-terra-max":"Correct, with a slow scalar cost stage and limited memory tests","gpt-6.1-sol-max":"Correct and clearly documented, but the direction-cost stage remains unnecessarily scalar.","deepseek-v4.1-flash-max":"correct and well laid out, but its direction search is scalar where the field is vector, and its header cites a file that does not exist","muse-wonder-full-xhigh":"Correct and well documented, but direction search is slow and own tests lack guard pages","opus-5.5-xhigh":"correct and well laid out, but slow direction search, slowish filter, a reference to a file that doesn't exist, and no guard tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction costs are reduced with scalar loops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No protected-boundary tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Memory-contract link points to an absent file","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Weighted costs use scalar loops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Read-boundary violations remain invisible","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Header points at a nonexistent document","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Cost stage of find_dir is scalar","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`padding()` written out three times","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No over-read test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Scalar find_dir cost stage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Direction-search partial sums go through memory","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Structured patterns only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Filter overhead","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"\"See the brief for the memory contract\"","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.9,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":0.0}},"defect_points":3.5,"score_cross_vendor":6.75,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.9,"tokens":{"uncached_input":15886,"cache_read":156800,"cache_write":0,"output":6770,"reasoning":1534},"cost_usd":0.6542,"cost_breakdown":{"input":0.15886,"cache_read":0.1568,"cache_write":0.0,"output":0.3385},"cost_usd_as_run":0.6542,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":300,"code":274,"comment":20,"blank":6,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":429,"code":375,"comment":24,"blank":30,"path":"tests/selftest.c","role":"test"}],"impl_lines":300,"impl_code":274,"impl_comment":20,"test_lines":429,"comment_ratio":0.068,"tests_declared":4,"object":{"text_bytes":3042,"rodata_bytes":254}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":292.77,"reference":3039.06,"speedup":10.38,"baselines":{"dav1d AVX2":136.43,"dav1d AVX-512ICL":52.2}},{"id":"find_dir","candidate":178.59,"reference":372.72,"speedup":2.087,"baselines":{"dav1d AVX2":38.59}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":6.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, though direction accumulation and memory tests need work","gpt-6.1-sol-max":"Correct and compact, with avoidable direction-search memory traffic and a test gap.","deepseek-v4.1-flash-max":"correct and tidy, but the direction search keeps every accumulator in memory and its tests never check bounds","muse-wonder-full-xhigh":"Correct and concise, but terse, with a memory-bound direction search and thin structured tests","opus-5.5-xhigh":"compact and correct, but the direction search goes through memory, vpmulld by 1 and 2, sparse docs and tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Partial sums repeatedly round-trip through the stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No protected-boundary tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Partial sums repeatedly round-trip through memory","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Forbidden reads escape the tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Partial sums live in memory and are load-modify-stored every row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The added suite never checks out-of-bounds access","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Documentation is thin","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Stack-resident find_dir partials","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Flat structured neighbours and no guard test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Unexplained tap indices","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Direction-search accumulation through memory","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Structured patterns only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Every tap goes through vpmulld","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"A one-line header, and no frame or register map for find_dir beyond a sentence","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Confusing comment","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":274,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.3,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":3.4,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.29,"tokens":{"uncached_input":18635,"cache_read":250624,"cache_write":0,"output":5753,"reasoning":364},"cost_usd":0.0599,"cost_breakdown":{"input":0.018635,"cache_read":0.012531,"cache_write":0.0,"output":0.028765},"cost_usd_as_run":0.0599,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":340,"code":328,"comment":10,"blank":2,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":393,"code":344,"comment":19,"blank":30,"path":"tests/selftest.c","role":"test"}],"impl_lines":340,"impl_code":328,"impl_comment":10,"test_lines":393,"comment_ratio":0.03,"tests_declared":4,"object":{"text_bytes":1669,"rodata_bytes":536}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":316.5,"reference":3038.89,"speedup":9.602,"baselines":{"dav1d AVX2":135.97,"dav1d AVX-512ICL":52.24}},{"id":"find_dir","candidate":432.14,"reference":365.84,"speedup":0.847,"baselines":{"dav1d AVX2":38.51}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.0,"deepseek-v4.1-flash-max":6.0,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":4.5},"council_summaries":{"gpt-6.1-terra-max":"Correct output, but direction search is slower than the reference","gpt-6.1-sol-max":"Correct, but direction search retains substantial scalar work and tests need strengthening.","deepseek-v4.1-flash-max":"correct and compact, but the direction search is slower than the C reference and the filter does clamp work it then discards","muse-wonder-full-xhigh":"Correct filter, but the scalar direction search is slower than the C reference","opus-5.5-xhigh":"correct and safe, but its direction search is slower than the C reference and the filter loop is spilled to memory"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Scalar per-pixel direction accumulation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Filter mode decisions remain inside the tap loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No guarded reads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Partial-sum construction is scalar","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Filter invariants are rebuilt inside the tap loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Pattern cases consume previously filtered input","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Padded fixtures conceal forbidden reads","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"find_dir is slower than the plain C reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Filter clamps with results that are thrown away","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"Comment claims work is done once that is repeated per row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"find_dir's eight accumulators are bare offsets with no family map","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The only added test is thin","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`TAP` has an unused macro parameter","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"Scalar find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Added pattern test decays its input","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Repeated work in the unified filter loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unused macro argument","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"find_dir is a scalar per-pixel loop, slower than the C it replaces","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Filter loop work isn't hoisted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"\"Compute shifts and weights once\" is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Wasted work per tap","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header and no notes on the frame layout or register use","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":328,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-04","reviewed":false},"score":5.8,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":2.0,"muse-wonder-full-xhigh":1.5,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":0.0}},"defect_points":6.4,"score_cross_vendor":5.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.43,"tokens":{"uncached_input":16,"cache_read":598715,"cache_write":105153,"output":61182,"reasoning":40610},"cost_usd":2.1847,"cost_breakdown":{"input":6.4e-05,"cache_read":0.119743,"cache_write":0.841224,"output":1.22364},"cost_usd_as_run":2.1847,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":629,"code":495,"comment":92,"blank":42,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":595,"code":516,"comment":35,"blank":44,"path":"tests/selftest.c","role":"test"}],"impl_lines":629,"impl_code":495,"impl_comment":92,"test_lines":595,"comment_ratio":0.157,"tests_declared":6,"object":{"text_bytes":12181,"rodata_bytes":1580}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":240.36,"reference":3128.38,"speedup":13.015,"baselines":{"dav1d AVX2":139.79,"dav1d AVX-512ICL":53.46}},{"id":"find_dir","candidate":46.36,"reference":382.34,"speedup":8.247,"baselines":{"dav1d AVX2":39.56}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Fast filter and correct checks, with test and comment gaps","opus-5.5-xhigh":"Fastest filter (two rows per zmm), clean vectorised find_dir, accurate docs; but the filter is 10.9 KB of unrolled code and still uses 32-bit lanes","muse-wonder-full-xhigh":"Bit-exact with the fastest filter; slower find_dir, large unrolled code, slightly thinner tests","deepseek-v4.1-flash-max":"fastest filter (≈16.7× the C reference) and a clever, well-documented direction search, held back by two test/hygiene gaps","gpt-6.1-sol-max":"Correct and fast, with minor test and documentation gaps."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Interior-row over-reads escape the guard tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI state is not explicitly tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Slop: guard-page coverage is overstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Slop: constant-block costs are not always zero","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Filter still widens to 32 bits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fully unrolled filter bodies","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Coverage gaps in find_dir and ABI checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Absent-neighbour trap doesn't cover row 1 at large strides","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"find_dir re-permutes from tables for every row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Both strengths' shifts/weights computed unconditionally, including `lzcnt(0)`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Fully unrolled filter triples object size","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard/ABI coverage gaps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Stale self-test header","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"1.1 MB core dump shipped in the entry","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Guard-page test kills the run instead of reporting, and never guards the `*var` store","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"`src/cdef_avx512.asm:346`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Guard-page guarantee is overstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Constant-block costs are usually nonzero","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.2061,"lines":102,"significant":495,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.5,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":1.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":1.5}},"defect_points":3.1,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1-b2","task":"cdef-avx512","lead":"opus-5.5-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":8.35,"tokens":{"cache_read":5207808,"cache_write":0,"output":19225,"reasoning":10477,"uncached_input":142731},"cost_usd":0.4992,"cost_breakdown":{"cache_read":0.26039,"cache_write":0.0,"input":0.142731,"output":0.096125},"cost_usd_as_run":0.4992,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":12.48,"tokens":{"cache_read":5286584,"cache_write":0,"output":49081,"reasoning":28331,"uncached_input":255192},"cost_usd":3.0597,"cost_breakdown":{"cache_read":1.057317,"cache_write":0.0,"input":1.020768,"output":0.98162},"cost_usd_as_run":3.0597,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":12.88,"tokens":{"cache_read":3052672,"cache_write":0,"output":23065,"reasoning":15214,"uncached_input":140977},"cost_usd":0.8179,"cost_breakdown":{"cache_read":0.305267,"cache_write":0.0,"input":0.281954,"output":0.23065},"cost_usd_as_run":0.8179,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":16.03,"tokens":{"cache_read":6145868,"cache_write":270868,"output":98601,"reasoning":71344,"uncached_input":68},"cost_usd":5.3684,"cost_breakdown":{"cache_read":1.229174,"cache_write":2.166944,"input":0.000272,"output":1.97202},"cost_usd_as_run":5.3684,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":16.25,"tokens":{"cache_read":12836096,"cache_write":0,"output":163969,"reasoning":128264,"uncached_input":131725},"cost_usd":0.1566,"cost_breakdown":{"cache_read":0.038508,"cache_write":0.0,"input":0.019759,"output":0.098381},"cost_usd_as_run":0.1566,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"F","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"F","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"K","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"F","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"K","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.0}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","opus-5.5-xhigh","deepseek-v4.1-flash-max"]},"entries":[{"letter":"B","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":78.48,"tokens":{"uncached_input":290180,"cache_read":4525972,"cache_write":0,"output":225898,"reasoning":105321},"cost_usd":3.2919,"cost_breakdown":{"input":0.58036,"cache_read":0.452597,"cache_write":0.0,"output":2.25898},"cost_usd_as_run":3.2919,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"lines":1088,"code":830,"comment":156,"blank":102,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":964,"code":839,"comment":49,"blank":76,"path":"tests/selftest.c","role":"test"}],"impl_lines":1088,"impl_code":830,"impl_comment":156,"test_lines":964,"comment_ratio":0.158,"tests_declared":8,"object":{"text_bytes":3004,"rodata_bytes":8432}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":109.73,"reference":3186.24,"speedup":29.037,"baselines":{"dav1d AVX2":141.01,"dav1d AVX-512ICL":53.75}},{"id":"find_dir","candidate":32.59,"reference":383.18,"speedup":11.758,"baselines":{"dav1d AVX2":39.72}}]},"isolation":{"tool_calls":46,"flags":8,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":9.0,"muse-wonder-full-xhigh":9.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and fast, with a small ABI-test gap and minor test slop","muse-wonder-full-xhigh":"fastest entry with strong guard/ABI tests; only test-coverage nits remain","gpt-6.1-sol-max":"Efficient, correct assembly with an unsafe ABI-test wrapper.","opus-5.5-xhigh":"Fastest entry, correct, accurately documented and well tested; only minor test gaps","deepseek-v4.1-flash-max":"fastest on both functions with the best invariant documentation; the only real gap is that its own suite never checks the upper-state rule."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper vector state is not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Tie patterns run twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Direction-pattern comment claims a stripe test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Extremes patterns decay for checkerboards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Guard test hardcodes the page size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Inline assembly hides vector clobbers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The ABI test misses vzeroupper and is fragile","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The extremes test restores only 3 of its 8 patterns","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Rows 4..7 are biased twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No upper-state/vzeroupper check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"6 KiB of precomputed mask data","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"The diag gather block is written out twice","status":"valid","detail":"","location":""}],"score":9.1,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":1.0}},"defect_points":2.0,"score_cross_vendor":9.1,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","task":"cdef-avx512","lead":"opus-5.5-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":33.83,"tokens":{"cache_read":23754752,"cache_write":0,"output":83082,"reasoning":43084,"uncached_input":598597},"cost_usd":2.2018,"cost_breakdown":{"cache_read":1.187738,"cache_write":0.0,"input":0.598597,"output":0.41541},"cost_usd_as_run":2.2018,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":90.57,"tokens":{"cache_read":22934752,"cache_write":0,"output":171808,"reasoning":91204,"uncached_input":1094933},"cost_usd":12.4028,"cost_breakdown":{"cache_read":4.586951,"cache_write":0.0,"input":4.379732,"output":3.43616},"cost_usd_as_run":12.4028,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":51.88,"tokens":{"cache_read":17555200,"cache_write":0,"output":104535,"reasoning":66680,"uncached_input":667973},"cost_usd":4.1369,"cost_breakdown":{"cache_read":1.75552,"cache_write":0.0,"input":1.335946,"output":1.04535},"cost_usd_as_run":4.1369,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":53.29,"tokens":{"cache_read":47225344,"cache_write":0,"output":444801,"reasoning":309284,"uncached_input":589890},"cost_usd":0.4971,"cost_breakdown":{"cache_read":0.141676,"cache_write":0.0,"input":0.088483,"output":0.26688},"cost_usd_as_run":0.4971,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":61.44,"tokens":{"cache_read":32116811,"cache_write":1029075,"output":275965,"reasoning":180490,"uncached_input":286},"cost_usd":20.1763,"cost_breakdown":{"cache_read":6.423362,"cache_write":8.2326,"input":0.001144,"output":5.5193},"cost_usd_as_run":20.1763,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-cdef-avx512-opus-5.5-xhigh-r1","letter":"B","score":8.0}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max","opus-5.5-xhigh"]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":20.58,"tokens":{"uncached_input":46,"cache_read":2865122,"cache_write":167566,"output":113852,"reasoning":74335},"cost_usd":4.1908,"cost_breakdown":{"input":0.000184,"cache_read":0.573024,"cache_write":1.340528,"output":2.27704},"cost_usd_as_run":4.1908,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":938,"code":726,"comment":155,"blank":57,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1003,"code":886,"comment":57,"blank":60,"path":"tests/selftest.c","role":"test"}],"impl_lines":938,"impl_code":726,"impl_comment":155,"test_lines":1003,"comment_ratio":0.176,"tests_declared":9,"object":{"text_bytes":3191,"rodata_bytes":3212}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":121.65,"reference":3172.01,"speedup":26.075,"baselines":{"dav1d AVX2":142.1,"dav1d AVX-512ICL":54.3}},{"id":"find_dir","candidate":38.46,"reference":380.58,"speedup":9.895,"baselines":{"dav1d AVX2":40.06}}]},"scores":{"gpt-6.1-terra-max":9.0,"muse-wonder-full-xhigh":9.5,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Fast, comprehensive implementation with a portable allocation error in its tests.","muse-wonder-full-xhigh":"fastest find_dir, strongest ABI harness","gpt-6.1-sol-max":"Strong kernels and ABI checks, with a test-allocation defect and a minor documentation error.","deepseek-v4.1-flash-max":"the most rigorously tested and fastest direction search; only boilerplate to trim","opus-5.5-xhigh":"fastest, cleanest and best-tested: per-tap masks derived in vector form, red-zone find_dir, thorough and accurate documentation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Aligned allocation has an invalid size","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Undef hygiene block","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Aligned allocation requests an invalid size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Claimed word layout exceeds one register","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"`%undef` blocks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Mixed register naming","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0165,"lines":12,"significant":726,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":21,"flags":1,"audited":"2026-11-04","reviewed":true},"score":9.3,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.0}},"defect_points":0.8,"score_cross_vendor":9.25,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":23.42,"tokens":{"uncached_input":43398,"cache_read":1143808,"cache_write":0,"output":35168,"reasoning":17079},"cost_usd":0.5529,"cost_breakdown":{"input":0.086796,"cache_read":0.114381,"cache_write":0.0,"output":0.35168},"cost_usd_as_run":0.5529,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":658,"code":591,"comment":41,"blank":26,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":738,"code":662,"comment":31,"blank":45,"path":"tests/selftest.c","role":"test"}],"impl_lines":658,"impl_code":591,"impl_comment":41,"test_lines":738,"comment_ratio":0.065,"tests_declared":7,"object":{"text_bytes":3488,"rodata_bytes":2944}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":124.88,"reference":3173.64,"speedup":25.414,"baselines":{"dav1d AVX2":142.48,"dav1d AVX-512ICL":54.42}},{"id":"find_dir","candidate":39.14,"reference":381.92,"speedup":9.758,"baselines":{"dav1d AVX2":40.06}}]},"scores":{"gpt-6.1-terra-max":9.0,"muse-wonder-full-xhigh":9.5,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Fast and well tested, with an ABI coverage gap.","muse-wonder-full-xhigh":"fastest filter, generated tables, strong tests","gpt-6.1-sol-max":"Fast, clearly structured kernels with extensive correctness and boundary coverage.","deepseek-v4.1-flash-max":"fast and correct on both entry points; one needlessly cryptic table expression","opus-5.5-xhigh":"fast, correct, tables generated by NASM macros, strong tests; documentation is terse"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Direction ABI and upper state are not checked","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Cryptic index generator","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Cleanup instructions have no regression check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`dw (w = 24)`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The header overstates how memory is read","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"damping is spilled and reloaded for no reason","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":591,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":20,"flags":0,"audited":"2026-11-04","reviewed":false},"score":9.1,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0}},"defect_points":0.9,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.54,"tokens":{"uncached_input":40324,"cache_read":169984,"cache_write":0,"output":2102,"reasoning":151},"cost_usd":0.0593,"cost_breakdown":{"input":0.040324,"cache_read":0.008499,"cache_write":0.0,"output":0.01051},"cost_usd_as_run":0.0593,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":142,"code":134,"comment":6,"blank":2,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":142,"impl_code":134,"impl_comment":6,"test_lines":348,"comment_ratio":0.043,"tests_declared":3,"object":{"text_bytes":403,"rodata_bytes":104}},"hidden":{"passed":3,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3163.27,"speedup":null,"baselines":{"dav1d AVX2":142.1,"dav1d AVX-512ICL":54.34}},{"id":"find_dir","candidate":568.11,"reference":381.46,"speedup":0.671,"baselines":{"dav1d AVX2":39.85}}]},"scores":{"gpt-6.1-terra-max":3.0,"muse-wonder-full-xhigh":3.0,"gpt-6.1-sol-max":3.0,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":1.0},"council_summaries":{"gpt-6.1-terra-max":"Direction search works in the exercised cases, but the entire filter is missing.","muse-wonder-full-xhigh":"find_dir correct but filter entirely missing","gpt-6.1-sol-max":"Direction search works, but the filter is missing and search remains unnecessarily scalar.","deepseek-v4.1-flash-max":"only the direction search exists; the filter is an unimplemented stub","opus-5.5-xhigh":"filter not implemented; direction search is scalar and slower than the C reference"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter returns without filtering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Direction search is scalar and slower than the reference","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Only the basic differential cases are present","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter unimplemented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Scalar find_dir slower than C","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter is a no-op","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Direction search performs scalar stack accumulation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"filter does nothing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"find_dir is scalar and slower than the C it replaces","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"tests/selftest.c is the untouched scaffold (byte-identical to entry-Q's)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the header advertises \"direction search and filter\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"The filter is an empty stub","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"find_dir is scalar","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unneeded save and no-op instruction","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The test file is the scaffold, unchanged","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Only a few one-line comments","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":134,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-04","reviewed":false},"score":2.4,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":6.8,"score_cross_vendor":2.0,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.92,"tokens":{"uncached_input":54264,"cache_read":623360,"cache_write":0,"output":13187,"reasoning":2881},"cost_usd":0.1514,"cost_breakdown":{"input":0.054264,"cache_read":0.031168,"cache_write":0.0,"output":0.065935},"cost_usd_as_run":0.1514,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":841,"code":818,"comment":18,"blank":5,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":492,"code":434,"comment":24,"blank":34,"path":"tests/selftest.c","role":"test"}],"impl_lines":841,"impl_code":818,"impl_comment":18,"test_lines":492,"comment_ratio":0.022,"tests_declared":6,"object":{"text_bytes":3610,"rodata_bytes":23872}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":143.53,"reference":3124.47,"speedup":21.769,"baselines":{"dav1d AVX2":140.13,"dav1d AVX-512ICL":53.58}},{"id":"find_dir","candidate":65.34,"reference":377.13,"speedup":5.772,"baselines":{"dav1d AVX2":39.64}}]},"scores":{"gpt-6.1-terra-max":8.0,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but its scalar setup and duplicated tables add avoidable work.","muse-wonder-full-xhigh":"correct, but magic tables and thin tests","gpt-6.1-sol-max":"Correct and bounded, with avoidable scalar staging and incomplete ABI coverage.","deepseek-v4.1-flash-max":"correct and clean, but pays for a 20 KB precomputed (edges,dir) table and has no ABI test","opus-5.5-xhigh":"correct with exact-width scalar accesses, but scalar, branchy plane construction, a 20 KB opaque mask table and thin docs and tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Shifted planes are copied one row at a time","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Direction offsets are repeated for every edge combination","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI state is not exercised","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"style","title":"Unauditable literal tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI coverage, narrow guards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Slowest find_dir of vector group","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Padding bloat","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Plane construction uses nested scalar copy loops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI hygiene lacks explicit coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"per-(edges,dir) mask/offset table instead of deriving masks per tap","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"the 1,536 table constants have no derivation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no callee-saved / upper-state check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"find_dir ≈14.7 ns vs 8.2 ns best","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Rows and planes are built with scalar, branchy copies","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"A 20 KB hand-written lookup table","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Thin documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Small wasted work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks and narrow guard/extremes coverage","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0293,"lines":24,"significant":818,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":16,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.6,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":3.7,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":17.58,"tokens":{"uncached_input":31329,"cache_read":1305984,"cache_write":0,"output":33501,"reasoning":16699},"cost_usd":0.5283,"cost_breakdown":{"input":0.062658,"cache_read":0.130598,"cache_write":0.0,"output":0.33501},"cost_usd_as_run":0.5283,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":612,"code":569,"comment":27,"blank":16,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":690,"code":626,"comment":24,"blank":40,"path":"tests/selftest.c","role":"test"}],"impl_lines":612,"impl_code":569,"impl_comment":27,"test_lines":690,"comment_ratio":0.045,"tests_declared":8,"object":{"text_bytes":3629,"rodata_bytes":3072}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":125.81,"reference":3191.7,"speedup":25.369,"baselines":{"dav1d AVX2":143.32,"dav1d AVX-512ICL":54.93}},{"id":"find_dir","candidate":53.71,"reference":384.52,"speedup":7.159,"baselines":{"dav1d AVX2":40.4}}]},"scores":{"gpt-6.1-terra-max":9.0,"muse-wonder-full-xhigh":8.5,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Fast and correct in the exercised cases, with incomplete ABI tests.","muse-wonder-full-xhigh":"correct and fast, minor test and find_dir gaps","gpt-6.1-sol-max":"Correct, efficient kernels backed by extensive layout and guard tests.","deepseek-v4.1-flash-max":"compact, fast and correct; docs are terse but accurate","opus-5.5-xhigh":"correct, robust, strong tests; find_dir loses time to a serial load chain; documentation is sparse"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI coverage omits direction and upper state","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"ABI test only half-effective","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"find_dir middling","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state hygiene is untested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"find_dir ≈11.7 ns vs 8.2 ns best","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir loads its rows through a serial dependency chain","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"The header is two lines","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":569,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":25,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.9,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.667,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":34.38,"tokens":{"uncached_input":131337,"cache_read":51546368,"cache_write":0,"output":338920,"reasoning":253422},"cost_usd":0.3777,"cost_breakdown":{"input":0.019701,"cache_read":0.154639,"cache_write":0.0,"output":0.203352},"cost_usd_as_run":0.3777,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.3777,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":1081,"code":851,"comment":173,"blank":57,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":811,"code":703,"comment":55,"blank":53,"path":"tests/selftest.c","role":"test"}],"impl_lines":1081,"impl_code":851,"impl_comment":173,"test_lines":811,"comment_ratio":0.169,"tests_declared":9,"object":{"text_bytes":3779,"rodata_bytes":3072}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":126.9,"reference":3128.04,"speedup":24.65,"baselines":{"dav1d AVX2":140.42,"dav1d AVX-512ICL":53.58}},{"id":"find_dir","candidate":44.85,"reference":377.26,"speedup":8.412,"baselines":{"dav1d AVX2":39.51}}]},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"The implementation passes differential checks, but its ABI and guard tests miss important failures.","muse-wonder-full-xhigh":"excellent code and docs, ABI test harness is broken","gpt-6.1-sol-max":"Strong kernels, undermined by an incorrectly constructed ABI test.","deepseek-v4.1-flash-max":"excellent, readable and correct; one delivered test is broken","opus-5.5-xhigh":"excellent kernel and documentation, but the test suite has a real ABI-probe bug and weaker guard coverage than it claims"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI probe forwards the wrong stack arguments","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Absent edges still receive accessible pointers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper vector state is not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Claimed flat filter patterns are not flat","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Both conditional branches assign the same index","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"ABI probe passes wrong stack arguments","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin guard/layout coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Slowest filter in top group","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead generator branch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI probe reads beyond its argument array","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is unchecked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Identical conditional branches","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"ABI probe reads past its argument array and passes out-of-domain arguments","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"both arms of the `%if HALF = 0` in `alt23_idx` assign the same index","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The ABI probe passes wrong, out-of-contract stack arguments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The guard tests cover less than their comment (276) says","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The extremes sweep sees each pattern only once per 32 cases","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"64-byte index loads read past 16- and 32-byte tables","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"A table generator branch is a no-op","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Comments repeat themselves","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0329,"lines":28,"significant":851,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":179,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.1,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":1.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0}},"defect_points":4.4,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":50.65,"tokens":{"uncached_input":235245,"cache_read":4971716,"cache_write":0,"output":82439,"reasoning":44181},"cost_usd":3.5841,"cost_breakdown":{"input":0.94098,"cache_read":0.994343,"cache_write":0.0,"output":1.64878},"cost_usd_as_run":3.5841,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":715,"code":513,"comment":158,"blank":44,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":826,"code":735,"comment":45,"blank":46,"path":"tests/selftest.c","role":"test"}],"impl_lines":715,"impl_code":513,"impl_comment":158,"test_lines":826,"comment_ratio":0.235,"tests_declared":8,"object":{"text_bytes":4092,"rodata_bytes":2752}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":128.37,"reference":3074.33,"speedup":23.949,"baselines":{"dav1d AVX2":137.48,"dav1d AVX-512ICL":52.66}},{"id":"find_dir","candidate":63.7,"reference":369.32,"speedup":5.798,"baselines":{"dav1d AVX2":38.84}}]},"scores":{"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":8.5,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and well documented, with avoidable direction-search work and an upper-state test gap.","muse-wonder-full-xhigh":"clean and well documented, slower find_dir","gpt-6.1-sol-max":"Correct, well-documented vector kernels with minor test and redundancy issues.","deepseek-v4.1-flash-max":"correct, well documented; the direction search is the slow part","opus-5.5-xhigh":"correct, safe and the best-documented of the vector designs; find_dir is slower than peers and the ABI test skips dirty register upper halves"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction costs make a stack round trip","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper vector state is not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Costs are loaded twice","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"find_dir goes through memory","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Modest added coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is unchecked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Cost array is loaded twice consecutively","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"find_dir ≈14.6 ns vs 8.2 ns best","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir round-trips its costs through the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The ABI wrapper never puts garbage in r9's upper half and nothing checks XINUSE","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0234,"lines":12,"significant":513,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":42,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.5,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":1.8,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":8.86,"tokens":{"uncached_input":42507,"cache_read":1397120,"cache_write":0,"output":39692,"reasoning":19983},"cost_usd":0.3108,"cost_breakdown":{"input":0.042507,"cache_read":0.069856,"cache_write":0.0,"output":0.19846},"cost_usd_as_run":0.3108,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":661,"code":599,"comment":38,"blank":24,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":840,"code":769,"comment":28,"blank":43,"path":"tests/selftest.c","role":"test"}],"impl_lines":661,"impl_code":599,"impl_comment":38,"test_lines":840,"comment_ratio":0.06,"tests_declared":8,"object":{"text_bytes":3763,"rodata_bytes":3968}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":128.92,"reference":3163.73,"speedup":24.54,"baselines":{"dav1d AVX2":141.51,"dav1d AVX-512ICL":54.04}},{"id":"find_dir","candidate":43.71,"reference":380.83,"speedup":8.713,"baselines":{"dav1d AVX2":40.02}}]},"scores":{"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":9.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct and fast, with a small test gap and a no-op clamp instruction.","muse-wonder-full-xhigh":"correct, well-generated tables, strong tests","gpt-6.1-sol-max":"Efficient, carefully bounded kernels with strong tests and one coverage gap.","deepseek-v4.1-flash-max":"fastest filter, correct throughout, strongest ABI test; nothing to fix","opus-5.5-xhigh":"same high-quality byte-domain design; good tests; only small redundancies"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper vector state is not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Conditional move cannot fire","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Wasteful mask-table stride","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI tests omit upper-vector state","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The edges argument is re-read from memory over and over","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0267,"lines":16,"significant":599,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":25,"flags":0,"audited":"2026-11-04","reviewed":false},"score":9.0,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":0.7,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.26,"tokens":{"uncached_input":12410,"cache_read":110720,"cache_write":0,"output":511,"reasoning":103},"cost_usd":0.0205,"cost_breakdown":{"input":0.01241,"cache_read":0.005536,"cache_write":0.0,"output":0.002555},"cost_usd_as_run":0.0205,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":25,"code":10,"comment":10,"blank":5,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":25,"impl_code":10,"impl_comment":10,"test_lines":348,"comment_ratio":0.5,"tests_declared":3,"object":{"text_bytes":4,"rodata_bytes":0}},"hidden":{"passed":1,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":3160.08,"speedup":null,"baselines":{"dav1d AVX2":141.47,"dav1d AVX-512ICL":54.0}},{"id":"find_dir","candidate":null,"reference":380.95,"speedup":null,"baselines":{"dav1d AVX2":39.85}}]},"scores":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":0.0},"council_summaries":{"gpt-6.1-terra-max":"Both public functions are unimplemented stubs.","muse-wonder-full-xhigh":"untouched stub, neither function implemented","gpt-6.1-sol-max":"Both exported operations remain nonfunctional stubs.","deepseek-v4.1-flash-max":"the untouched scaffold stub; nothing is implemented","opus-5.5-xhigh":"the unmodified scaffold stub"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Direction search does not write variance","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filter returns without filtering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Only the basic differential cases are present","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"find_dir stub","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Filter stub","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Direction search returns a constant and leaves variance unwritten","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Filter performs no filtering","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"both entry points are the scaffold stub","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"tests/selftest.c is byte-identical to entry-D's untouched scaffold copy","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Neither entry point is implemented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The test file is the scaffold, unchanged","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-04","reviewed":false},"score":0.5,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":6.6,"score_cross_vendor":0.667,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.78,"tokens":{"uncached_input":21858,"cache_read":163200,"cache_write":0,"output":4814,"reasoning":38},"cost_usd":0.6225,"cost_breakdown":{"input":0.21858,"cache_read":0.1632,"cache_write":0.0,"output":0.2407},"cost_usd_as_run":0.6225,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":1168,"code":1160,"comment":6,"blank":2,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":415,"code":363,"comment":21,"blank":31,"path":"tests/selftest.c","role":"test"}],"impl_lines":1168,"impl_code":1160,"impl_comment":6,"test_lines":415,"comment_ratio":0.005,"tests_declared":5,"object":{"text_bytes":5988,"rodata_bytes":4896}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":322.29,"reference":3152.44,"speedup":9.781,"baselines":{"dav1d AVX2":140.72,"dav1d AVX-512ICL":53.67}},{"id":"find_dir","candidate":216.13,"reference":377.59,"speedup":1.747,"baselines":{"dav1d AVX2":39.64}}]},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":6.0,"opus-5.5-xhigh":5.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but substantially slower and less well protected by its tests.","muse-wonder-full-xhigh":"correct everywhere probed, well behind on speed and clarity","gpt-6.1-sol-max":"Correct in probes, but substantial hot-path inefficiency and incomplete ABI tests.","deepseek-v4.1-flash-max":"correct, but 2.7–4x slower than its peers and much harder to maintain","opus-5.5-xhigh":"bit-exact and memory-safe, but a row-at-a-time word design 3–6× slower than peers, almost no documentation, and thin tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal direction sums use 64 separate permutations","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Filter repeats setup for every row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Missing-left reads are not guarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper-state violations are undetected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Direction tables lack their defining invariants","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Unused stack-pointer spill","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"3–5x slower than peers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Boundary test cannot see wrong values","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Stack pointer relocated below the frame","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Unmaintainable find_dir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Saved rsp never read","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Comment claims a copy that never happens","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Filter underuses vector lanes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal sums use 64 single-lane permutations","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI cleanup regressions escape testing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Saved stack pointer is never consumed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"filter and direction search are far from the class of the other entries","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"the filter rewrites `rsp` to a fake base below its own frame","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"~840 lines of fully unrolled direction-search code with magic 16-bit mask immediates (src:183-1021) and 64 mostly identical index tables (src:1024-1166","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the `directions` byte table (src:1023) and the mask immediates have no explanation (units are words","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The filter processes one row per iteration with 8 of 32 word lanes live","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"find_dir does 136 `vpermw` gathers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"rsp is repointed as a scratch base register","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"A dead store","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Duplicated index tables and no-op permutes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Essentially undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"A comment describes a copy that never happens","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"test_boundaries cannot detect wrong output and misses absent-left reads","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0552,"lines":64,"significant":1160,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.2,"slop":{"score":1.2,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":2.5}},"defect_points":7.5,"score_cross_vendor":5.833,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.38,"tokens":{"uncached_input":43122,"cache_read":281856,"cache_write":0,"output":10416,"reasoning":792},"cost_usd":0.2186,"cost_breakdown":{"input":0.086244,"cache_read":0.028186,"cache_write":0.0,"output":0.10416},"cost_usd_as_run":0.2186,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":517,"code":499,"comment":18,"blank":0,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":446,"code":391,"comment":23,"blank":32,"path":"tests/selftest.c","role":"test"}],"impl_lines":517,"impl_code":499,"impl_comment":18,"test_lines":446,"comment_ratio":0.035,"tests_declared":5,"object":{"text_bytes":4108,"rodata_bytes":2432}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":135.3,"reference":3126.7,"speedup":23.109,"baselines":{"dav1d AVX2":140.34,"dav1d AVX-512ICL":53.58}},{"id":"find_dir","candidate":66.14,"reference":377.38,"speedup":5.706,"baselines":{"dav1d AVX2":39.51}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the shared suite, with slower direction search and limited maintainability of its tables.","deepseek-v4.1-flash-max":"correct and compact, but effectively undocumented: no header, decimal magic masks, and four copy-pasted cost blocks","gpt-6.1-sol-max":"Correct kernels, but redundant work, opaque tables and incomplete ABI tests.","opus-5.5-xhigh":"Correct, but undocumented, decimal magic tables, avoidable stalls, thin tests","muse-wonder-full-xhigh":"bit-exact and safe, but the least documented and most copy-pasted of the correct entries"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction families run through a serial accumulator","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI hygiene is not exercised","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Direction tables lack their generating invariant","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"No file header and no explanation of the algorithm","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Unreadable generated constants","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Four identical cost reductions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No callee-saved-register test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Left handling repeats per row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Padding initialization immediately overwrites most cleared bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Direction-table invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No file header, no entry-point docs, almost no comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Store-forwarding stalls in both functions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Four copy-pasted square/weight/reduce blocks with repeated comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Essentially undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"find_dir is four copy-pasted blocks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Slow tier on both functions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinnest added suite","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0481,"lines":24,"significant":499,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.1,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0}},"defect_points":4.6,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":57.68,"tokens":{"uncached_input":268650,"cache_read":5267568,"cache_write":0,"output":69711,"reasoning":34394},"cost_usd":3.5223,"cost_breakdown":{"input":1.0746,"cache_read":1.053514,"cache_write":0.0,"output":1.39422},"cost_usd_as_run":3.5223,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":608,"code":462,"comment":106,"blank":40,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":726,"code":650,"comment":34,"blank":42,"path":"tests/selftest.c","role":"test"}],"impl_lines":608,"impl_code":462,"impl_comment":106,"test_lines":726,"comment_ratio":0.187,"tests_declared":9,"object":{"text_bytes":4088,"rodata_bytes":2560}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":130.89,"reference":3124.76,"speedup":23.873,"baselines":{"dav1d AVX2":140.25,"dav1d AVX-512ICL":53.58}},{"id":"find_dir","candidate":53.33,"reference":376.88,"speedup":7.067,"baselines":{"dav1d AVX2":39.51}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the shared suite, with redundant plane construction.","deepseek-v4.1-flash-max":"correct and clean, but it builds and stores plane 4 twice, and that shows in the filter timing","gpt-6.1-sol-max":"Correct and well tested, with redundant plane work and an ABI-test gap.","opus-5.5-xhigh":"Correct, reasonably fast; opaque hand-typed tables, redundant plane store, thin ABI test","muse-wonder-full-xhigh":"bit-exact and compact; hand-packed tables are the maintainability weak spot"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Plane 4 is written twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Plane 4 is constructed and stored twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Tap descriptors are opaque hex","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Plane four is constructed twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Register checks omit upper state","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Plane 4 is built and stored twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Hand-typed opaque tables","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Minimal ABI test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The find_dir row loads are serially merged masked broadcasts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Opaque hand-typed tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Plane 4 built and stored twice","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":462,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":43,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.3,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0}},"defect_points":2.0,"score_cross_vendor":8.25,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":206.72,"tokens":{"uncached_input":705156,"cache_read":76304576,"cache_write":0,"output":529816,"reasoning":446053},"cost_usd":2.6598,"cost_breakdown":{"input":0.105773,"cache_read":2.289137,"cache_write":0.0,"output":0.264908},"cost_usd_as_run":2.6598,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":1183,"code":951,"comment":171,"blank":61,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":1183,"impl_code":951,"impl_comment":171,"test_lines":348,"comment_ratio":0.152,"tests_declared":3,"object":{"text_bytes":3785,"rodata_bytes":16926}},"hidden":{"passed":3,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":null,"speedup":null,"baselines":{"dav1d AVX2":null,"dav1d AVX-512ICL":null}},{"id":"find_dir","candidate":null,"reference":null,"speedup":null,"baselines":{"dav1d AVX2":null}}]},"scores":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0},"council_summaries":{"gpt-6.1-terra-max":"Both entry points fail on valid inputs.","deepseek-v4.1-flash-max":"both entry points are non-functional: the filter writes its planes to a wild address and segfaults on every call; find_dir always returns dir 0 with a garbage var","gpt-6.1-sol-max":"Filtering crashes, and direction search returns incorrect results.","opus-5.5-xhigh":"Neither function works: the filter segfaults on every call, find_dir is wrong on every block","muse-wonder-full-xhigh":"unfinished: filter wild-writes and crashes, find_dir returns garbage, own suite red"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"critical","category":"robustness","title":"Filter writes through a non-pointer","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Direction selection loses the original costs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"abi","title":"Callee-saved state is overwritten before saving","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Direction-load description is stale","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"robustness","title":"Plane base register never points at the frame","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"abi","title":"rbx/rbp clobbered before they are saved","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"robustness","title":"Saved-rsp register is reused as scratch","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"Direction-table base is computed from itself","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"find_dir stores 16 bytes but indexes 32","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"Tie-break compares the unfolded fold against the maximum","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"The header describes a different program","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Selftest is the unmodified scaffold (348 lines, byte-identical to entry-I's)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"critical","category":"robustness","title":"Plane stores use an edge offset as a pointer","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"critical","category":"abi","title":"Frame bookkeeping cannot restore caller state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Maximum reduction destroys the direction-selection inputs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Filter crashes on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"abi","title":"rbx and rbp are clobbered","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"find_dir is wrong for every input","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Wrong shift masks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"misleading-docs","title":"Wrong comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"tests/selftest.c is the unmodified scaffold, and `make test` fails: a find_dir mismatch, then a segfault.","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"robustness","title":"Plane stores write through a non-pointer","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"abi","title":"Saved-rsp and callee-saved registers destroyed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"find_dir picks the maximum from the wrong vector","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Tap validity masks lack the dy=−2 case","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"`left[6]`/`left[7]` never inserted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Suite is the untouched scaffold and it fails","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"Header guarantees the opposite of the behaviour","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0786,"lines":64,"significant":814,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":212,"flags":0,"audited":"2026-11-04","reviewed":false},"score":0.7,"slop":{"score":1.3,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":2.0}},"defect_points":17.7,"score_cross_vendor":0.7,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":108.86,"tokens":{"uncached_input":2395,"cache_read":342656,"cache_write":0,"output":2567,"reasoning":893},"cost_usd":0.1915,"cost_breakdown":{"input":0.00479,"cache_read":0.171328,"cache_write":0.0,"output":0.015402},"cost_usd_as_run":0.1915,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"lines":938,"code":769,"comment":103,"blank":66,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1122,"code":1033,"comment":38,"blank":51,"path":"tests/selftest.c","role":"test"}],"impl_lines":938,"impl_code":769,"impl_comment":103,"test_lines":1122,"comment_ratio":0.118,"tests_declared":9,"object":{"text_bytes":4600,"rodata_bytes":2368}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":123.58,"reference":3163.19,"speedup":25.596,"baselines":{"dav1d AVX2":146.01,"dav1d AVX-512ICL":54.34}},{"id":"find_dir","candidate":76.59,"reference":381.29,"speedup":4.978,"baselines":{"dav1d AVX2":39.85}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"The assembly passes the shared suite, but its guard test corrupts heap memory.","deepseek-v4.1-flash-max":"correct, the most thorough test suite, but the slowest direction search and a heavy xmm chain","gpt-6.1-sol-max":"Assembly passes the common checks, but its guard tests corrupt the heap.","opus-5.5-xhigh":"Correct, fast filter, strong tests; find_dir is the slowest of the working entries","muse-wonder-full-xhigh":"bit-exact with a novel find_dir, but that find_dir is twice the leaders' cost"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guard snapshots overflow their allocation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction costs are reduced and spilled individually","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"find_dir is the slowest of the working entries","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"tests","title":"Snapshot buffer is too small","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI tests omit upper-state checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unused reference snapshot","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Slow find_dir","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The `sum8_store` reduction is written out twice by hand for hv0 and hv1","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Dead code in the guard test: `dst` is assigned, then overwritten on both branches; `gplane` is never used","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"find_dir is the slowest of the correct entries","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"\"Every tap is one masked 64-byte load\" is not what the code does","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Magic frame slots","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":56,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.5,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":2.9,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":16.52,"tokens":{"uncached_input":26537,"cache_read":613632,"cache_write":0,"output":27827,"reasoning":11597},"cost_usd":2.2704,"cost_breakdown":{"input":0.26537,"cache_read":0.613632,"cache_write":0.0,"output":1.39135},"cost_usd_as_run":2.2704,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":664,"code":598,"comment":41,"blank":25,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":734,"code":659,"comment":33,"blank":42,"path":"tests/selftest.c","role":"test"}],"impl_lines":664,"impl_code":598,"impl_comment":41,"test_lines":734,"comment_ratio":0.064,"tests_declared":7,"object":{"text_bytes":3499,"rodata_bytes":3392}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":122.83,"reference":3118.26,"speedup":25.387,"baselines":{"dav1d AVX2":140.0,"dav1d AVX-512ICL":53.29}},{"id":"find_dir","candidate":44.72,"reference":374.7,"speedup":8.379,"baselines":{"dav1d AVX2":39.26}}]},"scores":{"gpt-6.1-terra-max":10.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct, well-tested, and efficient across the reviewed paths.","deepseek-v4.1-flash-max":"strong code and tests; one dead alternate store path and a stray comment artefact","gpt-6.1-sol-max":"Correct implementation with strong coverage apart from upper-state regression checks.","opus-5.5-xhigh":"Correct, fast, generated tables, strong tests; thin docs and leftover alternative code","muse-wonder-full-xhigh":"bit-exact, fast, interleaved find_dir; leftover dead store strategy"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"Never-assembled second store strategy","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Broken comment indentation/claim","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Trampolines clear upper state without checking it","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"A benchmarking alternative is left in","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Header is 2 lines","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The ABI wrapper misaligns rsp by 0..15 bytes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"A guarded find_dir mismatch returns 0 with no message and no unmap","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unassembled alternative store path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Overlapping register aliases","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0201,"lines":12,"significant":598,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.9,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0}},"defect_points":1.5,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":403.8,"tokens":{"uncached_input":1257040,"cache_read":11105472,"cache_write":0,"output":120634,"reasoning":70020},"cost_usd":9.6619,"cost_breakdown":{"input":5.02816,"cache_read":2.221094,"cache_write":0.0,"output":2.41268},"cost_usd_as_run":9.6619,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"lines":771,"code":537,"comment":188,"blank":46,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":657,"code":589,"comment":29,"blank":39,"path":"tests/selftest.c","role":"test"}],"impl_lines":771,"impl_code":537,"impl_comment":188,"test_lines":657,"comment_ratio":0.259,"tests_declared":8,"object":{"text_bytes":4146,"rodata_bytes":2624}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":134.0,"reference":3151.35,"speedup":23.518,"baselines":{"dav1d AVX2":141.35,"dav1d AVX-512ICL":53.96}},{"id":"find_dir","candidate":65.8,"reference":380.7,"speedup":5.786,"baselines":{"dav1d AVX2":39.77}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the shared suite, with avoidable direction-search overhead and an ABI test gap.","deepseek-v4.1-flash-max":"correct, clearly documented and thoroughly tested; no ABI check in its own tests","gpt-6.1-sol-max":"Correct and readable, with repeated left-edge work and an ABI-test gap.","opus-5.5-xhigh":"Correct and well commented; store-forwarding stall in find_dir, hand-typed tables, no ABI tests","muse-wonder-full-xhigh":"bit-exact, best-documented design; find_dir on the slow side"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Direction costs are reduced and spilled separately","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI hygiene is not exercised","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No callee-saved-register test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Mixed-case hex literals","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Eight separate left-pair loads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir spills its costs and reloads them wide","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Hand-typed find_dir index tables, k-masks and row/col masks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI test (callee-saved, dirty upper halves, XINUSE); the guard test compares only the 8×8 output","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"`left` goes in through 8 separate 2-byte masked loads","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"find_dir well behind the leaders","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No callee-saved/XINUSE/upper-bits test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":538,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":88,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.2,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":2.2,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":17.84,"tokens":{"uncached_input":42,"cache_read":2685506,"cache_write":161460,"output":111076,"reasoning":73992},"cost_usd":4.0505,"cost_breakdown":{"input":0.000168,"cache_read":0.537101,"cache_write":1.29168,"output":2.22152},"cost_usd_as_run":4.0505,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":942,"code":717,"comment":158,"blank":67,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":933,"code":812,"comment":61,"blank":60,"path":"tests/selftest.c","role":"test"}],"impl_lines":942,"impl_code":717,"impl_comment":158,"test_lines":933,"comment_ratio":0.181,"tests_declared":9,"object":{"text_bytes":3217,"rodata_bytes":4820}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":122.74,"reference":3194.22,"speedup":26.024,"baselines":{"dav1d AVX2":147.94,"dav1d AVX-512ICL":54.67}},{"id":"find_dir","candidate":37.71,"reference":379.65,"speedup":10.068,"baselines":{"dav1d AVX2":40.14}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":9.5,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Fast and correct in the shared suite, with an out-of-domain test and a stale comment.","deepseek-v4.1-flash-max":"merge-ready: correct, fastest, best-documented and best-tested; only a duplicated address computation","gpt-6.1-sol-max":"Correct, efficient kernels and broad tests, with minor coverage and documentation issues.","opus-5.5-xhigh":"Fastest, fully generated tables, excellent comments and tests","muse-wonder-full-xhigh":"bit-exact, fastest tier, generated tables, strongest all-round suite"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Special direction tests use stride 8","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Constant blocks do not all have zero costs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Frame addresses recomputed for the stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI tests omit upper-state checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Constant blocks do not generally have zero costs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The ABI trampoline misaligns rsp by 0..15 bytes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Phase lettering out of order","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0195,"lines":14,"significant":717,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":20,"flags":1,"audited":"2026-11-04","reviewed":true},"score":9.2,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.0}},"defect_points":0.9,"score_cross_vendor":9.125,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":161.99,"tokens":{"uncached_input":1386,"cache_read":89212619,"cache_write":584849,"output":477988,"reasoning":0},"cost_usd":1.7692,"cost_breakdown":{"input":0.000208,"cache_read":1.427402,"cache_write":0.11697,"output":0.224654},"cost_usd_as_run":1.7692,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"lines":957,"code":724,"comment":188,"blank":45,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":348,"code":300,"comment":19,"blank":29,"path":"tests/selftest.c","role":"test"}],"impl_lines":957,"impl_code":724,"impl_comment":188,"test_lines":348,"comment_ratio":0.206,"tests_declared":3,"object":{"text_bytes":3858,"rodata_bytes":6400}},"hidden":{"passed":9,"total":15,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":null,"reference":null,"speedup":null,"baselines":{"dav1d AVX2":null,"dav1d AVX-512ICL":null}},{"id":"find_dir","candidate":null,"reference":null,"speedup":null,"baselines":{"dav1d AVX2":null}}]},"scores":{"gpt-6.1-terra-max":2.0,"deepseek-v4.1-flash-max":2.0,"gpt-6.1-sol-max":2.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":2.0},"council_summaries":{"gpt-6.1-terra-max":"Direction search works in the tested cases, but the filter has wrong border handling, can fault, and violates the ABI.","deepseek-v4.1-flash-max":"destroys r12/r13, which makes its own `make test` hang, and the filter is wrong for every edge flag except TOP-only","gpt-6.1-sol-max":"Valid filter calls corrupt registers, overread memory and produce wrong pixels.","opus-5.5-xhigh":"Clobbers r12/r13, reads outside the contract with BOTTOM, wrong output with LEFT or RIGHT; own `make test` hangs","muse-wonder-full-xhigh":"correct core math, but neighbour loads and callee-saved handling are broken; own suite hangs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Bottom loads use the wrong rows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"The first row’s left pixels are omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"The right-edge mask admits only one neighbour","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"abi","title":"Incoming `r12` and `r13` are lost","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Multiply operand types are reversed in the description","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"abi","title":"r12/r13 loaded before they are pushed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"Bottom rows read two rows too low","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"HAVE_RIGHT enables only column 8","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"Left splice mask for R0 is missing block row 0","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"Comment contradicts the code","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Selftest is the unmodified scaffold and no longer terminates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"abi","title":"Registers are overwritten before being saved","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Bottom rows are read two strides too late","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"The second right-hand pixel is omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Left pixels for block row zero are omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Multiply-add signedness is reversed in the overview","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"abi","title":"r12 and r13 are clobbered","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"With HAVE_BOTTOM, the bottom rows are read from bottom+2·stride and +3·stride","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"With HAVE_RIGHT, column 9 is never loaded","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"With HAVE_LEFT, block row 0 never gets left[0]","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"misleading-docs","title":"Several false comments: src/cdef_avx512.asm:25-26 (says \"unsigned accumulator pairs, signed weights\", which is reversed)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"tests/selftest.c is the unmodified scaffold","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Bottom rows read two strides too far","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"abi","title":"`r12`/`r13` overwritten before being saved","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"RIGHT mask misses column 9","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"`left[0]` never spliced into block row 0","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Shipped suite does not pass","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"Header claims no out-of-contract byte is ever accessed","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":10,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":232,"flags":1,"audited":"2026-11-04","reviewed":true},"score":1.9,"slop":{"score":1.6,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":2.0}},"defect_points":16.4,"score_cross_vendor":1.9,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.09,"tokens":{"uncached_input":23616,"cache_read":669184,"cache_write":0,"output":17094,"reasoning":4638},"cost_usd":0.2851,"cost_breakdown":{"input":0.047232,"cache_read":0.066918,"cache_write":0.0,"output":0.17094},"cost_usd_as_run":0.2851,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":621,"code":591,"comment":21,"blank":9,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":541,"code":479,"comment":25,"blank":37,"path":"tests/selftest.c","role":"test"}],"impl_lines":621,"impl_code":591,"impl_comment":21,"test_lines":541,"comment_ratio":0.034,"tests_declared":6,"object":{"text_bytes":3949,"rodata_bytes":6336}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":127.32,"reference":3146.05,"speedup":24.71,"baselines":{"dav1d AVX2":141.26,"dav1d AVX-512ICL":53.88}},{"id":"find_dir","candidate":45.73,"reference":381.0,"speedup":8.332,"baselines":{"dav1d AVX2":39.77}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the shared suite, with a missing ABI test.","deepseek-v4.1-flash-max":"correct and fast, but the least documented of the working entries and visibly copy-pasted in the plane builder","gpt-6.1-sol-max":"Correct kernels; some avoidable work and incomplete ABI tests.","opus-5.5-xhigh":"Correct and fast, but nearly undocumented, with literal tables and no ABI tests","muse-wonder-full-xhigh":"bit-exact and fast; thin docs, unrolled repetition, no ABI test"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI hygiene is not exercised","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"One-line header, no register map, unexplained tap offsets","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"Plane construction is 45 lines of copy-paste","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No callee-saved-register test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Repeated left-edge setup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI test (callee-saved, dirty upper halves, XINUSE) in tests/selftest.c.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"The find_dir gather masks are magic immediates (src/cdef_avx512.asm:386-405), although the index tables beside them are generated.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"A guarded find_dir mismatch returns 0 with no message","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Sparse documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Hand-unrolled plane/store blocks with no-op shifts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No callee-saved/XINUSE/upper-bits test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0491,"lines":29,"significant":591,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.0,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":0.0}},"defect_points":2.6,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":34.81,"tokens":{"uncached_input":52,"cache_read":5992756,"cache_write":410536,"output":488359,"reasoning":449212},"cost_usd":1.4302,"cost_breakdown":{"input":1.9e-05,"cache_read":0.28828,"cache_write":0.366501,"output":0.775358},"cost_usd_as_run":1.4302,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":896,"code":710,"comment":143,"blank":43,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":764,"code":670,"comment":43,"blank":51,"path":"tests/selftest.c","role":"test"}],"impl_lines":896,"impl_code":710,"impl_comment":143,"test_lines":764,"comment_ratio":0.168,"tests_declared":7,"object":{"text_bytes":3989,"rodata_bytes":2944}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":144.29,"reference":3159.28,"speedup":21.895,"baselines":{"dav1d AVX2":141.72,"dav1d AVX-512ICL":54.42}},{"id":"find_dir","candidate":70.67,"reference":381.29,"speedup":5.395,"baselines":{"dav1d AVX2":40.14}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Outputs match the shared suite, but its guard tests leave important holes and zero-strength paths do extra work.","deepseek-v4.1-flash-max":"correct, well-tested, robust; the slowest filter of the working entries and ten redundant constant reloads in the hot path","gpt-6.1-sol-max":"Correct outputs, but unnecessary tap work and incomplete guard/ABI tests.","opus-5.5-xhigh":"Correct and robust; branchless filter and find_dir row loads leave real speed unused; no ABI tests","muse-wonder-full-xhigh":"bit-exact, but the slowest filter and it never skips a tap set"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests use an invalid stride","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Open guard pages conceal forbidden reads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Disabled tap families still run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Index constants reloaded for every plane","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Zero-strength families still execute every tap","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard setup permits forbidden right overreads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Filter always runs all 12 taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"find_dir loads its rows with 8 serially merged masked broadcasts","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong cost naming","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The filter prototype comment appears twice, the first copy sitting over the TAP macro","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Every guard case uses stride 16384","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Slowest filter of the correct entries","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Index vectors reloaded per plane pair","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No callee-saved/XINUSE/upper-bits test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Duplicated and wrong-sign comments","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0296,"lines":21,"significant":710,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":37,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.6,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0}},"defect_points":3.6,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.7,"tokens":{"uncached_input":37403,"cache_read":439680,"cache_write":0,"output":14798,"reasoning":3241},"cost_usd":1.5536,"cost_breakdown":{"input":0.37403,"cache_read":0.43968,"cache_write":0.0,"output":0.7399},"cost_usd_as_run":1.5536,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"lines":629,"code":603,"comment":20,"blank":6,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":501,"code":442,"comment":22,"blank":37,"path":"tests/selftest.c","role":"test"}],"impl_lines":629,"impl_code":603,"impl_comment":20,"test_lines":501,"comment_ratio":0.032,"tests_declared":6,"object":{"text_bytes":3626,"rodata_bytes":2693}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":126.86,"reference":3088.65,"speedup":24.347,"baselines":{"dav1d AVX2":138.49,"dav1d AVX-512ICL":52.91}},{"id":"find_dir","candidate":42.24,"reference":373.39,"speedup":8.84,"baselines":{"dav1d AVX2":39.18}}]},"scores":{"gpt-6.1-terra-max":7.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":7.0,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and fast in testing, but ABI coverage and table documentation are thinner","muse-wonder-full-xhigh":"bit-exact but tersest: copy-pasted gathers, magic tables, thinnest tests, slowest","gpt-6.1-sol-max":"Correct vectorized kernels; ABI regression coverage and filter-table documentation need improvement.","deepseek-v4.1-flash-max":"correct but the least tested and least documented of the four, and the slowest","opus-5.5-xhigh":"correct, but built from hand-written magic tables, thinly documented, with weak tests and no ABI coverage"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI and upper-state cleanup lack assertions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard test leaves row gaps readable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Filter tables lack their layout invariant","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Direction search reloads unchanged masks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Twenty copy-pasted find_dir gather blocks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Opaque magic tables and no layout documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Slowest on both functions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinnest added tests, no ABI coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI requirements lack explicit checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Filter tap metadata is undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"`tap_offsets` is 32 hand-computed records with no documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"No calling-convention test at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Guard pages only at `stride` 16","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Failures are silent","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`find_dir` is 18 % slower than the fastest entry","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"The padding-row mask is computed twice verbatim","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The tests never check the calling convention","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"style","title":"Tables and masks are hand-coded numbers with no derivation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Documentation is sparse","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Direction search does avoidable work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"It is consistently the slowest entry","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1177,"lines":71,"significant":603,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.3,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0}},"defect_points":4.8,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":24.07,"tokens":{"uncached_input":46,"cache_read":3564922,"cache_write":202701,"output":146466,"reasoning":103926},"cost_usd":5.2641,"cost_breakdown":{"input":0.000184,"cache_read":0.712984,"cache_write":1.621608,"output":2.92932},"cost_usd_as_run":5.2641,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"lines":905,"code":694,"comment":149,"blank":62,"path":"src/cdef_avx512.asm","role":"impl"},{"lines":1023,"code":894,"comment":61,"blank":68,"path":"tests/selftest.c","role":"test"}],"impl_lines":905,"impl_code":694,"impl_comment":149,"test_lines":1023,"comment_ratio":0.177,"tests_declared":9,"object":{"text_bytes":3226,"rodata_bytes":3264}},"hidden":{"passed":15,"total":15,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"filter","candidate":122.58,"reference":3169.28,"speedup":25.855,"baselines":{"dav1d AVX2":146.93,"dav1d AVX-512ICL":54.25}},{"id":"find_dir","candidate":38.55,"reference":377.13,"speedup":9.783,"baselines":{"dav1d AVX2":39.81}}]},"scores":{"gpt-6.1-terra-max":8.0,"muse-wonder-full-xhigh":9.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and well-explained, with a structured-test domain error","muse-wonder-full-xhigh":"bit-exact, well documented and strongly tested; two nits","gpt-6.1-sol-max":"Sound, well-documented assembly; two ABI test checks need correction.","deepseek-v4.1-flash-max":"excellent docs and structure, strong tests, a hair behind the leader","opus-5.5-xhigh":"correct, clean and well documented with broad tests; only the upper-state check is missing"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Structured direction tests use an unsupported stride","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Upper-state cleanup is not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard test leaves row gaps readable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Constant-block costs are not always zero","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Uncommented ones-materialisation trick","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"ABI check omits XINUSE","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Control-state verification compares post-call values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper-state cleanup is unchecked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Constant-block costs are not generally zero","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"ABI test does not check the upper vector state","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`vpternlogd zONES,zONES,zONES,0xff` + `vpabsb zONES,zONES`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Several physical registers carry multiple `%define` aliases whose live ranges must be tracked by hand","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"The guard-page fault handler `_exit(3)`s","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Nothing checks the upper state","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Direction-search pattern tests use an out-of-contract stride","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0202,"lines":14,"significant":694,"window":6},"complexity":null,"files":["src/cdef_avx512.asm"]},"isolation":{"tool_calls":22,"flags":1,"audited":"2026-11-04","reviewed":true},"conduct":{"severity":"minor","summary":"While working it ran `pkill -f 'build/selftest test'`, a host-wide pattern that also matches other contestants' running self-tests, not just its own. No other process is known to have been hit; scores stand (owner decision, 2026-11-02)."},"score":8.5,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":2.4,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1-b2","task":"exr-piz","lead":"gpt-6-astra-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":11.03,"tokens":{"cache_read":4370432,"cache_write":0,"output":26345,"reasoning":15137,"uncached_input":91672},"cost_usd":0.4419,"cost_breakdown":{"cache_read":0.218522,"cache_write":0.0,"input":0.091672,"output":0.131725},"cost_usd_as_run":0.4419,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":43.37,"tokens":{"cache_read":5424128,"cache_write":0,"output":51826,"reasoning":26723,"uncached_input":370970},"cost_usd":3.6052,"cost_breakdown":{"cache_read":1.084826,"cache_write":0.0,"input":1.48388,"output":1.03652},"cost_usd_as_run":3.6052,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":16.18,"tokens":{"cache_read":3840768,"cache_write":0,"output":30298,"reasoning":17723,"uncached_input":143849},"cost_usd":0.9748,"cost_breakdown":{"cache_read":0.384077,"cache_write":0.0,"input":0.287698,"output":0.30298},"cost_usd_as_run":0.9748,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":16.65,"tokens":{"cache_read":11612928,"cache_write":0,"output":126816,"reasoning":92942,"uncached_input":112649},"cost_usd":0.1278,"cost_breakdown":{"cache_read":0.034839,"cache_write":0.0,"input":0.016897,"output":0.07609},"cost_usd_as_run":0.1278,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":23.55,"tokens":{"cache_read":7200533,"cache_write":234593,"output":73533,"reasoning":41915,"uncached_input":92},"cost_usd":4.7879,"cost_breakdown":{"cache_read":1.440107,"cache_write":1.876744,"input":0.000368,"output":1.47066},"cost_usd_as_run":4.7879,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"D","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"V","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"D","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"V","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"D","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"V","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"D","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"V","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"D","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"V","score":9.0}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max","opus-5.5-xhigh"]},"entries":[{"letter":"I","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":8.85,"tokens":{"uncached_input":91475,"cache_read":736520,"cache_write":0,"output":51670,"reasoning":20928},"cost_usd":0.7733,"cost_breakdown":{"input":0.18295,"cache_read":0.073652,"cache_write":0.0,"output":0.5167},"cost_usd_as_run":0.7733,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":609,"code":545,"comment":12,"blank":52,"role":"impl"},{"path":"piz_test.go","lines":910,"code":807,"comment":29,"blank":74,"role":"test"}],"impl_lines":609,"impl_code":545,"impl_comment":12,"test_lines":910,"comment_ratio":0.022,"tests_declared":9},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":278.4,"reference":88.6,"speedup":3.142,"baselines":{"oracle":88.6}},{"id":"decode_float","candidate":68.8,"reference":42.5,"speedup":1.619,"baselines":{"oracle":42.5}}]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Misses the cancellation deadline on a permitted layout and has allocation and coverage gaps","muse-wonder-full-xhigh":"Correct and fastest data decode, marred by upfront 1.8 MB zeroing and copy-pasted refill logic","gpt-6.1-sol-max":"Correct decoding and useful decomposition, with the same cancellation defect and some duplication.","deepseek-v4.1-flash-max":"correct, quickest on large chunks, well documented; one big per-call allocation and duplicated refill logic","opus-5.5-xhigh":"correct and fast; fixed 1.8 MB per-call tables, a cancellation gap, duplicated refill code"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Poll across degenerate component planes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Defer and size Huffman scratch allocation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Exercise a populated full alphabet","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"The LUT is described backwards","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Document the exported error classes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"1.83 MB `hufTables` zeroed before block-length validation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Accumulator refill block duplicated three times","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No randomized or dense-table robustness coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Wavelet test comment states wrong dimensions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Decoder LUT called \"forward\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"No-op reserved-word read","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Cancellation skips one-sample planes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Every compressed call reserves maximum scratch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Bit-reservoir refill is copied three times","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"LUT direction is reversed in the comment","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"1.83 MB `hufTables` zeroed per PIZ decode","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"the byte-refill loop is written out three times","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"`ctx != nil` guards on every poll","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"discarded reserved-word read","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Wavelet channel loop never polls ctx when min(W,H)=1","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"1.8 MB of tables allocated and zeroed on every PIZ call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Codes longer than 10 bits restart the trie walk from the root","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Accumulator refill loop pasted three times","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Nothing tests the full 65,537-symbol alphabet that SPEC §5.1 requires","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"No-op read of the reserved word (piz.go:177) and an unreachable `sumOverflow` carry path (piz.go:80–88)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"`ctx != nil` guards in about eight places and a stored copy split into 64 KiB pieces with polls (piz.go:100–119)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"Build forward LUT\" (piz.go:154) builds the reverse (index→value) LUT","status":"valid","detail":"","location":""}],"score":7.4,"slop":{"score":1.8,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":2.5,"gpt-6.1-sol-max":1.5,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":2.5}},"defect_points":5.1,"score_cross_vendor":7.4,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","task":"exr-piz","lead":"gpt-6-astra-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":38.45,"tokens":{"cache_read":16358144,"cache_write":0,"output":99284,"reasoning":49918,"uncached_input":401549},"cost_usd":1.7158,"cost_breakdown":{"cache_read":0.817908,"cache_write":0.0,"input":0.401549,"output":0.49642},"cost_usd_as_run":1.7158,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":78.4,"tokens":{"cache_read":23500672,"cache_write":0,"output":118744,"reasoning":64773,"uncached_input":675629},"cost_usd":4.8887,"cost_breakdown":{"cache_read":2.350067,"cache_write":0.0,"input":1.351258,"output":1.18744},"cost_usd_as_run":4.8887,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":115.84,"tokens":{"cache_read":50391296,"cache_write":0,"output":500590,"reasoning":343230,"uncached_input":410748},"cost_usd":0.5131,"cost_breakdown":{"cache_read":0.151174,"cache_write":0.0,"input":0.061613,"output":0.300353},"cost_usd_as_run":0.5131,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":109.67,"tokens":{"cache_read":35279889,"cache_write":940281,"output":351215,"reasoning":228697,"uncached_input":336},"cost_usd":21.6038,"cost_breakdown":{"cache_read":7.055978,"cache_write":7.522248,"input":0.001344,"output":7.0243},"cost_usd_as_run":21.6038,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":110.32,"tokens":{"cache_read":23789472,"cache_write":0,"output":224383,"reasoning":130312,"uncached_input":880874},"cost_usd":12.769,"cost_breakdown":{"cache_read":4.757894,"cache_write":0.0,"input":3.523496,"output":4.48766},"cost_usd_as_run":12.769,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":5.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":5.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":6.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":5.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":6.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"A","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-gpt-6-astra-xhigh-r1","letter":"B","score":7.5}],"panel":["gpt-6.1-terra-max","gpt-6.1-sol-max","deepseek-v4.1-flash-max","opus-5.5-xhigh","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.01,"tokens":{"uncached_input":7093,"cache_read":102656,"cache_write":0,"output":3160,"reasoning":14},"cost_usd":0.3316,"cost_breakdown":{"input":0.07093,"cache_read":0.102656,"cache_write":0.0,"output":0.158},"cost_usd_as_run":0.3316,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":360,"code":348,"comment":2,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":77,"code":75,"comment":0,"blank":2,"role":"test"}],"impl_lines":360,"impl_code":348,"impl_comment":2,"test_lines":77,"comment_ratio":0.006,"tests_declared":2},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":125,"reference":88.5,"speedup":1.412,"baselines":{"oracle":88.5}},{"id":"decode_float","candidate":59.1,"reference":44.6,"speedup":1.325,"baselines":{"oracle":44.6}}]},"quality":{"duplication":{"ratio":0.0465,"lines":12,"significant":258,"window":6},"complexity":{"functions":3,"mean":33.33,"max":95,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.0,"deepseek-v4.1-flash-max":5.0,"gpt-6.1-sol-max":5.5,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":4.5},"council_summaries":{"gpt-6.1-terra-max":"Functional probes pass, but cancellation, performance, documentation, and coverage need work.","deepseek-v4.1-flash-max":"correct but the thinnest of the set: 2 comments in 360 lines, 2 tests, slowest (1.0–1.4×)","gpt-6.1-sol-max":"Golden decoding succeeds, but cancellation and verification coverage are insufficient.","muse-wonder-full-xhigh":"Correct minimal decoder whose tests and documentation are not commensurate with the format","opus-5.5-xhigh":"correct, but one 307-line undocumented function, bit-serial decoding and 2 tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Empty wavelet planes do not poll across channels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocation and bitwise decoding are paid for small tables","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Important contract cases have no coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public types and limit lack documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"essentially undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"2 tests / 77 lines","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"1.62 MB trie + 64 KiB lengths + 128 KiB LUT per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"bit-at-a-time decode","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`s := 2; if t == Half { s = 1 }` repeated three times","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Poll across channels without wavelet levels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Avoid maximum trie allocation for every chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Avoid per-bit traversal for ordinary codes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Add adversarial and resource tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Explain the public API and internal invariants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Minimal and partly masked tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Almost no documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Bit-serial state machine and fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Single-function implementation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"bit-at-a-time trie walk, one `read(1)` per loop step","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"fixed 135,000-node trie per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"style","title":"the whole decoder is a single 307-line `Decompress`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"no doc comments on `PixelType`, the constants, `Layout`, `MaxChunkBytes` or the sentinels","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"77 lines, 2 tests","status":"valid","detail":"","location":""}],"score":5.4,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":7.1,"score_cross_vendor":5.167,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":318.41,"tokens":{"uncached_input":270341,"cache_read":2759140,"cache_write":0,"output":70589,"reasoning":45547},"cost_usd":3.045,"cost_breakdown":{"input":1.081364,"cache_read":0.551828,"cache_write":0.0,"output":1.41178},"cost_usd_as_run":3.045,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":593,"code":542,"comment":23,"blank":28,"role":"impl"},{"path":"piz_test.go","lines":1038,"code":943,"comment":40,"blank":55,"role":"test"}],"impl_lines":593,"impl_code":542,"impl_comment":23,"test_lines":1038,"comment_ratio":0.041,"tests_declared":26},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":252,"reference":89.8,"speedup":2.806,"baselines":{"oracle":89.8}},{"id":"decode_float","candidate":95.4,"reference":45.4,"speedup":2.101,"baselines":{"oracle":45.4}}]},"quality":{"duplication":{"ratio":0.053,"lines":22,"significant":415,"window":6},"complexity":{"functions":5,"mean":26.0,"max":107,"over_10":2,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":28,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Broad functional coverage, but channel-heavy cancellation and trie allocation need attention.","deepseek-v4.1-flash-max":"correct and 2.2–2.5×, but a 450-line single function with almost no docs","gpt-6.1-sol-max":"Correct decoding, but cancellation can stall across large channel lists.","muse-wonder-full-xhigh":"Correct with very targeted tests, but implemented as one long function","opus-5.5-xhigh":"correct, 3.4× oracle, but a fixed trie, a 449-line function, and valid-path tests that barely exercise decoding"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-pixel component planes skip cancellation checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocation is paid on every compressed chunk","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"`Decompress` is one 450-line function (piz.go:63-511) doing layout validation, bitmap parse, table unpack, trie build, 10-bit lookup build, data decode, wavelet and serialisation inline","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"1.62 MB trie preallocated per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"bit-at-a-time data decode with a 10-bit prefix table","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Poll when a plane has no wavelet levels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Avoid reserving the maximum trie for small tables","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Consolidate the duplicated fallback walks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Monolithic `Decompress`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unused sentinel-node initialization","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"fixed 135,000-node trie per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"10-bit lookup, then a per-bit `consumeBit` closure walk for longer codes and for run counts","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`Decompress` is one 449-line function","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"the valid-path code-table tests decode only symbol 0","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"no doc comments on `tableBitReader`, `invPair` or `wavDecode`","status":"valid","detail":"","location":""}],"score":7.0,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":4.1,"score_cross_vendor":6.875,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.98,"tokens":{"uncached_input":20406,"cache_read":190336,"cache_write":0,"output":4043,"reasoning":173},"cost_usd":0.0501,"cost_breakdown":{"input":0.020406,"cache_read":0.009517,"cache_write":0.0,"output":0.020215},"cost_usd_as_run":0.0501,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":372,"code":362,"comment":0,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":98,"code":95,"comment":0,"blank":3,"role":"test"}],"impl_lines":372,"impl_code":362,"impl_comment":0,"test_lines":98,"comment_ratio":0,"tests_declared":3},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":128.5,"reference":85,"speedup":1.512,"baselines":{"oracle":85}},{"id":"decode_float","candidate":60.4,"reference":42.9,"speedup":1.408,"baselines":{"oracle":42.9}}]},"quality":{"duplication":{"ratio":0.0446,"lines":12,"significant":269,"window":6},"complexity":{"functions":3,"mean":34.33,"max":98,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.0,"deepseek-v4.1-flash-max":5.5,"gpt-6.1-sol-max":5.5,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":4.5},"council_summaries":{"gpt-6.1-terra-max":"The decoder handles the probes, but cancellation, performance, and coverage need work.","deepseek-v4.1-flash-max":"correct but undocumented and barely tested; a 372-line comment-free decoder","gpt-6.1-sol-max":"Ordinary decoding works, but cancellation, tests, and documentation need changes.","muse-wonder-full-xhigh":"Functionally correct compact decoder held back by absent docs, thin tests, and bit-serial speed","opus-5.5-xhigh":"correct, but one 320-line undocumented function, bit-serial decoding and thin tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Many one-pixel channels delay cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Every compressed chunk reserves a maximum trie and decodes bit by bit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Contract edge cases are largely untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Exported interface lacks documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no documentation at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"3 tests / 98 lines","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"1.62 MB trie + 128 KiB LUT + 64 KiB lengths per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"bit-at-a-time trie walk over the byte slice with a 4096-step context check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Poll across channels with no transform levels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Avoid allocating the maximum trie unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Add a common-code lookup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Cover crafted format and resource cases","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Document the API and codec invariants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Thin contract tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Bit-serial decode and fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Whole decoder in one function","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"bit-at-a-time Huffman decode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"fixed 135,000-node trie per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"style","title":"the whole decoder is a single 320-line `Decompress`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"no doc comments on any exported identifier","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"tests are thin","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"shadows the builtins `min`/`max`","status":"valid","detail":"","location":""}],"score":5.6,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":7.1,"score_cross_vendor":5.5,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.32,"tokens":{"uncached_input":15456,"cache_read":224768,"cache_write":0,"output":9848,"reasoning":1537},"cost_usd":0.8717,"cost_breakdown":{"input":0.15456,"cache_read":0.224768,"cache_write":0.0,"output":0.4924},"cost_usd_as_run":0.8717,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":401,"code":381,"comment":6,"blank":14,"role":"impl"},{"path":"piz_test.go","lines":501,"code":483,"comment":10,"blank":8,"role":"test"}],"impl_lines":401,"impl_code":381,"impl_comment":6,"test_lines":501,"comment_ratio":0.016,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":129.6,"reference":87.9,"speedup":1.474,"baselines":{"oracle":87.9}},{"id":"decode_float","candidate":60,"reference":44.1,"speedup":1.361,"baselines":{"oracle":44.1}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":285,"window":6},"complexity":{"functions":6,"mean":17.17,"max":45,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.0,"deepseek-v4.1-flash-max":6.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":5.0},"council_summaries":{"gpt-6.1-terra-max":"Correct decoding in the probes, but expensive allocation and a flaky race-test gate need changes.","deepseek-v4.1-flash-max":"correct code, but its own test suite fails the required `go test -race ./...` gate about 40 % of the time","gpt-6.1-sol-max":"Sound decoder behavior, but slower Huffman code, missing docs, and a failing race-test gate.","muse-wonder-full-xhigh":"Correct and targeted, but slow, sparsely documented, and with a flaky timing test","opus-5.5-xhigh":"the code is correct, but its own suite fails the `go test -race` gate in 7 of 10 runs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Allocates the full word buffer before validating the Huffman block","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Reserves a maximum-size trie for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Cancellation timing test intermittently fails the required race gate","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public types and limits lack documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"`TestCancellation` is flaky","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"1.62 MB trie per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"6 comment lines in 401.","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`work.check()` inside the per-bit trie walk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Avoid maximum trie allocation for tiny alphabets","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Replace per-bit decoding on the common path","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Synchronize the cancellation timing test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Document the public types and size limit","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Flaky cancellation wall-clock test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Missing exported-item documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Bit-serial decode and fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"a wall-clock cancellation assert fails most `-race` runs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"bit-at-a-time decode with `w.check()` per bit and per run word","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"fixed 135,000-node trie per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`PixelType`, its constants, `MaxChunkBytes` and the error sentinels have no doc comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"shadows the builtins `min`/`max`","status":"valid","detail":"","location":""}],"score":6.3,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":5.7,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":32.76,"tokens":{"uncached_input":68,"cache_read":6314260,"cache_write":236253,"output":185805,"reasoning":130126},"cost_usd":6.8692,"cost_breakdown":{"input":0.000272,"cache_read":1.262852,"cache_write":1.890024,"output":3.7161},"cost_usd_as_run":6.8692,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":770,"code":642,"comment":83,"blank":45,"role":"impl"},{"path":"piz_test.go","lines":1837,"code":1644,"comment":88,"blank":105,"role":"test"}],"impl_lines":770,"impl_code":642,"impl_comment":83,"test_lines":1837,"comment_ratio":0.114,"tests_declared":30},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":433.8,"reference":87,"speedup":4.986,"baselines":{"oracle":87}},{"id":"decode_float","candidate":144.7,"reference":42.8,"speedup":3.381,"baselines":{"oracle":42.8}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":493,"window":6},"complexity":{"functions":23,"mean":6.74,"max":18,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":36,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":9.5,"opus-5.5-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Fast and extensively tested, with a cancellation gap on channel-heavy chunks.","deepseek-v4.1-flash-max":"fastest (3.2–4.6×), best-tested on the pathological table rules, well documented","gpt-6.1-sol-max":"Efficient, carefully tested decoding with one large-channel cancellation gap.","muse-wonder-full-xhigh":"Fastest entry, with excellent documentation and the broadest tests","opus-5.5-xhigh":"correct, fast (6.9× oracle), small allocations, the most thorough tests and accurate docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"The component loop can run without polling","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`lutCapacity` is declared and never used","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Account for components without wavelet work","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Tautological trie-length assertion","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"codes longer than 14 bits are decoded one bit at a time","status":"valid","detail":"","location":""}],"score":8.5,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.375,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":16.28,"tokens":{"uncached_input":15154,"cache_read":593920,"cache_write":0,"output":35624,"reasoning":16016},"cost_usd":0.4459,"cost_breakdown":{"input":0.030308,"cache_read":0.059392,"cache_write":0.0,"output":0.35624},"cost_usd_as_run":0.4459,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":520,"code":479,"comment":14,"blank":27,"role":"impl"},{"path":"piz_test.go","lines":865,"code":815,"comment":15,"blank":35,"role":"test"}],"impl_lines":520,"impl_code":479,"impl_comment":14,"test_lines":865,"comment_ratio":0.028,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":266.1,"reference":87.1,"speedup":3.055,"baselines":{"oracle":87.1}},{"id":"decode_float","candidate":93.9,"reference":43.8,"speedup":2.144,"baselines":{"oracle":43.8}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":354,"window":6},"complexity":{"functions":14,"mean":9.36,"max":21,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Broad tests and prompt cancellation, with avoidable allocations.","deepseek-v4.1-flash-max":"correct with a clean helper decomposition; thin docs, fixed 1.6 MB trie, 1.45–2.57×","gpt-6.1-sol-max":"Correct, robust, and well tested, with oversized trie scratch.","muse-wonder-full-xhigh":"Clean lookup-accelerated decoder with strong tests and one wasteful fixed allocation","opus-5.5-xhigh":"correct, 3.3× oracle, but a fixed 1.6 MB trie on every call and sparse docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Allocates the word buffer before checking the Huffman header","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Small tables allocate the maximum trie","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"1.62 MB trie + 128 KiB LUT per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"18 comment lines in 520","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"bit-at-a-time trie walk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Size the trie from the parsed table","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed trie allocation and per-packet context calls","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"fixed 135,000-node trie allocated per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"10-bit lookup, then a bit-at-a-time walk","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"`ctx.Err()` on every code-table entry","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"internal functions have no doc comments","status":"valid","detail":"","location":""}],"score":7.8,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.8,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":40.18,"tokens":{"uncached_input":121458,"cache_read":2232328,"cache_write":0,"output":46175,"reasoning":25175},"cost_usd":1.8558,"cost_breakdown":{"input":0.485832,"cache_read":0.446466,"cache_write":0.0,"output":0.9235},"cost_usd_as_run":1.8558,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":526,"code":480,"comment":18,"blank":28,"role":"impl"},{"path":"piz_test.go","lines":975,"code":869,"comment":61,"blank":45,"role":"test"}],"impl_lines":526,"impl_code":480,"impl_comment":18,"test_lines":975,"comment_ratio":0.036,"tests_declared":19},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":106.5,"reference":83.6,"speedup":1.274,"baselines":{"oracle":83.6}},{"id":"decode_float","candidate":55.4,"reference":42.5,"speedup":1.304,"baselines":{"oracle":42.5}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":365,"window":6},"complexity":{"functions":10,"mean":11.7,"max":77,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":25,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":5.5},"council_summaries":{"gpt-6.1-terra-max":"Functional probes pass, but cancellation and the bitwise hot path need improvement.","deepseek-v4.1-flash-max":"correct; the only trie entry that sizes its allocation sensibly (0.36 MB/call), but 1.4–2.0× and thin docs","gpt-6.1-sol-max":"Correct decoding and useful tests, but a cancellation gap and branchy expansion remain.","muse-wonder-full-xhigh":"Correct and well tested, with a monolithic bit-serial decoder","opus-5.5-xhigh":"correct, but bit-serial decoding through closures, polling on every element, and a 330-line function"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-pixel planes skip polling across channels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Huffman data is consumed one bit at a time","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"22 comment lines in 526","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"bit-at-a-time Huffman decode","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Poll before skipping one-dimensional planes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Add a prefix lookup to Huffman expansion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-bit closures on hot paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Monolithic `decodePIZ`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable capacity guard","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"bit-at-a-time decode through the `readDataBit` closure","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"a `cc.poll()` call on every LUT value, table entry, symbol, wavelet block and output pixel","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`decodePIZ` is one 330-line function","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"no doc comments on `canceller`, `decodePIZ`, `invPair` or `waveletPlane`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"the random-input robustness test asserts nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`if capNodes < 1` can never be true","status":"valid","detail":"","location":""}],"score":6.9,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":3.4,"score_cross_vendor":6.625,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":27.88,"tokens":{"uncached_input":44,"cache_read":3574288,"cache_write":299526,"output":387291,"reasoning":351398},"cost_usd":0.9842,"cost_breakdown":{"input":1.5e-05,"cache_read":0.169288,"cache_write":0.26936,"output":0.545564},"cost_usd_as_run":0.9842,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":600,"code":514,"comment":53,"blank":33,"role":"impl"},{"path":"piz_test.go","lines":1046,"code":927,"comment":52,"blank":67,"role":"test"}],"impl_lines":600,"impl_code":514,"impl_comment":53,"test_lines":1046,"comment_ratio":0.093,"tests_declared":18},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":123.5,"reference":87.3,"speedup":1.415,"baselines":{"oracle":87.3}},{"id":"decode_float","candidate":60,"reference":43.7,"speedup":1.373,"baselines":{"oracle":43.7}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":385,"window":6},"complexity":{"functions":23,"mean":5.74,"max":12,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":39,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":6.0},"council_summaries":{"gpt-6.1-terra-max":"Well-tested and robust in the probes, but decoding performs avoidable allocations and bitwise work.","deepseek-v4.1-flash-max":"very good tests and docs, correct, but the slowest hot path and a 1.6 MB per-call allocation","gpt-6.1-sol-max":"Correct, robust, and extensively tested; Huffman decoding carries avoidable costs.","muse-wonder-full-xhigh":"Very clear and thoroughly tested, but the slowest hot path","opus-5.5-xhigh":"well structured, documented and tested, but bit-serial decoding with no lookup table"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Allocates the output-sized word buffer before checking the Huffman header","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Full trie allocation and bitwise decoding dominate small chunks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"1.62 MB code trie preallocated on every call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Huffman data and table reads are bit-at-a-time through the byte slice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`buildLUT` allocates a 128 KiB slice before it can know `k`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Scale trie allocation to the table","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Add a prefix lookup for ordinary codes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-bit Huffman walk and fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Zero-length symbols counted into an unused bucket","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"every symbol walks the trie one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"fixed `trieCapacity` (135,000) trie per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"the final `return ErrMalformed` in `insertCode` can never be reached","status":"valid","detail":"","location":""}],"score":7.4,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":2.9,"score_cross_vendor":7.75,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":13.74,"tokens":{"uncached_input":64818,"cache_read":335360,"cache_write":0,"output":28016,"reasoning":10610},"cost_usd":0.4433,"cost_breakdown":{"input":0.129636,"cache_read":0.033536,"cache_write":0.0,"output":0.28016},"cost_usd_as_run":0.4433,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":474,"code":445,"comment":12,"blank":17,"role":"impl"},{"path":"piz_test.go","lines":883,"code":842,"comment":9,"blank":32,"role":"test"}],"impl_lines":474,"impl_code":445,"impl_comment":12,"test_lines":883,"comment_ratio":0.026,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":224.7,"reference":87.3,"speedup":2.574,"baselines":{"oracle":87.3}},{"id":"decode_float","candidate":77.8,"reference":43.3,"speedup":1.797,"baselines":{"oracle":43.3}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":327,"window":6},"complexity":{"functions":8,"mean":15.38,"max":54,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Comprehensive coverage and sound decoding, with avoidable trie allocation.","deepseek-v4.1-flash-max":"correct; same shape as A with fewer tests and slightly lower throughput (1.5–2.1×)","gpt-6.1-sol-max":"Correct and thoroughly tested, with unnecessarily large Huffman scratch.","muse-wonder-full-xhigh":"Solid structured lookup decoder with strong tests and a fixed-size trie cost","opus-5.5-xhigh":"correct, 2.8× oracle, but a fixed trie, per-bit peeks, and sparse docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum-size trie is allocated for small tables","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"1.62 MB trie per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"15 comment lines in 474","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"bit-at-a-time walk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Allocate the trie for the actual alphabet","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"fixed 135,000-node trie per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"`peek(10)` assembles 10 bits through a per-byte loop for every symbol, and longer codes go bit by bit through `read(1)`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"internal functions undocumented","status":"valid","detail":"","location":""}],"score":7.5,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.2,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.06,"tokens":{"uncached_input":12330,"cache_read":398976,"cache_write":0,"output":23674,"reasoning":7526},"cost_usd":1.706,"cost_breakdown":{"input":0.1233,"cache_read":0.398976,"cache_write":0.0,"output":1.1837},"cost_usd_as_run":1.706,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":458,"code":421,"comment":12,"blank":25,"role":"impl"},{"path":"piz_test.go","lines":777,"code":736,"comment":9,"blank":32,"role":"test"}],"impl_lines":458,"impl_code":421,"impl_comment":12,"test_lines":777,"comment_ratio":0.028,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":225,"reference":87.7,"speedup":2.566,"baselines":{"oracle":87.7}},{"id":"decode_float","candidate":79.9,"reference":42.8,"speedup":1.867,"baselines":{"oracle":42.8}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":312,"window":6},"complexity":{"functions":7,"mean":16.43,"max":53,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Strong coverage and cancellation handling, with unnecessary trie allocation.","deepseek-v4.1-flash-max":"correct, 1.6–2.0×; stack-sized 128 KiB LUT + 64 KiB lengths plus a fixed trie","gpt-6.1-sol-max":"Correct and well covered, with excessive fixed Huffman allocation.","muse-wonder-full-xhigh":"Solid lookup decoder with strong edge tests and only minor waste","opus-5.5-xhigh":"correct, 2.9× oracle, but a fixed trie, a slow per-symbol peek, and sparse docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Small code tables still allocate the maximum trie","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"1.62 MB trie (`piz.go:283`) plus `var lut [65536]uint16` and `var lengths [65537]uint8` zeroed on entry (piz.go:84, 235)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"17 comment lines in 458","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`count[length]++` also counts length 0","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Allocate only the required trie capacity","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Zero-length symbols counted into an unused bucket","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"fixed 135,000-node trie per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"each symbol reads 10 bits through the generic loop, then rewinds `data.pos -= 10 - entry.bits`, and codes over 10 bits walk bit by bit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"internal functions undocumented","status":"valid","detail":"","location":""}],"score":7.6,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":2.2,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":60.93,"tokens":{"uncached_input":325315,"cache_read":14413632,"cache_write":0,"output":201000,"reasoning":154234},"cost_usd":0.5817,"cost_breakdown":{"input":0.048797,"cache_read":0.432409,"cache_write":0.0,"output":0.1005},"cost_usd_as_run":0.5817,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":614,"code":521,"comment":61,"blank":32,"role":"impl"},{"path":"piz_test.go","lines":1345,"code":1132,"comment":128,"blank":85,"role":"test"}],"impl_lines":614,"impl_code":521,"impl_comment":61,"test_lines":1345,"comment_ratio":0.105,"tests_declared":23},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":124.2,"reference":89,"speedup":1.396,"baselines":{"oracle":89}},{"id":"decode_float","candidate":60.5,"reference":44.2,"speedup":1.369,"baselines":{"oracle":44.2}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":421,"window":6},"complexity":{"functions":19,"mean":7.79,"max":21,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":69,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct on checked streams, with slow cancellation on wide output","gpt-6.1-sol-max":"Correct decoding, but cancellation and test coverage need changes","opus-5.5-xhigh":"correct and well documented, but bit-at-a-time decoding (~113 MB/s) and coarse output polling","muse-wonder-full-xhigh":"correct and well documented, cancellation gap and slow decoder","deepseek-v4.1-flash-max":"clear and well documented at ~oracle speed"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Serialization checks only every 256 rows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocated for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Error conditions are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Serialization does not poll within a row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie allocated for small tables","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Missing-repeat test exits before Huffman validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"“Incomplete” table is complete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time Huffman decode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Cancel can exceed 10 ms","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Output loop polls only per row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Bit-at-a-time decode and fixed-size tables","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"1.35 MB trie allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`parseChunk` returns the 8192-byte bitmap by value","status":"valid","detail":"","location":""}],"score":6.9,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.7,"score_cross_vendor":6.9,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.32,"tokens":{"uncached_input":23862,"cache_read":331776,"cache_write":0,"output":9542,"reasoning":2022},"cost_usd":0.0882,"cost_breakdown":{"input":0.023862,"cache_read":0.016589,"cache_write":0.0,"output":0.04771},"cost_usd_as_run":0.0882,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":389,"code":377,"comment":2,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":441,"code":425,"comment":7,"blank":9,"role":"test"}],"impl_lines":389,"impl_code":377,"impl_comment":2,"test_lines":441,"comment_ratio":0.005,"tests_declared":8},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":108.2,"reference":86.3,"speedup":1.254,"baselines":{"oracle":86.3}},{"id":"decode_float","candidate":55.3,"reference":43.9,"speedup":1.26,"baselines":{"oracle":43.9}}]},"quality":{"duplication":{"ratio":0.0427,"lines":12,"significant":281,"window":6},"complexity":{"functions":3,"mean":35.0,"max":99,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":5.0,"muse-wonder-full-xhigh":6.5,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct on checked streams, but cancellation and coverage need work","gpt-6.1-sol-max":"Correct decoding with cancellation and boundary-test weaknesses","opus-5.5-xhigh":"correct, but the slowest entry (≈93 MB/s); undocumented monolith","muse-wonder-full-xhigh":"correct, but slowest hot path and undocumented exports","deepseek-v4.1-flash-max":"correct, but the slowest entry and the thinnest docs after P"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-dimensional channel loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocated for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"“Full alphabet” test is sparse","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API types and errors lack documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate planes bypass polling","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie reserved on every decode","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"LUT-threshold test never executes a wavelet level","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public types and decoding invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Context checked per symbol","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported API undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Monolithic `Decompress`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Weak cancellation test, no round trips","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Per-symbol context check and bit-at-a-time trie decode","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Exported API undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"context checked once per Huffman symbol","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"1.35 MB trie allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"public API undocumented","status":"valid","detail":"","location":""}],"score":6.1,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.4,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":15.93,"tokens":{"uncached_input":80364,"cache_read":10477312,"cache_write":0,"output":169513,"reasoning":124472},"cost_usd":0.1452,"cost_breakdown":{"input":0.012055,"cache_read":0.031432,"cache_write":0.0,"output":0.101708},"cost_usd_as_run":0.1452,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1452,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":579,"code":501,"comment":42,"blank":36,"role":"impl"},{"path":"piz_test.go","lines":1292,"code":1110,"comment":92,"blank":90,"role":"test"}],"impl_lines":579,"impl_code":501,"impl_comment":42,"test_lines":1292,"comment_ratio":0.077,"tests_declared":26},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":212.8,"reference":87.3,"speedup":2.438,"baselines":{"oracle":87.3}},{"id":"decode_float","candidate":56.5,"reference":44.6,"speedup":1.267,"baselines":{"oracle":44.6}}]},"quality":{"duplication":{"ratio":0.0313,"lines":12,"significant":383,"window":6},"complexity":{"functions":9,"mean":13.33,"max":78,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":73,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Broad coverage, with a cancellation gap and fixed trie cost","gpt-6.1-sol-max":"Correct decoding, with cancellation and allocation problems","opus-5.5-xhigh":"correct, mid-speed (~220 MB/s), thorough tests; one 320-line function","muse-wonder-full-xhigh":"correct and well tested, fixed 1.9 MB/call trie cost","deepseek-v4.1-flash-max":"correct, well documented; fixed-capacity buffers per call"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-dimensional channel loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocated for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Sentinel conditions are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Channel traversal can exceed cancellation deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie allocated for every table","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Monolithic `decodePIZ`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Avoidable work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unused branch","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Full-size lengths and trie per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Comment states 65536 is the repeat code","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Error sentinels undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Repeated w14 branch blocks in the wavelet","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"1.35 MB trie allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`bitmap` heap array (piz.go:114) and always-65537 `lengths` (piz.go:175).","status":"valid","detail":"","location":""}],"score":7.3,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":2.9,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.04,"tokens":{"uncached_input":30,"cache_read":1141597,"cache_write":90253,"output":56999,"reasoning":22025},"cost_usd":2.0904,"cost_breakdown":{"input":0.00012,"cache_read":0.228319,"cache_write":0.722024,"output":1.13998},"cost_usd_as_run":2.0904,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":582,"code":526,"comment":25,"blank":31,"role":"impl"},{"path":"piz_test.go","lines":1013,"code":910,"comment":53,"blank":50,"role":"test"}],"impl_lines":582,"impl_code":526,"impl_comment":25,"test_lines":1013,"comment_ratio":0.045,"tests_declared":17},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":300.1,"reference":88.2,"speedup":3.402,"baselines":{"oracle":88.2}},{"id":"decode_float","candidate":93.6,"reference":44.3,"speedup":2.113,"baselines":{"oracle":44.3}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":401,"window":6},"complexity":{"functions":11,"mean":11.91,"max":63,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":10.0,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Solid decoder with minor allocation and documentation costs","gpt-6.1-sol-max":"Complete decoder with strong tests and bounded work","opus-5.5-xhigh":"correct, fast (~285 MB/s), proportional trie, strong tests","muse-wonder-full-xhigh":"correct and fast, well tested, one extra pass","deepseek-v4.1-flash-max":"correct, fast, memory-frugal"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"LUT always reserves the full value space","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Sentinel conditions are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Indirect call in the hottest loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Inaccurate comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`goto`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Separate LUT-range validation pass","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Error sentinels undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"output words scanned twice","status":"valid","detail":"","location":""}],"score":8.6,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.01,"tokens":{"uncached_input":35291,"cache_read":169856,"cache_write":0,"output":3330,"reasoning":91},"cost_usd":0.0604,"cost_breakdown":{"input":0.035291,"cache_read":0.008493,"cache_write":0.0,"output":0.01665},"cost_usd_as_run":0.0604,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":360,"code":350,"comment":0,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":57,"code":55,"comment":0,"blank":2,"role":"test"}],"impl_lines":360,"impl_code":350,"impl_comment":0,"test_lines":57,"comment_ratio":0,"tests_declared":1},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":118,"reference":84.4,"speedup":1.398,"baselines":{"oracle":84.4}},{"id":"decode_float","candidate":56.1,"reference":43,"speedup":1.305,"baselines":{"oracle":43}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":257,"window":6},"complexity":{"functions":4,"mean":25.5,"max":95,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":5.0,"opus-5.5-xhigh":4.5,"muse-wonder-full-xhigh":5.5,"deepseek-v4.1-flash-max":5.5},"council_summaries":{"gpt-6.1-terra-max":"Basic decoding passes, but cancellation and test coverage need work","gpt-6.1-sol-max":"Functional decoder with substantial testing and maintenance gaps","opus-5.5-xhigh":"correct, but slow, wholly undocumented, and tested only on the golden files","muse-wonder-full-xhigh":"correct on probes, but golden-only tests and no docs","deepseek-v4.1-flash-max":"correct but least documented and least tested"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-dimensional channel loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Only golden vectors are tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocated for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API lacks documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Context is checked twice consecutively","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate planes bypass context checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Only golden fixtures are tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API and transform invariants lack documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Duplicate cancellation check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Context checked per symbol","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Nothing documented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Golden test only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Redundant code","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Only a golden-vector test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-symbol context check, per-bit trie walk, fixed trie","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicate context check and reinvented `min`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no comments at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"only the golden replay","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"the second, quantised ctx check can never add value","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"1.35 MB trie allocated unconditionally","status":"valid","detail":"","location":""}],"score":5.2,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5}},"defect_points":6.5,"score_cross_vendor":5.167,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":70.71,"tokens":{"uncached_input":191191,"cache_read":4318336,"cache_write":0,"output":310836,"reasoning":141157},"cost_usd":4.4066,"cost_breakdown":{"input":0.382382,"cache_read":2.159168,"cache_write":0.0,"output":1.865016},"cost_usd_as_run":4.4066,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"piz.go","lines":703,"code":655,"comment":14,"blank":34,"role":"impl"},{"path":"piz_test.go","lines":1053,"code":973,"comment":36,"blank":44,"role":"test"}],"impl_lines":703,"impl_code":655,"impl_comment":14,"test_lines":1053,"comment_ratio":0.021,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":219.8,"reference":88.6,"speedup":2.481,"baselines":{"oracle":88.6}},{"id":"decode_float","candidate":78.6,"reference":44.8,"speedup":1.754,"baselines":{"oracle":44.8}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":29,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct on checked streams, with a cancellation gap and fixed trie cost","gpt-6.1-sol-max":"Correct output with cancellation and allocation overhead","opus-5.5-xhigh":"correct, mid-speed, but no helper is documented and there are several unreachable checks","muse-wonder-full-xhigh":"correct, slower poll-heavy design, some dead code","deepseek-v4.1-flash-max":"correct; slowest of the fast-lookup group"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-dimensional channel loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocated for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Sentinel conditions are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"No polling across degenerate component planes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie allocated for every chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Existing-node success arm is unreachable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Avoidable overhead","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Helpers undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"LUT built in two passes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Poll function on every loop iteration and extra passes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable checks in trie building","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Error sentinels and most internal functions undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"1.35 MB trie allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`bl < 0` can never hold","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"reverse LUT built in two full 65536-value passes","status":"valid","detail":"","location":""}],"score":6.8,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":0.5}},"defect_points":3.2,"score_cross_vendor":6.8,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.89,"tokens":{"uncached_input":11832,"cache_read":307328,"cache_write":0,"output":12225,"reasoning":2690},"cost_usd":0.1766,"cost_breakdown":{"input":0.023664,"cache_read":0.030733,"cache_write":0.0,"output":0.12225},"cost_usd_as_run":0.1766,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":447,"code":427,"comment":6,"blank":14,"role":"impl"},{"path":"piz_test.go","lines":612,"code":590,"comment":11,"blank":11,"role":"test"}],"impl_lines":447,"impl_code":427,"impl_comment":6,"test_lines":612,"comment_ratio":0.014,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":192.6,"reference":85.1,"speedup":2.263,"baselines":{"oracle":85.1}},{"id":"decode_float","candidate":72,"reference":44.3,"speedup":1.625,"baselines":{"oracle":44.3}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":321,"window":6},"complexity":{"functions":6,"mean":19.0,"max":54,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Good edge tests, with a cancellation gap and fixed trie cost","gpt-6.1-sol-max":"Correct core decoding with a cancellation gap","opus-5.5-xhigh":"correct with compact, targeted tests; heavy per-element polling overhead and sparse docs","muse-wonder-full-xhigh":"correct with solid tests, partial docs, fixed trie cost","deepseek-v4.1-flash-max":"correct and compact; thin docs, size-independent trie"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-dimensional channel loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocated for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Limit and sentinel conditions are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate planes bypass cancellation polling","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie reserved unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public format semantics are underspecified","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Poll call on every element","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Missing and inaccurate docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Builtins shadowed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Slice returned with an error","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Half the exports undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135,000-node trie per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"1.35 MB trie allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"sparse","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`return out, ctx.Err()`","status":"valid","detail":"","location":""}],"score":6.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.5,"score_cross_vendor":6.833,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":20.46,"tokens":{"uncached_input":34,"cache_read":1782878,"cache_write":141667,"output":103669,"reasoning":40876},"cost_usd":3.5634,"cost_breakdown":{"input":0.000136,"cache_read":0.356576,"cache_write":1.133336,"output":2.07338},"cost_usd_as_run":3.5634,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":551,"code":489,"comment":29,"blank":33,"role":"impl"},{"path":"piz_test.go","lines":1571,"code":1409,"comment":79,"blank":83,"role":"test"}],"impl_lines":551,"impl_code":489,"impl_comment":29,"test_lines":1571,"comment_ratio":0.056,"tests_declared":26},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":291.3,"reference":89,"speedup":3.273,"baselines":{"oracle":89}},{"id":"decode_float","candidate":95.1,"reference":44.4,"speedup":2.142,"baselines":{"oracle":44.4}}]},"quality":{"duplication":{"ratio":0.032,"lines":12,"significant":375,"window":6},"complexity":{"functions":15,"mean":7.87,"max":30,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":16,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong implementation and extensive edge coverage","gpt-6.1-sol-max":"Robust implementation with one masked collision test","opus-5.5-xhigh":"correct, fast (~277 MB/s), proportional trie, very thorough tests","muse-wonder-full-xhigh":"correct, exemplary tests and polling, middling speed","deepseek-v4.1-flash-max":"correct, fast, most allocation-frugal (3 KB per call)"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Sentinel conditions are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Large collision fixture bypasses table validation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Per-element poll calls","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Slow suite","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-symbol prefix reload and second full LUT scan","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Error sentinels undocumented","status":"valid","detail":"","location":""}],"score":8.6,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.1,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.49,"tokens":{"uncached_input":29372,"cache_read":167552,"cache_write":0,"output":6721,"reasoning":570},"cost_usd":0.1427,"cost_breakdown":{"input":0.058744,"cache_read":0.016755,"cache_write":0.0,"output":0.06721},"cost_usd_as_run":0.1427,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":387,"code":372,"comment":3,"blank":12,"role":"impl"},{"path":"piz_test.go","lines":391,"code":380,"comment":5,"blank":6,"role":"test"}],"impl_lines":387,"impl_code":372,"impl_comment":3,"test_lines":391,"comment_ratio":0.008,"tests_declared":7},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":123.1,"reference":88.5,"speedup":1.391,"baselines":{"oracle":88.5}},{"id":"decode_float","candidate":58.6,"reference":44.1,"speedup":1.329,"baselines":{"oracle":44.1}}]},"quality":{"duplication":{"ratio":0.0435,"lines":12,"significant":276,"window":6},"complexity":{"functions":5,"mean":20.8,"max":86,"over_10":2,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":5.0,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Robust checked behavior, with avoidable allocation and test gaps","gpt-6.1-sol-max":"Correct and cancellation-aware, with avoidable allocation","opus-5.5-xhigh":"correct but slow (~109 MB/s), undocumented, partly self-confirming tests","muse-wonder-full-xhigh":"correct and compact, slow bit reader, no export docs","deepseek-v4.1-flash-max":"correct and compact, under-documented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocated for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Wavelet boundary test shares the implementation under test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API types and errors lack documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie allocated for small tables","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public format contract is undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Poll call per bit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported API undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Self-confirming wavelet test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Monolithic `Decompress`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Bit-by-bit reader and fixed-size tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Exported API undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"sparse","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"1.35 MB trie allocated unconditionally","status":"valid","detail":"","location":""}],"score":6.9,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.0,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":83.26,"tokens":{"uncached_input":354,"cache_read":10866766,"cache_write":324500,"output":278335,"reasoning":0},"cost_usd":0.3696,"cost_breakdown":{"input":5.3e-05,"cache_read":0.173868,"cache_write":0.0649,"output":0.130817},"cost_usd_as_run":0.3696,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.3696,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":688,"code":601,"comment":53,"blank":34,"role":"impl"},{"path":"piz_test.go","lines":1412,"code":1256,"comment":78,"blank":78,"role":"test"}],"impl_lines":688,"impl_code":601,"impl_comment":53,"test_lines":1412,"comment_ratio":0.081,"tests_declared":20},"hidden":{"passed":4,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":262.6,"reference":88.1,"speedup":2.981,"baselines":{"oracle":88.1}},{"id":"decode_float","candidate":94,"reference":45,"speedup":2.089,"baselines":{"oracle":45}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":59,"flags":1,"audited":"2026-11-02","reviewed":true},"conduct":{"severity":"minor","summary":"While checking its own files it ran `find` over the folder that holds the other contestants' entries (recently changed file names only) and listed a cache folder outside its own. It read no other entry's files; scores stand (owner policy for gainless deviations, 2026-11-02)."},"scores":{"gpt-6.1-terra-max":3.5,"gpt-6.1-sol-max":4.0,"opus-5.5-xhigh":4.0,"muse-wonder-full-xhigh":4.0,"deepseek-v4.1-flash-max":5.5},"council_summaries":{"gpt-6.1-terra-max":"A bitmap edge produces wrong output and can panic","gpt-6.1-sol-max":"Valid bitmaps can corrupt output or panic","opus-5.5-xhigh":"counts bitmap bit 0 as a second zero: wrong output, and a panic on a full bitmap","muse-wonder-full-xhigh":"wrong output whenever bitmap bit 0 is set","deepseek-v4.1-flash-max":"the only entry with wrong output"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Zero is inserted twice into the LUT","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Bitmap builder excludes the failing bit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Maximum trie allocated for every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Sentinel conditions are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Wavelet threshold comment is reversed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Explicit zero is inserted twice into the LUT","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie allocated before structural validation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Wavelet-mode explanation is reversed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"LUT counts value 0 twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Tests never set bit 0","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Very frequent polling and a fixed trie","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Odd identifiers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"LUT duplicates value 0 when bitmap bit 0 is set","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Crafted-chunk helper never sets bitmap bit 0","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Context polled every 16 work units and fixed trie","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Error sentinels undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"value 0 inserted twice when bitmap bit 0 is set","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"\"the zero value appears once\"","status":"valid","detail":"","location":""}],"score":4.2,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0}},"defect_points":6.5,"score_cross_vendor":4.2,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":369.2,"tokens":{"uncached_input":426805,"cache_read":3595240,"cache_write":0,"output":62583,"reasoning":34123},"cost_usd":3.6779,"cost_breakdown":{"input":1.70722,"cache_read":0.719048,"cache_write":0.0,"output":1.25166},"cost_usd_as_run":3.6779,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":526,"code":479,"comment":23,"blank":24,"role":"impl"},{"path":"piz_test.go","lines":1104,"code":1014,"comment":37,"blank":53,"role":"test"}],"impl_lines":526,"impl_code":479,"impl_comment":23,"test_lines":1104,"comment_ratio":0.046,"tests_declared":16},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":142.3,"reference":89.7,"speedup":1.586,"baselines":{"oracle":89.7}},{"id":"decode_float","candidate":66,"reference":45.3,"speedup":1.457,"baselines":{"oracle":45.3}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":350,"window":6},"complexity":{"functions":5,"mean":22.8,"max":89,"over_10":2,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":35,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks, with avoidable per-chunk allocation.","gpt-6.1-sol-max":"Correct and robust, with substantial Huffman overhead and one unreachable check.","opus-5.5-xhigh":"correct with good tests; slow bit-serial decoder and dead branches","deepseek-v4.1-flash-max":"correct, fastest of the trie-based decoders on large chunks, and well tested; a fixed 1.62 MB allocation and two small warts.","muse-wonder-full-xhigh":"Correct, fastest of the trie entries, well tested; only fixed-cost allocation and small slop items"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Worst-case trie allocated for every chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Unreachable bitmap check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Huffman decoding walks every bit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unreachable short-header branch","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman decode walks the trie one bit per step","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"380-line monolithic `Decompress`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable or always-true branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Cancellation tests hard-code exact poll counts","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"fixed 1.62 MB trie and 128 KiB LUT per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"unreachable branch","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"transposed block-corner names","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135,000-node trie allocation and per-bit trie walk","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable length re-check in bitmap parsing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"No-op plane-count ctx poll and always-true wavelet edge guards","status":"valid","detail":"","location":""}],"score":7.6,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":2.8,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.02,"tokens":{"uncached_input":9686,"cache_read":660608,"cache_write":0,"output":28801,"reasoning":13367},"cost_usd":0.1867,"cost_breakdown":{"input":0.009686,"cache_read":0.03303,"cache_write":0.0,"output":0.144005},"cost_usd_as_run":0.1867,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":467,"code":430,"comment":13,"blank":24,"role":"impl"},{"path":"piz_test.go","lines":665,"code":622,"comment":16,"blank":27,"role":"test"}],"impl_lines":467,"impl_code":430,"impl_comment":13,"test_lines":665,"comment_ratio":0.029,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":257.6,"reference":87.1,"speedup":2.958,"baselines":{"oracle":87.1}},{"id":"decode_float","candidate":94.4,"reference":43.9,"speedup":2.15,"baselines":{"oracle":43.9}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":322,"window":6},"complexity":{"functions":6,"mean":19.5,"max":54,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":16,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Strong correctness and cancellation coverage, with avoidable allocation overhead.","gpt-6.1-sol-max":"Strong contract coverage and bounded cancellation; oversized Huffman scratch adds avoidable cost.","opus-5.5-xhigh":"correct and reasonably fast; trie-based decoder with per-word overhead, thin internal docs","deepseek-v4.1-flash-max":"correct, compact and well tested; fixed allocation plus two nits.","muse-wonder-full-xhigh":"Correct and well decomposed, but slower than the oracle with high fixed allocation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Worst-case trie allocated for every chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Polling comment overstates coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated for tiny tables","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Trie decoder with per-word function-call polling","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public constants and internals undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"fixed 1.62 MB trie and 128 KiB LUT per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`checkWork` is called once per bit during trie construction","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the three PixelType constants are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Fixed ~1.83 MB allocation per call and per-bit/per-word ctx helper in hot loops","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No random-valid or encoder round-trip coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Exported constants and errors undocumented","status":"valid","detail":"","location":""}],"score":8.0,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.3,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1-b2","task":"exr-piz","lead":"gpt-6.1-sol-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":12.92,"tokens":{"cache_read":4244480,"cache_write":0,"output":29464,"reasoning":17464,"uncached_input":98324},"cost_usd":0.4579,"cost_breakdown":{"cache_read":0.212224,"cache_write":0.0,"input":0.098324,"output":0.14732},"cost_usd_as_run":0.4579,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":14.73,"tokens":{"cache_read":3431680,"cache_write":0,"output":27241,"reasoning":14474,"uncached_input":123846},"cost_usd":0.8633,"cost_breakdown":{"cache_read":0.343168,"cache_write":0.0,"input":0.247692,"output":0.27241},"cost_usd_as_run":0.8633,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":18.85,"tokens":{"cache_read":5401504,"cache_write":0,"output":62334,"reasoning":35244,"uncached_input":203253},"cost_usd":3.14,"cost_breakdown":{"cache_read":1.080301,"cache_write":0.0,"input":0.813012,"output":1.24668},"cost_usd_as_run":3.14,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":25.77,"tokens":{"cache_read":6593180,"cache_write":223880,"output":74264,"reasoning":44071,"uncached_input":82},"cost_usd":4.5953,"cost_breakdown":{"cache_read":1.318636,"cache_write":1.79104,"input":0.000328,"output":1.48528},"cost_usd_as_run":4.5953,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":28.54,"tokens":{"cache_read":18384256,"cache_write":0,"output":161218,"reasoning":122729,"uncached_input":102800},"cost_usd":0.1673,"cost_breakdown":{"cache_read":0.055153,"cache_write":0.0,"input":0.01542,"output":0.096731},"cost_usd_as_run":0.1673,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"X","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"W","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"K","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"X","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"W","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"K","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"X","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"W","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"K","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"X","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"W","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"K","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"X","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"W","score":9.0}],"panel":["gpt-6.1-terra-max","gpt-6.1-sol-max","muse-wonder-full-xhigh","opus-5.5-xhigh","deepseek-v4.1-flash-max"]},"entries":[{"letter":"X","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":14.52,"tokens":{"uncached_input":130275,"cache_read":1071561,"cache_write":0,"output":87817,"reasoning":41305},"cost_usd":1.2459,"cost_breakdown":{"input":0.26055,"cache_read":0.107156,"cache_write":0.0,"output":0.87817},"cost_usd_as_run":1.2459,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":529,"code":475,"comment":9,"blank":45,"role":"impl"},{"path":"piz_test.go","lines":983,"code":863,"comment":46,"blank":74,"role":"test"}],"impl_lines":529,"impl_code":475,"impl_comment":9,"test_lines":983,"comment_ratio":0.019,"tests_declared":10},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":144.8,"reference":90.3,"speedup":1.604,"baselines":{"oracle":90.3}},{"id":"decode_float","candidate":67.1,"reference":45.2,"speedup":1.485,"baselines":{"oracle":45.2}}]},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checked cases, with avoidable allocation and minor test and documentation gaps","gpt-6.1-sol-max":"Compatible behavior and substantial tests, but excessive scratch allocation and a monolithic implementation.","muse-wonder-full-xhigh":"oracle-correct and well tested with an encoder, but one 464-line function and the same trie preallocation","opus-5.5-xhigh":"correct and robust, but bit-at-a-time Huffman decoding and one monolithic 465-line function","deepseek-v4.1-flash-max":"correct and second-fastest (1.4–2.0× the oracle) with a strong suite, but its decoder internals are undocumented and it over-allocates the trie."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Always reserves the maximum trie","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Mutation test performs only truncations","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Exported error classes lack documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Incorrect run-symbol number","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Every table reserves the largest trie","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"style","title":"Entire pipeline occupies one large function","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Incorrect run-symbol description","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Full-size trie backing allocated on every decode","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Stored-chunk copy never polls the context","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"The whole decoder is one 464-line function with duplicated boilerplate","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"`TestTruncationsAndMutations` contains no mutations and only samples truncations","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Cancellation test can pass vacuously and is wall-clock flaky","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman decoding walks the trie one bit per loop step, with no lookup table","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The trie is always preallocated at 135,000 nodes (1.6 MB, zeroed)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`Decompress` holds every stage in one 465-line body, with six pasted copies of the `ops++; if ops >= 4096 {…}` poll block, about ten `if ctx != nil` guards, and the four-line wavelet step written twice for 14-/16-bit mode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No comments on the invariants: why 135,000 nodes is enough, why `bitmap[0] |= 1`, what `ops` means","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The synthetic round trips only use sample values 0..3, so the 16-bit wavelet path is covered only by the one golden chunk","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"`TestTruncationsAndMutations` contains no mutations, only truncations","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The 58-bit test comment says \"sym 59 is RLE\", but the run-length symbol is 58. The table also carries a 60th packet past iM, and the test passes only because 354 and 360 bits both round to 45 bytes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"135000-node trie allocated per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"decoder invariants unexplained","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`if ctx != nil` at every poll site (piz.go:101, 236, 296, 382, 411, 450, 472, 484, 495, 514)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`Decompress` is a single 464-line function (piz.go:65-529) with table parse, trie build, bit decode, wavelet, LUT map and pack inline","status":"valid","detail":"","location":""}],"isolation":{"tool_calls":18,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.5,"score_cross_vendor":7.8,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","task":"exr-piz","lead":"gpt-6.1-sol-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":39.06,"tokens":{"cache_read":19878400,"cache_write":0,"output":109375,"reasoning":55109,"uncached_input":440052},"cost_usd":1.9809,"cost_breakdown":{"cache_read":0.99392,"cache_write":0.0,"input":0.440052,"output":0.546875},"cost_usd_as_run":1.9809,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":107.07,"tokens":{"cache_read":74609152,"cache_write":0,"output":625481,"reasoning":436816,"uncached_input":421965},"cost_usd":0.6624,"cost_breakdown":{"cache_read":0.223827,"cache_write":0.0,"input":0.063295,"output":0.375289},"cost_usd_as_run":0.6624,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":86.92,"tokens":{"cache_read":20476672,"cache_write":0,"output":128999,"reasoning":77260,"uncached_input":677865},"cost_usd":4.6934,"cost_breakdown":{"cache_read":2.047667,"cache_write":0.0,"input":1.35573,"output":1.28999},"cost_usd_as_run":4.6934,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":97.69,"tokens":{"cache_read":27931778,"cache_write":942665,"output":332450,"reasoning":212600,"uncached_input":268},"cost_usd":19.7778,"cost_breakdown":{"cache_read":5.586356,"cache_write":7.54132,"input":0.001072,"output":6.649},"cost_usd_as_run":19.7778,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":67.02,"tokens":{"cache_read":16596464,"cache_write":0,"output":159803,"reasoning":93204,"uncached_input":1188551},"cost_usd":11.2696,"cost_breakdown":{"cache_read":3.319293,"cache_write":0.0,"input":4.754204,"output":3.19606},"cost_usd_as_run":11.2696,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":4.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":5.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":5.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":6.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B","score":5.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5}],"panel":["gpt-6.1-terra-max","deepseek-v4.1-flash-max","gpt-6.1-sol-max","opus-5.5-xhigh","muse-wonder-full-xhigh"],"guardrail_blocks":[{"date":"2026-11-03T07:41:00+00:00","provider":"openai","attempt":1,"message":"This content was flagged for possible cybersecurity risk.","outcome":"retry-passed","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3"}]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":12.32,"tokens":{"uncached_input":42,"cache_read":1807132,"cache_write":108247,"output":69980,"reasoning":26413},"cost_usd":2.6272,"cost_breakdown":{"input":0.000168,"cache_read":0.361426,"cache_write":0.865976,"output":1.3996},"cost_usd_as_run":2.6272,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":600,"code":512,"comment":48,"blank":40,"role":"impl"},{"path":"piz_test.go","lines":1053,"code":955,"comment":50,"blank":48,"role":"test"}],"impl_lines":600,"impl_code":512,"impl_comment":48,"test_lines":1053,"comment_ratio":0.086,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":248.8,"reference":91.8,"speedup":2.71,"baselines":{"oracle":91.8}},{"id":"decode_float","candidate":84.8,"reference":46.1,"speedup":1.839,"baselines":{"oracle":46.1}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Extensive coverage and sound sampled decoding, with a cancellation gap","gpt-6.1-sol-max":"Strong tests and clear structure; channel traversal can ignore cancellation.","opus-5.5-xhigh":"correct; 8-bit root table plus trie; thorough docs and tests","deepseek-v4.1-flash-max":"complete, clean, well documented and well tested; one avoidable per-call allocation","muse-wonder-full-xhigh":"Correct, thoroughly documented and tested; no defects found"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Unpolled channel loop","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack descriptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Skipped wavelet planes consume no polling budget","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"`wav2Decode` calls the butterfly through a function variable (`dec := wdec16` / `wdec14`), which blocks inlining in the hottest loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:393","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"piz.go:446-448","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"piz.go:110-112","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":391,"window":6},"complexity":{"functions":17,"mean":7.12,"max":24,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":21,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.2,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.6,"score_cross_vendor":8.25,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":53.8,"tokens":{"uncached_input":204935,"cache_read":4774912,"cache_write":0,"output":121830,"reasoning":96411},"cost_usd":0.2349,"cost_breakdown":{"input":0.03074,"cache_read":0.143247,"cache_write":0.0,"output":0.060915},"cost_usd_as_run":0.2349,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":551,"code":436,"comment":81,"blank":34,"role":"impl"},{"path":"piz_test.go","lines":852,"code":727,"comment":56,"blank":69,"role":"test"}],"impl_lines":551,"impl_code":436,"impl_comment":81,"test_lines":852,"comment_ratio":0.157,"tests_declared":14},"hidden":{"passed":11,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":121.6,"reference":90,"speedup":1.351,"baselines":{"oracle":90}},{"id":"decode_float","candidate":56.8,"reference":45,"speedup":1.262,"baselines":{"oracle":45}}]},"scores":{"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":5.5,"opus-5.5-xhigh":5.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Accepts a malformed table, and several tests mask the relevant validation paths","gpt-6.1-sol-max":"Accepts a malformed table, and several test fixtures mask their intended checks.","opus-5.5-xhigh":"accepts chunks whose run-length symbol has no code; checks an invented rule instead","deepseek-v4.1-flash-max":"well documented and tested; adds one rejection rule the contract does not have","muse-wonder-full-xhigh":"Correct with good docs/tests; one dead check and 32-bit-unsafe code values"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Missing run-symbol code is accepted","status":"valid","defect":"EXR-004","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Unpolled channel loop","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Single zero length is encoded incorrectly","status":"valid","defect":"EXR-017","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Framing tests fail before reaching their target fields","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Incorrect run-code rule","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack descriptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Missing run-symbol codes are accepted","status":"valid","defect":"EXR-004","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate channel planes delay cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Huffman symbols are decoded bit by bit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Single zero lengths are encoded incorrectly","status":"valid","defect":"EXR-017","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Header fixtures fail before reaching their target checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Padding helper’s documented size is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Missing \"run-length symbol must have a code\" check","status":"valid","defect":"EXR-004","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time decode, one `bit()` call per bit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"The substitute rule \"run symbol must have a nonzero code\" never changes an outcome","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"A width that is not a multiple of p leaves a trailing vertical pair\" is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"robustness","title":"Trie construction (up to 65537×58 inserts) has no ctx polling","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The comment says `dataPadding` returns one data byte; it returns 22","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"piz.go:337-339","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:294","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"piz.go:303-305","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"abi","title":"Huffman code values held in `int`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Unreachable run-symbol check","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":328,"window":6},"complexity":{"functions":10,"mean":10.9,"max":58,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":41,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.4,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":2.0,"muse-wonder-full-xhigh":0.5}},"defect_points":6.2,"score_cross_vendor":6.4,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.88,"tokens":{"uncached_input":19281,"cache_read":188416,"cache_write":0,"output":4141,"reasoning":242},"cost_usd":0.0494,"cost_breakdown":{"input":0.019281,"cache_read":0.009421,"cache_write":0.0,"output":0.020705},"cost_usd_as_run":0.0494,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":352,"code":342,"comment":0,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":107,"code":104,"comment":0,"blank":3,"role":"test"}],"impl_lines":352,"impl_code":342,"impl_comment":0,"test_lines":107,"comment_ratio":0,"tests_declared":4},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":119.3,"reference":90.3,"speedup":1.321,"baselines":{"oracle":90.3}},{"id":"decode_float","candidate":56.2,"reference":44.2,"speedup":1.271,"baselines":{"oracle":44.2}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Sampled decoding matches, but cancellation, allocation, tests, and API docs need work","gpt-6.1-sol-max":"Core decoding matches, but tests, documentation, and hot paths need work.","opus-5.5-xhigh":"correct, but undocumented, monolithic, slow bit-serial Huffman decode, thin tests","deepseek-v4.1-flash-max":"correct and fast enough, but undocumented, monolithic and thinly tested","muse-wonder-full-xhigh":"Correct implementation with no documentation and thin tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Unpolled channel loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Crafted edge cases are absent","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API is undocumented","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate planes bypass cancellation polling","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Huffman decoding incurs per-bit reader and polling work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Contract edge cases lack direct coverage","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API and decoding invariants are undocumented","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman data decoded one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No documentation anywhere","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin tests","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"The whole decoder is one ~300-line function, with a local `min` shadowing the builtin","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The `pair` closure re-tests `len(lut) <= 16384` on every butterfly","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"piz.go:1-352","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"piz.go:52-352","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:166","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"piz_test.go:1-107","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"piz.go:190-192","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:266-269","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests would not catch the classic PIZ pitfalls","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Builtin shadowed","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":252,"window":6},"complexity":{"functions":3,"mean":32.67,"max":93,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.3,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":5.8,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.04,"tokens":{"uncached_input":11737,"cache_read":490752,"cache_write":0,"output":23566,"reasoning":10060},"cost_usd":0.1541,"cost_breakdown":{"input":0.011737,"cache_read":0.024538,"cache_write":0.0,"output":0.11783},"cost_usd_as_run":0.1541,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":458,"code":421,"comment":16,"blank":21,"role":"impl"},{"path":"piz_test.go","lines":540,"code":506,"comment":13,"blank":21,"role":"test"}],"impl_lines":458,"impl_code":421,"impl_comment":16,"test_lines":540,"comment_ratio":0.037,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":136.6,"reference":88.7,"speedup":1.54,"baselines":{"oracle":88.7}},{"id":"decode_float","candidate":64.8,"reference":45.7,"speedup":1.418,"baselines":{"oracle":45.7}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Sound decoding and responsive cancellation, with avoidable trie allocation","gpt-6.1-sol-max":"Robust validation and cancellation, with a conservative Huffman hot path.","opus-5.5-xhigh":"correct, good tests; bit-serial decode","deepseek-v4.1-flash-max":"clean and correct; average tests, and two fixed 128 KB/64 KB buffers per call","muse-wonder-full-xhigh":"Correct, clean and well tested; no defects found"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack descriptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Huffman decoding remains a serial bit walk","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-serial trie decode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"A magic `after: 154` is claimed to cancel \"inside the single wide output row\", but the test only asserts Canceled, so the claim is unverified and will drift as poll sites move","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:148, 250, 299","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"piz.go:44-70","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"the suite is good but has no fuzz target and no encoder-driven round trip","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0382,"lines":12,"significant":314,"window":6},"complexity":{"functions":5,"mean":22.6,"max":47,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.6,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":15.79,"tokens":{"uncached_input":18094,"cache_read":228992,"cache_write":0,"output":31198,"reasoning":15128},"cost_usd":0.3711,"cost_breakdown":{"input":0.036188,"cache_read":0.022899,"cache_write":0.0,"output":0.31198},"cost_usd_as_run":0.3711,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":432,"code":396,"comment":14,"blank":22,"role":"impl"},{"path":"piz_test.go","lines":876,"code":827,"comment":11,"blank":38,"role":"test"}],"impl_lines":432,"impl_code":396,"impl_comment":14,"test_lines":876,"comment_ratio":0.034,"tests_declared":16},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":124.9,"reference":90.1,"speedup":1.386,"baselines":{"oracle":90.1}},{"id":"decode_float","candidate":60.2,"reference":45.6,"speedup":1.32,"baselines":{"oracle":45.6}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Sound decoding and cancellation, with avoidable allocation and a misleading test comment","gpt-6.1-sol-max":"Strong robustness and edge-case tests; expensive per-bit Huffman polling.","opus-5.5-xhigh":"correct, very thorough tests; bit-serial decode plus a poll call on every unit of work","deepseek-v4.1-flash-max":"compact, documented and well tested; the same eager trie reservation","muse-wonder-full-xhigh":"Correct, well-tested trie decoder; no defects found"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Stored-path comment describes the wrong case","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack descriptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Huffman hot path polls at every trie edge","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie walk, with `poll()` called per bit, per run word, per LUT word and per output word","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:293","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":294,"window":6},"complexity":{"functions":8,"mean":13.12,"max":46,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.4,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":364.32,"tokens":{"uncached_input":336533,"cache_read":3988208,"cache_write":0,"output":59257,"reasoning":35111},"cost_usd":3.3289,"cost_breakdown":{"input":1.346132,"cache_read":0.797642,"cache_write":0.0,"output":1.18514},"cost_usd_as_run":3.3289,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":521,"code":497,"comment":8,"blank":16,"role":"impl"},{"path":"piz_test.go","lines":604,"code":570,"comment":1,"blank":33,"role":"test"}],"impl_lines":521,"impl_code":497,"impl_comment":8,"test_lines":604,"comment_ratio":0.016,"tests_declared":14},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":160,"reference":90.8,"speedup":1.762,"baselines":{"oracle":90.8}},{"id":"decode_float","candidate":70.2,"reference":45.6,"speedup":1.539,"baselines":{"oracle":45.6}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Broad tests and sound decoding, with a cancellation gap and excessive scratch allocation","gpt-6.1-sol-max":"Solid format support; cancellation and bit-at-a-time lookahead need improvement.","opus-5.5-xhigh":"correct; 12-bit prefix-table decoder; one long function, internals undocumented","deepseek-v4.1-flash-max":"solid and well tested; the prefix fast path is slower than a plain trie walk for short codes","muse-wonder-full-xhigh":"Correct; no internal documentation and a bit-by-bit hot peek"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Unpolled channel loop","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack descriptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Per-channel transforms omit aggregate cancellation checks","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Fast-table lookahead still reads individual bits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Decompress is ~355 lines covering bitmap, table parse, trie, prefix table, decode, wavelet dispatch and pack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Internals undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The wavelet is tested only through the goldens","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:82-89, 395-411","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:327","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"piz.go:348-350","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"piz.go:190","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No internal comments","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Bit-by-bit peek in decode hot loop","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":379,"window":6},"complexity":{"functions":6,"mean":19.83,"max":96,"over_10":2,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":38,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.9,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":9.81,"tokens":{"uncached_input":10732,"cache_read":251648,"cache_write":0,"output":22963,"reasoning":7412},"cost_usd":1.5071,"cost_breakdown":{"input":0.10732,"cache_read":0.251648,"cache_write":0.0,"output":1.14815},"cost_usd_as_run":1.5071,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":437,"code":398,"comment":18,"blank":21,"role":"impl"},{"path":"piz_test.go","lines":822,"code":776,"comment":10,"blank":36,"role":"test"}],"impl_lines":437,"impl_code":398,"impl_comment":18,"test_lines":822,"comment_ratio":0.043,"tests_declared":17},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":141.3,"reference":90.2,"speedup":1.567,"baselines":{"oracle":90.2}},{"id":"decode_float","candidate":66.3,"reference":45.7,"speedup":1.451,"baselines":{"oracle":45.7}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Sound decoding and cancellation, with avoidable trie allocation","gpt-6.1-sol-max":"Correct and robust in probes; Huffman throughput remains avoidably limited.","opus-5.5-xhigh":"correct, strong tests, clear docs; bit-serial Huffman decode","deepseek-v4.1-flash-max":"textbook-clean and thoroughly tested, but no faster than the oracle","muse-wonder-full-xhigh":"Correct with exemplary edge-case tests; no defects found"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack descriptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Every Huffman code uses a serial bit walk","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie walk for every data symbol","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"`inversePair` branches on `small` for every butterfly","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:287","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"piz.go:312-314","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":293,"window":6},"complexity":{"functions":7,"mean":15.71,"max":45,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.4,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.5,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":7.55,"tokens":{"uncached_input":40,"cache_read":1378913,"cache_write":81912,"output":48178,"reasoning":15860},"cost_usd":1.8948,"cost_breakdown":{"input":0.00016,"cache_read":0.275783,"cache_write":0.655296,"output":0.96356},"cost_usd_as_run":1.8948,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":526,"code":476,"comment":20,"blank":30,"role":"impl"},{"path":"piz_test.go","lines":953,"code":874,"comment":35,"blank":44,"role":"test"}],"impl_lines":526,"impl_code":476,"impl_comment":20,"test_lines":953,"comment_ratio":0.04,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":277.7,"reference":89.5,"speedup":3.103,"baselines":{"oracle":89.5}},{"id":"decode_float","candidate":95.3,"reference":45,"speedup":2.118,"baselines":{"oracle":45}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Sound sampled decoding, but invalid-layout precedence and cancellation fail at large channel counts","gpt-6.1-sol-max":"Good buffered decoding, with layout-precedence and cancellation defects.","opus-5.5-xhigh":"correct; 10-bit fast table with a buffered bit reader; strong encoder round-trip tests","deepseek-v4.1-flash-max":"fast and well tested, but the layout check can report the wrong error","muse-wonder-full-xhigh":"One wrong error class for invalid-type-plus-oversize layouts"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Size rejection hides an invalid channel","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Unpolled channel loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack descriptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Size rejection precedes complete type validation","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Channel traversal performs uncounted work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"TestCancellation accepts a 50 ms cancellation lag where the contract allows 10 ms","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"piz.go:75-88","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:334-335","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"piz.go:98-100","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Invalid layout masked by size error","status":"valid","defect":"EXR-006","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":366,"window":6},"complexity":{"functions":9,"mean":12.44,"max":51,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":19,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.2,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.9,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":39.65,"tokens":{"uncached_input":111875,"cache_read":1766992,"cache_write":0,"output":42309,"reasoning":23083},"cost_usd":1.6471,"cost_breakdown":{"input":0.4475,"cache_read":0.353398,"cache_write":0.0,"output":0.84618},"cost_usd_as_run":1.6471,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":615,"code":555,"comment":29,"blank":31,"role":"impl"},{"path":"piz_test.go","lines":707,"code":648,"comment":23,"blank":36,"role":"test"}],"impl_lines":615,"impl_code":555,"impl_comment":29,"test_lines":707,"comment_ratio":0.05,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":259.7,"reference":87.7,"speedup":2.961,"baselines":{"oracle":87.7}},{"id":"decode_float","candidate":94.8,"reference":43.2,"speedup":2.194,"baselines":{"oracle":43.2}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Broad decoding coverage, with a cancellation gap and excessive scratch allocation","gpt-6.1-sol-max":"Broad format validation; cancellation accounting misses small channel planes.","opus-5.5-xhigh":"correct 12-bit prefix decoder with good tests; some comment and redundancy slop","deepseek-v4.1-flash-max":"correct and well documented; the trie insert is harder to read than the reference's","muse-wonder-full-xhigh":"Correct with strong tests; two cosmetic slips"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Unpolled channel loop","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack descriptions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Per-plane polling misses aggregate channel work","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`decompressPIZ` is ~380 lines, and `invPair` re-tests `lutSize` on every butterfly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Comment names the wrong function","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The LUT is built with two 65536-iteration passes, one to count and one to fill, just to size the slice exactly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"No wavelet round trip beyond the goldens","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:301","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"piz.go:315-342","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"piz.go:46-51","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Comment names a different function","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Bytes-per-pixel recomputed","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0477,"lines":20,"significant":419,"window":6},"complexity":{"functions":8,"mean":16.38,"max":89,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":2.2,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.96,"tokens":{"uncached_input":20589,"cache_read":166144,"cache_write":0,"output":4578,"reasoning":119},"cost_usd":0.6009,"cost_breakdown":{"input":0.20589,"cache_read":0.166144,"cache_write":0.0,"output":0.2289},"cost_usd_as_run":0.6009,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":378,"code":365,"comment":2,"blank":11,"role":"impl"},{"path":"piz_test.go","lines":174,"code":168,"comment":2,"blank":4,"role":"test"}],"impl_lines":378,"impl_code":365,"impl_comment":2,"test_lines":174,"comment_ratio":0.005,"tests_declared":4},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":129.3,"reference":88.4,"speedup":1.463,"baselines":{"oracle":88.4}},{"id":"decode_float","candidate":61.2,"reference":44.2,"speedup":1.385,"baselines":{"oracle":44.2}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct on sampled decoding cases, with cancellation, allocation, test, and documentation gaps","gpt-6.1-sol-max":"Correct decoding in probes, with cancellation, throughput, and documentation weaknesses.","opus-5.5-xhigh":"correct, but monolithic, barely documented, bit-serial, with masked tests","deepseek-v4.1-flash-max":"correct but the thinnest deliverable: no docs, one giant function, thin tests","muse-wonder-full-xhigh":"Correct and compact; sparse docs and partly masked tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Unpolled channel loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Oversized trie reservation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Edge coverage is incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public types are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Empty transforms bypass cancellation checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Huffman decoding walks every bit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public format semantics are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Truncation and mutation loops discard their results","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-serial trie decode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported PixelType, constants, Layout, MaxChunkBytes and errors have no doc comments, and the one-line Decompress doc says nothing about its behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"One ~315-line Decompress, with `pair(..., nl <= 16384)` repeated at six call sites","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The comment calls the table \"legal\" while the assertion expects rejection (because the output is short)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"piz.go:9-63","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"piz.go:64-378","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"piz_test.go:1-174","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:202","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"piz.go:227-229","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Exported items undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Truncation/mutation loops assert nothing","status":"valid","defect":"EXR-015","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0449,"lines":12,"significant":267,"window":6},"complexity":{"functions":3,"mean":35.67,"max":102,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.6,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.6,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.52,"tokens":{"uncached_input":19378,"cache_read":265344,"cache_write":0,"output":7736,"reasoning":1363},"cost_usd":0.0713,"cost_breakdown":{"input":0.019378,"cache_read":0.013267,"cache_write":0.0,"output":0.03868},"cost_usd_as_run":0.0713,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":393,"code":379,"comment":2,"blank":12,"role":"impl"},{"path":"piz_test.go","lines":307,"code":298,"comment":1,"blank":8,"role":"test"}],"impl_lines":393,"impl_code":379,"impl_comment":2,"test_lines":307,"comment_ratio":0.005,"tests_declared":8},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":122.8,"reference":86.1,"speedup":1.426,"baselines":{"oracle":86.1}},{"id":"decode_float","candidate":58.9,"reference":43.8,"speedup":1.345,"baselines":{"oracle":43.8}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":5.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with avoidable allocation and sparse public documentation","deepseek-v4.1-flash-max":"Correct, compact, reasonable tests; undocumented API and an unexplained table reservation.","gpt-6.1-sol-max":"Conformant decoding in the probes, with allocation and documentation weaknesses.","opus-5.5-xhigh":"Correct; the slowest entry, with an undocumented API and a monolith","muse-wonder-full-xhigh":"correct compact decoder; no exported docs, worst small-chunk overhead"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Trie is always reserved at maximum size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Public layout types and limits lack comments","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"piz.go:10-23","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:203","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"piz_test.go:196-207","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Full-size trie reserved unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API omits essential explanations","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time decoding, with a poller call on every bit and every output word","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Allocates a fixed 1.6 MB trie capacity on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported types, constants and errors are undocumented","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`Decompress` is a 270-line monolith","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"A local variable `min` shadows the builtin","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"All exported identifiers undocumented","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135000-node trie plus 65536-entry LUT reservations per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Nil context panics","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0426,"lines":12,"significant":282,"window":6},"complexity":{"functions":5,"mean":20.6,"max":84,"over_10":2,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.8,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.0,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":21.69,"tokens":{"uncached_input":25509,"cache_read":9203712,"cache_write":0,"output":212042,"reasoning":184553},"cost_usd":0.1587,"cost_breakdown":{"input":0.003826,"cache_read":0.027611,"cache_write":0.0,"output":0.127225},"cost_usd_as_run":0.1587,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1587,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":519,"code":462,"comment":30,"blank":27,"role":"impl"},{"path":"piz_test.go","lines":1255,"code":1132,"comment":46,"blank":77,"role":"test"}],"impl_lines":519,"impl_code":462,"impl_comment":30,"test_lines":1255,"comment_ratio":0.061,"tests_declared":26},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":131.8,"reference":87.7,"speedup":1.503,"baselines":{"oracle":87.7}},{"id":"decode_float","candidate":58.6,"reference":43.2,"speedup":1.356,"baselines":{"oracle":43.2}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Broad tests, but cancellation and trie allocation need work","deepseek-v4.1-flash-max":"Solid, well documented and very thoroughly tested; duplicated bit extraction.","gpt-6.1-sol-max":"Broad decoding coverage, with small-plane cancellation and allocation issues.","opus-5.5-xhigh":"Correct and very thoroughly tested, but slow (bit-at-a-time trie) and written as one 300-line function","muse-wonder-full-xhigh":"correct, readable oracle-style decoder; fixed trie reservation and thin API docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-pixel channel loop does not poll cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Trie is always reserved at maximum size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack public descriptions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"piz.go:436-447 vs piz.go:325","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"piz.go:320-366","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Polling counter resets for every plane","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated for minimal tables","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman decoding walks the trie one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Allocates a fixed 1.6 MB trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`decodeChunk` is a 310-line monolith","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The layout test allocates a 512 MiB slice","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Trie reserved at fixed 135000 nodes (1.62 MB zeroed) on every call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Decompress and error vars do not state the contract","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":347,"window":6},"complexity":{"functions":6,"mean":19.17,"max":74,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":54,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.9,"score_cross_vendor":7.125,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.04,"tokens":{"uncached_input":7277,"cache_read":162944,"cache_write":0,"output":7559,"reasoning":849},"cost_usd":0.6137,"cost_breakdown":{"input":0.07277,"cache_read":0.162944,"cache_write":0.0,"output":0.37795},"cost_usd_as_run":0.6137,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":395,"code":377,"comment":6,"blank":12,"role":"impl"},{"path":"piz_test.go","lines":395,"code":383,"comment":5,"blank":7,"role":"test"}],"impl_lines":395,"impl_code":377,"impl_comment":6,"test_lines":395,"comment_ratio":0.016,"tests_declared":10},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":128,"reference":87.4,"speedup":1.465,"baselines":{"oracle":87.4}},{"id":"decode_float","candidate":59.6,"reference":43.5,"speedup":1.37,"baselines":{"oracle":43.5}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Broad edge coverage with excessive trie reservation","deepseek-v4.1-flash-max":"Correct and compact; docs are thin and the stored path returns a slice together with an error.","gpt-6.1-sol-max":"Conformant and robust in the probes, with oversized scratch and incomplete API docs.","opus-5.5-xhigh":"Correct, with compact but meaningful tests; weak docs, slow decode, and returns data alongside an error","muse-wonder-full-xhigh":"correct output, but can return data and an error together"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Trie is always reserved at maximum size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Limit and sentinel errors lack public descriptions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"piz.go:26","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:189 (also piz.go:99)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocation on every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public layout and pixel representation are underspecified","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Calls `r.read(1)` once per bit through the trie","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Allocates a fixed 1.6 MB trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported constants and `MaxChunkBytes` are undocumented, and the `Layout` doc is trimmed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Returns the output together with a context error","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The length-field check cannot be reached","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Returns non-nil output together with a context error","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Constants and error vars undocumented, Decompress is one line","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135000-node trie reservation per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Direct ctx.Err() per row in pack (x==0) and per trailing-column block-row in wavelet","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Nil context panics","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0432,"lines":12,"significant":278,"window":6},"complexity":{"functions":5,"mean":20.8,"max":45,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.2,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":2.9,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":24.65,"tokens":{"uncached_input":54,"cache_read":260910,"cache_write":40011,"output":85594,"reasoning":0},"cost_usd":0.0524,"cost_breakdown":{"input":8e-06,"cache_read":0.004175,"cache_write":0.008002,"output":0.040229},"cost_usd_as_run":0.0524,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0524,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":45,"code":26,"comment":11,"blank":8,"role":"impl"}],"impl_lines":45,"impl_code":26,"impl_comment":11,"test_lines":0,"comment_ratio":0.297,"tests_declared":0},"hidden":{"passed":0,"total":12,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":8758606.1,"reference":90.2,"speedup":97102.063,"baselines":{"oracle":90.2}},{"id":"decode_float","candidate":13681484.5,"reference":45.4,"speedup":301354.284,"baselines":{"oracle":45.4}}]},"scores":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5},"council_summaries":{"gpt-6.1-terra-max":"Decoder is unimplemented","deepseek-v4.1-flash-max":"The scaffold stub, unmodified: no implementation and no tests.","gpt-6.1-sol-max":"The required component remains an unimplemented stub.","opus-5.5-xhigh":"Not implemented: the stub is unchanged and there are no tests","muse-wonder-full-xhigh":"unimplemented stub; nothing decodes"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"Every call returns an unrelated error","status":"valid","defect":"EXR-001","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Test deliverable is absent","status":"valid","defect":"EXR-002","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"piz.go:43-45","status":"valid","defect":"EXR-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"tests","title":"piz_test.go is missing","status":"valid","defect":"EXR-002","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"scratch/main.go, scratch/go.mod","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"critical","category":"correctness","title":"No contract behavior is implemented","status":"valid","defect":"EXR-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"`Decompress` is not implemented","status":"valid","defect":"EXR-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"There is no test file","status":"valid","defect":"EXR-002","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Decompress is the scaffold stub","status":"valid","defect":"EXR-001","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"No test file delivered","status":"valid","defect":"EXR-002","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Extra scratch module beyond the deliverables","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":9,"flags":2,"audited":"2026-11-02","reviewed":false},"score":0.1,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":6.8,"score_cross_vendor":0.1,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":23.5,"tokens":{"uncached_input":60,"cache_read":7779755,"cache_write":358762,"output":321461,"reasoning":278586},"cost_usd":1.327,"cost_breakdown":{"input":2.6e-05,"cache_read":0.383435,"cache_write":0.33223,"output":0.611303},"cost_usd_as_run":1.327,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":603,"code":494,"comment":72,"blank":37,"role":"impl"},{"path":"piz_test.go","lines":1243,"code":1092,"comment":84,"blank":67,"role":"test"}],"impl_lines":603,"impl_code":494,"impl_comment":72,"test_lines":1243,"comment_ratio":0.127,"tests_declared":16},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":132.3,"reference":85.5,"speedup":1.547,"baselines":{"oracle":85.5}},{"id":"decode_float","candidate":61.7,"reference":43,"speedup":1.435,"baselines":{"oracle":43}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Well-tested decoder with a cancellation gap for many small planes","deepseek-v4.1-flash-max":"Clean, fast, well documented and tested; a per-pair branch in the hot wavelet loop.","gpt-6.1-sol-max":"Thorough implementation and tests, with a verified cancellation gap.","opus-5.5-xhigh":"Correct and cleanly structured, with good docs and tests; held back by bit-at-a-time decoding","muse-wonder-full-xhigh":"clean proportional-trie implementation, thorough tests and docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-pixel channel loop does not poll cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack public descriptions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"piz.go:528-534","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Identity planes bypass cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"`pairInverse` branches on `w14` for every pair inside the wavelet loops","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Direct ctx.Err() once per scanline when rows are narrower than packBlock","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Comment claims whole stream is consumed including tokens after output fills","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":374,"window":6},"complexity":{"functions":21,"mean":5.81,"max":19,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":37,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":1.8,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.69,"tokens":{"uncached_input":19729,"cache_read":200192,"cache_write":0,"output":5892,"reasoning":510},"cost_usd":0.1184,"cost_breakdown":{"input":0.039458,"cache_read":0.020019,"cache_write":0.0,"output":0.05892},"cost_usd_as_run":0.1184,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":379,"code":367,"comment":2,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":260,"code":255,"comment":2,"blank":3,"role":"test"}],"impl_lines":379,"impl_code":367,"impl_comment":2,"test_lines":260,"comment_ratio":0.005,"tests_declared":5},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":124.4,"reference":84.7,"speedup":1.469,"baselines":{"oracle":84.7}},{"id":"decode_float","candidate":57.3,"reference":42.2,"speedup":1.358,"baselines":{"oracle":42.2}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with excessive trie reservation","deepseek-v4.1-flash-max":"Correct and simple, but the public API is undocumented and it is the slowest decoder.","gpt-6.1-sol-max":"Decoding passed the probes; allocation efficiency and API documentation lag.","opus-5.5-xhigh":"Correct, with compact tests that hit the key edge cases; undocumented API, a monolith, and slow decoding","muse-wonder-full-xhigh":"compact correct decoder let down by missing exported docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Trie is always reserved at maximum size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Public layout types and limits lack comments","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"piz.go:10-30","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:227-263","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:184","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated per call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public types and layout are undocumented","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Allocates a fixed 1.6 MB trie capacity on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`PixelType`, `Layout`, `MaxChunkBytes` and the error variables are undocumented","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`Decompress` is a 265-line monolith","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The cancellation test accepts a nil error","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"A local variable `min` shadows the builtin","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"All exported identifiers undocumented","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135000-node trie reservation per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Pack loop calls ctx.Err() at x==0 of every row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Nil context panics","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0444,"lines":12,"significant":270,"window":6},"complexity":{"functions":4,"mean":26.25,"max":87,"over_10":2,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.9,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.4,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.67,"tokens":{"uncached_input":15410,"cache_read":187392,"cache_write":0,"output":3167,"reasoning":57},"cost_usd":0.0406,"cost_breakdown":{"input":0.01541,"cache_read":0.00937,"cache_write":0.0,"output":0.015835},"cost_usd_as_run":0.0406,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":345,"code":335,"comment":0,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":47,"code":45,"comment":0,"blank":2,"role":"test"}],"impl_lines":345,"impl_code":335,"impl_comment":0,"test_lines":47,"comment_ratio":0,"tests_declared":1},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":114.4,"reference":87,"speedup":1.315,"baselines":{"oracle":87}},{"id":"decode_float","candidate":55.4,"reference":42.8,"speedup":1.294,"baselines":{"oracle":42.8}}]},"scores":{"gpt-6.1-terra-max":5.5,"deepseek-v4.1-flash-max":5.5,"gpt-6.1-sol-max":5.5,"opus-5.5-xhigh":5.0,"muse-wonder-full-xhigh":5.0},"council_summaries":{"gpt-6.1-terra-max":"Incorrect layout precedence and delayed cancellation, with only golden tests","deepseek-v4.1-flash-max":"Complete and mostly correct, but undocumented, barely tested, and with one verified layout-check deviation.","gpt-6.1-sol-max":"Functional decoding, but validation, cancellation, tests, and documentation need changes.","opus-5.5-xhigh":"Correct, but tested only on the golden files, with no documentation and one 290-line function","muse-wonder-full-xhigh":"behavior nearly correct but almost untested and wholly undocumented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Size error can precede an invalid channel","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-pixel channel loop does not poll cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Only golden fixtures are tested","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Trie is always reserved at maximum size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Context is checked twice at the same checkpoint","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Exported API lacks documentation","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"piz.go:68-70","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"piz_test.go:12","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"piz.go:1-45","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"piz.go:207-213","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Size rejection precedes complete channel validation","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Identity transforms leave cancellation unchecked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Only golden decoding is tested","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API lacks documentation","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Duplicate cancellation check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The only test is the golden-vector check","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No package comment and no doc on any exported identifier","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Context polled on every token plus a second, redundant poll","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Allocates a fixed 1.6 MB trie capacity on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Everything is inlined into one 290-line `Decompress`, wavelet included","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The `len(src)-off < 4` check cannot be reached","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Own tests cover only the golden vectors","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No comments anywhere","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"ErrTooLarge returned before later channel types are validated","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"ctx.Err() called on every Huffman token","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135000-node trie and 65536-entry LUT reservations per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Nil context panics","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Second context check in the same decode iteration","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":248,"window":6},"complexity":{"functions":3,"mean":31.0,"max":88,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.3,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":0.5}},"defect_points":8.0,"score_cross_vendor":5.167,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":13.12,"tokens":{"uncached_input":18300,"cache_read":424576,"cache_write":0,"output":28685,"reasoning":12159},"cost_usd":0.3659,"cost_breakdown":{"input":0.0366,"cache_read":0.042458,"cache_write":0.0,"output":0.28685},"cost_usd_as_run":0.3659,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":418,"code":387,"comment":12,"blank":19,"role":"impl"},{"path":"piz_test.go","lines":799,"code":754,"comment":9,"blank":36,"role":"test"}],"impl_lines":418,"impl_code":387,"impl_comment":12,"test_lines":799,"comment_ratio":0.03,"tests_declared":17},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":134.9,"reference":84.9,"speedup":1.589,"baselines":{"oracle":84.9}},{"id":"decode_float","candidate":63.3,"reference":43.2,"speedup":1.465,"baselines":{"oracle":43.2}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Strong coverage and robust decoding, with excessive trie reservation","deepseek-v4.1-flash-max":"Correct, tidy trie decoder; polls the context far more often than needed.","gpt-6.1-sol-max":"Strong conformance and useful tests; scratch allocation is unnecessarily large.","opus-5.5-xhigh":"Correct, with solid tests; bit-at-a-time decoding and a fixed trie allocation limit speed","muse-wonder-full-xhigh":"correct with boundary-focused tests; over-eager polling"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Trie is always reserved at maximum size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack public descriptions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"piz.go:319","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"piz.go:31-35","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated for every table","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Allocates a fixed 1.6 MB trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`Decompress` holds framing, LUT, wavelet dispatch and packing in one 145-line body","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Direct ctx.Err() every 64 tokens, every wavelet block-row and every pack row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135000-node trie reservation per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Error vars undocumented and Decompress omits stored/error semantics","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Nil context panics","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":287,"window":6},"complexity":{"functions":5,"mean":21.8,"max":46,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.2,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":28.82,"tokens":{"uncached_input":56,"cache_read":4755135,"cache_write":216644,"output":172596,"reasoning":119664},"cost_usd":6.1363,"cost_breakdown":{"input":0.000224,"cache_read":0.951027,"cache_write":1.733152,"output":3.45192},"cost_usd_as_run":6.1363,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":717,"code":609,"comment":65,"blank":43,"role":"impl"},{"path":"piz_test.go","lines":1929,"code":1779,"comment":68,"blank":82,"role":"test"}],"impl_lines":717,"impl_code":609,"impl_comment":65,"test_lines":1929,"comment_ratio":0.096,"tests_declared":21},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":347.1,"reference":86.6,"speedup":4.008,"baselines":{"oracle":86.6}},{"id":"decode_float","candidate":97.6,"reference":42.9,"speedup":2.275,"baselines":{"oracle":42.9}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Comprehensive tests and adaptive tables, with a cancellation gap","deepseek-v4.1-flash-max":"Very fast with an outstanding test suite; heavier than it needs to be.","gpt-6.1-sol-max":"Efficient implementation and extensive tests, with a cancellation hole between planes.","opus-5.5-xhigh":"Correct and fast, with well-designed tests, accurate docs and an early rejection of streams that cannot be filled","muse-wonder-full-xhigh":"fast lookup+trie decoder, best-tested; only a context-poll shortcut to fix"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"One-pixel channel loop does not poll cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack public descriptions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"piz.go:55-78","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:554 and piz.go:589","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Identity planes bypass the shared polling budget","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Codes longer than 12 bits fall back to a bit-by-bit trie walk","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Poll short-circuits when Done()==nil","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":468,"window":6},"complexity":{"functions":20,"mean":7.2,"max":27,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":27,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.5,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.5,"score_cross_vendor":8.375,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":44.76,"tokens":{"uncached_input":134347,"cache_read":2185200,"cache_write":0,"output":49180,"reasoning":28923},"cost_usd":1.958,"cost_breakdown":{"input":0.537388,"cache_read":0.43704,"cache_write":0.0,"output":0.9836},"cost_usd_as_run":1.958,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":523,"code":484,"comment":14,"blank":25,"role":"impl"},{"path":"piz_test.go","lines":701,"code":652,"comment":17,"blank":32,"role":"test"}],"impl_lines":523,"impl_code":484,"impl_comment":14,"test_lines":701,"comment_ratio":0.028,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":138.4,"reference":86.1,"speedup":1.607,"baselines":{"oracle":86.1}},{"id":"decode_float","candidate":59.1,"reference":40.3,"speedup":1.467,"baselines":{"oracle":40.3}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Solid decoding with an avoidable per-call allocation","deepseek-v4.1-flash-max":"Correct, well organised and documented; the cancellation helper carries unreachable logic.","gpt-6.1-sol-max":"Conformant and robust in the probes, with excessive fixed scratch allocation.","opus-5.5-xhigh":"Correct, but slow bit-at-a-time decoding plus some slop in context polling","muse-wonder-full-xhigh":"correct straightforward decoder with a convoluted context helper"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Trie is always reserved at maximum size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Sentinel errors lack public descriptions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"piz.go:52-60","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:38","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie reserved for small tables","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Allocates a fixed 1.6 MB trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Redundant context checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `else` branch","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`decodePIZ` is a 316-line monolith","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"ctxErr double-checks Err/Done and invents context.Canceled","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135000-node trie reservation per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Callable contract barely documented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable symbol-range branch","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0332,"lines":12,"significant":361,"window":6},"complexity":{"functions":7,"mean":17.43,"max":82,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":27,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.3,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.5}},"defect_points":2.5,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.27,"tokens":{"uncached_input":43099,"cache_read":132864,"cache_write":0,"output":9370,"reasoning":2259},"cost_usd":0.1932,"cost_breakdown":{"input":0.086198,"cache_read":0.013286,"cache_write":0.0,"output":0.0937},"cost_usd_as_run":0.1932,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":394,"code":371,"comment":10,"blank":13,"role":"impl"},{"path":"piz_test.go","lines":395,"code":385,"comment":5,"blank":5,"role":"test"}],"impl_lines":394,"impl_code":371,"impl_comment":10,"test_lines":395,"comment_ratio":0.026,"tests_declared":10},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":122.6,"reference":83.9,"speedup":1.461,"baselines":{"oracle":83.9}},{"id":"decode_float","candidate":55.7,"reference":41,"speedup":1.359,"baselines":{"oracle":41}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and robust in the probes, with avoidable per-call allocation.","gpt-6.1-sol-max":"Correct and responsive, with avoidable Huffman allocation overhead.","opus-5.5-xhigh":"Compact and correct, essentially the oracle's algorithm; bit-at-a-time Huffman decoding, sparse docs","deepseek-v4.1-flash-max":"the simplest, most readable implementation and correct, at the cost of speed and precision of style.","muse-wonder-full-xhigh":"compact and correct, with thin tests/docs and a non-idiomatic (output, error) return"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Always reserves the maximum trie","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie allocated for tiny tables","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman data is decoded one bit at a time by walking the trie","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Same 135,000-node trie allocated every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Internal functions are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin coverage of the wavelet and layout paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Returns a non-nil output together with an error","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"No table-driven Huffman decode; a 1.6 MB trie is reserved per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`return out, ctx.Err()` returns a usable result together with a non-nil error","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"bit-at-a-time reader and fixed-capacity trie","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"successful output returned together with a context error","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"thinnest suite and a self-confirming wavelet test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"minimal documentation and unnamed magic numbers","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":272,"window":6},"complexity":{"functions":5,"mean":20.8,"max":45,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.1,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":68.78,"tokens":{"uncached_input":200088,"cache_read":4024192,"cache_write":0,"output":298021,"reasoning":132033},"cost_usd":4.2004,"cost_breakdown":{"input":0.400176,"cache_read":2.012096,"cache_write":0.0,"output":1.788126},"cost_usd_as_run":4.2004,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"piz.go","lines":627,"code":588,"comment":9,"blank":30,"role":"impl"},{"path":"piz_test.go","lines":917,"code":839,"comment":34,"blank":44,"role":"test"}],"impl_lines":627,"impl_code":588,"impl_comment":9,"test_lines":917,"comment_ratio":0.015,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":108.4,"reference":90.4,"speedup":1.199,"baselines":{"oracle":90.4}},{"id":"decode_float","candidate":45.4,"reference":45.5,"speedup":0.998,"baselines":{"oracle":45.5}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in differential checks, with cancellation, allocation, and test gaps.","gpt-6.1-sol-max":"Correct decoding, but cancellation, allocation and internal documentation need work.","opus-5.5-xhigh":"Correct but slower than the oracle, undocumented internals, thin and masked tests","deepseek-v4.1-flash-max":"correct and clear, but it is the only entry that does not beat the oracle's speed, and it carries a few dead checks.","muse-wonder-full-xhigh":"correct and robust, but the slowest entry with sparse docs and partly masked wavelet tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Identity planes do not poll cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Always reserves the maximum trie","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The compressed-input alias check is ineffective","status":"valid","defect":"EXR-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Negative bitmap span is impossible","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Identity planes leave channel traversal uninterruptible","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Maximum trie reserved independently of table size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Non-obvious Huffman bounds are unexplained","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Impossible negative bitmap length","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Slower than the reference on realistic chunks","status":"valid","defect":"EXR-011","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"135,000-node trie allocated every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Masked and thin tests","status":"valid","defect":"EXR-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost nothing is documented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Non-idiomatic structure","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Bit-at-a-time trie decode with no lookup table","status":"valid","defect":"EXR-011","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Two unreachable guards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"135k-capacity trie allocation plus bit-at-a-time Huffman walk","status":"valid","defect":"EXR-011","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"almost no internal documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"wavelet tests are masked","status":"valid","defect":"EXR-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"inconsistent little-endian decoding and naming","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":48,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.9,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":4.8,"score_cross_vendor":6.9,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1-b2","task":"exr-piz","lead":"gpt-6.1-terra-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":15.91,"tokens":{"cache_read":7211144,"cache_write":0,"output":62787,"reasoning":36682,"uncached_input":306691},"cost_usd":3.9247,"cost_breakdown":{"cache_read":1.442229,"cache_write":0.0,"input":1.226764,"output":1.25574},"cost_usd_as_run":3.9247,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":13.35,"tokens":{"cache_read":4713856,"cache_write":0,"output":31640,"reasoning":16562,"uncached_input":101221},"cost_usd":0.4951,"cost_breakdown":{"cache_read":0.235693,"cache_write":0.0,"input":0.101221,"output":0.1582},"cost_usd_as_run":0.4951,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":17.81,"tokens":{"cache_read":5796118,"cache_write":241900,"output":92073,"reasoning":62545,"uncached_input":68},"cost_usd":4.9362,"cost_breakdown":{"cache_read":1.159224,"cache_write":1.9352,"input":0.000272,"output":1.84146},"cost_usd_as_run":4.9362,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":17.8,"tokens":{"cache_read":3521024,"cache_write":0,"output":35872,"reasoning":23076,"uncached_input":150302},"cost_usd":1.0114,"cost_breakdown":{"cache_read":0.352102,"cache_write":0.0,"input":0.300604,"output":0.35872},"cost_usd_as_run":1.0114,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":24.42,"tokens":{"cache_read":12954624,"cache_write":0,"output":197847,"reasoning":163998,"uncached_input":120230},"cost_usd":0.1756,"cost_breakdown":{"cache_read":0.038864,"cache_write":0.0,"input":0.018034,"output":0.118708},"cost_usd_as_run":0.1756,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"G","score":6.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"D","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"Y","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"G","score":4.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"D","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"Y","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"G","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"D","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"Y","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"G","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"D","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"Y","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"G","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"D","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"Y","score":9.0}],"panel":["muse-wonder-full-xhigh","gpt-6.1-terra-max","opus-5.5-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"E","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":8.53,"tokens":{"uncached_input":98244,"cache_read":725390,"cache_write":0,"output":41703,"reasoning":0},"cost_usd":0.6861,"cost_breakdown":{"input":0.196488,"cache_read":0.072539,"cache_write":0.0,"output":0.41703},"cost_usd_as_run":0.6861,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":551,"code":484,"comment":19,"blank":48,"role":"impl"},{"path":"piz_test.go","lines":832,"code":696,"comment":76,"blank":60,"role":"test"}],"impl_lines":551,"impl_code":484,"impl_comment":19,"test_lines":832,"comment_ratio":0.038,"tests_declared":8},"hidden":{"passed":3,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":175,"reference":90.3,"speedup":1.938,"baselines":{"oracle":90.3}},{"id":"decode_float","candidate":89,"reference":44.8,"speedup":1.987,"baselines":{"oracle":44.8}}]},"isolation":{"tool_calls":21,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":5.0,"gpt-6.1-terra-max":3.5,"opus-5.5-xhigh":4.5,"gpt-6.1-sol-max":4.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"muse-wonder-full-xhigh":"fast and lean, but spuriously rejects every valid chunk with a full 65,536-entry LUT","gpt-6.1-terra-max":"Valid full-LUT chunks fail, and an entire wavelet test silently skips.","opus-5.5-xhigh":"Rejects every valid chunk whose LUT holds all 65,536 values; otherwise correct and about 2× the oracle's speed","gpt-6.1-sol-max":"Rejects valid full-LUT chunks and misses cancellation deadlines.","deepseek-v4.1-flash-max":"fast and mostly correct, but rejects every legal 65,536-entry LUT, with a dead 16-bit test branch"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"LUT size truncated to uint16","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"16-bit wavelet path silently skipped in odd-dimension test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Overwritten exploratory code in sparse-code test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Wrong justification for the bitCount early exit","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"A full reverse LUT rejects every word","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Run payloads can skip cancellation checkpoints","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"The odd-dimension wavelet test exercises no cases","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Huffman decoding traverses one bit at a time","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Canonical-code explanation states a false rule","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Abandoned test tables are overwritten","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Full LUT is truncated to 0","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The synthetic wavelet test checks nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The Huffman decoder walks the trie one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"One function does everything","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Internal code is undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Test comments read like thinking aloud","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"A comment doesn't match the check below it","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Full LUT size wraps to zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Run payloads skip cancellation checkpoints","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Synthetic wavelet test skips every case","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Discarded test construction drafts remain","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"correctness","title":"`k16 := uint16(k)` truncates 65536 to 0, rejecting all full-bitmap chunks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"the 16-bit half of `TestOddDimensionsAndWaveletPaths` never executes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"dead test constructions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"stream-of-consciousness test comments","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"undocumented trie cap and garbled early-exhaustion check","status":"valid","detail":"","location":""}],"score":4.7,"slop":{"score":1.8,"per_seat":{"muse-wonder-full-xhigh":3.0,"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":2.0}},"defect_points":9.0,"score_cross_vendor":4.7,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","task":"exr-piz","lead":"gpt-6.1-terra-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":51.59,"tokens":{"cache_read":19755008,"cache_write":0,"output":121654,"reasoning":73560,"uncached_input":616453},"cost_usd":4.425,"cost_breakdown":{"cache_read":1.975501,"cache_write":0.0,"input":1.232906,"output":1.21654},"cost_usd_as_run":4.425,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":63.09,"tokens":{"cache_read":24138188,"cache_write":0,"output":209623,"reasoning":122104,"uncached_input":935612},"cost_usd":12.7625,"cost_breakdown":{"cache_read":4.827637,"cache_write":0.0,"input":3.742448,"output":4.19246},"cost_usd_as_run":12.7625,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":36.86,"tokens":{"cache_read":19971968,"cache_write":0,"output":92331,"reasoning":43695,"uncached_input":470459},"cost_usd":1.9307,"cost_breakdown":{"cache_read":0.998598,"cache_write":0.0,"input":0.470459,"output":0.461655},"cost_usd_as_run":1.9307,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":80.16,"tokens":{"cache_read":37428509,"cache_write":1009487,"output":378438,"reasoning":240585,"uncached_input":338},"cost_usd":23.1317,"cost_breakdown":{"cache_read":7.485702,"cache_write":8.075896,"input":0.001352,"output":7.56876},"cost_usd_as_run":23.1317,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":94.94,"tokens":{"cache_read":76281088,"cache_write":0,"output":565028,"reasoning":395999,"uncached_input":407499},"cost_usd":0.6289,"cost_breakdown":{"cache_read":0.228843,"cache_write":0.0,"input":0.061126,"output":0.339017},"cost_usd_as_run":0.6289,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1b","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1b","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":10.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":5.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"A","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-gpt-6.1-terra-max-r1","letter":"B","score":7.0}],"panel":["gpt-6.1-sol-max","muse-wonder-full-xhigh","gpt-6.1-terra-max","opus-5.5-xhigh","deepseek-v4.1-flash-max"],"guardrail_blocks":[{"date":"2026-11-04T00:38:28+00:00","provider":"openai","attempt":2,"message":"This content was flagged for possible cybersecurity risk. If this seems wrong, try rephrasing your request.","outcome":"retry-blocked","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1"},{"date":"2026-11-04T00:49:06+00:00","provider":"openai","attempt":1,"message":"This content was flagged for possible cybersecurity risk. If this seems wrong, try rephrasing your request.","outcome":"retry-blocked","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1"},{"date":"2026-11-04T00:49:06+00:00","provider":"openai","attempt":3,"message":"Reshuffled session (same entries, new labels and order) passed.","outcome":"retry-passed","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1b"},{"date":"2026-11-04T01:11:12+00:00","provider":"openai","attempt":1,"message":"This content was flagged for possible cybersecurity risk. If this seems wrong, try rephrasing your request.","outcome":"retry-passed","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2"}]},"entries":[{"letter":"A","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.02,"tokens":{"uncached_input":43725,"cache_read":364032,"cache_write":0,"output":27698,"reasoning":14826},"cost_usd":0.4008,"cost_breakdown":{"input":0.08745,"cache_read":0.036403,"cache_write":0.0,"output":0.27698},"cost_usd_as_run":0.4008,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":428,"code":395,"comment":12,"blank":21,"role":"impl"},{"path":"piz_test.go","lines":499,"code":468,"comment":8,"blank":23,"role":"test"}],"impl_lines":428,"impl_code":395,"impl_comment":12,"test_lines":499,"comment_ratio":0.029,"tests_declared":9},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":143.2,"reference":89.7,"speedup":1.596,"baselines":{"oracle":89.7}},{"id":"decode_float","candidate":67.1,"reference":45.2,"speedup":1.485,"baselines":{"oracle":45.2}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":291,"window":6},"complexity":{"functions":8,"mean":13.75,"max":32,"over_10":5,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.5,"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-sol-max":"Careful validation and cancellation; oversized Huffman scratch.","muse-wonder-full-xhigh":"oracle-identical with one of the best crafted test tables; arena cost only","gpt-6.1-terra-max":"Correct and carefully tested, with unnecessary fixed Huffman allocation","opus-5.5-xhigh":"correct, well-structured and well-tested; thin internal docs","deepseek-v4.1-flash-max":"Correct, fully documented, solid tests; same oversized per-call arena, no fuzz."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated for tiny tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 140k-node arena plus full LUT allocation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Internals undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"huffmanTable returns five values","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"No allocation-bound test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`tree := make([]huffmanNode, maxHuffmanNodes)`","status":"valid","detail":"","location":""}],"score":8.1,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.7,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.14,"tokens":{"uncached_input":8605,"cache_read":279168,"cache_write":0,"output":8834,"reasoning":2660},"cost_usd":0.0667,"cost_breakdown":{"input":0.008605,"cache_read":0.013958,"cache_write":0.0,"output":0.04417},"cost_usd_as_run":0.0667,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":367,"code":351,"comment":5,"blank":11,"role":"impl"},{"path":"piz_test.go","lines":290,"code":279,"comment":1,"blank":10,"role":"test"}],"impl_lines":367,"impl_code":351,"impl_comment":5,"test_lines":290,"comment_ratio":0.014,"tests_declared":7},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":133.1,"reference":90.2,"speedup":1.476,"baselines":{"oracle":90.2}},{"id":"decode_float","candidate":63.8,"reference":45.8,"speedup":1.393,"baselines":{"oracle":45.8}}]},"quality":{"duplication":{"ratio":0.0536,"lines":14,"significant":261,"window":6},"complexity":{"functions":3,"mean":33.33,"max":95,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":7.5,"gpt-6.1-terra-max":6.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-sol-max":"Correct decoding, but degenerate planes bypass cancellation.","muse-wonder-full-xhigh":"correct on all probes, but thin tests, no exported docs, one monolithic function","gpt-6.1-terra-max":"Decoding checks are sound, but cancellation and documentation need work","opus-5.5-xhigh":"correct, but the exported API is undocumented, the decoder is one function, and tests are thin","deepseek-v4.1-flash-max":"Correct and compact, but the exported API is undocumented and the per-call arena is oversized."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Channel traversal can exceed cancellation deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie reserved unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public types lack documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Exported identifiers undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No fuzz, mutation or allocation coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Whole decoder is one function","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 135001-node trie arena","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Degenerate channel planes defer cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public types and limits are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported API undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"300-line Decompress","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 135,001-node trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"No docs on the exported surface","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Unexplained limit","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`nodes := make([]trieNode, maxNodes+1)`","status":"valid","detail":"","location":""}],"score":7.0,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.9,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.26,"tokens":{"uncached_input":7586,"cache_read":215040,"cache_write":0,"output":8452,"reasoning":1497},"cost_usd":0.7135,"cost_breakdown":{"input":0.07586,"cache_read":0.21504,"cache_write":0.0,"output":0.4226},"cost_usd_as_run":0.7135,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":386,"code":370,"comment":4,"blank":12,"role":"impl"},{"path":"piz_test.go","lines":362,"code":355,"comment":2,"blank":5,"role":"test"}],"impl_lines":386,"impl_code":370,"impl_comment":4,"test_lines":362,"comment_ratio":0.011,"tests_declared":9},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":137.7,"reference":90.4,"speedup":1.523,"baselines":{"oracle":90.4}},{"id":"decode_float","candidate":63.4,"reference":45.8,"speedup":1.384,"baselines":{"oracle":45.8}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":274,"window":6},"complexity":{"functions":5,"mean":20.6,"max":46,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.0,"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-sol-max":"Solid behavior, with allocation and API-documentation gaps.","muse-wonder-full-xhigh":"decoder is correct, but its own suite is not reliably green and exports are undocumented","gpt-6.1-terra-max":"Sound decoding, but expensive for small chunks and lightly documented","opus-5.5-xhigh":"correct with reasonable tests; the exported API is undocumented","deepseek-v4.1-flash-max":"Correct, clean and well tested, but the exported API is undocumented, the per-call arena is 1.7 MB, and its cancellation test is flaky under `-race`."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum-sized trie used for every chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API descriptions are missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Flaky wall-clock cancellation test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Exported identifiers undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 140k-node trie arena","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Stored path can return data with an error","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public types and limits are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported API undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Stored path returns a buffer together with an error","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node trie and a 65,537-entry length table on every call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Exported API undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Fixed 1.68 MB trie per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Fragile 10 ms assertion","status":"valid","detail":"","location":""}],"score":7.4,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.1,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":72.07,"tokens":{"uncached_input":295745,"cache_read":2940652,"cache_write":0,"output":73682,"reasoning":46934},"cost_usd":3.2447,"cost_breakdown":{"input":1.18298,"cache_read":0.58813,"cache_write":0.0,"output":1.47364},"cost_usd_as_run":3.2447,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":506,"code":454,"comment":25,"blank":27,"role":"impl"},{"path":"piz_test.go","lines":493,"code":428,"comment":34,"blank":31,"role":"test"}],"impl_lines":506,"impl_code":454,"impl_comment":25,"test_lines":493,"comment_ratio":0.052,"tests_declared":15},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":140,"reference":90,"speedup":1.556,"baselines":{"oracle":90}},{"id":"decode_float","candidate":67.3,"reference":45.3,"speedup":1.486,"baselines":{"oracle":45.3}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":27,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":8.0,"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-sol-max":"Strong format checks; cancellation and unnecessary work remain.","muse-wonder-full-xhigh":"correct and fastest of the trie entries, held back by a 423-line Decompress","gpt-6.1-terra-max":"Correct in the checked streams, with cancellation and allocation costs","opus-5.5-xhigh":"correct, but everything sits in one 420-line function and the tests are moderate","deepseek-v4.1-flash-max":"Thoroughly tested and documented, fast, with one dead branch and a fixed-size arena to tidy."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Wavelet traversal can go unpolled across channels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated for every table","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unreachable fallback checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Monolithic Decompress","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 140k-node trie arena","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead size re-check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Channel loop can defer cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Monolithic Decompress","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin robustness coverage","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable size check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable size check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Fixed trie","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"Step banners","status":"valid","detail":"","location":""}],"score":7.3,"slop":{"score":0.5,"per_seat":{"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":1.0}},"defect_points":3.2,"score_cross_vendor":7.125,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.59,"tokens":{"uncached_input":9892,"cache_read":124160,"cache_write":0,"output":4189,"reasoning":155},"cost_usd":0.4325,"cost_breakdown":{"input":0.09892,"cache_read":0.12416,"cache_write":0.0,"output":0.20945},"cost_usd_as_run":0.4325,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":361,"code":343,"comment":8,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":163,"code":159,"comment":0,"blank":4,"role":"test"}],"impl_lines":361,"impl_code":343,"impl_comment":8,"test_lines":163,"comment_ratio":0.023,"tests_declared":4},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":137.3,"reference":90.1,"speedup":1.524,"baselines":{"oracle":90.1}},{"id":"decode_float","candidate":63.2,"reference":45.7,"speedup":1.383,"baselines":{"oracle":45.7}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":257,"window":6},"complexity":{"functions":4,"mean":23.75,"max":88,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":7.0,"gpt-6.1-terra-max":6.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-sol-max":"Correct decoding; cancellation and regression coverage need work.","muse-wonder-full-xhigh":"correct on all probes, but the thinnest test suite of the twelve","gpt-6.1-terra-max":"Basic decoding works, but cancellation and test coverage need work","opus-5.5-xhigh":"correct, but tests are thin and the code is one function with closures","deepseek-v4.1-flash-max":"Correct, tidy and documented, but its tests are thin for a format this deep and the arena is fixed."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate planes perform no cancellation checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Fixed trie dominates small calls","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Important Huffman cases lack explicit coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Minimal contract coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Whole decoder is one function","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 140k-node trie arena","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Degenerate channel planes defer cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Important contract edges are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"300-line Decompress with closures","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Internals undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node trie on every call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Thin suite","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`trie := make([]node, 140000)`","status":"valid","detail":"","location":""}],"score":6.6,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.1,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.47,"tokens":{"uncached_input":8322,"cache_read":278656,"cache_write":0,"output":17276,"reasoning":4243},"cost_usd":1.2257,"cost_breakdown":{"input":0.08322,"cache_read":0.278656,"cache_write":0.0,"output":0.8638},"cost_usd_as_run":1.2257,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":409,"code":375,"comment":13,"blank":21,"role":"impl"},{"path":"piz_test.go","lines":606,"code":570,"comment":10,"blank":26,"role":"test"}],"impl_lines":409,"impl_code":375,"impl_comment":13,"test_lines":606,"comment_ratio":0.034,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":134.7,"reference":90.2,"speedup":1.493,"baselines":{"oracle":90.2}},{"id":"decode_float","candidate":63.3,"reference":45.3,"speedup":1.397,"baselines":{"oracle":45.3}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":279,"window":6},"complexity":{"functions":8,"mean":12.88,"max":42,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.5,"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-sol-max":"Robust decoding and thorough tests, with one allocation inefficiency.","muse-wonder-full-xhigh":"oracle-identical on every probe; only costs are trie-tier speed/allocation","gpt-6.1-terra-max":"Correct in the cases checked, with excessive fixed Huffman workspace","opus-5.5-xhigh":"correct and robust, with good tests; internal docs are thin","deepseek-v4.1-flash-max":"Correct and well explained, with allocation and cancellation tests; only the tiny exported identifiers lack docs."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Tiny tables receive a maximum-sized trie","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 140k-node trie arena","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Zero-length count is a no-op","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Oversized trie on every compressed chunk","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Internals undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Partial API docs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`nodes := make([]huffNode, 140000)`","status":"valid","detail":"","location":""}],"score":8.2,"slop":{"score":0.1,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.8,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":89.47,"tokens":{"uncached_input":416390,"cache_read":8453760,"cache_write":0,"output":433377,"reasoning":196460},"cost_usd":7.6599,"cost_breakdown":{"input":0.83278,"cache_read":4.22688,"cache_write":0.0,"output":2.600262},"cost_usd_as_run":7.6599,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"piz.go","lines":674,"code":610,"comment":27,"blank":37,"role":"impl"},{"path":"piz_test.go","lines":831,"code":743,"comment":55,"blank":33,"role":"test"}],"impl_lines":674,"impl_code":610,"impl_comment":27,"test_lines":831,"comment_ratio":0.042,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":76.6,"reference":90.1,"speedup":0.85,"baselines":{"oracle":90.1}},{"id":"decode_float","candidate":34.4,"reference":45.6,"speedup":0.754,"baselines":{"oracle":45.6}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":71,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":5.5,"muse-wonder-full-xhigh":6.0,"gpt-6.1-terra-max":6.0,"opus-5.5-xhigh":5.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-sol-max":"Cancellation failures and unnecessary scratch undermine otherwise correct decoding.","muse-wonder-full-xhigh":"output-correct, but cancellation can be ignored for the whole stream and it is 3× slower than its peers","gpt-6.1-terra-max":"Decodes the checked vectors but has cancellation and workspace problems","opus-5.5-xhigh":"output is correct, but cancellation can lag 18× past the bound, it is slower than the oracle, and some comments are false over no-op code","deepseek-v4.1-flash-max":"Correct but the slowest entry (31 MB/s, below the oracle), with a >20 ms cancellation gap on wide planes and a padded buffer whose justifying comments are false."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Output-dependent polling stalls on valid runs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Wide wavelet rows exceed cancellation deadline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie allocated for trivial tables","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Nonexistent border reads justify unnecessary padding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Always-successful LUT error plumbing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Cancellation polling is tied to output count, not work","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Slowest hot paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Wavelet polls only once per block row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"One-past-end padding machinery guards an impossible access","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Long work runs between cancellation checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"redundancy","title":"Unneeded wavelet padding and border save","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Huffman context check keyed to the output count","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"wav2 checks the context once per row","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Slower than the oracle","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False \"one-past-end\" claim on no-op padding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Test helper slop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"Cancellation gap of 17-26 ms","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"31 MB/s on a realistic 1000×32 half chunk versus 47 for the oracle and 60-83 for every other entry","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable guards","status":"valid","detail":"","location":""}],"score":5.8,"slop":{"score":1.4,"per_seat":{"gpt-6.1-sol-max":1.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":2.0,"opus-5.5-xhigh":2.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":6.4,"score_cross_vendor":5.8,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.09,"tokens":{"uncached_input":28297,"cache_read":338688,"cache_write":0,"output":21229,"reasoning":8417},"cost_usd":0.3028,"cost_breakdown":{"input":0.056594,"cache_read":0.033869,"cache_write":0.0,"output":0.21229},"cost_usd_as_run":0.3028,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":403,"code":371,"comment":15,"blank":17,"role":"impl"},{"path":"piz_test.go","lines":524,"code":493,"comment":10,"blank":21,"role":"test"}],"impl_lines":403,"impl_code":371,"impl_comment":15,"test_lines":524,"comment_ratio":0.039,"tests_declared":10},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":117,"reference":90.2,"speedup":1.297,"baselines":{"oracle":90.2}},{"id":"decode_float","candidate":58.2,"reference":45.3,"speedup":1.285,"baselines":{"oracle":45.3}}]},"quality":{"duplication":{"ratio":0.0433,"lines":12,"significant":277,"window":6},"complexity":{"functions":5,"mean":20.2,"max":45,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.0,"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-sol-max":"Thoroughly validated implementation with avoidable fixed scratch.","muse-wonder-full-xhigh":"correct and well tested; its bit reader makes it the slowest trie entry after J","gpt-6.1-terra-max":"Correct and well tested, with a costly fixed trie","opus-5.5-xhigh":"correct, robust and thoroughly tested; thin internal docs and a fixed-size trie","deepseek-v4.1-flash-max":"Best-in-class docs and tests among the trie decoders, let down only by a fixed 2 MB-per-call allocation and the slowest inner bit reader of that group."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Trie allocation ignores actual table size","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Generic per-bit reader on the hot path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 140k-node trie arena","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Zero-length count is a no-op","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node trie zeroed on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Internals undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Fixed arena and LUT","status":"valid","detail":"","location":""}],"score":8.3,"slop":{"score":0.1,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.8,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.38,"tokens":{"uncached_input":20531,"cache_read":222976,"cache_write":0,"output":6583,"reasoning":796},"cost_usd":0.1292,"cost_breakdown":{"input":0.041062,"cache_read":0.022298,"cache_write":0.0,"output":0.06583},"cost_usd_as_run":0.1292,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":375,"code":350,"comment":11,"blank":14,"role":"impl"},{"path":"piz_test.go","lines":291,"code":276,"comment":7,"blank":8,"role":"test"}],"impl_lines":375,"impl_code":350,"impl_comment":11,"test_lines":291,"comment_ratio":0.03,"tests_declared":9},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":129.4,"reference":90,"speedup":1.438,"baselines":{"oracle":90}},{"id":"decode_float","candidate":61.7,"reference":45.7,"speedup":1.35,"baselines":{"oracle":45.7}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":261,"window":6},"complexity":{"functions":5,"mean":19.2,"max":78,"over_10":2,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":7.5,"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-sol-max":"Correct format handling; many degenerate planes defeat cancellation.","muse-wonder-full-xhigh":"compact and correct; tests are the thin part","gpt-6.1-terra-max":"Correct in the checked cases, but has a cancellation gap and costly trie","opus-5.5-xhigh":"correct and compact; tests and docs are moderate","deepseek-v4.1-flash-max":"Correct and properly documented; the only real cost is the same fixed 1.7 MB trie."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Skipped wavelet planes never poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Shallow negative/oracle coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 140k-node trie arena","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Degenerate channel planes defer cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Fixed trie dominates small decodes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node trie on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Cancellation test allows 1 s","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Helpers undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Fixed 140000-node trie","status":"valid","detail":"","location":""}],"score":7.3,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.7,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":31.59,"tokens":{"uncached_input":46,"cache_read":3895002,"cache_write":259341,"output":345258,"reasoning":314523},"cost_usd":0.8995,"cost_breakdown":{"input":1.7e-05,"cache_read":0.187309,"cache_write":0.232274,"output":0.479942},"cost_usd_as_run":0.8995,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":576,"code":486,"comment":59,"blank":31,"role":"impl"},{"path":"piz_test.go","lines":496,"code":425,"comment":42,"blank":29,"role":"test"}],"impl_lines":576,"impl_code":486,"impl_comment":59,"test_lines":496,"comment_ratio":0.108,"tests_declared":10},"hidden":{"passed":11,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":135.5,"reference":92,"speedup":1.473,"baselines":{"oracle":92}},{"id":"decode_float","candidate":63.3,"reference":45.2,"speedup":1.4,"baselines":{"oracle":45.2}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":369,"window":6},"complexity":{"functions":18,"mean":6.67,"max":15,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":27,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-sol-max":5.5,"muse-wonder-full-xhigh":5.5,"gpt-6.1-terra-max":5.5,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"gpt-6.1-sol-max":"Two validation mismatches and a cancellation gap.","muse-wonder-full-xhigh":"clean and adaptive, but it accepts a stream the oracle rejects and its test enshrines the bug","gpt-6.1-terra-max":"Accepts a malformed stream and tests that behavior as valid","opus-5.5-xhigh":"clean, well-documented design, but it accepts a run with no previous word, which the oracle rejects, and its own test locks that in","deepseek-v4.1-flash-max":"Clean, documented and fast, but it accepts one malformed run the oracle rejects, and its own latency test is unstable under instrumentation."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Initial zero-length run is incorrectly accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Size error hides a later invalid channel","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate planes bypass cancellation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Zero-length run before any word is accepted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Self-confirming test masks that bug","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"ErrTooLarge wins over a later invalid channel type","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Double formatting in malformedf","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A zero run is accepted before any literal","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Size error masks a later invalid channel","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Degenerate planes defer cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"The suite requires the invalid zero-run behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"Zero-length run before any word is accepted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"A test case enforces the divergence","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Size error shadows a later invalid type","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The trie cap never binds","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"Zero-length run with no previous word is accepted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Instrumentation-sensitive latency assertion","status":"valid","detail":"","location":""}],"score":5.7,"slop":{"score":0.1,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":6.1,"score_cross_vendor":5.75,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.23,"tokens":{"uncached_input":17213,"cache_read":190208,"cache_write":0,"output":4748,"reasoning":683},"cost_usd":0.0505,"cost_breakdown":{"input":0.017213,"cache_read":0.00951,"cache_write":0.0,"output":0.02374},"cost_usd_as_run":0.0505,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":368,"code":357,"comment":1,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":110,"code":107,"comment":0,"blank":3,"role":"test"}],"impl_lines":368,"impl_code":357,"impl_comment":1,"test_lines":110,"comment_ratio":0.003,"tests_declared":4},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":145.4,"reference":90.3,"speedup":1.61,"baselines":{"oracle":90.3}},{"id":"decode_float","candidate":67.8,"reference":45.8,"speedup":1.48,"baselines":{"oracle":45.8}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":268,"window":6},"complexity":{"functions":3,"mean":33.67,"max":96,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-terra-max":6.5},"council_summaries":{"muse-wonder-full-xhigh":"behaviourally correct everywhere probed, held back by missing docs, thin tests and fixed arenas","opus-5.5-xhigh":"correct, but no docs, thin tests, slow","gpt-6.1-sol-max":"Correct in the probes, with allocation, testing and documentation weaknesses.","deepseek-v4.1-flash-max":"behaviourally correct but under-delivered: almost no documentation and thin tests","gpt-6.1-terra-max":"Golden chunks work, but cancellation coverage and tests are thin."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No doc comments on any exported item","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Full-size arenas allocated per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Thin suite","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Single ~300-line Decompress monolith","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Only golden, four layout cases, truncations of one chunk, and a concurrency test that doesn't check output","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No doc comments on any exported identifier (the scaffold's were removed)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time decode through a method call per bit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"The whole decoder, wavelet included, is one 300-line function","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 135,000-node arena zeroed per call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Full trie arena allocated for tiny tables","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Malformed tests stop before Huffman parsing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public decoding interface lacks documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"The file has one comment line in 368","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"Four tests, 87.4 % coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`make([]node, 135000)` per call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Many-channel wavelet loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Important decoder cases are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Exported API lacks documentation","status":"valid","detail":"","location":""}],"score":6.8,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.8,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":80.93,"tokens":{"uncached_input":239180,"cache_read":3141180,"cache_write":0,"output":65926,"reasoning":39721},"cost_usd":2.9035,"cost_breakdown":{"input":0.95672,"cache_read":0.628236,"cache_write":0.0,"output":1.31852},"cost_usd_as_run":2.9035,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":496,"code":450,"comment":20,"blank":26,"role":"impl"},{"path":"piz_test.go","lines":824,"code":717,"comment":44,"blank":63,"role":"test"}],"impl_lines":496,"impl_code":450,"impl_comment":20,"test_lines":824,"comment_ratio":0.043,"tests_declared":21},"hidden":{"passed":11,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":139.8,"reference":90.2,"speedup":1.55,"baselines":{"oracle":90.2}},{"id":"decode_float","candidate":66,"reference":45.7,"speedup":1.444,"baselines":{"oracle":45.7}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":34,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":5.5,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-terra-max":6.0},"council_summaries":{"muse-wonder-full-xhigh":"strong engineering with one narrow oracle divergence on a leading zero-length run","opus-5.5-xhigh":"accepts a zero-length run at stream start; slow; monolithic","gpt-6.1-sol-max":"Accepts an explicitly forbidden run sequence.","deepseek-v4.1-flash-max":"correct except one narrow run rule; otherwise tidy","gpt-6.1-terra-max":"It accepts a malformed leading run and can delay cancellation."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Zero-length run as first symbol accepted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Fixed 140k-capacity trie plus 128 KiB LUT buffer per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Zero-length run tested only after a word","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"No-op GC-hint assignments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"correctness","title":"A zero-length run with no previous word is accepted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decode, and `decodePair` branches on the mode for every pair","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"A fixed 140,000-node arena (1.68 MB, zeroed) is allocated on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The run-without-previous test uses count 5 only, so it misses this bug; the wavelet is tested only through the golden vectors","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Decompress is about 300 lines with numbered section banners","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`bitmap = nil` and `trie = nil` are dead stores, and the `x < nx` / `y < ny` guards are always true","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Initial zero-length run is accepted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie arena allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Legal long-code table is called oversubscribed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"correctness","title":"`if runLen > 0 && outCount == 0` lets a run of length 0 open the stream","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`bitmap = nil` (`piz.go:131`) and `trie = nil` (`piz.go:300`) before returning","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"A zero run is accepted without a previous word","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Many-channel wavelet loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The oversubscription test describes a valid table as invalid","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not compile on 32-bit Go","status":"valid","detail":"","location":""}],"score":6.3,"slop":{"score":0.7,"per_seat":{"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.5}},"defect_points":5.1,"score_cross_vendor":6.25,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.61,"tokens":{"uncached_input":38356,"cache_read":294528,"cache_write":0,"output":9101,"reasoning":1932},"cost_usd":0.1972,"cost_breakdown":{"input":0.076712,"cache_read":0.029453,"cache_write":0.0,"output":0.09101},"cost_usd_as_run":0.1972,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":407,"code":384,"comment":10,"blank":13,"role":"impl"},{"path":"piz_test.go","lines":310,"code":303,"comment":2,"blank":5,"role":"test"}],"impl_lines":407,"impl_code":384,"impl_comment":10,"test_lines":310,"comment_ratio":0.025,"tests_declared":8},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":137.6,"reference":89.2,"speedup":1.543,"baselines":{"oracle":89.2}},{"id":"decode_float","candidate":64.5,"reference":45.8,"speedup":1.408,"baselines":{"oracle":45.8}}]},"quality":{"duplication":{"ratio":0.042,"lines":12,"significant":286,"window":6},"complexity":{"functions":5,"mean":21.0,"max":48,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-terra-max":7.5},"council_summaries":{"muse-wonder-full-xhigh":"correct and well tested for its size; fixed arena is the only real cost","opus-5.5-xhigh":"correct and compact; slow decoder","gpt-6.1-sol-max":"Correct and well tested, with oversized Huffman scratch.","deepseek-v4.1-flash-max":"correct and readable, same fixed 140k-node arena","gpt-6.1-terra-max":"The decoder passed the behavioral probes, but its timing test is flaky and its trie is expensive."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Fixed 140k-node trie per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Offset comment in wrong units","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time decode with a context-tick branch on every bit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node arena zeroed per call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No wavelet round trip beyond golden and no allocation-bound test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Only Decompress has a doc comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Fixed maximum trie allocation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`make([]huffNode, 140000)` per call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Cancellation test fails intermittently under the required race run","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Fixed trie allocation","status":"valid","detail":"","location":""}],"score":7.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.5,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":33.24,"tokens":{"uncached_input":240,"cache_read":5380899,"cache_write":217222,"output":196093,"reasoning":0},"cost_usd":0.2217,"cost_breakdown":{"input":3.6e-05,"cache_read":0.086094,"cache_write":0.043444,"output":0.092164},"cost_usd_as_run":0.2217,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2217,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":45,"code":26,"comment":11,"blank":8,"role":"impl"}],"impl_lines":45,"impl_code":26,"impl_comment":11,"test_lines":0,"comment_ratio":0.297,"tests_declared":0},"hidden":{"passed":0,"total":12,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":9740927.8,"reference":89.2,"speedup":109203.226,"baselines":{"oracle":89.2}},{"id":"decode_float","candidate":14269935.5,"reference":44.9,"speedup":317815.935,"baselines":{"oracle":44.9}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":39,"flags":4,"audited":"2026-11-02","reviewed":true},"scores":{"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-terra-max":0.0},"council_summaries":{"muse-wonder-full-xhigh":"unimplemented stub, no tests","opus-5.5-xhigh":"not implemented","gpt-6.1-sol-max":"Unimplemented stub with no test deliverable.","deepseek-v4.1-flash-max":"not implemented; stub plus leftover scratch files","gpt-6.1-terra-max":"The implementation and tests are missing."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Decompress is the scaffold stub","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"No test file","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Prototype scratch directory shipped","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"`Decompress` returns `errUnimplemented` for every input","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"No piz_test.go","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"critical","category":"correctness","title":"Decompress is entirely unimplemented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Required tests are absent","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`Decompress` returns a private `errUnimplemented`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"tests","title":"No `piz_test.go` at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"dead-code","title":"A `scratch/` directory holding 29 Python probe files (164 KB)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"`Decompress` is unimplemented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Test deliverable is absent","status":"valid","detail":"","location":""}],"score":0.4,"slop":{"score":0.6,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":2.0}},"defect_points":7.4,"score_cross_vendor":0.4,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.62,"tokens":{"uncached_input":5627,"cache_read":121600,"cache_write":0,"output":2912,"reasoning":76},"cost_usd":0.0263,"cost_breakdown":{"input":0.005627,"cache_read":0.00608,"cache_write":0.0,"output":0.01456},"cost_usd_as_run":0.0263,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":321,"code":311,"comment":0,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":31,"code":29,"comment":0,"blank":2,"role":"test"}],"impl_lines":321,"impl_code":311,"impl_comment":0,"test_lines":31,"comment_ratio":0,"tests_declared":1},"hidden":{"passed":5,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":131.7,"reference":89.1,"speedup":1.478,"baselines":{"oracle":89.1}},{"id":"decode_float","candidate":60.9,"reference":45.3,"speedup":1.344,"baselines":{"oracle":45.3}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":231,"window":6},"complexity":{"functions":3,"mean":29.0,"max":82,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":3.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":3.5,"deepseek-v4.1-flash-max":3.0,"gpt-6.1-terra-max":2.5},"council_summaries":{"muse-wonder-full-xhigh":"wavelet edge bug rejects valid OpenEXR data, and no test ever decoded a PIZ chunk","opus-5.5-xhigh":"wrong output on valid input; almost no tests or docs","gpt-6.1-sol-max":"Rejects valid image data, and its tests never exercise PIZ decoding.","deepseek-v4.1-flash-max":"valid chunks decode wrongly: the wavelet's block loop bounds are off, which the thin tests never touch","gpt-6.1-terra-max":"It rejects a valid golden chunk and lacks tests for compressed decoding."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Wavelet block bounds use p instead of p2","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Only test never exercises PIZ decoding","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Fixed arenas per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"Size error can pre-empt layout error","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"No cancellation polls in table/trie phases","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"The wavelet loops use `y+p < h` and `x+p < w` instead of `y+p2 <= h` and `x+p2 <= w`, so extra 2×2 blocks are decoded whenever the dimension mod 2p is greater than p","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"A single smoke test and no golden test, which would have caught the wavelet bug","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No doc comments anywhere","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time decode, and the `pair` closure branches on the mode for every pair","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Types and the size cap are checked in the same loop, so 16.7M+ Float channels followed by type 9 return ErrTooLarge instead of ErrInvalidLayout (verified)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Partial wavelet blocks are reconstructed incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Size errors preempt invalid channel types","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie storage allocated for every table","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"No successful compressed decoding is tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API and layout invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"Wavelet block loops use `p` instead of `p2` in their bounds","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"Sole test is 31 lines and 8.3 % coverage, with no golden vectors","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"No package doc and no doc comment on any exported identifier","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`make([]node, 1, 135000)` allocates 1.6 MB per call regardless of chunk size","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Partial wavelet blocks are decoded twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Size errors precede a later invalid channel","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Many-channel wavelet loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"No compressed chunk is tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Exported API lacks documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Redundant conversion","status":"valid","detail":"","location":""}],"score":2.9,"slop":{"score":0.1,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":10.9,"score_cross_vendor":2.833,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.52,"tokens":{"uncached_input":26,"cache_read":934205,"cache_write":93739,"output":61308,"reasoning":30353},"cost_usd":2.163,"cost_breakdown":{"input":0.000104,"cache_read":0.186841,"cache_write":0.749912,"output":1.22616},"cost_usd_as_run":2.163,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":584,"code":521,"comment":29,"blank":34,"role":"impl"},{"path":"piz_test.go","lines":1062,"code":963,"comment":40,"blank":59,"role":"test"}],"impl_lines":584,"impl_code":521,"impl_comment":29,"test_lines":1062,"comment_ratio":0.053,"tests_declared":17},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":337,"reference":90.9,"speedup":3.707,"baselines":{"oracle":90.9}},{"id":"decode_float","candidate":122.2,"reference":45.2,"speedup":2.704,"baselines":{"oracle":45.2}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":400,"window":6},"complexity":{"functions":15,"mean":7.93,"max":45,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":9.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-terra-max":7.5},"council_summaries":{"muse-wonder-full-xhigh":"correct fast-table decoder with a proved trie bound and strong tests","opus-5.5-xhigh":"correct, fast (12-bit table plus trie), strong tests; small hot-path overhead","gpt-6.1-sol-max":"Efficient, robust decoder with one masked regression test.","deepseek-v4.1-flash-max":"correct, fast, well documented; the most complete self-testing of the field","gpt-6.1-terra-max":"Thorough and relatively fast, with an unpolled many-channel path."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Run-fill work undercounted for polling","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Wavelet calls its step through a function value, plus a poller method per block","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Placeholder test cases overwritten later (\"// replaced below\", `\"table bits unused\": nil`)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Wavelet/LUT regression fails at the outer size check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"No cancellation checks between empty planes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not compile on 32-bit Go","status":"valid","detail":"","location":""}],"score":8.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.2,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":22.94,"tokens":{"uncached_input":41047,"cache_read":17698944,"cache_write":0,"output":228391,"reasoning":174811},"cost_usd":0.1963,"cost_breakdown":{"input":0.006157,"cache_read":0.053097,"cache_write":0.0,"output":0.137035},"cost_usd_as_run":0.1963,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1963,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":538,"code":462,"comment":49,"blank":27,"role":"impl"},{"path":"piz_test.go","lines":782,"code":692,"comment":38,"blank":52,"role":"test"}],"impl_lines":538,"impl_code":462,"impl_comment":49,"test_lines":782,"comment_ratio":0.096,"tests_declared":14},"hidden":{"passed":8,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":113.8,"reference":88.8,"speedup":1.282,"baselines":{"oracle":88.8}},{"id":"decode_float","candidate":55.8,"reference":43.9,"speedup":1.271,"baselines":{"oracle":43.9}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":351,"window":6},"complexity":{"functions":13,"mean":8.92,"max":32,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":90,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":5.0,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-terra-max":4.0},"council_summaries":{"muse-wonder-full-xhigh":"clean proportional design undermined by the trailing-bits defect","opus-5.5-xhigh":"ignores trailing bits; otherwise clean and documented","gpt-6.1-sol-max":"Good valid-data coverage, but incomplete stream validation.","deepseek-v4.1-flash-max":"fast and leanest on memory, but accepts over-long Huffman streams","gpt-6.1-terra-max":"It can panic on malformed 32-bit input and accepts extra Huffman data."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Trailing Huffman bits ignored","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Synthetic malformed set lacks the excess case","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"`for pos < nWords` stops decoding once the output is full and never checks the remaining bits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time decode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No trailing-data case; `TestGoldenLiftingVariant` only checks the fixtures; no allocation test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The `k == 0` branch in `readBits` is never taken, and the `numCodes == 0` check duplicates the run-symbol check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Trailing Huffman data is ignored","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"`for pos < nWords` ends decoding at exactly `n/2` words and never verifies the stream was consumed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Negative table length on 32-bit Go panics","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Trailing Huffman bits are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Many-channel wavelet loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Bit-at-a-time Huffman traversal","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not compile on 32-bit Go","status":"valid","detail":"","location":""}],"score":5.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.9,"score_cross_vendor":5.5,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":49.09,"tokens":{"uncached_input":234593,"cache_read":6094592,"cache_write":0,"output":173153,"reasoning":144106},"cost_usd":0.3046,"cost_breakdown":{"input":0.035189,"cache_read":0.182838,"cache_write":0.0,"output":0.086577},"cost_usd_as_run":0.3046,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":553,"code":461,"comment":59,"blank":33,"role":"impl"},{"path":"piz_test.go","lines":1178,"code":1032,"comment":79,"blank":67,"role":"test"}],"impl_lines":553,"impl_code":461,"impl_comment":59,"test_lines":1178,"comment_ratio":0.113,"tests_declared":20},"hidden":{"passed":8,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":130.7,"reference":91,"speedup":1.436,"baselines":{"oracle":91}},{"id":"decode_float","candidate":62.5,"reference":46.4,"speedup":1.347,"baselines":{"oracle":46.4}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":45,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":5.5,"opus-5.5-xhigh":4.5,"gpt-6.1-sol-max":4.5,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-terra-max":3.5},"council_summaries":{"muse-wonder-full-xhigh":"polished and well tested, but the same trailing-bits defect plus a cancellation blind spot","opus-5.5-xhigh":"ignores trailing bits; cancellation gap on zero-run streams","gpt-6.1-sol-max":"Multiple contract violations, compounded by misleading tests.","deepseek-v4.1-flash-max":"excellent documentation, but the same stream-exhaustion gap and no speed advantage","gpt-6.1-terra-max":"Malformed input can panic, and valid large layouts exceed the allocation and cancellation limits."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Trailing Huffman bits ignored","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Cancellation poll keyed to output words","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Slowest entry with the largest footprint","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Extensive suite misses the excess-symbol case","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Canonical codes narrowed to uint32 without stating why safe","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"The Huffman loop stops once the output is full, so leftover bits are never checked","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Context polling is keyed to output words, which zero-length runs don't advance","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time decode, and the wavelet uses a pointer-returning closure","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The comment \"Data: exactly bitCount bits decoding to exactly n/2 words\" is false; nothing checks the bit count","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No too-many-words or trailing-bits case, which hides the bug; `_ = lay` is a dead variable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Trailing Huffman data is ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Channel planning exceeds the allocation budget","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Zero-length runs suppress cancellation polling","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie arena allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Structured malformed cases never reach their intended checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Claimed 16-bit test actually selects 14-bit mode","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"`for w := 0; w < total;` stops at `n/2` words without checking the remaining bits","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`make([]trieNode, 1, maxTrieNodes)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Negative symbol index on 32-bit Go panics","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Trailing Huffman bits are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Channel plan exceeds the allocation budget","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Cancellation depends on output progress","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Malformed-stream cases are masked by the size rule","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not compile on 32-bit Go","status":"valid","detail":"","location":""}],"score":4.9,"slop":{"score":0.4,"per_seat":{"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":9.1,"score_cross_vendor":4.9,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":146.79,"tokens":{"uncached_input":219082,"cache_read":5391640,"cache_write":0,"output":85201,"reasoning":52619},"cost_usd":3.6587,"cost_breakdown":{"input":0.876328,"cache_read":1.078328,"cache_write":0.0,"output":1.70402},"cost_usd_as_run":3.6587,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":539,"code":490,"comment":18,"blank":31,"role":"impl"},{"path":"piz_test.go","lines":428,"code":392,"comment":10,"blank":26,"role":"test"}],"impl_lines":539,"impl_code":490,"impl_comment":18,"test_lines":428,"comment_ratio":0.035,"tests_declared":11},"hidden":{"passed":8,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":123.8,"reference":89.4,"speedup":1.385,"baselines":{"oracle":89.4}},{"id":"decode_float","candidate":60.1,"reference":45.3,"speedup":1.327,"baselines":{"oracle":45.3}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":51,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":4.5,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-terra-max":6.0},"council_summaries":{"muse-wonder-full-xhigh":"correct on valid data but accepts excess Huffman symbols the oracle rejects","opus-5.5-xhigh":"ignores trailing bits; slow; modest tests","gpt-6.1-sol-max":"Valid chunks decode correctly, but malformed tails are accepted.","deepseek-v4.1-flash-max":"sound decoder that accepts malformed Huffman streams","gpt-6.1-terra-max":"Golden chunks work, but extra Huffman data is accepted and cancellation can be delayed."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Trailing Huffman bits ignored","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Fixed 140k-node trie backing allocated every call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No excess-symbol/trailing-bits case","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"`for out < wordsNeeded` stops decoding once the output is full and never checks the remaining bits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time decode through the `bitAt` closure","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No trailing or too-many-words case, no encoder round trip (wavelet only through golden), no allocation test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Most helpers undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Fixed 140,000-node arena per call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Huffman decoding stops before validating the tail","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Maximum trie arena allocated for tiny tables","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"correctness","title":"The decode loop stops as soon as `wordsNeeded` words exist and never checks the bit stream is exhausted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`make([]trieNode, 1, maxTrieNodes)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Trailing Huffman bits are ignored","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Many-channel wavelet loop does not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Fixed trie allocation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not compile on 32-bit Go","status":"valid","detail":"","location":""}],"score":5.9,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.4,"score_cross_vendor":5.875,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":11.81,"tokens":{"uncached_input":38,"cache_read":1363913,"cache_write":91572,"output":56026,"reasoning":22216},"cost_usd":2.126,"cost_breakdown":{"input":0.000152,"cache_read":0.272783,"cache_write":0.732576,"output":1.12052},"cost_usd_as_run":2.126,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":488,"code":443,"comment":17,"blank":28,"role":"impl"},{"path":"piz_test.go","lines":972,"code":879,"comment":43,"blank":50,"role":"test"}],"impl_lines":488,"impl_code":443,"impl_comment":17,"test_lines":972,"comment_ratio":0.037,"tests_declared":17},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":166.4,"reference":89.3,"speedup":1.863,"baselines":{"oracle":89.3}},{"id":"decode_float","candidate":83.1,"reference":44.2,"speedup":1.88,"baselines":{"oracle":44.2}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":338,"window":6},"complexity":{"functions":10,"mean":10.4,"max":46,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":18,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-terra-max":5.5},"council_summaries":{"muse-wonder-full-xhigh":"correct, well tested, only an over-generous trie reservation","opus-5.5-xhigh":"correct and well tested; decoder reads one bit at a time; helpers undocumented","gpt-6.1-sol-max":"Correct, bounded implementation with extensive edge coverage.","deepseek-v4.1-flash-max":"correct and quick, with clean docs and a well-sized trie","gpt-6.1-terra-max":"Size validation fails on 32-bit Go, and many small planes delay cancellation."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Trie capacity formula over-reserves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman data is decoded by walking the trie one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Wavelet uses a function value plus `tick` per block","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`decoder`, `huffDecode`, `wav2Decode`, `wdec14/16` have no doc comments","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Dimension multiplication overflows on 32-bit Go","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Many-channel wavelet loop does not poll","status":"valid","detail":"","location":""}],"score":8.0,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.0,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":29.51,"tokens":{"uncached_input":70,"cache_read":5304814,"cache_write":202711,"output":148602,"reasoning":91317},"cost_usd":5.655,"cost_breakdown":{"input":0.00028,"cache_read":1.060963,"cache_write":1.621688,"output":2.97204},"cost_usd_as_run":5.655,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":783,"code":659,"comment":77,"blank":47,"role":"impl"},{"path":"piz_test.go","lines":1869,"code":1692,"comment":87,"blank":90,"role":"test"}],"impl_lines":783,"impl_code":659,"impl_comment":77,"test_lines":1869,"comment_ratio":0.105,"tests_declared":23},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":460.4,"reference":89,"speedup":5.173,"baselines":{"oracle":89}},{"id":"decode_float","candidate":161.5,"reference":43.6,"speedup":3.704,"baselines":{"oracle":43.6}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":509,"window":6},"complexity":{"functions":21,"mean":7.29,"max":23,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":36,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":9.5,"gpt-6.1-terra-max":9.5,"muse-wonder-full-xhigh":9.5,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-sol-max":"Robust, efficient implementation with only trivial test padding.","gpt-6.1-terra-max":"Correct and well tested, with minor cleanup.","muse-wonder-full-xhigh":"No contract defects found; fastest tier with the strongest tests","opus-5.5-xhigh":"Correct, about 6× the oracle's speed, robust on every layout I probed, and the strongest test suite","deepseek-v4.1-flash-max":"Correct, exhaustively tested and well documented; only small slop and a stale comment."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Discarded LUT fixture is slop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Unused test fixture","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Message-only trie walk","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Tests assert free-form message text","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable error path","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"emptyBeyond walks the trie only to choose between two ErrMalformed messages","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The cancellation-latency test asserts 60 ms while the contract says 10 ms","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"dead variable in test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"error-message-only helper","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"oversized bound in comment","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"no-op context poll","status":"valid","detail":"","location":""}],"score":9.3,"slop":{"score":0.7,"per_seat":{"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":1.2,"score_cross_vendor":9.375,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.28,"tokens":{"uncached_input":26994,"cache_read":659968,"cache_write":0,"output":29010,"reasoning":14833},"cost_usd":0.205,"cost_breakdown":{"input":0.026994,"cache_read":0.032998,"cache_write":0.0,"output":0.14505},"cost_usd_as_run":0.205,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":439,"code":401,"comment":17,"blank":21,"role":"impl"},{"path":"piz_test.go","lines":579,"code":547,"comment":3,"blank":29,"role":"test"}],"impl_lines":439,"impl_code":401,"impl_comment":17,"test_lines":579,"comment_ratio":0.041,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":153.8,"reference":84.8,"speedup":1.814,"baselines":{"oracle":84.8}},{"id":"decode_float","candidate":67.6,"reference":41.9,"speedup":1.613,"baselines":{"oracle":41.9}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":300,"window":6},"complexity":{"functions":6,"mean":18.0,"max":48,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":16,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-sol-max":8.0,"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-sol-max":"Clear, correct implementation with a slow Huffman path and masked malformed-table coverage.","gpt-6.1-terra-max":"Correct and robust, with avoidable decoding cost.","muse-wonder-full-xhigh":"Compact and correct everywhere probed; minor perf and docs gaps only","opus-5.5-xhigh":"Correct and robust, but it decodes one bit at a time and its malformed-table tests are masked","deepseek-v4.1-flash-max":"Solid, well-tested decoder; correctness is clean, but the naive bit-at-a-time Huffman loop and full-size buffers cost 5x throughput."},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Bitwise decoding dominates realistic chunks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Trailing-table-byte test cannot detect its missing check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Huffman decoding walks one bit at a time","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public decoding behavior is underdocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Bit-at-a-time decode with fixed 192 KiB of tables","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Sentinel and behaviour docs are thin","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No fuzz, benchmark or allocation test; brittle poll counts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Ignored bit-reader success flag","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman decode walks the trie one bit per iteration","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The table-rule tests pass whether or not the rule is enforced","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"One stage does too much","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"full-size buffers on every call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"bit-at-a-time Huffman decode","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"discarded error return","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no encoder round-trip","status":"valid","detail":"","location":""}],"score":7.7,"slop":{"score":0.0,"per_seat":{"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.4,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1-b2","task":"exr-piz","lead":"muse-wonder-full-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":42.33,"tokens":{"cache_read":4815940,"cache_write":0,"output":59818,"reasoning":28759,"uncached_input":244482},"cost_usd":3.1375,"cost_breakdown":{"cache_read":0.963188,"cache_write":0.0,"input":0.977928,"output":1.19636},"cost_usd_as_run":3.1375,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":16.58,"tokens":{"cache_read":4386567,"cache_write":222895,"output":75239,"reasoning":46192,"uncached_input":58},"cost_usd":4.1655,"cost_breakdown":{"cache_read":0.877313,"cache_write":1.78316,"input":0.000232,"output":1.50478},"cost_usd_as_run":4.1655,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":20.28,"tokens":{"cache_read":3462016,"cache_write":0,"output":25605,"reasoning":16579,"uncached_input":140569},"cost_usd":0.8834,"cost_breakdown":{"cache_read":0.346202,"cache_write":0.0,"input":0.281138,"output":0.25605},"cost_usd_as_run":0.8834,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":32.03,"tokens":{"cache_read":19311232,"cache_write":0,"output":159564,"reasoning":110085,"uncached_input":114044},"cost_usd":0.1708,"cost_breakdown":{"cache_read":0.057934,"cache_write":0.0,"input":0.017107,"output":0.095738},"cost_usd_as_run":0.1708,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"H","score":5.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"F","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"X","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"H","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"F","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"X","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"H","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"F","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"X","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"H","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"F","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"X","score":9.0}],"panel":["muse-wonder-full-xhigh","opus-5.5-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"L","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":3.4,"tokens":{"uncached_input":60435,"cache_read":310011,"cache_write":0,"output":20903,"reasoning":0},"cost_usd":0.3609,"cost_breakdown":{"input":0.12087,"cache_read":0.031001,"cache_write":0.0,"output":0.20903},"cost_usd_as_run":0.3609,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":502,"code":436,"comment":26,"blank":40,"role":"impl"},{"path":"piz_test.go","lines":511,"code":464,"comment":6,"blank":41,"role":"test"}],"impl_lines":502,"impl_code":436,"impl_comment":26,"test_lines":511,"comment_ratio":0.056,"tests_declared":6},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":151.7,"reference":91.5,"speedup":1.658,"baselines":{"oracle":91.5}},{"id":"decode_float","candidate":68.3,"reference":46.1,"speedup":1.482,"baselines":{"oracle":46.1}}]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"muse-wonder-full-xhigh":"Correct and robust everywhere, held back by monolith structure, thin docs and some lazy tests","opus-5.5-xhigh":"Correct and robust, but one 450-line function, a bit-by-bit decoder, thin tests and no docs","gpt-6.1-sol-max":"Correct in probes, with maintainability and cancellation-test weaknesses","deepseek-v4.1-flash-max":"correct, robust and 2–3× the oracle on real chunks, but the public doc states none of the contract's guarantees and the test suite skips mid‑decode cancellation and the memory bound."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Full-size LUT allocated per call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Single ~450-line `Decompress`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Stub-level public documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"`TestRandomGarbage` asserts nothing, and several contract areas are untested","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable and redundant guards","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman decoding walks the trie one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"style","title":"The whole decoder is one function","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No doc comments except on `Decompress`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Two no-ops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Mid-decode cancellation is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"style","title":"Every decoding phase occupies one large function","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Decompress documents nothing beyond \"decodes one PIZ chunk\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"cancellation and the allocation bound are only exercised at the entry point","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`lut[0] = 0`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`wh > MaxChunkBytes`","status":"valid","detail":"","location":""}],"score":7.75,"slop":{"score":0.5,"per_seat":{"muse-wonder-full-xhigh":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":4.25,"score_cross_vendor":7.75,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","task":"exr-piz","lead":"muse-wonder-full-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":10.84,"tokens":{"cache_read":5125376,"cache_write":0,"output":24573,"reasoning":13543,"uncached_input":110533},"cost_usd":0.4897,"cost_breakdown":{"cache_read":0.256269,"cache_write":0.0,"input":0.110533,"output":0.122865},"cost_usd_as_run":0.4897,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":200.72,"tokens":{"cache_read":20267040,"cache_write":0,"output":201315,"reasoning":106660,"uncached_input":1614571},"cost_usd":14.5379,"cost_breakdown":{"cache_read":4.053408,"cache_write":0.0,"input":6.458284,"output":4.0263},"cost_usd_as_run":14.5379,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":4}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":83.42,"tokens":{"cache_read":22146688,"cache_write":0,"output":160321,"reasoning":94867,"uncached_input":800585},"cost_usd":5.4191,"cost_breakdown":{"cache_read":2.214669,"cache_write":0.0,"input":1.60117,"output":1.60321},"cost_usd_as_run":5.4191,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":105.84,"tokens":{"cache_read":74869760,"cache_write":0,"output":669718,"reasoning":472057,"uncached_input":505004},"cost_usd":0.7022,"cost_breakdown":{"cache_read":0.224609,"cache_write":0.0,"input":0.075751,"output":0.401831},"cost_usd_as_run":0.7022,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":4}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":128.07,"tokens":{"cache_read":51135212,"cache_write":1248108,"output":438546,"reasoning":260722,"uncached_input":476},"cost_usd":28.9848,"cost_breakdown":{"cache_read":10.227042,"cache_write":9.984864,"input":0.001904,"output":8.77092},"cost_usd_as_run":28.9848,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":4}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"H","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"F","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"X","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"H","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"F","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"X","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"H","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"F","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"X","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"H","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"F","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfK","source_run":"2026-11-02-exr-piz-muse-wonder-full-xhigh-r1","letter":"X","score":9.5}],"panel":["muse-wonder-full-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max","opus-5.5-xhigh"],"guardrail_blocks":[{"date":"2026-11-03T19:44:03+00:00","provider":"openai","attempt":1,"message":"This content was flagged for possible cybersecurity risk (OpenAI cyber safeguards; suggests Daybreak access)","outcome":"retry-blocked","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1"},{"date":"2026-11-03T19:46:23+00:00","provider":"openai","attempt":1,"message":"This content was flagged for possible cybersecurity risk (OpenAI cyber safeguards; suggests Daybreak access); retry cancelled after the seat was dropped","outcome":"dnf","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2"},{"date":"2026-11-03T19:51:31+00:00","provider":"openai","attempt":2,"message":"This content was flagged for possible cybersecurity risk (OpenAI cyber safeguards; suggests Daybreak access)","outcome":"retry-blocked","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1"},{"date":"2026-11-03T19:58:17+00:00","provider":"openai","attempt":3,"message":"This content was flagged for possible cybersecurity risk (OpenAI cyber safeguards; reshuffled retry also blocked; seat dropped from this run's panel)","outcome":"dnf","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1b"}]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":6.87,"tokens":{"uncached_input":28,"cache_read":965272,"cache_write":83013,"output":44836,"reasoning":21566},"cost_usd":1.754,"cost_breakdown":{"input":0.000112,"cache_read":0.193054,"cache_write":0.664104,"output":0.89672},"cost_usd_as_run":1.754,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":492,"code":425,"comment":36,"blank":31,"role":"impl"},{"path":"piz_test.go","lines":691,"code":604,"comment":38,"blank":49,"role":"test"}],"impl_lines":492,"impl_code":425,"impl_comment":36,"test_lines":691,"comment_ratio":0.078,"tests_declared":14},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":138.3,"reference":92.6,"speedup":1.494,"baselines":{"oracle":92.6}},{"id":"decode_float","candidate":63.1,"reference":46.4,"speedup":1.36,"baselines":{"oracle":46.4}}]},"scores":{"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":8.0},"council_summaries":{"muse-wonder-full-xhigh":"Solid trie decoder; only test-depth gaps on long codes","deepseek-v4.1-flash-max":"good docs and 98.6% coverage; the trie is over-preallocated for small alphabets","gpt-6.1-sol-max":"Thorough decoding checks, with a cancellation gap and one ineffective positive test","opus-5.5-xhigh":"correct, clear and well-commented decoder, good tests"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No valid long-code test; random-valid accepts malformed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`nodes := make([]trieNode, 1, 2*coded+maxCodeLen*(maxCodeLen+1)+2)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate planes bypass cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"“Valid streams” test accepts universal rejection","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Bit-at-a-time trie decoding with inline bit extraction","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"There is no valid 58-bit or full-alphabet decode, and `TestRandomValidStreams` checks only error class, not output","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"`malformed`, `decodePIZ` and `dec` have no doc comments","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0492,"lines":16,"significant":325,"window":6},"complexity":{"functions":9,"mean":11.67,"max":64,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.125,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":1.75,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":15.0,"tokens":{"uncached_input":14890,"cache_read":502912,"cache_write":0,"output":34275,"reasoning":17703},"cost_usd":0.4228,"cost_breakdown":{"input":0.02978,"cache_read":0.050291,"cache_write":0.0,"output":0.34275},"cost_usd_as_run":0.4228,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":421,"code":387,"comment":14,"blank":20,"role":"impl"},{"path":"piz_test.go","lines":648,"code":607,"comment":11,"blank":30,"role":"test"}],"impl_lines":421,"impl_code":387,"impl_comment":14,"test_lines":648,"comment_ratio":0.035,"tests_declared":14},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":146.4,"reference":91.5,"speedup":1.6,"baselines":{"oracle":91.5}},{"id":"decode_float","candidate":67.5,"reference":46,"speedup":1.467,"baselines":{"oracle":46}}]},"scores":{"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":10.0,"opus-5.5-xhigh":8.0},"council_summaries":{"muse-wonder-full-xhigh":"Compact correct decoder with the sharpest white-box tests","deepseek-v4.1-flash-max":"thorough tests and clean structure; only a fragile-looking slice bound","gpt-6.1-sol-max":"Strong implementation and unusually effective adversarial tests","opus-5.5-xhigh":"correct and robust, very good tests, internals undocumented"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`inverseWavelet(ctx, words[base+j:base+planeWords], …)`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Huffman decoding still walks a trie one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`decodeHuffman`, `inverseWavelet`, `inversePair` and `sampleBytes` have no doc comments, and the invariants (relative symbol indexing, step layout) are not explained","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`if c&1023 == 0` inside the per-channel loop polls only at channel 0, 1024, …","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":288,"window":6},"complexity":{"functions":6,"mean":18.33,"max":45,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.75,"slop":{"score":0.125,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":0.75,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":9.82,"tokens":{"uncached_input":25959,"cache_read":395520,"cache_write":0,"output":21986,"reasoning":9923},"cost_usd":0.3113,"cost_breakdown":{"input":0.051918,"cache_read":0.039552,"cache_write":0.0,"output":0.21986},"cost_usd_as_run":0.3113,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":424,"code":395,"comment":12,"blank":17,"role":"impl"},{"path":"piz_test.go","lines":476,"code":447,"comment":7,"blank":22,"role":"test"}],"impl_lines":424,"impl_code":395,"impl_comment":12,"test_lines":476,"comment_ratio":0.029,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":126.3,"reference":90.8,"speedup":1.391,"baselines":{"oracle":90.8}},{"id":"decode_float","candidate":65.7,"reference":45.7,"speedup":1.438,"baselines":{"oracle":45.7}}]},"scores":{"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":7.0},"council_summaries":{"muse-wonder-full-xhigh":"Leanest allocations; tests built from explicit bits","deepseek-v4.1-flash-max":"smallest heap footprint (128 KiB LUT lives on the stack), solid code, a few untested rejection paths","gpt-6.1-sol-max":"Compact, carefully bounded implementation with strong edge-case tests","opus-5.5-xhigh":"correct; compact trie with leaves stored on edges; modest docs and tests"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"three error returns are never reached","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`var lut [65536]uint16`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Bit-at-a-time decoding through the general `read(1)`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`sampleBytes`, `decodeHuffman`, `inverseWavelet` and `waveletPair` have no doc comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Every valid handcrafted case decodes to all zeros through an empty bitmap, so symbol identity is checked only through the LUT range; there are no wavelet vectors beyond golden","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":291,"window":6},"complexity":{"functions":6,"mean":18.5,"max":46,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.375,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":1.125,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":4.2,"tokens":{"uncached_input":22,"cache_read":621590,"cache_write":64773,"output":29078,"reasoning":9412},"cost_usd":1.2241,"cost_breakdown":{"input":8.8e-05,"cache_read":0.124318,"cache_write":0.518184,"output":0.58156},"cost_usd_as_run":1.2241,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":460,"code":410,"comment":21,"blank":29,"role":"impl"},{"path":"piz_test.go","lines":535,"code":472,"comment":22,"blank":41,"role":"test"}],"impl_lines":460,"impl_code":410,"impl_comment":21,"test_lines":535,"comment_ratio":0.049,"tests_declared":10},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":144.1,"reference":91.3,"speedup":1.578,"baselines":{"oracle":91.3}},{"id":"decode_float","candidate":65.3,"reference":45.8,"speedup":1.426,"baselines":{"oracle":45.8}}]},"scores":{"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":7.0},"council_summaries":{"muse-wonder-full-xhigh":"Clean decoder state; one row-wise poll gap","deepseek-v4.1-flash-max":"correct and reasonably tested, but the docs are thin and the row stride is derived by division","gpt-6.1-sol-max":"Correct decoding, with cancellation checks measured in unsuitable units","opus-5.5-xhigh":"correct and fairly fast; polls per row; partial docs"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Reorder can skip context checks for 2048 rows","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`rowBytes := n / h`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"the internals are largely undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"5 error returns never reached (coverage 95.1%)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Rows and skipped transforms leave long cancellation gaps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"The output loop calls `d.poll()` once per row (it fires every 2048 calls)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`bytesPerSample`, `decode`, `bits`, `huffmanDecode`, `dec14` and `dec16` have no doc comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Long codes appear only in the {1,58} rejection case; there is no valid 58-bit or full-alphabet decode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Bit-at-a-time decoding; the wavelet goes through a `dec` function value and an `idx` closure","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":310,"window":6},"complexity":{"functions":11,"mean":10.09,"max":45,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.5,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":2.5,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.18,"tokens":{"uncached_input":19731,"cache_read":235648,"cache_write":0,"output":5595,"reasoning":652},"cost_usd":0.0595,"cost_breakdown":{"input":0.019731,"cache_read":0.011782,"cache_write":0.0,"output":0.027975},"cost_usd_as_run":0.0595,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":365,"code":353,"comment":2,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":183,"code":179,"comment":0,"blank":4,"role":"test"}],"impl_lines":365,"impl_code":353,"impl_comment":2,"test_lines":183,"comment_ratio":0.006,"tests_declared":4},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":118.9,"reference":91.2,"speedup":1.304,"baselines":{"oracle":91.2}},{"id":"decode_float","candidate":60.3,"reference":45.8,"speedup":1.317,"baselines":{"oracle":45.8}}]},"scores":{"muse-wonder-full-xhigh":6.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":5.5},"council_summaries":{"muse-wonder-full-xhigh":"Correct on every probe, but undocumented and thinly tested","deepseek-v4.1-flash-max":"correct decoder, but it ships no documentation at all and its tests miss most of the malformed rules","gpt-6.1-sol-max":"Compact implementation, but cancellation and documentation need work","opus-5.5-xhigh":"correct, but no docs at all, one monolithic function, thin tests"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinnest assertion coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no package comment and no doc comment on a single exported identifier","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"the suite never exercises most of the rules it exists to guard","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the boundary rules are untested and a mutation survives","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"`TestRandomAndConcurrent`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Skipped wavelet levels leave channel traversal unpolled","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public interface is undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No exported identifier has a doc comment (PixelType and its constants, Layout, MaxChunkBytes, the errors, Decompress), and there is no package comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The tests are thin","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`Decompress` is a single ~310-line function covering framing, table, trie, Huffman, wavelet, LUT and output; the 14/16-bit choice lives in a closure checked on every lifting step","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":258,"window":6},"complexity":{"functions":3,"mean":34.33,"max":98,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.25,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":4.375,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":44.93,"tokens":{"uncached_input":95546,"cache_read":1744848,"cache_write":0,"output":32507,"reasoning":13676},"cost_usd":1.3813,"cost_breakdown":{"input":0.382184,"cache_read":0.34897,"cache_write":0.0,"output":0.65014},"cost_usd_as_run":1.3813,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":509,"code":453,"comment":24,"blank":32,"role":"impl"},{"path":"piz_test.go","lines":621,"code":548,"comment":35,"blank":38,"role":"test"}],"impl_lines":509,"impl_code":453,"impl_comment":24,"test_lines":621,"comment_ratio":0.05,"tests_declared":21},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":118.9,"reference":80.2,"speedup":1.483,"baselines":{"oracle":80.2}},{"id":"decode_float","candidate":55.4,"reference":41.3,"speedup":1.341,"baselines":{"oracle":41.3}}]},"scores":{"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":6.5},"council_summaries":{"muse-wonder-full-xhigh":"Clean and correct; tests stay on Height=1 layouts","deepseek-v4.1-flash-max":"correct and well tested, but it allocates a per-symbol code array that costs ~0.5 MiB per chunk","gpt-6.1-sol-max":"Correct decoding, with avoidable scratch storage and a cancellation gap","opus-5.5-xhigh":"correct, oracle-shaped code, sparse docs, middling tests"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Narrow handcrafted and cancellation coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"`codes := make([]uint64, nsym)`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"six validation paths are never exercised","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate channel transforms do not poll","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Materialized codes duplicate temporary storage","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Huffman decoding is one bit per `read(1)` call, and it builds an extra `codes []uint64` array (512 KiB for a full alphabet) that the trie loop could compute inline","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`malformed`, `bytesPerSample`, `decompressPIZ`, `huffmanDecode`, `waveletDec` and `wavDecode` have no doc comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"There is no valid long-code or full-alphabet decode and no mid-decode cancellation test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.25,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":2.5,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":22.83,"tokens":{"uncached_input":82,"cache_read":6441101,"cache_write":216709,"output":299852,"reasoning":267857},"cost_usd":0.858,"cost_breakdown":{"input":3.4e-05,"cache_read":0.312234,"cache_write":0.186436,"output":0.35933},"cost_usd_as_run":0.858,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":518,"code":423,"comment":59,"blank":36,"role":"impl"},{"path":"piz_test.go","lines":481,"code":432,"comment":19,"blank":30,"role":"test"}],"impl_lines":518,"impl_code":423,"impl_comment":59,"test_lines":481,"comment_ratio":0.122,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":132.6,"reference":91.5,"speedup":1.449,"baselines":{"oracle":91.5}},{"id":"decode_float","candidate":62.2,"reference":46,"speedup":1.352,"baselines":{"oracle":46}}]},"scores":{"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":7.5},"council_summaries":{"muse-wonder-full-xhigh":"Tidy poller design; tests miss mid-decode paths","deepseek-v4.1-flash-max":"best-in-class validation coverage, but the 14/16-bit boundary is untested","gpt-6.1-sol-max":"Readable and careful, but cancellation coverage misses degenerate planes","opus-5.5-xhigh":"correct, well documented, clean stages"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No mid-decode cancel or valid long-code test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"`small := len(lut) <= 16384`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Channel traversal never advances the poller","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The only cancellation test is `TestCancelledContextOnEntry`; there is no mid-decode cancellation, and no valid long-code or 16-bit-wavelet decode beyond golden","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Bit-at-a-time decoding: `bits()` loops over `bit()`, and `tick()` runs on every word","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":317,"window":6},"complexity":{"functions":15,"mean":7.47,"max":31,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":43,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.75,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":1.5,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":24.09,"tokens":{"uncached_input":216,"cache_read":3687206,"cache_write":143049,"output":123895,"reasoning":0},"cost_usd":0.1459,"cost_breakdown":{"input":3.2e-05,"cache_read":0.058995,"cache_write":0.02861,"output":0.058231},"cost_usd_as_run":0.1459,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1459,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":519,"code":444,"comment":43,"blank":32,"role":"impl"},{"path":"piz_test.go","lines":1080,"code":932,"comment":78,"blank":70,"role":"test"}],"impl_lines":519,"impl_code":444,"impl_comment":43,"test_lines":1080,"comment_ratio":0.088,"tests_declared":30},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":90.6,"reference":89.3,"speedup":1.015,"baselines":{"oracle":89.3}},{"id":"decode_float","candidate":38.2,"reference":44.3,"speedup":0.862,"baselines":{"oracle":44.3}}]},"scores":{"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":6.0},"council_summaries":{"muse-wonder-full-xhigh":"Correct everywhere probed, but cancellation can lag ~67 ms","deepseek-v4.1-flash-max":"the leanest allocations of the field with the most complete test suite","gpt-6.1-sol-max":"Core decoding matches, but cancellation can stall and a test comment misstates coverage","opus-5.5-xhigh":"correct, but cancellation fails on zero-length runs; slowest entry"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Zero-run flood starves context polls","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Wavelet values and mid-decode cancel unverified","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"`TestHandcraftedOverSubscribedPassesArithmeticChecks`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Polling depends on output growth","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Claimed 14-bit test exercises 16-bit mode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Huffman loop polls the context only when the output count is a multiple of 4096","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Huffman decoding reads one bit per `read(1)` call while walking a trie; the wavelet calls `dec` through a function variable plus an `idx` closure","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"`TestHandcraftedFullRoundTripWithWavelet` only checks that two decodes give the same bytes, and its comment says \"real 14-bit wavelet run\" although its full bitmap (k=65536) forces 16-bit mode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`if capacity < 1` can never be true, since capacity is 2*coded+128","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`equalBytes` reimplements `bytes.Equal`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":35,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.0,"slop":{"score":0.5,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0}},"defect_points":2.875,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":10.35,"tokens":{"uncached_input":42170,"cache_read":6140032,"cache_write":0,"output":109556,"reasoning":77842},"cost_usd":0.0905,"cost_breakdown":{"input":0.006326,"cache_read":0.01842,"cache_write":0.0,"output":0.065734},"cost_usd_as_run":0.0905,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0905,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":586,"code":521,"comment":35,"blank":30,"role":"impl"},{"path":"piz_test.go","lines":941,"code":823,"comment":40,"blank":78,"role":"test"}],"impl_lines":586,"impl_code":521,"impl_comment":35,"test_lines":941,"comment_ratio":0.063,"tests_declared":17},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":142.9,"reference":92.7,"speedup":1.542,"baselines":{"oracle":92.7}},{"id":"decode_float","candidate":64.8,"reference":46,"speedup":1.409,"baselines":{"oracle":46}}]},"scores":{"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":7.0},"council_summaries":{"muse-wonder-full-xhigh":"Well-structured; duplicated wavelet and a row-wise poll gap","deepseek-v4.1-flash-max":"clear, well documented and well tested; only a growing LUT slice","gpt-6.1-sol-max":"Clear stage separation, with insufficient cancellation checks in output and plane traversal","opus-5.5-xhigh":"correct and well documented; output loop polls every 1024 rows"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Reorder polls only every 1024 rows","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated inverse-wavelet bodies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"16-bit wavelet test checks length only","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`reverseLUT` starts at `make([]uint16, 0, 256)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Cancellation intervals ignore row and channel cost","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"`convertAndInterleave` polls only every 1024 rows","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The \"sixteen bit wavelet with odd sides\" test checks only output length","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0925,"lines":36,"significant":389,"window":6},"complexity":{"functions":16,"mean":7.94,"max":40,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":53,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.625,"slop":{"score":0.125,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":1.875,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.66,"tokens":{"uncached_input":15493,"cache_read":187776,"cache_write":0,"output":5136,"reasoning":273},"cost_usd":0.1011,"cost_breakdown":{"input":0.030986,"cache_read":0.018778,"cache_write":0.0,"output":0.05136},"cost_usd_as_run":0.1011,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":398,"code":382,"comment":5,"blank":11,"role":"impl"},{"path":"piz_test.go","lines":192,"code":186,"comment":1,"blank":5,"role":"test"}],"impl_lines":398,"impl_code":382,"impl_comment":5,"test_lines":192,"comment_ratio":0.013,"tests_declared":7},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":133.1,"reference":91.1,"speedup":1.461,"baselines":{"oracle":91.1}},{"id":"decode_float","candidate":63.2,"reference":46.3,"speedup":1.365,"baselines":{"oracle":46.3}}]},"scores":{"muse-wonder-full-xhigh":6.5,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":5.0},"council_summaries":{"muse-wonder-full-xhigh":"Correct but wasteful per call, with thin docs and tests","deepseek-v4.1-flash-max":"correct, but it allocates a 2.4 MB arena per call, documents almost nothing, and tests least","gpt-6.1-sol-max":"Functional decoder, but oversized scratch allocation and sparse documentation hurt quality","opus-5.5-xhigh":"correct, but undocumented API, 2.4 MB allocated every call, thin tests"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Fixed 200k-node arena per decode","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Exported API mostly undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Weak assertions, no mid-decode cancel","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"`nodes := make([]node, 1, 200000)`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"the public surface is effectively undocumented and one comment claims a bound the code does not enforce","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"coverage 90.8% with 10 error returns unreached","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`lengths := make([]uint8, imax+1)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"One-pixel channel traversal ignores cancellation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Every table reserves a 2.4 MB trie arena","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public types lack their contract documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Exported types, constants and errors have no doc comments, and the only `Decompress` doc (\"returns an independent copy…\") describes just the stored path","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Truncation and concurrency run only on the all-zero chunk, `TestGarbage` ignores results, and the only content check is one zero-run case","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"`make([]node, 1, 200000)` allocates and zeroes a 2.4 MB arena on every PIZ call regardless of table size","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"The parameter named `max` shadows the builtin, and `max < 16384` is re-evaluated on every `lift` call","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0429,"lines":12,"significant":280,"window":6},"complexity":{"functions":5,"mean":22.4,"max":48,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.0,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":4.75,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":35.28,"tokens":{"uncached_input":210564,"cache_read":4310464,"cache_write":0,"output":103991,"reasoning":83019},"cost_usd":0.2129,"cost_breakdown":{"input":0.031585,"cache_read":0.129314,"cache_write":0.0,"output":0.051996},"cost_usd_as_run":0.2129,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":496,"code":403,"comment":62,"blank":31,"role":"impl"},{"path":"piz_test.go","lines":764,"code":648,"comment":61,"blank":55,"role":"test"}],"impl_lines":496,"impl_code":403,"impl_comment":62,"test_lines":764,"comment_ratio":0.133,"tests_declared":10},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":129.5,"reference":87.8,"speedup":1.475,"baselines":{"oracle":87.8}},{"id":"decode_float","candidate":59.6,"reference":44.1,"speedup":1.351,"baselines":{"oracle":44.1}}]},"scores":{"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":4.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"opus-5.5-xhigh":"Matches the oracle with good docs, but the decoder is slow and cancellation can lag on very wide rows","muse-wonder-full-xhigh":"Clean and correct on amd64, but same 32-bit panic, wasteful LUT allocation and thinner edge tests","gpt-6.1-sol-max":"Decoding matched the oracle, but 32-bit panics and unpolled loops violate robustness requirements.","deepseek-v4.1-flash-max":"correct and clean; one context-polling gap and a needless full-size LUT"},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman decoding walks the trie one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"The reorder stage checks the context only once per row","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`pixelBytes` is computed a second time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Coverage misses several areas","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"A test comment says the cancelled context is caught at \"the reverse LUT loop's first poll\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The `g := g` copy is unnecessary","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"abi","title":"Panic on 32-bit from crafted Huffman header","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Huffman data decoded one bit at a time","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Reverse LUT always allocates 65536-entry capacity (128 KiB)","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Missing crafted edge coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Per-pixel byte count recomputed in `decompressPIZ`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Every malformed path returns the bare sentinel","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Huffman header panics on 32-bit Go","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Row and plane loops exceed the cancellation budget","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Huffman decoding branches for every bit","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"final reorder polls only once per scanline","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"full-size LUT allocated unconditionally","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"bytes-per-pixel computed three times","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":49,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.75,"slop":{"score":0.625,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.5}},"defect_points":5.25,"score_cross_vendor":6.75,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":58.52,"tokens":{"uncached_input":0,"cache_read":0,"cache_write":0,"output":0,"reasoning":0},"cost_usd":0.0,"cost_breakdown":{"input":0.0,"cache_read":0.0,"cache_write":0.0,"output":0.0},"cost_usd_as_run":0.0,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":null},"status":"scored","code":{"files":[{"path":"piz.go","lines":585,"code":538,"comment":19,"blank":28,"role":"impl"},{"path":"piz_test.go","lines":785,"code":719,"comment":17,"blank":49,"role":"test"}],"impl_lines":585,"impl_code":538,"impl_comment":19,"test_lines":785,"comment_ratio":0.034,"tests_declared":10},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":102,"reference":90.4,"speedup":1.128,"baselines":{"oracle":90.4}},{"id":"decode_float","candidate":42.9,"reference":45.1,"speedup":0.951,"baselines":{"oracle":45.1}}]},"scores":{"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":6.0},"council_summaries":{"gpt-6.1-sol-max":"Correct decoding with avoidable scratch allocation and unnecessary code.","muse-wonder-full-xhigh":"correct; slowest entry due to extra materialisation","deepseek-v4.1-flash-max":"correct, well documented and well tested; pays for a growing LUT and carries an unreachable error path.","opus-5.5-xhigh":"Correct but the slowest entry (at or below oracle speed), with convoluted trie code and dead branches"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Intermediate codes duplicate construction state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unreachable node-limit rejection","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Cancellation coverage stops at entry","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"redundant code list and extra passes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"convoluted trie insertion","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"reverse LUT is grown by `append` on the hot path","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"node cap returns `ErrMalformed` for a table the spec says must decode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"At or below oracle speed: 64 MB/s RGBA half (oracle 61) and 32 MB/s float (oracle 38)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Three branches can never run","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Trie insertion takes about 50 lines because it re-derives the parent's child pointer after each `alloc`, since the slice may move","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test cancels during decoding, and the wavelet is exercised only by the goldens and all-zero images","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":37,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.25,"slop":{"score":0.375,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0}},"defect_points":2.75,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":71.53,"tokens":{"uncached_input":98265,"cache_read":1889672,"cache_write":0,"output":34704,"reasoning":15987},"cost_usd":1.4651,"cost_breakdown":{"input":0.39306,"cache_read":0.377934,"cache_write":0.0,"output":0.69408},"cost_usd_as_run":1.4651,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":501,"code":451,"comment":25,"blank":25,"role":"impl"},{"path":"piz_test.go","lines":625,"code":557,"comment":33,"blank":35,"role":"test"}],"impl_lines":501,"impl_code":451,"impl_comment":25,"test_lines":625,"comment_ratio":0.053,"tests_declared":19},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":149.1,"reference":90,"speedup":1.657,"baselines":{"oracle":90}},{"id":"decode_float","candidate":65.8,"reference":43.5,"speedup":1.513,"baselines":{"oracle":43.5}}]},"scores":{"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":7.0},"council_summaries":{"gpt-6.1-sol-max":"Correct, economical implementation with gaps in robustness tests.","muse-wonder-full-xhigh":"correct and well-labelled errors, but one giant function","deepseek-v4.1-flash-max":"correct, fast and well documented, with a systematic hand-crafted malformed suite.","opus-5.5-xhigh":"Correct and documented, but one 370-line function and a bit-at-a-time decoder"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Resource guarantees lack regression coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"monolithic Decompress","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"cancellation only tested on entry","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`bytesPerSample` silently returns 0 for an impossible type","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding through a `getBit` closure","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Every phase lives in one ~370-line `Decompress` with three closures (`readBits`, `newNode`, `getBit`)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test decodes valid 58-bit codes (only the {1,58} rejection), and none covers the full alphabet or cancellation during decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"A test comment names `symWriter` but the type is `symEncoder`, and `layout4x1` is actually a 32×1 layout","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":21,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.0,"slop":{"score":0.125,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":2.0,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.45,"tokens":{"uncached_input":9905,"cache_read":241536,"cache_write":0,"output":9872,"reasoning":2103},"cost_usd":0.1427,"cost_breakdown":{"input":0.01981,"cache_read":0.024154,"cache_write":0.0,"output":0.09872},"cost_usd_as_run":0.1427,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":419,"code":401,"comment":6,"blank":12,"role":"impl"},{"path":"piz_test.go","lines":384,"code":366,"comment":10,"blank":8,"role":"test"}],"impl_lines":419,"impl_code":401,"impl_comment":6,"test_lines":384,"comment_ratio":0.015,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":136.7,"reference":90.5,"speedup":1.51,"baselines":{"oracle":90.5}},{"id":"decode_float","candidate":64,"reference":46.2,"speedup":1.385,"baselines":{"oracle":46.2}}]},"scores":{"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-sol-max":"Robust decoding with an unnecessarily large allocation on every compressed call.","muse-wonder-full-xhigh":"correct; fixed 2.4 MB trie allocation drags every call","deepseek-v4.1-flash-max":"correct, but a fixed 2.36 MiB allocation per call makes it the slowest entry, and the exported API is undocumented.","opus-5.5-xhigh":"Correct, but allocates 2.3 MB on every call and one test is masked"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Tiny tables allocate a maximum-sized trie","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public data-model semantics are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"constant 196608-node trie capacity","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"sparse public documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"the Huffman trie is preallocated at a fixed 196 608 nodes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"exported identifiers carry no documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The trie is preallocated at a fixed 196,608 nodes, about 2.36 MB, zeroed on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The \"table length mismatch\" case never reaches the table-length rule","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported identifiers other than `Decompress` have no doc comments","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0404,"lines":12,"significant":297,"window":6},"complexity":{"functions":5,"mean":22.4,"max":47,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.875,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":3.5,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":48.84,"tokens":{"uncached_input":143334,"cache_read":2074096,"cache_write":0,"output":46686,"reasoning":27711},"cost_usd":1.9219,"cost_breakdown":{"input":0.573336,"cache_read":0.414819,"cache_write":0.0,"output":0.93372},"cost_usd_as_run":1.9219,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":534,"code":465,"comment":31,"blank":38,"role":"impl"},{"path":"piz_test.go","lines":516,"code":447,"comment":24,"blank":45,"role":"test"}],"impl_lines":534,"impl_code":465,"impl_comment":31,"test_lines":516,"comment_ratio":0.063,"tests_declared":21},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":140.8,"reference":90.5,"speedup":1.556,"baselines":{"oracle":90.5}},{"id":"decode_float","candidate":62.8,"reference":44.7,"speedup":1.405,"baselines":{"oracle":44.7}}]},"scores":{"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":7.0},"council_summaries":{"gpt-6.1-sol-max":"Correct output, but cancellation polling has substantial gaps.","muse-wonder-full-xhigh":"well-factored, fast for a trie decoder","deepseek-v4.1-flash-max":"correct, clean, and the LUT is sized exactly; no findings.","opus-5.5-xhigh":"Clean and correct; bit-at-a-time decoding and test gaps"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Small channels bypass cancellation checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"discarded read error","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"One `readBit` call per bit while walking the trie","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test cancels during decoding, decodes valid 58-bit codes, or uses the full alphabet","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`c := c` loop-variable copies are redundant since Go 1.22","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":21,"window":6},"complexity":{"functions":1,"mean":1.0,"max":1,"over_10":0,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.625,"slop":{"score":0.125,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":1.5,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.88,"tokens":{"uncached_input":14583,"cache_read":652800,"cache_write":0,"output":26880,"reasoning":12933},"cost_usd":0.1816,"cost_breakdown":{"input":0.014583,"cache_read":0.03264,"cache_write":0.0,"output":0.1344},"cost_usd_as_run":0.1816,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":439,"code":405,"comment":12,"blank":22,"role":"impl"},{"path":"piz_test.go","lines":545,"code":507,"comment":8,"blank":30,"role":"test"}],"impl_lines":439,"impl_code":405,"impl_comment":12,"test_lines":545,"comment_ratio":0.029,"tests_declared":14},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":139.2,"reference":90.9,"speedup":1.531,"baselines":{"oracle":90.9}},{"id":"decode_float","candidate":63.8,"reference":45.9,"speedup":1.39,"baselines":{"oracle":45.9}}]},"scores":{"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":7.5},"council_summaries":{"gpt-6.1-sol-max":"Carefully bounded implementation with strong edge-case coverage.","muse-wonder-full-xhigh":"correct, well-tested trie decoder; one dead guard","deepseek-v4.1-flash-max":"correct and clean; thin on one doc comment and re-tests the wavelet mode in the inner loop.","opus-5.5-xhigh":"Correct and well tested; bit-at-a-time decoding"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"unreachable length guard","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`Layout` doc drops the contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"the 14/16-bit mode is re-decided four times per 2×2 block","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding with a work counter checked on every bit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Context polled once every 2,048 channels before each wavelet pass, which does nothing useful","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":302,"window":6},"complexity":{"functions":5,"mean":22.2,"max":51,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":16,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.375,"slop":{"score":0.25,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":1.125,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.87,"tokens":{"uncached_input":28384,"cache_read":288512,"cache_write":0,"output":10274,"reasoning":2117},"cost_usd":0.0942,"cost_breakdown":{"input":0.028384,"cache_read":0.014426,"cache_write":0.0,"output":0.05137},"cost_usd_as_run":0.0942,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":411,"code":388,"comment":10,"blank":13,"role":"impl"},{"path":"piz_test.go","lines":297,"code":278,"comment":4,"blank":15,"role":"test"}],"impl_lines":411,"impl_code":388,"impl_comment":10,"test_lines":297,"comment_ratio":0.025,"tests_declared":8},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":135.3,"reference":91.2,"speedup":1.484,"baselines":{"oracle":91.2}},{"id":"decode_float","candidate":73,"reference":45.9,"speedup":1.59,"baselines":{"oracle":45.9}}]},"scores":{"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":7.0},"council_summaries":{"gpt-6.1-sol-max":"Compact, correct decoding with a cancellation hole and incomplete API documentation.","muse-wonder-full-xhigh":"compact correct decoder; thin docs on constants","deepseek-v4.1-flash-max":"correct and pleasantly small, but the exported API is under-documented and the tests are the thinnest of the \"serious\" entries.","opus-5.5-xhigh":"Matches the oracle everywhere, but decodes Huffman one bit at a time and the docs and tests are thin"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Degenerate channel traversal is uninterruptible","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Channel representation is insufficiently documented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"undocumented exported constants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"no cancellation-during-decode or allocation test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"three exported items are documented below the contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the suite never exercises the allocation bound or a maximal alphabet","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`min` is shadowed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Huffman decoding walks the trie one bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Gaps in what the tests exercise","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Several exported names have no doc comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"A local variable named `min` shadows the builtin","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.042,"lines":12,"significant":286,"window":6},"complexity":{"functions":5,"mean":21.6,"max":45,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.25,"slop":{"score":0.0,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":2.875,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.74,"tokens":{"uncached_input":17164,"cache_read":181632,"cache_write":0,"output":4479,"reasoning":86},"cost_usd":0.5772,"cost_breakdown":{"input":0.17164,"cache_read":0.181632,"cache_write":0.0,"output":0.22395},"cost_usd_as_run":0.5772,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":378,"code":358,"comment":10,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":174,"code":169,"comment":1,"blank":4,"role":"test"}],"impl_lines":378,"impl_code":358,"impl_comment":10,"test_lines":174,"comment_ratio":0.027,"tests_declared":4},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":127.2,"reference":91.6,"speedup":1.389,"baselines":{"oracle":91.6}},{"id":"decode_float","candidate":61.6,"reference":45.8,"speedup":1.345,"baselines":{"oracle":45.8}}]},"scores":{"gpt-6.1-sol-max":4.0,"muse-wonder-full-xhigh":6.0,"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":5.5},"council_summaries":{"gpt-6.1-sol-max":"Malformed input can panic on 32-bit Go, and cancellation is insufficient.","muse-wonder-full-xhigh":"correct core undermined by false stub doc and thin tests","deepseek-v4.1-flash-max":"correct, but the package doc still calls the file a stub, the whole decoder is one 315-line function, and the tests are thin.","opus-5.5-xhigh":"Correct, but a false package doc, one 315-line function and thin tests"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Header conversion permits negative allocation lengths","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"No-op planes leave cancellation unpolled","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Completed implementation still calls itself a stub","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"package comment claims file is a stub","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"thin contract coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"single-function decoder","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"the package comment says the file is a stub","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"one 315-line function holds the entire algorithm","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"174 lines and four tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The package doc says \"This file is a stub that fixes the interface. Replace it with the implementation.\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"The whole decoder is one 315-line `Decompress`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0455,"lines":12,"significant":264,"window":6},"complexity":{"functions":3,"mean":34.0,"max":97,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-04","reviewed":false},"score":5.5,"slop":{"score":1.5,"per_seat":{"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":1.0}},"defect_points":5.25,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.8,"tokens":{"uncached_input":11102,"cache_read":178432,"cache_write":0,"output":3454,"reasoning":122},"cost_usd":0.0373,"cost_breakdown":{"input":0.011102,"cache_read":0.008922,"cache_write":0.0,"output":0.01727},"cost_usd_as_run":0.0373,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":375,"code":365,"comment":0,"blank":10,"role":"impl"},{"path":"piz_test.go","lines":47,"code":45,"comment":0,"blank":2,"role":"test"}],"impl_lines":375,"impl_code":365,"impl_comment":0,"test_lines":47,"comment_ratio":0,"tests_declared":1},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":130.8,"reference":90.7,"speedup":1.442,"baselines":{"oracle":90.7}},{"id":"decode_float","candidate":62.4,"reference":45.9,"speedup":1.359,"baselines":{"oracle":45.9}}]},"scores":{"gpt-6.1-sol-max":3.5,"muse-wonder-full-xhigh":5.5,"deepseek-v4.1-flash-max":5.5,"opus-5.5-xhigh":5.0},"council_summaries":{"gpt-6.1-sol-max":"Panic exposure, cancellation gaps, and a substantially incomplete test suite.","muse-wonder-full-xhigh":"correct but undocumented and almost untested","deepseek-v4.1-flash-max":"correct decoder, but a single 47-line test, no documentation at all, and the largest avoidable allocation.","opus-5.5-xhigh":"Behaviour is exact, but there are no docs, the only tests are the goldens, and the code is one long function"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Unsigned Huffman bounds become negative on 32-bit Go","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Channel traversal exceeds cancellation latency","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Tests cover only golden outputs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API lacks documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"golden-only test file","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"zero documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"315-line single-letter monolith","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"reinvented builtin","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"the whole test suite is one golden-vector loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"the decode trie is sized by the *alphabet*, not by the coded symbols","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no documentation on any item","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"builtin `min`/`max` are shadowed, forcing a hand-rolled helper","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"The only test decodes the eight goldens","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No doc comment on the package or on any exported identifier, including `Decompress`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Confusing names","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Trie capacity is sized from the alphabet, not the number of coded symbols","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":268,"window":6},"complexity":{"functions":4,"mean":26.0,"max":97,"over_10":1,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-04","reviewed":false},"score":4.875,"slop":{"score":0.125,"per_seat":{"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":7.625,"score_cross_vendor":5.333,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":26.88,"tokens":{"uncached_input":60,"cache_read":5047100,"cache_write":205495,"output":157005,"reasoning":109119},"cost_usd":5.7937,"cost_breakdown":{"input":0.00024,"cache_read":1.00942,"cache_write":1.64396,"output":3.1401},"cost_usd_as_run":5.7937,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":739,"code":606,"comment":91,"blank":42,"role":"impl"},{"path":"piz_test.go","lines":1512,"code":1329,"comment":89,"blank":94,"role":"test"}],"impl_lines":739,"impl_code":606,"impl_comment":91,"test_lines":1512,"comment_ratio":0.131,"tests_declared":19},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":399.3,"reference":91.7,"speedup":4.354,"baselines":{"oracle":91.7}},{"id":"decode_float","candidate":132.8,"reference":46,"speedup":2.887,"baselines":{"oracle":46}}]},"scores":{"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":9.5,"deepseek-v4.1-flash-max":9.5,"opus-5.5-xhigh":9.0},"council_summaries":{"gpt-6.1-sol-max":"Complete, efficient implementation with unusually strong tests.","muse-wonder-full-xhigh":"exemplary docs, hybrid table decoder, deepest test suite","deepseek-v4.1-flash-max":"fastest large-chunk decoder and the most complete test suite; only stylistic friction.","opus-5.5-xhigh":"Exact, fast (3–5× the oracle), best-specified tests (each rejection reason is checked) and excellent docs"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"OpenEXR limitations are stated too broadly","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"the `huffman` field shadows the package constant","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"`wav2Decode` branches on `w14` inside the innermost loop and calls `poll.tick(1)` on every block","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0256,"lines":12,"significant":468,"window":6},"complexity":{"functions":22,"mean":6.23,"max":18,"over_10":5,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":34,"flags":1,"audited":"2026-11-04","reviewed":true},"score":9.375,"slop":{"score":0.125,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0}},"defect_points":0.375,"score_cross_vendor":9.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.22,"tokens":{"uncached_input":9158,"cache_read":232064,"cache_write":0,"output":7472,"reasoning":971},"cost_usd":0.6972,"cost_breakdown":{"input":0.09158,"cache_read":0.232064,"cache_write":0.0,"output":0.3736},"cost_usd_as_run":0.6972,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":396,"code":376,"comment":6,"blank":14,"role":"impl"},{"path":"piz_test.go","lines":292,"code":282,"comment":4,"blank":6,"role":"test"}],"impl_lines":396,"impl_code":376,"impl_comment":6,"test_lines":292,"comment_ratio":0.016,"tests_declared":7},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":129.5,"reference":91.1,"speedup":1.422,"baselines":{"oracle":91.1}},{"id":"decode_float","candidate":61.2,"reference":45.9,"speedup":1.333,"baselines":{"oracle":45.9}}]},"scores":{"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.0,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-sol-max":"Correct decoding with a cancellation hole and documentation shortcomings.","muse-wonder-full-xhigh":"correct; sparse docs, poller overhead on hot path","deepseek-v4.1-flash-max":"correct and compact, but the exported API has no documentation.","opus-5.5-xhigh":"Correct, but the exported API is undocumented and it polls on every operation"},"findings":[{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Plane traversal performs unaccounted work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public representation semantics are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Nil-context support is falsely attributed to the format contract","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"undocumented exported types/constants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"function call per bit and per word","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no doc comments on any exported item","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Bit-at-a-time trie decoding, plus a `poller.check()` call on every bit, LUT entry, run word and output word","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`PixelType`, `Layout`, `MaxChunkBytes`, the constants and the errors have no doc comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test cancels during decoding, and none covers a full or sparse 65,537-symbol alphabet","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The comment says `poll` but the type is `poller`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":277,"window":6},"complexity":{"functions":7,"mean":15.0,"max":43,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.875,"slop":{"score":0.25,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":2.75,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.04,"tokens":{"uncached_input":10785,"cache_read":392704,"cache_write":0,"output":24487,"reasoning":10696},"cost_usd":1.7249,"cost_breakdown":{"input":0.10785,"cache_read":0.392704,"cache_write":0.0,"output":1.22435},"cost_usd_as_run":1.7249,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":423,"code":385,"comment":15,"blank":23,"role":"impl"},{"path":"piz_test.go","lines":634,"code":591,"comment":14,"blank":29,"role":"test"}],"impl_lines":423,"impl_code":385,"impl_comment":15,"test_lines":634,"comment_ratio":0.038,"tests_declared":14},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":130,"reference":91.7,"speedup":1.418,"baselines":{"oracle":91.7}},{"id":"decode_float","candidate":62.2,"reference":46.1,"speedup":1.349,"baselines":{"oracle":46.1}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with avoidable Huffman overhead.","gpt-6.1-sol-max":"Correct and robust in the checks run, with avoidable Huffman decoding cost.","opus-5.5-xhigh":"correct, the slowest of the trie decoders because it polls on every bit; thinner tests and docs","deepseek-v4.1-flash-max":"correct and clean, but the transform is its least-tested part and the public docs are thin","muse-wonder-full-xhigh":"Correct and well tested, but polls the context on every bit and word"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Bit-at-a-time Huffman decoding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"tests","title":"Allocation limit is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Ordinary codes require a trie walk per bit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The trie walk calls `poll()` and a general `read(1)` on every bit, and polls on every word of a run copy","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"There is no randomized valid round trip, no allocation-bound test and no benchmark, and only the goldens exercise the wavelet","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`decodeHuffman`, `inverseWavelet`, `inversePair` and `poll` have no doc comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The context is checked again after a stored copy or a decode has already finished, throwing away a complete result","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"exported errors and the decode/wavelet helpers undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"no round-trip encoder and no crafted *valid* multi-level transform","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"per-bit `poll()` and per-element 14/16-bit branch","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Poll call per Huffman edge and per copied word","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Layout doc drops half the contract wording","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":288,"window":6},"complexity":{"functions":7,"mean":15.14,"max":44,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.25,"slop":{"score":0.125,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":2.25,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-opus-5.5-xhigh-r1-b2","task":"exr-piz","lead":"opus-5.5-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6-astra-xhigh","judge":"gpt-6-astra-xhigh","model":"gpt-6-astra","effort":"xhigh","label":"GPT‑6 Astra xhigh","usage":{"wall_minutes":9.55,"tokens":{"cache_read":1663104,"cache_write":0,"output":15605,"reasoning":7642,"uncached_input":92250},"cost_usd":3.3659,"cost_breakdown":{"cache_read":1.663104,"cache_write":0.0,"input":0.9225,"output":0.78025},"cost_usd_as_run":3.3659,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":11.02,"tokens":{"cache_read":8552960,"cache_write":0,"output":36373,"reasoning":17799,"uncached_input":121001},"cost_usd":0.7305,"cost_breakdown":{"cache_read":0.427648,"cache_write":0.0,"input":0.121001,"output":0.181865},"cost_usd_as_run":0.7305,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":41.33,"tokens":{"cache_read":3384060,"cache_write":0,"output":45875,"reasoning":27154,"uncached_input":194298},"cost_usd":2.3715,"cost_breakdown":{"cache_read":0.676812,"cache_write":0.0,"input":0.777192,"output":0.9175},"cost_usd_as_run":2.3715,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":15.91,"tokens":{"cache_read":2510080,"cache_write":0,"output":34347,"reasoning":18348,"uncached_input":123104},"cost_usd":0.8407,"cost_breakdown":{"cache_read":0.251008,"cache_write":0.0,"input":0.246208,"output":0.34347},"cost_usd_as_run":0.8407,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":21.61,"tokens":{"cache_read":6537691,"cache_write":232367,"output":105853,"reasoning":62198,"uncached_input":84},"cost_usd":5.2839,"cost_breakdown":{"cache_read":1.307538,"cache_write":1.858936,"input":0.000336,"output":2.11706},"cost_usd_as_run":5.2839,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":42.5,"tokens":{"cache_read":15347072,"cache_write":0,"output":167725,"reasoning":122679,"uncached_input":98117},"cost_usd":0.1614,"cost_breakdown":{"cache_read":0.046041,"cache_write":0.0,"input":0.014718,"output":0.100635},"cost_usd_as_run":0.1614,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"T","score":8.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"E","score":10.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"I","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"T","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"I","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"T","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"I","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"T","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"E","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"I","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"T","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"E","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"I","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"T","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"E","score":8.5}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","opus-5.5-xhigh","deepseek-v4.1-flash-max"]},"entries":[{"letter":"W","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":17.6,"tokens":{"uncached_input":134658,"cache_read":1691737,"cache_write":0,"output":103178,"reasoning":39620},"cost_usd":1.4703,"cost_breakdown":{"input":0.269316,"cache_read":0.169174,"cache_write":0.0,"output":1.03178},"cost_usd_as_run":1.4703,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":785,"code":708,"comment":23,"blank":54,"role":"impl"},{"path":"piz_test.go","lines":1276,"code":1198,"comment":17,"blank":61,"role":"test"}],"impl_lines":785,"impl_code":708,"impl_comment":23,"test_lines":1276,"comment_ratio":0.031,"tests_declared":9},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":547.5,"reference":91.8,"speedup":5.964,"baselines":{"oracle":91.8}},{"id":"decode_float","candidate":250.8,"reference":46,"speedup":5.452,"baselines":{"oracle":46}}]},"scores":{"gpt-6-astra-xhigh":5.5,"gpt-6.1-terra-max":6.0,"muse-wonder-full-xhigh":8.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6-astra-xhigh":"Fast decoding, but valid layouts violate the memory and cancellation limits.","gpt-6.1-terra-max":"Valid inputs can exceed the allocation bound and delay cancellation.","muse-wonder-full-xhigh":"Fastest and broadly tested, paid for with the most complex decoder","gpt-6.1-sol-max":"Correct, fast decoding, but violates allocation and cancellation limits.","opus-5.5-xhigh":"fastest and correct, but cancellation is only checked once per row, and some docs are wrong or padded","deepseek-v4.1-flash-max":"correct, fastest and best-tested, but the hand-rolled dual bit-buffer decoder is the hardest of the four to maintain and one doc comment names the LUT wrongly."},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Channel metadata exceeds the allocation budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"robustness","title":"Cancellation polling leaves entire wide rows unchecked","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Cancellation timing test cannot detect polling gaps","status":"valid","defect":"EXR-018","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"misleading-docs","title":"Reverse LUT described as forward","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Per-channel spans exceed the allocation limit","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Wide rows delay cancellation","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Prefix collision tests are masked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cancellation timing starts when cancellation is observed","status":"valid","defect":"EXR-018","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Bitmap parser calls its LUT forward","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public error and cancellation behavior is undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Marquee valid edges assert length only","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"One ~350-line Huffman function","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong LUT direction","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Repeated nil-context guards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Channel metadata exceeds the allocation bound","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Wide rows delay cancellation beyond 10 ms","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Prefix-validation regressions escape the suite","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Lookup direction is mislabeled","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Cancellation checked only once per row","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"`parseBitmap` is described as building the \"forward lookup table\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"`if ctx != nil` around every context check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`lut[0] = 0` straight after `make`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Stack-array micro-optimisations (`stackNZ [4096]`, `stackLong [512]`, `stackSpans [8]`, `maxPossibleNZ`)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Two copies of the wavelet, plus a checked and an unchecked copy of the output loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Robustness and fuzz tests don't check the error type","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"\"forward lookup table\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"Two undocumented bit-buffer conventions in one decoder","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"Ten `ctx != nil` guards","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Redundant size guard","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"`chanSpan` stack-array machinery","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Unexplained stack bound","status":"valid","detail":"","location":""}],"isolation":{"tool_calls":19,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.4,"slop":{"score":1.7,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":1.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":3.0,"deepseek-v4.1-flash-max":3.0}},"defect_points":5.4,"score_cross_vendor":7.4,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","task":"exr-piz","lead":"opus-5.5-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6-astra-xhigh","judge":"gpt-6-astra-xhigh","model":"gpt-6-astra","effort":"xhigh","label":"GPT‑6 Astra xhigh","usage":{"wall_minutes":50.09,"tokens":{"cache_read":11292672,"cache_write":0,"output":69586,"reasoning":32043,"uncached_input":608687},"cost_usd":20.8589,"cost_breakdown":{"cache_read":11.292672,"cache_write":0.0,"input":6.08687,"output":3.4793},"cost_usd_as_run":20.8589,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":33.5,"tokens":{"cache_read":21460224,"cache_write":0,"output":101250,"reasoning":51127,"uncached_input":392370},"cost_usd":1.9716,"cost_breakdown":{"cache_read":1.073011,"cache_write":0.0,"input":0.39237,"output":0.50625},"cost_usd_as_run":1.9716,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":68.83,"tokens":{"cache_read":24240512,"cache_write":0,"output":125374,"reasoning":73818,"uncached_input":1031308},"cost_usd":5.7404,"cost_breakdown":{"cache_read":2.424052,"cache_write":0.0,"input":2.062616,"output":1.25374},"cost_usd_as_run":5.7404,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":70.38,"tokens":{"cache_read":30872816,"cache_write":935620,"output":294323,"reasoning":181586,"uncached_input":308},"cost_usd":19.5472,"cost_breakdown":{"cache_read":6.174563,"cache_write":7.48496,"input":0.001232,"output":5.88646},"cost_usd_as_run":19.5472,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":107.37,"tokens":{"cache_read":54333824,"cache_write":0,"output":564661,"reasoning":435938,"uncached_input":473193},"cost_usd":0.5727,"cost_breakdown":{"cache_read":0.163002,"cache_write":0.0,"input":0.070979,"output":0.338797},"cost_usd_as_run":0.5727,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":177.28,"tokens":{"cache_read":20154560,"cache_write":0,"output":194127,"reasoning":110246,"uncached_input":748289},"cost_usd":10.9066,"cost_breakdown":{"cache_read":4.030912,"cache_write":0.0,"input":2.993156,"output":3.88254},"cost_usd_as_run":10.9066,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":5.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":6.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":4.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":4.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B","score":7.5}],"panel":["gpt-6.1-terra-max","gpt-6.1-sol-max","opus-5.5-xhigh","deepseek-v4.1-flash-max","muse-wonder-full-xhigh"],"guardrail_blocks":[{"date":"2026-11-02T18:20:00+00:00","provider":"openai","attempt":1,"message":"This content was flagged for possible cybersecurity risk (council review of an OpenEXR PIZ decompressor; probing with malformed input)","outcome":"retry-passed","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3"}]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":16.86,"tokens":{"uncached_input":58,"cache_read":3078088,"cache_write":125467,"output":83328,"reasoning":38618},"cost_usd":3.2861,"cost_breakdown":{"input":0.000232,"cache_read":0.615618,"cache_write":1.003736,"output":1.66656},"cost_usd_as_run":3.2861,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":744,"code":596,"comment":107,"blank":41,"role":"impl"},{"path":"piz_test.go","lines":1118,"code":989,"comment":66,"blank":63,"role":"test"}],"impl_lines":744,"impl_code":596,"impl_comment":107,"test_lines":1118,"comment_ratio":0.152,"tests_declared":14},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":413.8,"reference":89,"speedup":4.649,"baselines":{"oracle":89}},{"id":"decode_float","candidate":211.9,"reference":43.2,"speedup":4.905,"baselines":{"oracle":43.2}}]},"scores":{"gpt-6-astra-xhigh":5.0,"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Extensive correctness tests, but allocation and cancellation limits are violated.","gpt-6.1-terra-max":"The per-channel scratch arrays substantially exceed the memory bound.","gpt-6.1-sol-max":"Strong decoding and tests, but violates memory and cancellation limits.","opus-5.5-xhigh":"Best docs, rigorous tests and a fast branch-light reader; ctx is polled per row and the LUT build is slow","deepseek-v4.1-flash-max":"correct and extremely well tested; overshoots the allocation budget for very large channel lists, and one comment is false","muse-wonder-full-xhigh":"Correct with the best documentation; fixed 128 KiB LUT and per-row charging cost it"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Two channel arrays exceed the allocation limit","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Cancellation checks surround unbounded loops","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Two per-channel arrays exceed the allocation budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Whole wavelet rows run between cancellation checks","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Channel bookkeeping exceeds the allocation budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Singleton planes bypass cancellation polling","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Wavelet and output poll ctx once per row","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Full 65,536-step LUT scan into a fixed 128 KiB array","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"Allocation bound exceeded for large channel lists","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"False refill invariant","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Full-size LUT built on every call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Rows charged before processing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Unbounded cancellation loop","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0617,"lines":28,"significant":454,"window":6},"complexity":{"functions":22,"mean":6.77,"max":23,"over_10":5,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":28,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.5,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":3.2,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":55.68,"tokens":{"uncached_input":450,"cache_read":18049768,"cache_write":325540,"output":294855,"reasoning":0},"cost_usd":0.4926,"cost_breakdown":{"input":6.8e-05,"cache_read":0.288796,"cache_write":0.065108,"output":0.138582},"cost_usd_as_run":0.4926,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.4926,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":637,"code":532,"comment":75,"blank":30,"role":"impl"},{"path":"piz_test.go","lines":1453,"code":1275,"comment":96,"blank":82,"role":"test"}],"impl_lines":637,"impl_code":532,"impl_comment":75,"test_lines":1453,"comment_ratio":0.124,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":435,"reference":91.6,"speedup":4.749,"baselines":{"oracle":91.6}},{"id":"decode_float","candidate":255.4,"reference":45.8,"speedup":5.576,"baselines":{"oracle":45.8}}]},"scores":{"gpt-6-astra-xhigh":4.0,"gpt-6.1-terra-max":6.0,"gpt-6.1-sol-max":5.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6-astra-xhigh":"Allocation and cancellation violations, plus a reproduced 32-bit panic.","gpt-6.1-terra-max":"Correct on tested streams, but exceeds the allocation bound and misses cancellation deadlines.","gpt-6.1-sol-max":"Correct decoded results, but several resource and reviewability defects.","opus-5.5-xhigh":"Correct output, but the Huffman stage can ignore cancellation for 140+ ms on valid input; one false comment","deepseek-v4.1-flash-max":"excellent tests and docs, but cancellation polling is keyed to output words, and there is a per-channel allocation and a false comment","muse-wonder-full-xhigh":"Correct with exemplary crafted tests, but cancellation can be ignored for tens of ms"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Unsigned header narrowing can panic","status":"valid","defect":"EXR-007","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Per-channel allocation exceeds the budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Cancellation polling can stop advancing","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Bitmap tests mutate the wrong field","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Incorrect table-padding explanation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Per-channel bases exceed the memory budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Wide wavelet rows delay cancellation","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"LUT growth causes avoidable allocations","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Table padding is described as data-byte padding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Channel offsets exceed the allocation budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Output-position polling misses zero-length runs","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"LUT construction repeatedly reallocates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Bitmap-maximum test edits the minimum","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Table-padding explanation is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Huffman loop polls ctx only when the output word count is a multiple of 16384","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Wavelet and writeOutput poll ctx once per row","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"tblReader doc says \"the table's padding bits may lie in the first data byte, so cap is the whole block\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"LUT is built by scanning all 65,536 bits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`rlcLen` names the long-code marker, not anything to do with run-length codes, and 63 also appears as `rlePacket-1`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"robustness","title":"Cancellation can be ignored for tens of milliseconds","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"Allocation bound exceeded for large channel lists","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"False `tblReader` comment and needlessly permissive cap","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"Mid-decode cancellation test cannot catch the bug","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Context check keyed to words written, not work done","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Per-row-only context checks","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"False rationale for block-wide table reader","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1037,"lines":42,"significant":405,"window":6},"complexity":{"functions":14,"mean":9.36,"max":57,"over_10":2,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":74,"flags":1,"audited":"2026-11-02","reviewed":true},"score":6.8,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":5.8,"score_cross_vendor":6.8,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.2,"tokens":{"uncached_input":34768,"cache_read":420864,"cache_write":0,"output":11196,"reasoning":1397},"cost_usd":0.1118,"cost_breakdown":{"input":0.034768,"cache_read":0.021043,"cache_write":0.0,"output":0.05598},"cost_usd_as_run":0.1118,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":593,"code":558,"comment":17,"blank":18,"role":"impl"},{"path":"piz_test.go","lines":428,"code":385,"comment":24,"blank":19,"role":"test"}],"impl_lines":593,"impl_code":558,"impl_comment":17,"test_lines":428,"comment_ratio":0.03,"tests_declared":9},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":363.5,"reference":87.4,"speedup":4.159,"baselines":{"oracle":87.4}},{"id":"decode_float","candidate":122.5,"reference":42.8,"speedup":2.862,"baselines":{"oracle":42.8}}]},"scores":{"gpt-6-astra-xhigh":9.0,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6-astra-xhigh":"Efficient and correct in probes, with limited cancellation tests.","gpt-6.1-terra-max":"Correct on tested inputs, with small test and documentation gaps.","gpt-6.1-sol-max":"Sound decoder with weaker assertions around crafted edge cases.","opus-5.5-xhigh":"Correct and robust, but exported docs are missing, crafted tests are moderate, and there are shadowing nits","deepseek-v4.1-flash-max":"correct and very fast; two nits (docs, missing cancellation test)","muse-wonder-full-xhigh":"Correct and fast; crafted-valid tests don't check the bytes they decode"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Cancellation coverage uses only malformed input","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"One LUT-bound test is masked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Mid-decode cancellation is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Layout omits channel ordering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"The test’s size explanation is incorrect","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Crafted valid cases check length without checking content","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Long-code test describes absent shorter codes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported items are undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Crafted coverage has gaps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`x := lut[v]` shadows the range index `x`, and `min := nx` shadows the builtin","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"writeOutput polls ctx at x==0 of every row, which makes the `y&255` check redundant","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Chunked stored copy with ctx checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Pixel-type constants undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No mid-decode cancellation test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Valid crafted cases assert only error and length","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong test comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Opaque output addressing","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1976,"lines":82,"significant":415,"window":6},"complexity":{"functions":11,"mean":13.55,"max":56,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.3,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":2.3,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":57.49,"tokens":{"uncached_input":201147,"cache_read":4928716,"cache_write":0,"output":65921,"reasoning":39218},"cost_usd":3.1088,"cost_breakdown":{"input":0.804588,"cache_read":0.985743,"cache_write":0.0,"output":1.31842},"cost_usd_as_run":3.1088,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":683,"code":604,"comment":48,"blank":31,"role":"impl"},{"path":"piz_test.go","lines":806,"code":712,"comment":41,"blank":53,"role":"test"}],"impl_lines":683,"impl_code":604,"impl_comment":48,"test_lines":806,"comment_ratio":0.074,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":315.7,"reference":87.2,"speedup":3.62,"baselines":{"oracle":87.2}},{"id":"decode_float","candidate":184.7,"reference":42.1,"speedup":4.387,"baselines":{"oracle":42.1}}]},"scores":{"gpt-6-astra-xhigh":6.0,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Correct outputs, but cancellation misses work spread across many channels.","gpt-6.1-terra-max":"Good coverage and decoding, with a cancellation gap for many channels.","gpt-6.1-sol-max":"Correct decoder with a geometry-dependent cancellation gap.","opus-5.5-xhigh":"Correct and robust with an efficient LUT build; small redundancies","deepseek-v4.1-flash-max":"correct, clear and well tested; only a docs nit","muse-wonder-full-xhigh":"Correct, fast, and clean; only a vacuous allocation test"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Channel traversal has no aggregate cancellation budget","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"dead-code","title":"Unused test fixture","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Channel loops can run past the cancellation deadline","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Cancellation disappears across singleton planes","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unused layout setup","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`contextErr` treats a nil ctx as live","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"writeOutput has two overlapping ctx-poll schemes (`sinceCheck` per row and `w&0x3FFF` per word)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unused test variable kept alive with `_ = half`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Pixel-type constants undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Allocation test cannot fail","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1905,"lines":88,"significant":462,"window":6},"complexity":{"functions":17,"mean":8.65,"max":29,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":45,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.2,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.5,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.4,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":15.58,"tokens":{"uncached_input":14680,"cache_read":410368,"cache_write":0,"output":30021,"reasoning":12270},"cost_usd":0.3706,"cost_breakdown":{"input":0.02936,"cache_read":0.041037,"cache_write":0.0,"output":0.30021},"cost_usd_as_run":0.3706,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":621,"code":570,"comment":28,"blank":23,"role":"impl"},{"path":"piz_test.go","lines":779,"code":719,"comment":31,"blank":29,"role":"test"}],"impl_lines":621,"impl_code":570,"impl_comment":28,"test_lines":779,"comment_ratio":0.047,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":402.1,"reference":86.8,"speedup":4.632,"baselines":{"oracle":86.8}},{"id":"decode_float","candidate":154.3,"reference":42.8,"speedup":3.605,"baselines":{"oracle":42.8}}]},"scores":{"gpt-6-astra-xhigh":10.0,"gpt-6.1-terra-max":10.0,"gpt-6.1-sol-max":10.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6-astra-xhigh":"Complete implementation with strong tests and bounded resource use.","gpt-6.1-terra-max":"Complete, well-tested implementation with bounded inner-loop work.","gpt-6.1-sol-max":"Complete, bounded implementation with strong contract coverage.","opus-5.5-xhigh":"Correct, fine-grained ctx, efficient LUT, accurate docs and thorough tests; only nits","deepseek-v4.1-flash-max":"correct on every probe, with exhaustive tests, accurate docs, exact-size LUT and best-in-class small-chunk cost","muse-wonder-full-xhigh":"Correct, fastest, and the most consistent cancellation hygiene"},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Ctx checks after all work is done throw away a finished result, and the stored copy is chunked","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Each wavelet row ends with a duplicate `blocks&4095` check that fires on every row once `blocks` sits at a multiple of 4096","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"peek silently returns 0 on reservoir underflow","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1374,"lines":58,"significant":422,"window":6},"complexity":{"functions":14,"mean":10.64,"max":29,"over_10":7,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.5,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.3,"score_cross_vendor":9.167,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":51.71,"tokens":{"uncached_input":209395,"cache_read":5565024,"cache_write":0,"output":61452,"reasoning":29266},"cost_usd":3.1796,"cost_breakdown":{"input":0.83758,"cache_read":1.113005,"cache_write":0.0,"output":1.22904},"cost_usd_as_run":3.1796,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":711,"code":639,"comment":44,"blank":28,"role":"impl"},{"path":"piz_test.go","lines":904,"code":783,"comment":70,"blank":51,"role":"test"}],"impl_lines":711,"impl_code":639,"impl_comment":44,"test_lines":904,"comment_ratio":0.064,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":367.1,"reference":88,"speedup":4.172,"baselines":{"oracle":88}},{"id":"decode_float","candidate":179.4,"reference":43.9,"speedup":4.087,"baselines":{"oracle":43.9}}]},"scores":{"gpt-6-astra-xhigh":6.5,"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Correct decoding with a cancellation hole across small channel planes.","gpt-6.1-terra-max":"Correct on tested streams, but many channels delay cancellation.","gpt-6.1-sol-max":"Correct decoding, with cancellation and allocation-efficiency issues.","opus-5.5-xhigh":"Correct and robust; the Huffman loop is hand-unrolled and repetitive","deepseek-v4.1-flash-max":"correct, clear, and its external test package covers the public contract well; only a docs nit","muse-wonder-full-xhigh":"Correct and joint-fastest; duplicated bit-reader code is the maintenance risk"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Small planes bypass cancellation polling","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Singleton planes do not poll between channels","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"LUT expansion repeatedly reallocates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Channel traversal can exceed the cancellation deadline","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dense LUTs repeatedly grow and copy","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Bit-buffer refill and mask code is pasted throughout decodeHuffman","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`else if cntBits < 64` is always true","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`out, err := writeOutput(…); if err != nil {return nil, err}; return out, nil`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`ctxErr` nil-ctx wrapper","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Pixel-type constants undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Reservoir refill/mask snippet copied four times","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Allocation test cannot fail","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1516,"lines":74,"significant":488,"window":6},"complexity":{"functions":12,"mean":12.67,"max":76,"over_10":5,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":50,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":2.5,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":1.0}},"defect_points":2.1,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.49,"tokens":{"uncached_input":40,"cache_read":1557661,"cache_write":89378,"output":51777,"reasoning":16471},"cost_usd":2.0623,"cost_breakdown":{"input":0.00016,"cache_read":0.311532,"cache_write":0.715024,"output":1.03554},"cost_usd_as_run":2.0623,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":636,"code":552,"comment":52,"blank":32,"role":"impl"},{"path":"piz_test.go","lines":1088,"code":989,"comment":38,"blank":61,"role":"test"}],"impl_lines":636,"impl_code":552,"impl_comment":52,"test_lines":1088,"comment_ratio":0.086,"tests_declared":16},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":475.8,"reference":87.9,"speedup":5.413,"baselines":{"oracle":87.9}},{"id":"decode_float","candidate":231.1,"reference":42.9,"speedup":5.387,"baselines":{"oracle":42.9}}]},"scores":{"gpt-6-astra-xhigh":6.0,"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Correct outputs, but cancellation remains unbounded across substantial work.","gpt-6.1-terra-max":"Strong decoding and tests, but large rows and channel counts delay cancellation.","gpt-6.1-sol-max":"Fast, well-tested decoding with incomplete cancellation polling.","opus-5.5-xhigh":"Fastest decoder with excellent round-trip tests; ctx is polled per row and the LUT build is slow","deepseek-v4.1-flash-max":"correct, very well tested and documented; one dead re-slice","muse-wonder-full-xhigh":"Correct with strong round-trip tests; small dead code and per-row cancellation"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Cancellation gaps across channels and wide rows","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"redundancy","title":"Unused capacity restriction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Cancellation is checked only after an output row","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"LUT capacity restriction has no effect","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Singleton channel traversal lacks context checks","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"LUT capacity restriction has no consumer","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Wavelet and output poll ctx only between rows","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"LUT is built by scanning all 65,536 bits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"`refill` doc says it \"tops up … to at least 57 bits\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"wavelet14 writes the 14-bit step out by hand six times while wavelet16 calls `dec16`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Dead re-slice in writeOutput","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Context checked only after each channel-row","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead zero-length count","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"No-op reslice","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0898,"lines":38,"significant":423,"window":6},"complexity":{"functions":14,"mean":9.71,"max":33,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":20,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":2.1,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.92,"tokens":{"uncached_input":15792,"cache_read":343424,"cache_write":0,"output":15016,"reasoning":3844},"cost_usd":0.2161,"cost_breakdown":{"input":0.031584,"cache_read":0.034342,"cache_write":0.0,"output":0.15016},"cost_usd_as_run":0.2161,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":530,"code":493,"comment":18,"blank":19,"role":"impl"},{"path":"piz_test.go","lines":509,"code":465,"comment":23,"blank":21,"role":"test"}],"impl_lines":530,"impl_code":493,"impl_comment":18,"test_lines":509,"comment_ratio":0.035,"tests_declared":10},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":337.9,"reference":86.6,"speedup":3.902,"baselines":{"oracle":86.6}},{"id":"decode_float","candidate":136,"reference":43.4,"speedup":3.134,"baselines":{"oracle":43.4}}]},"scores":{"gpt-6-astra-xhigh":9.0,"gpt-6.1-terra-max":9.5,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6-astra-xhigh":"Solid implementation; cancellation regression coverage is incomplete.","gpt-6.1-terra-max":"Solid implementation; its cancellation tests stop at pre-cancelled contexts.","gpt-6.1-sol-max":"Solid implementation with one minor cancellation-test gap.","opus-5.5-xhigh":"Correct, with fine-grained cancellation and good crafted tests; LUT build is slow and there is some redundant ctx plumbing","deepseek-v4.1-flash-max":"correct, clean, fastest overall; only a missing mid-decode cancellation test","muse-wonder-full-xhigh":"Correct and well-tested; slowest-tier decode, minor test gaps"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Cancellation is tested only before entry","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No test cancels during decoding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Cancellation coverage stops at entry","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"LUT is built by scanning all 65,536 bits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Stored path copies in 32 KiB slices with ctx checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Odd per-channel `c&16383` ctx check, and ctx is checked back to back after Huffman and at the start of writeOutput","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"No mid-decode cancellation test, only pre-cancelled contexts","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"No mid-decode cancellation test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Byte-at-a-time bit reservoir","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No during-cancellation or im≠0 coverage","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1444,"lines":52,"significant":360,"window":6},"complexity":{"functions":12,"mean":11.58,"max":55,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.8,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.4,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.96,"tokens":{"uncached_input":9022,"cache_read":163968,"cache_write":0,"output":4826,"reasoning":154},"cost_usd":0.0413,"cost_breakdown":{"input":0.009022,"cache_read":0.008198,"cache_write":0.0,"output":0.02413},"cost_usd_as_run":0.0413,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":457,"code":426,"comment":15,"blank":16,"role":"impl"},{"path":"piz_test.go","lines":152,"code":148,"comment":0,"blank":4,"role":"test"}],"impl_lines":457,"impl_code":426,"impl_comment":15,"test_lines":152,"comment_ratio":0.034,"tests_declared":5},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":231.8,"reference":83.2,"speedup":2.786,"baselines":{"oracle":83.2}},{"id":"decode_float","candidate":113.3,"reference":42.8,"speedup":2.647,"baselines":{"oracle":42.8}}]},"scores":{"gpt-6-astra-xhigh":8.0,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6-astra-xhigh":"Correct and robust in probes, with thin tests and avoidable wavelet overhead.","gpt-6.1-terra-max":"Correct on tested inputs, but its tests miss several contract boundaries.","gpt-6.1-sol-max":"Correct and bounded, but slower decoding and thin adversarial coverage.","opus-5.5-xhigh":"Correct and robust, but the tests are thin, the docs are sparse and it has the slowest Huffman loop","deepseek-v4.1-flash-max":"compact, correct decoder, but its own tests leave most of the contract’s hard rules unchecked and its docs are sparse","muse-wonder-full-xhigh":"Correct compact decoder, but slowest and by far the thinnest tests and docs"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Important contract cases lack assertions","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Indirect calls remain in the wavelet hot loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Pathological Huffman and cancellation cases lack assertions","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Huffman peeks repeatedly traverse bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Layout leaves channel ordering implicit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Prefix lookup consumes and rewinds the field reader","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Malformed coverage largely stops at outer framing","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Obsolete loop-capture shadow","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Tests leave most of the contract unchecked","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Bit reader reads 12 bits byte by byte for every symbol, then rewinds","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"LUT is built by testing all 65,536 bitmap bits","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported items lack contract docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`min, max :=` shadows the builtins","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Extra per-row ctx check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Tests stop short of the contract’s hard parts","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Public constants and sentinels undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Most of the contract is untested by its own suite","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Slowest decode","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Exported constants and Layout fields undocumented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":316,"window":6},"complexity":{"functions":9,"mean":12.89,"max":53,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.3,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.0,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":24.71,"tokens":{"uncached_input":58,"cache_read":4327497,"cache_write":239629,"output":329800,"reasoning":298701},"cost_usd":0.8084,"cost_breakdown":{"input":2.2e-05,"cache_read":0.206803,"cache_write":0.208807,"output":0.392745},"cost_usd_as_run":0.8084,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":698,"code":564,"comment":100,"blank":34,"role":"impl"},{"path":"piz_test.go","lines":775,"code":648,"comment":79,"blank":48,"role":"test"}],"impl_lines":698,"impl_code":564,"impl_comment":100,"test_lines":775,"comment_ratio":0.151,"tests_declared":13},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":412,"reference":86.2,"speedup":4.78,"baselines":{"oracle":86.2}},{"id":"decode_float","candidate":199,"reference":39.5,"speedup":5.038,"baselines":{"oracle":39.5}}]},"scores":{"gpt-6-astra-xhigh":6.0,"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":9.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Correct decoding, but cancellation checks leave long gaps.","gpt-6.1-terra-max":"Broad coverage, with cancellation gaps on wide rows.","gpt-6.1-sol-max":"Correct decoder with cancellation gaps and avoidable LUT allocation.","opus-5.5-xhigh":"Correct and clearly structured; ctx is polled per row, the LUT build is slow, and comments are slightly verbose","deepseek-v4.1-flash-max":"correct, clear, thoroughly documented and tested; no defects found","muse-wonder-full-xhigh":"Correct, best-documented structure; cancellation charging and allocation count are the costs"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Cancellation exceeds 10 ms","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Package overview misstates the pipeline","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Pacing happens before whole rows","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Repeated LUT growth adds allocations","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Pacing excludes singleton-plane work","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"LUT construction grows from one element","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Bit-peek documentation promises an unsupported width","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Pacer charges a whole block row before running it, so ctx is checked once per row","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"LUT built bit by bit with append growth","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`bitmapBytes` and `maxNZLimit` are separate constants that both equal 8192","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Comments restate code, and one explains that Mul64 isn't needed but is kept anyway","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Pacer charges work before doing it","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Most allocations in the field","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0425,"lines":18,"significant":424,"window":6},"complexity":{"functions":24,"mean":6.12,"max":17,"over_10":3,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":34,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.4,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":31.55,"tokens":{"uncached_input":98,"cache_read":8358667,"cache_write":224337,"output":166906,"reasoning":106106},"cost_usd":6.8049,"cost_breakdown":{"input":0.000392,"cache_read":1.671733,"cache_write":1.794696,"output":3.33812},"cost_usd_as_run":6.8049,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"piz.go","lines":849,"code":664,"comment":132,"blank":53,"role":"impl"},{"path":"piz_test.go","lines":1405,"code":1236,"comment":97,"blank":72,"role":"test"}],"impl_lines":849,"impl_code":664,"impl_comment":132,"test_lines":1405,"comment_ratio":0.166,"tests_declared":16},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":568.1,"reference":85.5,"speedup":6.644,"baselines":{"oracle":85.5}},{"id":"decode_float","candidate":410,"reference":40.4,"speedup":10.149,"baselines":{"oracle":40.4}}]},"scores":{"gpt-6-astra-xhigh":8.0,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6-astra-xhigh":"Strong decoding and test coverage, with one cancellation gap for identity transforms","gpt-6.1-terra-max":"Thorough implementation, but cancellation can stall across many identity planes.","gpt-6.1-sol-max":"Strong decoding and tests, with a cancellation gap across many identity planes.","opus-5.5-xhigh":"Correct, fastest on float and noisy data, best docs, and the only test suite that catches the prefix-rule mutant","deepseek-v4.1-flash-max":"Best-in-class docs and tests and in the fastest tier; the hand-rolled burst loop is the one maintenance risk.","muse-wonder-full-xhigh":"Correct and best documented, held back only by complexity and run-heavy speed"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"robustness","title":"Identity transforms bypass the shared cancellation ticker","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Identity planes do not advance the cancellation ticker","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not compile on 32-bit Go","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Identity planes bypass cancellation checks","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Long-code lookup implemented twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"piz.go:488-545","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:87","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Run tokens always leave the hot loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Precedence-dependent Haar expression","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":501,"window":6},"complexity":{"functions":32,"mean":5.38,"max":25,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":49,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.7,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.7,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":64.09,"tokens":{"uncached_input":407959,"cache_read":8579392,"cache_write":0,"output":157067,"reasoning":126916},"cost_usd":0.3971,"cost_breakdown":{"input":0.061194,"cache_read":0.257382,"cache_write":0.0,"output":0.078534},"cost_usd_as_run":0.3971,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":714,"code":585,"comment":93,"blank":36,"role":"impl"},{"path":"piz_test.go","lines":910,"code":798,"comment":56,"blank":56,"role":"test"}],"impl_lines":714,"impl_code":585,"impl_comment":93,"test_lines":910,"comment_ratio":0.137,"tests_declared":12},"hidden":{"passed":8,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":348.8,"reference":85,"speedup":4.104,"baselines":{"oracle":85}},{"id":"decode_float","candidate":187.4,"reference":43.2,"speedup":4.338,"baselines":{"oracle":43.2}}]},"scores":{"gpt-6-astra-xhigh":4.0,"gpt-6.1-terra-max":4.0,"gpt-6.1-sol-max":5.5,"opus-5.5-xhigh":4.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6-astra-xhigh":"Rejects valid 58-bit streams and has additional validation and cancellation defects","gpt-6.1-terra-max":"Rejects valid 58-bit streams and has validation and cancellation defects.","gpt-6.1-sol-max":"Error precedence and cancellation fail, while much of the malformed-test coverage is ineffective.","opus-5.5-xhigh":"Valid chunks with 58-bit codes are rejected or silently decoded wrong; the wavelet and output stages ignore cancellation; wrong error precedence","deepseek-v4.1-flash-max":"Excellent docs, careful code, strong tests.","muse-wonder-full-xhigh":"Decodes correctly, but returns the wrong error class in one layout edge and largely ignores late cancellation"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"correctness","title":"Unaligned 58-bit codes exceed the reservoir","status":"valid","defect":"EXR-003","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Size failure precedes complete layout validation","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"robustness","title":"Wavelet and output work can exceed the cancellation deadline","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Crafted malformed cases never reach the decoder","status":"valid","defect":"EXR-016","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Reserved header word is misidentified","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Reservoir cannot always hold a 58-bit code","status":"valid","defect":"EXR-003","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Size validation can hide an invalid channel","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Wavelet and output loops lack bounded cancellation checks","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Reserved header word is described as a bit-count copy","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Size rejection precedes complete layout validation","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Wavelet work between context checks is too large","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"All crafted malformed cases bypass PIZ decoding","status":"valid","defect":"EXR-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Reserved word is called a bit-count copy","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"`fill` stops once `cnt > 56`, so `peek(58)` zero-pads the 58th bit when `cnt == 57`","status":"valid","defect":"EXR-003","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"No cancellation in the wavelet or output stages","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"correctness","title":"Size cap checked before all channel types are validated","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Reserved header word misdescribed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Tests miss the alignment bug and include a no-op check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:89","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"correctness","title":"Layout-vs-size precedence inverted for huge channel lists","status":"valid","defect":"EXR-006","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"No effective cancellation polling in wavelet/output","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Tests mask both defects","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Fifth Huffman word misdescribed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0676,"lines":30,"significant":444,"window":6},"complexity":{"functions":19,"mean":7.84,"max":22,"over_10":5,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":58,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.9,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":1.0}},"defect_points":5.9,"score_cross_vendor":5.9,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":13.17,"tokens":{"uncached_input":14502,"cache_read":360576,"cache_write":0,"output":28253,"reasoning":10723},"cost_usd":1.9182,"cost_breakdown":{"input":0.14502,"cache_read":0.360576,"cache_write":0.0,"output":1.41265},"cost_usd_as_run":1.9182,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":623,"code":572,"comment":27,"blank":24,"role":"impl"},{"path":"piz_test.go","lines":778,"code":715,"comment":30,"blank":33,"role":"test"}],"impl_lines":623,"impl_code":572,"impl_comment":27,"test_lines":778,"comment_ratio":0.045,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":445.6,"reference":85.4,"speedup":5.218,"baselines":{"oracle":85.4}},{"id":"decode_float","candidate":168.1,"reference":41.9,"speedup":4.012,"baselines":{"oracle":41.9}}]},"scores":{"gpt-6-astra-xhigh":10.0,"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Complete, efficient implementation with no verified defects","gpt-6.1-terra-max":"Sound sampled behavior and thorough edge tests, with a cancellation coverage gap.","gpt-6.1-sol-max":"Careful decoding and cancellation handling, with a masked negative-test case.","opus-5.5-xhigh":"Correct, among the fastest, with clean code and thorough tests; prefix test masked","deepseek-v4.1-flash-max":"Fast, correct, clearly written; documentation is accurate but terse.","muse-wonder-full-xhigh":"Correct and fast; bit-reader invariant is the main maintenance risk"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cancellation is tested only before entry","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Prefix-overlap regression is not detected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Prefix case fails for another reason, and only a pre-cancelled context is tested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:52-54","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Hard-to-verify refill invariant","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1542,"lines":66,"significant":428,"window":6},"complexity":{"functions":15,"mean":9.8,"max":27,"over_10":7,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.7,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":1.0,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.37,"tokens":{"uncached_input":13850,"cache_read":245760,"cache_write":0,"output":10463,"reasoning":1840},"cost_usd":0.9074,"cost_breakdown":{"input":0.1385,"cache_read":0.24576,"cache_write":0.0,"output":0.52315},"cost_usd_as_run":0.9074,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":545,"code":506,"comment":20,"blank":19,"role":"impl"},{"path":"piz_test.go","lines":439,"code":399,"comment":19,"blank":21,"role":"test"}],"impl_lines":545,"impl_code":506,"impl_comment":20,"test_lines":439,"comment_ratio":0.038,"tests_declared":8},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":353,"reference":84.2,"speedup":4.192,"baselines":{"oracle":84.2}},{"id":"decode_float","candidate":133.3,"reference":41.6,"speedup":3.204,"baselines":{"oracle":41.6}}]},"scores":{"gpt-6-astra-xhigh":10.0,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Correct decoder with bounded allocations and consistent cancellation checks","gpt-6.1-terra-max":"Solid sampled behavior, with limited cancellation testing.","gpt-6.1-sol-max":"Sound implementation with a gap in malformed-prefix test isolation.","opus-5.5-xhigh":"Compact and correct with solid crafted tests, but its prefix test is masked and there is no mid-decode cancellation test","deepseek-v4.1-flash-max":"Fastest on small chunks and correct; the thinnest documentation of the strong group.","muse-wonder-full-xhigh":"Compact, correct and well tested; only documentation is thin"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cancellation is tested only before entry","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Prefix tests can pass on word-count failure","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Prefix and over-subscription cases fail for another reason","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`real` shadows a builtin","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"piz.go:11-24","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Sparse exported docs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1448,"lines":54,"significant":373,"window":6},"complexity":{"functions":12,"mean":11.83,"max":55,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.3,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":1.2,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":16.99,"tokens":{"uncached_input":38170,"cache_read":8182528,"cache_write":0,"output":153001,"reasoning":118274},"cost_usd":0.1221,"cost_breakdown":{"input":0.005725,"cache_read":0.024548,"cache_write":0.0,"output":0.091801},"cost_usd_as_run":0.1221,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1221,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"piz.go","lines":662,"code":582,"comment":47,"blank":33,"role":"impl"},{"path":"piz_test.go","lines":892,"code":737,"comment":85,"blank":70,"role":"test"}],"impl_lines":662,"impl_code":582,"impl_comment":47,"test_lines":892,"comment_ratio":0.075,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":429.9,"reference":86.6,"speedup":4.964,"baselines":{"oracle":86.6}},{"id":"decode_float","candidate":215.7,"reference":41,"speedup":5.261,"baselines":{"oracle":41}}]},"scores":{"gpt-6-astra-xhigh":6.5,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Correct output, but cancellation propagation and a test failure path are defective","gpt-6.1-terra-max":"Correct on sampled streams, with cancellation gaps on extreme layouts.","gpt-6.1-sol-max":"Correct sampled decoding, with two distinct cancellation defects.","opus-5.5-xhigh":"Correct, fast and well documented, but coarse context polling, avoidable allocation growth, and some test slop","deepseek-v4.1-flash-max":"Fastest implementation and clean throughout; docs are the thinnest of the strong entries.","muse-wonder-full-xhigh":"Fastest decoder probed and correct, with a fragile cancellation design in the wavelet"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"robustness","title":"Wavelet cancellation is not propagated between planes","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Concurrent failure reporting can deadlock","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Cancellation is checked too infrequently across planes and wide rows","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not compile on 32-bit Go","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Wavelet cancellation is not propagated","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Wide rows exceed the cancellation interval","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"The malformed-test control is already invalid","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Wavelet and output check the context every 16 rows","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"`pairs` grows by append from capacity 256","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Prefix case fails for another reason","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Test comments contradict the code","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"piz.go:1","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Silent wavelet abort and 16-row polling","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1091,"lines":48,"significant":440,"window":6},"complexity":{"functions":18,"mean":8.17,"max":22,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":57,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":2.8,"score_cross_vendor":7.75,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.02,"tokens":{"uncached_input":33347,"cache_read":248832,"cache_write":0,"output":8716,"reasoning":1095},"cost_usd":0.1787,"cost_breakdown":{"input":0.066694,"cache_read":0.024883,"cache_write":0.0,"output":0.08716},"cost_usd_as_run":0.1787,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":544,"code":502,"comment":23,"blank":19,"role":"impl"},{"path":"piz_test.go","lines":398,"code":368,"comment":13,"blank":17,"role":"test"}],"impl_lines":544,"impl_code":502,"impl_comment":23,"test_lines":398,"comment_ratio":0.044,"tests_declared":8},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":348.9,"reference":83.5,"speedup":4.178,"baselines":{"oracle":83.5}},{"id":"decode_float","candidate":141.9,"reference":42.6,"speedup":3.331,"baselines":{"oracle":42.6}}]},"scores":{"gpt-6-astra-xhigh":8.5,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6-astra-xhigh":"Sound decoder with misleading documentation and masked negative tests","gpt-6.1-terra-max":"Correct on sampled streams, but has avoidable bitmap work and misleading comments.","gpt-6.1-sol-max":"Correct and bounded in probes, with misleading documentation and ineffective prefix tests.","opus-5.5-xhigh":"Correct implementation, but the package doc is a leftover stub and its code-table tests never reach the table checks","deepseek-v4.1-flash-max":"Correct decoder shipped with the scaffold's \"this is a stub\" comment still in the package doc, plus the thinnest tests outside Q.","muse-wonder-full-xhigh":"Correct and adequately tested, marred by leftover stub documentation"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Prefix-validation tests fail before table validation","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Package documentation still calls the implementation a stub","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Bitmap expansion scans all 65,536 values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cancellation is tested only before entry","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Package comment still calls the implementation a stub","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"`peek` does not always pad at the logical end","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Completed implementation still claims to be a stub","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Prefix fixtures fail before table validation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Package comment is the scaffold stub text","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"\"prefix\" and \"oversubscribed\" cases are rejected by the header","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Test case tables are written as single 300-character lines","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"misleading-docs","title":"piz.go:3-4","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"piz.go:156-162","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"piz_test.go:64-77","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Stale stub package comment","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Double context polling per wavelet row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1609,"lines":60,"significant":373,"window":6},"complexity":{"functions":12,"mean":11.75,"max":55,"over_10":5,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":1.6,"per_seat":{"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":3.0,"muse-wonder-full-xhigh":2.0}},"defect_points":3.2,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":42.23,"tokens":{"uncached_input":156165,"cache_read":2867064,"cache_write":0,"output":47026,"reasoning":20415},"cost_usd":2.1386,"cost_breakdown":{"input":0.62466,"cache_read":0.573413,"cache_write":0.0,"output":0.94052},"cost_usd_as_run":2.1386,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"piz.go","lines":758,"code":659,"comment":61,"blank":38,"role":"impl"},{"path":"piz_test.go","lines":878,"code":775,"comment":58,"blank":45,"role":"test"}],"impl_lines":758,"impl_code":659,"impl_comment":61,"test_lines":878,"comment_ratio":0.085,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":395.8,"reference":86.6,"speedup":4.57,"baselines":{"oracle":86.6}},{"id":"decode_float","candidate":188,"reference":41.1,"speedup":4.574,"baselines":{"oracle":41.1}}]},"scores":{"gpt-6-astra-xhigh":8.0,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Correct decoding with a cancellation gap across identity planes","gpt-6.1-terra-max":"Broadly correct, but cancellation can stall across identity planes.","gpt-6.1-sol-max":"Correct decoding, with cancellation and rejection-test gaps.","opus-5.5-xhigh":"Correct, with a careful bit reader that zeroes padding bits, good docs and a solid crafted suite; prefix test masked","deepseek-v4.1-flash-max":"Correct, well tested; small style blemish in the error helper.","muse-wonder-full-xhigh":"Correct, well documented and responsive; small tails and formatting waste only"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"robustness","title":"Identity-plane traversal never polls cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Plane traversal can run without checking cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests do not compile on 32-bit Go","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Many identity planes delay cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Prefix validation lacks an isolating regression","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Prefix case fails for another reason, and only a pre-cancelled context is tested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:39-42","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:89","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Uncounted odd-row wavelet tail","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Double formatting in every malformed error","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1392,"lines":70,"significant":503,"window":6},"complexity":{"functions":19,"mean":8.37,"max":27,"over_10":7,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":33,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":1.0}},"defect_points":2.1,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.81,"tokens":{"uncached_input":8855,"cache_read":159232,"cache_write":0,"output":3599,"reasoning":89},"cost_usd":0.0348,"cost_breakdown":{"input":0.008855,"cache_read":0.007962,"cache_write":0.0,"output":0.017995},"cost_usd_as_run":0.0348,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":427,"code":407,"comment":7,"blank":13,"role":"impl"},{"path":"piz_test.go","lines":62,"code":59,"comment":0,"blank":3,"role":"test"}],"impl_lines":427,"impl_code":407,"impl_comment":7,"test_lines":62,"comment_ratio":0.017,"tests_declared":1},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":200.7,"reference":82.4,"speedup":2.436,"baselines":{"oracle":82.4}},{"id":"decode_float","candidate":103.3,"reference":41.3,"speedup":2.501,"baselines":{"oracle":41.3}}]},"scores":{"gpt-6-astra-xhigh":6.0,"gpt-6.1-terra-max":6.0,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":5.0,"muse-wonder-full-xhigh":6.0},"council_summaries":{"gpt-6-astra-xhigh":"Functional decoding, but cancellation and contract-test coverage need substantial work","gpt-6.1-terra-max":"Sampled decoding works, but cancellation handling, test coverage, and documentation need work.","gpt-6.1-sol-max":"Sampled decoding is correct; cancellation, tests and public documentation need work.","opus-5.5-xhigh":"Correct output, but tests are goldens only, exported items lack docs, it is the slowest entry, and cancellation can be ignored for most of a second","deepseek-v4.1-flash-max":"Correct core, but the deliverable is only half there: no doc comments on any exported name and a 62-line test file.","muse-wonder-full-xhigh":"Implementation decodes correctly, but tests, docs and cancellation are under-delivered"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"robustness","title":"Cancellation polling depends on output progress","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Tests cover only golden decoding","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Public types lack their essential semantics","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Zero-length runs can bypass cancellation checks","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Plane and output traversal can delay cancellation","status":"valid","defect":"EXR-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Tests cover only golden chunks","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Bit decoding repeatedly reads fields and bitmap expansion scans all values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public types, constants, and error sentinels are undocumented","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Output progress does not bound cancellation latency","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Tests cover only golden decoding","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public types lack their contract documentation","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Tests cover goldens and one benchmark only","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"Context is checked only when `len(out)` hits a multiple of 16384","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Exported identifiers have no doc comments","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Slow bit reader and branchy wavelet","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`min` and `max` shadow builtins","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"piz_test.go:31-62","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"piz.go:11-33","status":"valid","defect":"EXR-013","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"piz.go:93-99","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"piz_test.go:50","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:1","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Golden-only test suite","status":"valid","defect":"EXR-014","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"robustness","title":"Cancellation keyed to output length","status":"valid","defect":"EXR-010","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Per-row output cancellation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Byte-at-a-time bit reader","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Undocumented public API","status":"valid","defect":"EXR-013","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":299,"window":6},"complexity":{"functions":7,"mean":16.29,"max":54,"over_10":4,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.7,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":7.1,"score_cross_vendor":5.5,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":8.72,"tokens":{"uncached_input":15885,"cache_read":1563776,"cache_write":0,"output":41129,"reasoning":16694},"cost_usd":0.2997,"cost_breakdown":{"input":0.015885,"cache_read":0.078189,"cache_write":0.0,"output":0.205645},"cost_usd_as_run":0.2997,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":618,"code":574,"comment":22,"blank":22,"role":"impl"},{"path":"piz_test.go","lines":1117,"code":1031,"comment":44,"blank":42,"role":"test"}],"impl_lines":618,"impl_code":574,"impl_comment":22,"test_lines":1117,"comment_ratio":0.037,"tests_declared":18},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":444.3,"reference":84.3,"speedup":5.27,"baselines":{"oracle":84.3}},{"id":"decode_float","candidate":189.9,"reference":41.5,"speedup":4.576,"baselines":{"oracle":41.5}}]},"scores":{"gpt-6-astra-xhigh":10.0,"gpt-6.1-terra-max":9.5,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6-astra-xhigh":"Thorough validation, bounded work, and strong independent tests","gpt-6.1-terra-max":"Strong sampled behavior, bounded work, and comprehensive tests.","gpt-6.1-sol-max":"Fast, well structured implementation with one missed rejection regression.","opus-5.5-xhigh":"Correct and fast, with very strong tests (trie differential, wavelet round trip, counting-context cancellation); only the prefix test is masked","deepseek-v4.1-flash-max":"Widest test coverage in the set (18 test functions incl. Huffman-vs-trie differentials), clean code, fast.","muse-wonder-full-xhigh":"Correct on every differential probe, with strong cancellation, tests and speed"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Prefix rejection remains unprotected by the suite","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Prefix rule never isolated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"piz.go:587","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:553","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1492,"lines":64,"significant":429,"window":6},"complexity":{"functions":11,"mean":14.0,"max":60,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":28,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.2,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":0.8,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":17.03,"tokens":{"uncached_input":22373,"cache_read":676992,"cache_write":0,"output":37911,"reasoning":18356},"cost_usd":0.4916,"cost_breakdown":{"input":0.044746,"cache_read":0.067699,"cache_write":0.0,"output":0.37911},"cost_usd_as_run":0.4916,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":594,"code":547,"comment":26,"blank":21,"role":"impl"},{"path":"piz_test.go","lines":733,"code":679,"comment":26,"blank":28,"role":"test"}],"impl_lines":594,"impl_code":547,"impl_comment":26,"test_lines":733,"comment_ratio":0.045,"tests_declared":12},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":381.3,"reference":86.6,"speedup":4.403,"baselines":{"oracle":86.6}},{"id":"decode_float","candidate":149.9,"reference":42.6,"speedup":3.519,"baselines":{"oracle":42.6}}]},"scores":{"gpt-6-astra-xhigh":10.0,"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6-astra-xhigh":"Correct, well-tested implementation with effective resource controls","gpt-6.1-terra-max":"Strong decoder and broad tests, with a cancellation coverage gap.","gpt-6.1-sol-max":"Correct, bounded implementation with one rejection-test coverage gap.","opus-5.5-xhigh":"Correct, with clean structure and good tests (random trees reaching 58 bits, wavelet round trip); prefix test masked","deepseek-v4.1-flash-max":"Clean, fast, correct; brief but accurate documentation; nothing to flag.","muse-wonder-full-xhigh":"Lean, correct and independently tested wavelet/Huffman logic"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cancellation is tested only before entry","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"docs","title":"Error sentinels lack individual documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Prefix validation lacks an isolating test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Prefix case fails for another reason, and only a pre-cancelled context is tested","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated wavelet bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1369,"lines":56,"significant":409,"window":6},"complexity":{"functions":13,"mean":11.23,"max":60,"over_10":5,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.7,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":0.8,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.4,"tokens":{"uncached_input":18623,"cache_read":131456,"cache_write":0,"output":6102,"reasoning":28},"cost_usd":0.6228,"cost_breakdown":{"input":0.18623,"cache_read":0.131456,"cache_write":0.0,"output":0.3051},"cost_usd_as_run":0.6228,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"piz.go","lines":518,"code":479,"comment":21,"blank":18,"role":"impl"},{"path":"piz_test.go","lines":343,"code":313,"comment":14,"blank":16,"role":"test"}],"impl_lines":518,"impl_code":479,"impl_comment":21,"test_lines":343,"comment_ratio":0.042,"tests_declared":7},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":197.8,"reference":85.1,"speedup":2.324,"baselines":{"oracle":85.1}},{"id":"decode_float","candidate":92.1,"reference":41.7,"speedup":2.209,"baselines":{"oracle":41.7}}]},"scores":{"gpt-6-astra-xhigh":9.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-terra-max":8.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6-astra-xhigh":"Passed the targeted correctness and resource checks; stale documentation remains.","gpt-6.1-sol-max":"Correct and robust in probing; avoidable bit-reader overhead and stale documentation remain.","opus-5.5-xhigh":"Correct and the most responsive to cancellation, but the slowest Huffman path, thin tests and a stale package comment","deepseek-v4.1-flash-max":"behaviourally correct with the best cancellation granularity, but the deliverable still calls itself a stub, the public API is under-documented, the bit reader is ~1.8× slower than its peers and there is no wavelet-level test","gpt-6.1-terra-max":"Correct and well bounded in the checks, with a slower Huffman hot path and minor test and documentation gaps.","muse-wonder-full-xhigh":"Behaviourally identical to the oracle but slowest, with the thinnest tests and a stale stub package comment"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"misleading-docs","title":"Completed implementation still claims to be a stub","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Rebuilds lookahead for every symbol","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Implementation still described as a stub","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Slow bit reader on the hot path","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Tests are thin","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Package comment is the stub's","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Contract missing from docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicated sample-size logic","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"piz.go:2-4","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"piz.go:70","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"piz.go:160-181","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"piz_test.go (whole file)","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Bit peeks repeatedly reconstruct the same byte window","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Pathological Huffman tests discard decoded values","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Package comment still calls the implementation a stub","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Package comment is still the scaffold stub text","status":"valid","defect":"EXR-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Unbuffered bit reader recomputes peeks from the byte slice","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"LUT slice made with cap 65536 on every call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Crafted Huffman tests assert only error-or-nil on an internal function","status":"valid","defect":"EXR-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Terse naming throughout reduces readability","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.17,"lines":60,"significant":353,"window":6},"complexity":{"functions":11,"mean":12.0,"max":52,"over_10":5,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":1.1,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.5,"deepseek-v4.1-flash-max":2.0,"muse-wonder-full-xhigh":1.0}},"defect_points":4.2,"score_cross_vendor":6.833,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":65.81,"tokens":{"uncached_input":291945,"cache_read":5165568,"cache_write":0,"output":274869,"reasoning":120246},"cost_usd":4.8159,"cost_breakdown":{"input":0.58389,"cache_read":2.582784,"cache_write":0.0,"output":1.649214},"cost_usd_as_run":4.8159,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"piz.go","lines":806,"code":692,"comment":82,"blank":32,"role":"impl"},{"path":"piz_test.go","lines":1050,"code":949,"comment":40,"blank":61,"role":"test"}],"impl_lines":806,"impl_code":692,"impl_comment":82,"test_lines":1050,"comment_ratio":0.106,"tests_declared":11},"hidden":{"passed":12,"total":12,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"decode_half","candidate":392.6,"reference":90.6,"speedup":4.333,"baselines":{"oracle":90.6}},{"id":"decode_float","candidate":166.8,"reference":45,"speedup":3.707,"baselines":{"oracle":45}}]},"scores":{"gpt-6-astra-xhigh":6.0,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-terra-max":6.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6-astra-xhigh":"Correct core behavior, but resource limits fail on legal layouts.","gpt-6.1-sol-max":"Correct decoding in probing, with insufficient memory and cancellation bounds.","opus-5.5-xhigh":"Correct and well structured, but slow on long codes, wasteful when reading tables, and its tests are only partly independent","deepseek-v4.1-flash-max":"correct, fast and very well tested; the inverse wavelet is the only place context polling is coarse, plus two nits","gpt-6.1-terra-max":"Decoding matched the oracle in the probes, but valid large layouts exceed memory and cancellation limits.","muse-wonder-full-xhigh":"Identical to the oracle, best small-chunk performance and LUT construction; weaker robustness assertions and minor doc/clarity slips"},"findings":[{"seat":"gpt-6-astra-xhigh","severity":"high","category":"robustness","title":"Per-channel records exceed allocation budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"robustness","title":"Row-based polling leaves long cancellation gaps","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Per-channel output metadata exceeds the allocation budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Wavelet cancellation depends on row width","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"Wavelet row not interruptible","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Long codes decoded a bit at a time","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Code-length list grown by `append`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Robustness and cancellation tests are weak","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Two checks that never fire","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Nil-context wrapper and repeated checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"wavelet polls ctx once per level and every 256 rows","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"piz.go:210-211","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"piz.go:90-97","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Per-channel metadata exceeds the allocation budget","status":"valid","defect":"EXR-005","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Wide wavelet rows delay cancellation","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Long-code tests use only zero-valued literals","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Wavelet cancellation checked every 256 rows","status":"valid","defect":"EXR-009","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Robustness tests assert neither error class nor rejection","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Two inaccurate doc phrases","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Huffman tables passed as long parameter lists","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0838,"lines":44,"significant":525,"window":6},"complexity":{"functions":22,"mean":7.86,"max":23,"over_10":6,"tool":"lizard"},"files":["piz.go"]},"isolation":{"tool_calls":45,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.2,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":4.2,"score_cross_vendor":7.2,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1-b2","task":"range-coder","lead":"gpt-6-astra-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":7.46,"tokens":{"cache_read":2752427,"cache_write":184738,"output":44037,"reasoning":25326,"uncached_input":42},"cost_usd":2.9093,"cost_breakdown":{"cache_read":0.550485,"cache_write":1.477904,"input":0.000168,"output":0.88074},"cost_usd_as_run":2.9093,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":8.18,"tokens":{"cache_read":2334208,"cache_write":0,"output":21920,"reasoning":12023,"uncached_input":87768},"cost_usd":0.3141,"cost_breakdown":{"cache_read":0.11671,"cache_write":0.0,"input":0.087768,"output":0.1096},"cost_usd_as_run":0.3141,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":11.07,"tokens":{"cache_read":1776000,"cache_write":0,"output":21936,"reasoning":13266,"uncached_input":112675},"cost_usd":0.6223,"cost_breakdown":{"cache_read":0.1776,"cache_write":0.0,"input":0.22535,"output":0.21936},"cost_usd_as_run":0.6223,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":15.87,"tokens":{"cache_read":4040000,"cache_write":0,"output":52958,"reasoning":34440,"uncached_input":270213},"cost_usd":2.948,"cost_breakdown":{"cache_read":0.808,"cache_write":0.0,"input":1.080852,"output":1.05916},"cost_usd_as_run":2.948,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":17.49,"tokens":{"cache_read":20853760,"cache_write":0,"output":167372,"reasoning":109964,"uncached_input":130325},"cost_usd":0.1825,"cost_breakdown":{"cache_read":0.062561,"cache_write":0.0,"input":0.019549,"output":0.100423},"cost_usd_as_run":0.1825,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"I","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"H","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"I","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"H","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"I","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":10.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"H","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"I","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"H","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"I","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"H","score":9.0}],"panel":["opus-5.5-xhigh","gpt-6.1-terra-max","gpt-6.1-sol-max","muse-wonder-full-xhigh","deepseek-v4.1-flash-max"]},"entries":[{"letter":"Z","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":11.85,"tokens":{"uncached_input":74177,"cache_read":378086,"cache_write":0,"output":42487,"reasoning":23668},"cost_usd":0.611,"cost_breakdown":{"input":0.148354,"cache_read":0.037809,"cache_write":0.0,"output":0.42487},"cost_usd_as_run":0.611,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1399,"code":1199,"comment":57,"blank":143,"role":"impl+test"}],"impl_lines":627,"impl_code":548,"impl_comment":30,"test_lines":772,"comment_ratio":0.052,"tests_declared":12},"hidden":{"passed":9,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.26,"reference":9.89,"speedup":0.964,"baselines":{"oracle":9.89}},{"id":"dec","candidate":11.48,"reference":11.56,"speedup":1.007,"baselines":{"oracle":11.56}}]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":3.5,"gpt-6.1-terra-max":4.0,"gpt-6.1-sol-max":4.0,"muse-wonder-full-xhigh":4.0,"deepseek-v4.1-flash-max":4.0},"council_summaries":{"opus-5.5-xhigh":"the encoder drops the leading 0xFF bytes: wrong output and a failed round-trip on ordinary valid input","gpt-6.1-terra-max":"Valid encodings can silently lose leading range bytes","gpt-6.1-sol-max":"Discards leading carry runs and corrupts valid packets","muse-wonder-full-xhigh":"Drops leading 0xFF carry bytes; wrong output and wrong finish on valid input","deepseek-v4.1-flash-max":"The encoder silently drops a leading 0xFF run, so its output is not the oracle's."},"findings":[{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"`carry` discards pending 0xFF bytes before the first byte","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Tests can't catch this","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public methods don't state preconditions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `CorruptStream` branch in `decode_uint`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Leading carry bytes are discarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method preconditions are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Impossible small-uint corruption check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Empty-buffer comment is false","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Leading `0xff` bytes are discarded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Pending runs use an avoidable byte loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Pending 0xFF bytes dropped before first buffered byte","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests never emit 0xFF as first symbol from a fresh encoder","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No contract in any public doc comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unreachable CorruptStream check in decode_uint","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`carry` discards the pending 0xFF run when no byte is buffered yet","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"No test covers a stream whose first normalized symbol is 0xFF","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable `CorruptStream` branch","status":"valid","detail":"","location":""}],"score":3.9,"slop":{"score":0.6,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":6.2,"score_cross_vendor":3.9,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","task":"range-coder","lead":"gpt-6-astra-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":31.28,"tokens":{"cache_read":16479232,"cache_write":0,"output":80985,"reasoning":42815,"uncached_input":442964},"cost_usd":1.6718,"cost_breakdown":{"cache_read":0.823962,"cache_write":0.0,"input":0.442964,"output":0.404925},"cost_usd_as_run":1.6718,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":129.86,"tokens":{"cache_read":16096724,"cache_write":0,"output":137135,"reasoning":76175,"uncached_input":828364},"cost_usd":9.2755,"cost_breakdown":{"cache_read":3.219345,"cache_write":0.0,"input":3.313456,"output":2.7427},"cost_usd_as_run":9.2755,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":37.6,"tokens":{"cache_read":47782656,"cache_write":0,"output":395381,"reasoning":279931,"uncached_input":615703},"cost_usd":0.4729,"cost_breakdown":{"cache_read":0.143348,"cache_write":0.0,"input":0.092356,"output":0.237228},"cost_usd_as_run":0.4729,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":58.26,"tokens":{"cache_read":32148933,"cache_write":980626,"output":245308,"reasoning":145257,"uncached_input":274},"cost_usd":19.1821,"cost_breakdown":{"cache_read":6.429786,"cache_write":7.845008,"input":0.001096,"output":4.90616},"cost_usd_as_run":19.1821,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":72.19,"tokens":{"cache_read":16547968,"cache_write":0,"output":102999,"reasoning":57452,"uncached_input":630529},"cost_usd":3.9459,"cost_breakdown":{"cache_read":1.654797,"cache_write":0.0,"input":1.261058,"output":1.02999},"cost_usd_as_run":3.9459,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":10.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":4.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":4.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-gpt-6-astra-xhigh-r1","letter":"B","score":9.5}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","deepseek-v4.1-flash-max","opus-5.5-xhigh","gpt-6.1-sol-max"]},"entries":[{"letter":"A","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":14.47,"tokens":{"uncached_input":27289,"cache_read":7545088,"cache_write":0,"output":155623,"reasoning":132012},"cost_usd":0.1201,"cost_breakdown":{"input":0.004093,"cache_read":0.022635,"cache_write":0.0,"output":0.093374},"cost_usd_as_run":0.1201,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1201,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1371,"code":1081,"comment":196,"blank":94,"role":"impl+test"}],"impl_lines":744,"impl_code":521,"impl_comment":178,"test_lines":627,"comment_ratio":0.255,"tests_declared":18},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.39,"reference":10.22,"speedup":0.984,"baselines":{"oracle":10.22}},{"id":"dec","candidate":11.71,"reference":11.86,"speedup":1.013,"baselines":{"oracle":11.86}}]},"scores":{"gpt-6.1-terra-max":4.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Silent overflow after a very long carry run, plus avoidable flushing work.","deepseek-v4.1-flash-max":"The most complete precondition docs; three dead defensive fragments.","muse-wonder-full-xhigh":"bit-exact with exemplary pending docs; false wrapping claims and duplicated accounting code","opus-5.5-xhigh":"Exact, with good tests; one false public doc, plus a dead guard on the hot loop with a false comment","gpt-6.1-sol-max":"Compatible and readable, with misleading buffer documentation and limited ICDF fixtures."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Carry counter wraps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Failed writes are repeated for the whole carry run","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Constructor promises untouched storage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`finish`'s `.min(8)`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`decode_icdf`'s `index + 1 >= icdf.len()` break","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"`normalize`'s \"Only reachable through arithmetic on malformed tables\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"\"Arithmetic is wrapping\" claims are false","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"`TOP` misdescribed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"`tell_frac` body duplicated verbatim","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Slowest encoder in a same-mix timing probe","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"`new()` doc is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False justification for a guard inside the normalize loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`tell_frac` body duplicated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Misnamed test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Repeated failed carry writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Buffer changes before finishing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Successful ICDF fixtures use only precision 8","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1439,"lines":58,"significant":403,"window":6},"complexity":{"functions":36,"mean":3.22,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":51,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.2,"slop":{"score":1.6,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":2.5,"deepseek-v4.1-flash-max":1.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":1.0}},"defect_points":3.3,"score_cross_vendor":6.875,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":8.87,"tokens":{"uncached_input":21904,"cache_read":287360,"cache_write":0,"output":21016,"reasoning":5706},"cost_usd":0.2827,"cost_breakdown":{"input":0.043808,"cache_read":0.028736,"cache_write":0.0,"output":0.21016},"cost_usd_as_run":0.2827,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1247,"code":1112,"comment":48,"blank":87,"role":"impl+test"}],"impl_lines":551,"impl_code":456,"impl_comment":47,"test_lines":696,"comment_ratio":0.093,"tests_declared":17},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.31,"reference":10.35,"speedup":0.915,"baselines":{"oracle":10.35}},{"id":"dec","candidate":11.69,"reference":11.91,"speedup":1.019,"baselines":{"oracle":11.91}}]},"scores":{"gpt-6.1-terra-max":4.5,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.5},"council_summaries":{"gpt-6.1-terra-max":"Broad coverage and efficient carry flushing, but the carry count can wrap.","deepseek-v4.1-flash-max":"Accurate docs and broad tests; the `finish` bust path writes where libopus deliberately does not.","muse-wonder-full-xhigh":"bit-exact with strong tests; only guard-clause and validation nits","opus-5.5-xhigh":"Exact, good tests including wrap-around, terse but adequate docs","gpt-6.1-sol-max":"Clear, comprehensive, and efficient, with one unnecessary defensive operation."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Carry counter wraps","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"Latches the shared-byte overflow and then still ORs the *unmasked* window into the last range byte","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`icdf.is_empty()` in the `decode_icdf` guard","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"\"Equal or overflowing malformed boundaries must not produce a zero range\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unguarded degenerate-range paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Redundant emptiness check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False comment and dead `.max(1)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unnecessary range repair","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0458,"lines":16,"significant":349,"window":6},"complexity":{"functions":39,"mean":2.64,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.7,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5}},"defect_points":1.4,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":57.88,"tokens":{"uncached_input":170548,"cache_read":2068608,"cache_write":0,"output":215224,"reasoning":98471},"cost_usd":2.6667,"cost_breakdown":{"input":0.341096,"cache_read":1.034304,"cache_write":0.0,"output":1.291344},"cost_usd_as_run":2.6667,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1490,"code":1272,"comment":117,"blank":101,"role":"impl+test"}],"impl_lines":631,"impl_code":482,"impl_comment":104,"test_lines":859,"comment_ratio":0.177,"tests_declared":18},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.07,"reference":9.89,"speedup":0.982,"baselines":{"oracle":9.89}},{"id":"dec","candidate":11.73,"reference":11.52,"speedup":0.982,"baselines":{"oracle":11.52}}]},"scores":{"gpt-6.1-terra-max":4.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Long carry runs can disappear, and flushing an exhausted buffer is unnecessarily linear.","deepseek-v4.1-flash-max":"Thorough docs and the densest assertions of the set; two dead branches in the hot code.","muse-wonder-full-xhigh":"bit-exact, defensive, with explicit per-method pending documentation","opus-5.5-xhigh":"Exact and well documented; a few dead defensive branches and test nits","gpt-6.1-sol-max":"Compatible and well tested, with minor performance and test cleanup needed."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Carry counter wraps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry flush repeats dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Probability-coded bits are called raw bits","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`decode_icdf`'s extra break condition is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Leftover raw bits are merged outside the `error.is_none()` gate","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Range-coded bit called a \"raw bit\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"`finish` merges leftover bits before checking they fit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"raw bit\" for the range-coded bit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Slack tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Repeated failed carry writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Tautological assertion","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":29,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5}},"defect_points":2.1,"score_cross_vendor":7.6,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.9,"tokens":{"uncached_input":10906,"cache_read":128512,"cache_write":0,"output":4262,"reasoning":211},"cost_usd":0.0386,"cost_breakdown":{"input":0.010906,"cache_read":0.006426,"cache_write":0.0,"output":0.02131},"cost_usd_as_run":0.0386,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":481,"code":450,"comment":30,"blank":1,"role":"impl+test"}],"impl_lines":452,"impl_code":421,"impl_comment":30,"test_lines":29,"comment_ratio":0.067,"tests_declared":1},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.28,"reference":10.36,"speedup":1.008,"baselines":{"oracle":10.36}},{"id":"dec","candidate":12.28,"reference":11.81,"speedup":0.962,"baselines":{"oracle":11.81}}]},"scores":{"gpt-6.1-terra-max":6.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":6.5},"council_summaries":{"gpt-6.1-terra-max":"Ordinary behavior matches the oracle, but tests and public documentation are sparse.","deepseek-v4.1-flash-max":"Correct, robust and compact, but the test suite is a single vector and the docs state no contract at all.","muse-wonder-full-xhigh":"implementation verified bit-exact externally, but its own tests and docs do not verify the contract","opus-5.5-xhigh":"Correct and robust, but the decoder is never tested and docs are bare","gpt-6.1-sol-max":"Compatible core, substantially under-tested and under-documented."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry flush repeats dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Only one encoder vector is tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public methods omit their preconditions and error behavior","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"The whole `#[cfg(test)]` module is one 11-op vector","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Public docs are one-liners that omit every precondition and error contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`Display` delegates to `Debug`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Single test covers one encoder vector and nothing else","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Docs carry no contract information","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Plain arithmetic and unguarded shifts depend wholly on register invariants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Cramped layout hurts readability","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"One encoder-only test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public docs omit the contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable guards on the decode path","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Almost the entire decoder and robustness contract lack regression coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API contracts are largely undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Repeated failed carry writes","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0502,"lines":16,"significant":319,"window":6},"complexity":{"functions":32,"mean":3.06,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.4,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":4.9,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.9,"tokens":{"uncached_input":14644,"cache_read":207104,"cache_write":0,"output":10332,"reasoning":876},"cost_usd":0.1533,"cost_breakdown":{"input":0.029288,"cache_read":0.02071,"cache_write":0.0,"output":0.10332},"cost_usd_as_run":0.1533,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":896,"code":829,"comment":32,"blank":35,"role":"impl+test"}],"impl_lines":480,"impl_code":426,"impl_comment":32,"test_lines":416,"comment_ratio":0.07,"tests_declared":9},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.52,"reference":10.19,"speedup":0.969,"baselines":{"oracle":10.19}},{"id":"dec","candidate":11.7,"reference":11.68,"speedup":0.998,"baselines":{"oracle":11.68}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with bounded carry-overflow handling and incomplete API documentation.","deepseek-v4.1-flash-max":"Correct and compact, but tests leave the finish boundary half-unchecked and one defensive comment is false.","muse-wonder-full-xhigh":"bit-exact; docs omit preconditions and error semantics","opus-5.5-xhigh":"Exact and compact; the weakest docs after D","gpt-6.1-sol-max":"Efficient compatible implementation, with incomplete documentation and targeted test gaps."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method contracts omit important bounds","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Finish-size sweep asserts nothing between 10 and 23 bytes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"\"Malformed ICDFs may produce a zero range\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"Unguarded shift in `tell_frac`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Method docs state no numeric preconditions or rejection behaviour","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Encoder `normalize`/`tell_frac` unguarded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `decode_icdf` fallback","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Preconditions and rejection undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False comment on a dead guard","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public methods omit important preconditions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Incorrect zero-range explanation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Lower-precision ICDF tests are ineffective","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0495,"lines":16,"significant":323,"window":6},"complexity":{"functions":34,"mean":2.91,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":1.0}},"defect_points":2.2,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.14,"tokens":{"uncached_input":12997,"cache_read":209664,"cache_write":0,"output":10238,"reasoning":626},"cost_usd":0.8515,"cost_breakdown":{"input":0.12997,"cache_read":0.209664,"cache_write":0.0,"output":0.5119},"cost_usd_as_run":0.8515,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":897,"code":854,"comment":35,"blank":8,"role":"impl+test"}],"impl_lines":480,"impl_code":437,"impl_comment":35,"test_lines":417,"comment_ratio":0.074,"tests_declared":8},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.24,"reference":10.1,"speedup":0.986,"baselines":{"oracle":10.1}},{"id":"dec","candidate":11.87,"reference":11.97,"speedup":1.008,"baselines":{"oracle":11.97}}]},"scores":{"gpt-6.1-terra-max":6.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Good ordinary coverage, with a platform-sized carry counter and costly overflow handling.","deepseek-v4.1-flash-max":"Correct and lean; the finish sweep is thin and the error path deviates from `ec_enc_done`.","muse-wonder-full-xhigh":"bit-exact compact implementation; contract semantics largely undocumented","opus-5.5-xhigh":"Exact and compact, but docs are terse and one comment is false","gpt-6.1-sol-max":"Correct compact core, with documentation and ICDF coverage gaps."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Carry count wraps on the target’s `usize` boundary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry flush repeats dropped writes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Finish-size sweep stops asserting at the boundary","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Leftover raw bits are merged outside the `error.is_none()` gate","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"\"Malformed inverse CDFs can produce zero\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Public docs omit the error model","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"`fraction()` and encoder `normalize` have no degenerate-range defense","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin spots in an otherwise dense 8-test suite","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `decode_icdf` fallback","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Method docs never state what happens on misuse","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False comment on a dead guard","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Repeated failed carry writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Incorrect zero-range explanation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Nontrivial ICDF precision coverage is missing","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0483,"lines":16,"significant":331,"window":6},"complexity":{"functions":35,"mean":2.89,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":1.0}},"defect_points":3.0,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.45,"tokens":{"uncached_input":30,"cache_read":1325636,"cache_write":106567,"output":67793,"reasoning":35570},"cost_usd":2.4736,"cost_breakdown":{"input":0.00012,"cache_read":0.265127,"cache_write":0.852536,"output":1.35586},"cost_usd_as_run":2.4736,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1664,"code":1315,"comment":241,"blank":108,"role":"impl+test"}],"impl_lines":762,"impl_code":504,"impl_comment":209,"test_lines":902,"comment_ratio":0.293,"tests_declared":30},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.3,"reference":10.19,"speedup":0.989,"baselines":{"oracle":10.19}},{"id":"dec","candidate":11.87,"reference":11.66,"speedup":0.982,"baselines":{"oracle":11.66}}]},"scores":{"gpt-6.1-terra-max":4.5,"deepseek-v4.1-flash-max":9.5,"muse-wonder-full-xhigh":9.0,"opus-5.5-xhigh":9.5,"gpt-6.1-sol-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Thorough ordinary tests, but a long carry run can wrap and disappear.","deepseek-v4.1-flash-max":"The best documentation and widest test suite; only a cosmetic wart.","muse-wonder-full-xhigh":"bit-exact with the deepest test suite and invariant-level documentation","opus-5.5-xhigh":"Exact, robust, accurate docs that state the invariants, very thorough tests; nothing to fix","gpt-6.1-sol-max":"Strong invariants, documentation, and regression coverage."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Carry counter wraps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry flush repeats dropped writes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`self.rem.unsigned_abs()` inside `if self.rem >= 0`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Normalization and `finish` rely on the `val < rng`, `rng >= 1` invariant with no safety net","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Pending-cancellation rule undocumented on most mutators","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Repeated failed carry writes","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0415,"lines":16,"significant":386,"window":6},"complexity":{"functions":40,"mean":2.73,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.3,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.6,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.35,"tokens":{"uncached_input":14,"cache_read":387488,"cache_write":72888,"output":39495,"reasoning":16183},"cost_usd":1.4506,"cost_breakdown":{"input":5.6e-05,"cache_read":0.077498,"cache_write":0.583104,"output":0.7899},"cost_usd_as_run":1.4506,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1466,"code":1217,"comment":151,"blank":98,"role":"impl+test"}],"impl_lines":685,"impl_code":510,"impl_comment":126,"test_lines":781,"comment_ratio":0.198,"tests_declared":21},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.52,"reference":10.27,"speedup":0.976,"baselines":{"oracle":10.27}},{"id":"dec","candidate":11.92,"reference":11.73,"speedup":0.984,"baselines":{"oracle":11.73}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Strong ordinary behavior and tests, with a very large-buffer carry-count limit.","deepseek-v4.1-flash-max":"Bit-exact, robust, and thoroughly tested; only unreachable defensive no-ops.","muse-wonder-full-xhigh":"bit-exact, the most defensive implementation, strong randomized tests","opus-5.5-xhigh":"Exact, clear docs and extensive tests; only dead clamps with contradictory comments","gpt-6.1-sol-max":"Comprehensive and compatible, with efficient overflow handling and minor redundant arithmetic."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Saturating `u32` loses pending bytes on large buffers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`finish` clamps a shift its own comment already bounds","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Pending-cancellation rule undocumented on most mutators","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Dead `.max(1)` clamps and `rng != 0` guards on hot paths, with contradictory comments","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Repeated nonzero-range clamps do nothing","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0416,"lines":16,"significant":385,"window":6},"complexity":{"functions":40,"mean":2.88,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.7,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5}},"defect_points":0.6,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":317.34,"tokens":{"uncached_input":212065,"cache_read":1796048,"cache_write":0,"output":37753,"reasoning":17419},"cost_usd":1.9625,"cost_breakdown":{"input":0.84826,"cache_read":0.35921,"cache_write":0.0,"output":0.75506},"cost_usd_as_run":1.9625,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1444,"code":1265,"comment":96,"blank":83,"role":"impl+test"}],"impl_lines":699,"impl_code":571,"impl_comment":78,"test_lines":745,"comment_ratio":0.12,"tests_declared":18},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.03,"reference":9.9,"speedup":0.987,"baselines":{"oracle":9.9}},{"id":"dec","candidate":11.73,"reference":11.47,"speedup":0.978,"baselines":{"oracle":11.47}}]},"scores":{"gpt-6.1-terra-max":4.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Carry-count wrapping causes false success, with avoidable defensive branches and flushing work.","deepseek-v4.1-flash-max":"Well documented and well tested, with redundant saved state and an over-defended `finish`.","muse-wonder-full-xhigh":"bit-exact and thoroughly tested; over-defensive arithmetic obscures the hot paths","opus-5.5-xhigh":"Exact and well tested, but padded with unreachable defensive code","gpt-6.1-sol-max":"Compatible behavior, with unnecessary state and defensive scaffolding."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Carry counter wraps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry flush repeats dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Private helpers contain unreachable argument branches","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"`Pending.rng` duplicates `self.rng`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"`finish` is buried in defensive plumbing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`decode_value`'s `ft == 0` / `factor == 0` early returns","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Pervasive no-op defensiveness","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Pending pair stores a copy of `rng`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `decode_icdf` fallback","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Pending-cancellation rule undocumented on most mutators","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Defensive dead branches throughout","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`Pending.rng` duplicates `self.rng`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Repeated failed carry writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Pending decode duplicates the range register","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0375,"lines":16,"significant":427,"window":6},"complexity":{"functions":40,"mean":3.12,"max":18,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":23,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.2,"slop":{"score":1.4,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":2.0,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":1.0}},"defect_points":2.7,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":61.03,"tokens":{"uncached_input":282789,"cache_read":20148736,"cache_write":0,"output":216880,"reasoning":185053},"cost_usd":0.7553,"cost_breakdown":{"input":0.042418,"cache_read":0.604462,"cache_write":0.0,"output":0.10844},"cost_usd_as_run":0.7553,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1487,"code":1129,"comment":266,"blank":92,"role":"impl+test"}],"impl_lines":715,"impl_code":479,"impl_comment":192,"test_lines":772,"comment_ratio":0.286,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.54,"reference":10.77,"speedup":1.022,"baselines":{"oracle":10.77}},{"id":"dec","candidate":11.89,"reference":11.71,"speedup":0.985,"baselines":{"oracle":11.71}}]},"scores":{"gpt-6.1-terra-max":3.5,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":6.0},"council_summaries":{"gpt-6.1-terra-max":"A legal long sequence panics, with an additional linear carry flush.","deepseek-v4.1-flash-max":"Rich tests and good docs, undermined by leftover scratch code, two docs that overstate what is checked, and one vacuous assertion.","muse-wonder-full-xhigh":"bit-exact and defensive; slop confined to test leftovers and one wrong doc line","opus-5.5-xhigh":"Exact output, but a reachable debug-overflow panic, several false docs and leftover scratch tests","gpt-6.1-sol-max":"Ordinary compatibility is strong, but a valid long sequence panics."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Carry counter overflows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry flush repeats dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"`ilog(0)` is described incorrectly","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The success assertion in `tests2::buffer_size_sweep` is vacuous","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"Two test doc comments claim per-step accounting checks the code does not perform","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"Leftover probe module `scratch4` with a `println!`-driven test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"`normalize`'s \"`rng != 0` guards degenerate states produced by malformed tables\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`finish` increments `self.back` for the merged partial byte","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"`ilog(0)` documented as 32, code returns 0","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Leftover probe test duplicates the sweep","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Tautological assertion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"Plain `ext += 1` on a u32 counter","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"`ext` counter overflows","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False doc claims","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Leftover debug test module","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"No-op assertion and duplicated code","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Carry-run counter overflows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Repeated failed carry writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Comments claim behavior and checks absent from the code","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Leftover probe and tautological check","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0879,"lines":34,"significant":387,"window":6},"complexity":{"functions":38,"mean":2.95,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":96,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.5,"slop":{"score":1.4,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":1.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":2.0}},"defect_points":4.1,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.57,"tokens":{"uncached_input":13541,"cache_read":400896,"cache_write":0,"output":19915,"reasoning":6780},"cost_usd":0.1332,"cost_breakdown":{"input":0.013541,"cache_read":0.020045,"cache_write":0.0,"output":0.099575},"cost_usd_as_run":0.1332,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1135,"code":1001,"comment":51,"blank":83,"role":"impl+test"}],"impl_lines":567,"impl_code":467,"impl_comment":51,"test_lines":568,"comment_ratio":0.098,"tests_declared":12},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.14,"reference":10.26,"speedup":0.921,"baselines":{"oracle":10.26}},{"id":"dec","candidate":12.06,"reference":11.88,"speedup":0.985,"baselines":{"oracle":11.88}}]},"scores":{"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":7.0},"council_summaries":{"muse-wonder-full-xhigh":"sound implementation with partial precondition docs and fewer tests","deepseek-v4.1-flash-max":"Correct and solid; one redundant loop condition and docs that state less than the contract.","gpt-6.1-terra-max":"Efficient and well tested, with incomplete public docs","opus-5.5-xhigh":"Bit-exact, batched carry flush, thorough snapshot-based misuse and pending tests; one false comment","gpt-6.1-sol-max":"Efficient batching and useful tests; carry-counter semantics and API documentation need correction."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"preconditions stated only partially","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"12 tests, the fewest outside the Y/H/D group, with several contract areas folded into combined tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"finish merge runs after a latched error, unlike the reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"redundant break condition","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"preconditions are only partly stated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Method preconditions are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Malformed-table comment describes an unreachable zero range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False zero-width claim","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Redundant loop exit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Carry count diverges at the 32-bit boundary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Several enforced preconditions are undocumented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0455,"lines":16,"significant":352,"window":6},"complexity":{"functions":40,"mean":2.67,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.0,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.97,"tokens":{"uncached_input":13482,"cache_read":327040,"cache_write":0,"output":21549,"reasoning":4929},"cost_usd":1.5393,"cost_breakdown":{"input":0.13482,"cache_read":0.32704,"cache_write":0.0,"output":1.07745},"cost_usd_as_run":1.5393,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1356,"code":1194,"comment":75,"blank":87,"role":"impl+test"}],"impl_lines":583,"impl_code":462,"impl_comment":73,"test_lines":773,"comment_ratio":0.136,"tests_declared":17},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.08,"reference":10.19,"speedup":0.92,"baselines":{"oracle":10.19}},{"id":"dec","candidate":12.36,"reference":11.93,"speedup":0.965,"baselines":{"oracle":11.93}}]},"scores":{"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-terra-max":9.5,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":7.5},"council_summaries":{"muse-wonder-full-xhigh":"well documented and tested; only the on-error finish fidelity lags","deepseek-v4.1-flash-max":"Tightest implementation here: no dead code, no wrong docs, and a size-sweep test that pins the exact Ok/Err boundary.","gpt-6.1-terra-max":"Strong implementation and tests, with one misleading invariant comment","opus-5.5-xhigh":"Bit-exact, batched carry flush, thorough tests and docs; contradictory comments and a test of an impossible state","gpt-6.1-sol-max":"Strong documentation, tests, and batching; an extreme carry-counter boundary breaks compatibility."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"finish merges leftover bits after a latched error","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Malformed-table comment describes an unreachable state","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Zero-width claim contradicts L's own comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Test drives an unreachable state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Carry count does not wrap like the oracle","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0456,"lines":16,"significant":351,"window":6},"complexity":{"functions":39,"mean":2.72,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.6,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":0.9,"score_cross_vendor":8.667,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":324.95,"tokens":{"uncached_input":288287,"cache_read":2326672,"cache_write":0,"output":34388,"reasoning":12125},"cost_usd":2.3062,"cost_breakdown":{"input":1.153148,"cache_read":0.465334,"cache_write":0.0,"output":0.68776},"cost_usd_as_run":2.3062,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1363,"code":1155,"comment":128,"blank":80,"role":"impl+test"}],"impl_lines":711,"impl_code":552,"impl_comment":109,"test_lines":652,"comment_ratio":0.165,"tests_declared":20},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.19,"reference":10.08,"speedup":0.989,"baselines":{"oracle":10.08}},{"id":"dec","candidate":11.55,"reference":11.35,"speedup":0.983,"baselines":{"oracle":11.35}}]},"scores":{"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-terra-max":6.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":7.0},"council_summaries":{"muse-wonder-full-xhigh":"correct and well tested, with one wrong doc and finish-fidelity nits","deepseek-v4.1-flash-max":"Correct, well documented, but several branches are provably unreachable and one is wrong if it ever ran.","gpt-6.1-terra-max":"Correct in the probes, but the carry path, size tests, and docs need changes","opus-5.5-xhigh":"Correct with good tests, but about a dozen unreachable branches and a wrong rounding claim in the public `tell` docs","gpt-6.1-sol-max":"Ordinary behavior is compatible, but defensive padding and weak finish assertions reduce quality."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"tell() doc says \"rounded down\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"finish merges leftover raw bits outside the no-error block and writes them unmasked when the ends meet","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"decode_icdf maintains `chosen_t`/`chosen_s` updated on both branches plus an unreachable `get(symbol-1)` None arm in encode_icdf","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"dead \"fallback\" bookkeeping in `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Failed carry writes are replayed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Size sweep discards every finish result","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"`tell` is described as rounded down","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Raw-bit helpers contain impossible width branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"`tell` documented as \"rounded down\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Pervasive unreachable guards","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Carry resolution remains linear on an empty output","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Internal raw-bit modes are unreachable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Whole-bit accounting is described as rounding down","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Buffer sweep discards every finish result","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0381,"lines":16,"significant":420,"window":6},"complexity":{"functions":41,"mean":2.98,"max":15,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":38,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.3,"slop":{"score":1.7,"per_seat":{"gpt-6.1-terra-max":1.5,"muse-wonder-full-xhigh":1.5,"deepseek-v4.1-flash-max":1.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":2.0}},"defect_points":3.1,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":0.61,"tokens":{"uncached_input":24,"cache_read":99121,"cache_write":9940,"output":1445,"reasoning":0},"cost_usd":0.0043,"cost_breakdown":{"input":4e-06,"cache_read":0.001586,"cache_write":0.001988,"output":0.000679},"cost_usd_as_run":0.0043,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0043,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"dnf","dnf_reason":"Isolation violation: instead of working only in its folder, it explored the private benchmark repository (listing, git log, tracker) and other home folders, then stopped without writing code.","isolation":{"tool_calls":5,"flags":3,"audited":"2026-11-02","reviewed":true},"findings":[]},{"letter":"O","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.62,"tokens":{"uncached_input":17870,"cache_read":129152,"cache_write":0,"output":6447,"reasoning":42},"cost_usd":0.1131,"cost_breakdown":{"input":0.03574,"cache_read":0.012915,"cache_write":0.0,"output":0.06447},"cost_usd_as_run":0.1131,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":678,"code":644,"comment":32,"blank":2,"role":"impl+test"}],"impl_lines":465,"impl_code":432,"impl_comment":32,"test_lines":213,"comment_ratio":0.069,"tests_declared":6},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.56,"reference":10.22,"speedup":0.968,"baselines":{"oracle":10.22}},{"id":"dec","candidate":11.93,"reference":11.81,"speedup":0.99,"baselines":{"oracle":11.81}}]},"scores":{"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":6.5},"council_summaries":{"muse-wonder-full-xhigh":"functionally correct and terse, but thin tests and missing precondition docs","deepseek-v4.1-flash-max":"Correct and compact; docs stop short of the contract and the test suite stops at two buffer sizes.","gpt-6.1-terra-max":"Compact and correct in the probes, but tests and API docs are thin","opus-5.5-xhigh":"Correct and compact, but the thinnest tests and docs, plus a false comment","gpt-6.1-sol-max":"Efficient overflow resolution; counter compatibility, documentation, and boundary tests need changes."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"only 6 tests for the whole contract","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"public methods state no preconditions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"finish merges leftover bits even when an error is already latched","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"the budget sweep asserts nothing except two sizes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"thin docs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Carry bytes are written one at a time","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Budget sweep ignores most finish outcomes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public input constraints are omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Malformed-table comment describes an unreachable zero range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Decode/update never round-tripped; sweep mostly unchecked","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"One-line public docs with no preconditions or error rules","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"A malformed ICDF may produce a zero range.\" is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`if end | m >= …` without parentheses","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Saturating carry count changes finalization","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public contracts are largely unstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Size sweep misses the actual fit boundary","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0935,"lines":30,"significant":321,"window":6},"complexity":{"functions":36,"mean":3.0,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.0,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":3.5,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":3.76,"tokens":{"uncached_input":14,"cache_read":355320,"cache_write":62082,"output":28581,"reasoning":7622},"cost_usd":1.1394,"cost_breakdown":{"input":5.6e-05,"cache_read":0.071064,"cache_write":0.496656,"output":0.57162},"cost_usd_as_run":1.1394,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1337,"code":1143,"comment":109,"blank":85,"role":"impl+test"}],"impl_lines":665,"impl_code":521,"impl_comment":96,"test_lines":672,"comment_ratio":0.156,"tests_declared":20},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.14,"reference":10.31,"speedup":1.017,"baselines":{"oracle":10.31}},{"id":"dec","candidate":12.43,"reference":12.2,"speedup":0.981,"baselines":{"oracle":12.2}}]},"scores":{"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":8.5},"council_summaries":{"muse-wonder-full-xhigh":"clean, exact finish, guards and tests all in place","deepseek-v4.1-flash-max":"Correct and thoroughly tested; one naming problem that makes the file harder to read than it needs to be.","gpt-6.1-terra-max":"Broad tests and correct probed behavior, with avoidable carry latency","opus-5.5-xhigh":"Bit-exact with good docs and 20 solid tests, but several dead guards and confusing helper names","gpt-6.1-sol-max":"Broadly correct and well tested; carry resolution retains avoidable linear work."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"encode_icdf validates `cur`/`prev` via `get()` then re-indexes `icdf[symbol]`/`icdf[symbol-1]` after validation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"\"raw\" means two different things","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Failed carry writes are replayed individually","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False claim on decoder zero-range guards","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Impossible branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"\"raw\" names for range-coded helpers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Carry flush processes bytes that cannot fit","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0404,"lines":16,"significant":396,"window":6},"complexity":{"functions":39,"mean":2.87,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.5,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.3,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.51,"tokens":{"uncached_input":18514,"cache_read":128256,"cache_write":0,"output":6390,"reasoning":21},"cost_usd":0.6329,"cost_breakdown":{"input":0.18514,"cache_read":0.128256,"cache_write":0.0,"output":0.3195},"cost_usd_as_run":0.6329,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":722,"code":691,"comment":30,"blank":1,"role":"impl+test"}],"impl_lines":454,"impl_code":423,"impl_comment":30,"test_lines":268,"comment_ratio":0.066,"tests_declared":6},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.13,"reference":10.38,"speedup":1.025,"baselines":{"oracle":10.38}},{"id":"dec","candidate":12.11,"reference":11.79,"speedup":0.974,"baselines":{"oracle":11.79}}]},"scores":{"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":6.0},"council_summaries":{"muse-wonder-full-xhigh":"correct and compact, with thin tests and missing precondition docs","deepseek-v4.1-flash-max":"Correct and fast, but the public API is essentially undocumented and the tests are the thinnest here.","gpt-6.1-terra-max":"Compact and correct in the probes, but its carry path and coverage need work","opus-5.5-xhigh":"Compact and fully correct, but docs are one-liners and tests never check decode/update on a valid stream","gpt-6.1-sol-max":"Counter incompatibility, linear discarded writes, and thin documentation/testing require changes."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"only 6 grouped tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"public methods state no preconditions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"finish merges leftover bits after a latched error, unlike the reference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no method documents its domain or its failure behaviour","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"the size sweep checks nothing except two sizes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Failed carry writes are replayed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Budget sweep ignores most finish outcomes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public input constraints are omitted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The generic decode path and golden encodes are never verified by decoding","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public docs omit preconditions and error behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable zero-range guard","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Carry counter has incompatible wrap behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Carry resolution repeats every failed write","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Required method domains are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Finish boundary is not asserted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0498,"lines":16,"significant":321,"window":6},"complexity":{"functions":32,"mean":3.06,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":4.1,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":342.47,"tokens":{"uncached_input":202845,"cache_read":3177452,"cache_write":0,"output":49524,"reasoning":25248},"cost_usd":2.4373,"cost_breakdown":{"input":0.81138,"cache_read":0.63549,"cache_write":0.0,"output":0.99048},"cost_usd_as_run":2.4373,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1510,"code":1305,"comment":117,"blank":88,"role":"impl+test"}],"impl_lines":684,"impl_code":543,"impl_comment":91,"test_lines":826,"comment_ratio":0.144,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.53,"reference":9.98,"speedup":0.948,"baselines":{"oracle":9.98}},{"id":"dec","candidate":11.5,"reference":11.58,"speedup":1.007,"baselines":{"oracle":11.58}}]},"scores":{"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0},"council_summaries":{"muse-wonder-full-xhigh":"most defensive entry; pays for it in redundant branches and the slowest encoder","deepseek-v4.1-flash-max":"Correct and very fully documented; a few unreachable defensive branches.","gpt-6.1-terra-max":"Sound core behavior, with avoidable carry work and masked test failures","opus-5.5-xhigh":"Correct and well documented, but over-defensive throughout, and the only entry with an encoder clearly slower than the oracle","gpt-6.1-sol-max":"Compatible behavior and useful coverage; carry work and permissive edge assertions remain."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"pervasive unreachable `None`/`0` fallbacks after bounds already proven (bounded decode_icdf loop `get(index)` with `None => 0`, re-`get(symbol-1)` in encode_icdf)","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"finish latches BufferOverflow on the busted shared byte without the reference's masked partial write","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"slowest encoder in my bench (7 vs 3–6 ns/op) consistent with the extra `get_mut`/saturating checks on write paths","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Failed carry writes are replayed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Size sweep ignores most finish outcomes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Exhaustion test permits a spurious corruption error","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Encoder about 20% slower than the oracle","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False claim plus dead zero-range checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Defensive no-ops hide the libopus mapping","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Exhausted carry flushing is linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Edge tests admit incorrect outcomes","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0386,"lines":16,"significant":414,"window":6},"complexity":{"functions":41,"mean":2.98,"max":17,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":37,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.4,"score_cross_vendor":7.75,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.02,"tokens":{"uncached_input":13463,"cache_read":355328,"cache_write":0,"output":10214,"reasoning":800},"cost_usd":0.0823,"cost_breakdown":{"input":0.013463,"cache_read":0.017766,"cache_write":0.0,"output":0.05107},"cost_usd_as_run":0.0823,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":880,"code":845,"comment":30,"blank":5,"role":"impl+test"}],"impl_lines":483,"impl_code":448,"impl_comment":30,"test_lines":397,"comment_ratio":0.063,"tests_declared":7},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.27,"reference":10.19,"speedup":0.992,"baselines":{"oracle":10.19}},{"id":"dec","candidate":12.1,"reference":11.94,"speedup":0.987,"baselines":{"oracle":11.94}}]},"scores":{"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0},"council_summaries":{"muse-wonder-full-xhigh":"correct core; grouped-thin tests, no precondition docs, revalidation in uint paths","deepseek-v4.1-flash-max":"Correct, but the shortest docs and the thinnest test suite of the \"complete\" entries.","gpt-6.1-terra-max":"Correct in the probes, but carry resolution and documentation need work","opus-5.5-xhigh":"Correct and compact; tests are few but meaningful; public docs are bare one-liners","gpt-6.1-sol-max":"Compact and compatible; carry flushing and public documentation need improvement."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"only 7 grouped tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"public methods state no preconditions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"encode_uint/decode_uint route through the public validating `encode`/`encode_bits`/`decode`/`update`/`decode_bits`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"preconditions and error behaviour are mostly absent","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"redundant break condition","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Failed carry writes are replayed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Budget sweep accepts premature overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public ICDF requirements are omitted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public docs omit domains, InvalidArgument and pending rules","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Dead guard and redundant exit test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Exhausted carry runs remain linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public argument domains are omitted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0472,"lines":16,"significant":339,"window":6},"complexity":{"functions":35,"mean":2.86,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":2.7,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":19.54,"tokens":{"uncached_input":34,"cache_read":2101141,"cache_write":200715,"output":290670,"reasoning":265905},"cost_usd":0.5131,"cost_breakdown":{"input":1.1e-05,"cache_read":0.09742,"cache_write":0.172617,"output":0.243046},"cost_usd_as_run":0.5131,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1288,"code":1052,"comment":148,"blank":88,"role":"impl+test"}],"impl_lines":674,"impl_code":489,"impl_comment":137,"test_lines":614,"comment_ratio":0.219,"tests_declared":22},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.28,"reference":10.28,"speedup":1.0,"baselines":{"oracle":10.28}},{"id":"dec","candidate":11.92,"reference":11.78,"speedup":0.988,"baselines":{"oracle":11.78}}]},"scores":{"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":6.5},"council_summaries":{"muse-wonder-full-xhigh":"best pending-lifecycle tests; minor finish blemishes","deepseek-v4.1-flash-max":"Bit-exact with the best documentation and deepest test suite here; only cosmetic blemishes.","gpt-6.1-terra-max":"Broad tests and correct probed behavior, with carry latency and misleading comments","opus-5.5-xhigh":"Correct with very good docs, but no-op guards on the hot path and a self-confirming test of dead code","gpt-6.1-sol-max":"Careful validation and broad tests; carry saturation changes outcomes and flushing wastes work."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"finish merges leftover bits outside the no-error block","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"`self.back += 1` after the non-shared merge in finish","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`finish` bumps a field of a consumed value","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"test snapshot stores the pending decode twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Failed carry writes are replayed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Fractional accounting is described as a unit conversion of `tell`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Malformed-table comment describes an unreachable zero range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False comment on a per-iteration guard","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Self-confirming test of an impossible state","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"No-op defensive operations and a dead store","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Saturation changes the finish result","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Carry flushing loops over rejected writes","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0431,"lines":16,"significant":371,"window":6},"complexity":{"functions":39,"mean":2.82,"max":15,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":19,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":1.5,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.3,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":13.1,"tokens":{"uncached_input":30727,"cache_read":296576,"cache_write":0,"output":28755,"reasoning":11626},"cost_usd":0.3787,"cost_breakdown":{"input":0.061454,"cache_read":0.029658,"cache_write":0.0,"output":0.28755},"cost_usd_as_run":0.3787,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1391,"code":1236,"comment":62,"blank":93,"role":"impl+test"}],"impl_lines":564,"impl_code":457,"impl_comment":59,"test_lines":827,"comment_ratio":0.114,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.6,"reference":10.62,"speedup":1.002,"baselines":{"oracle":10.62}},{"id":"dec","candidate":11.96,"reference":12.13,"speedup":1.014,"baselines":{"oracle":12.13}}]},"scores":{"muse-wonder-full-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":8.5},"council_summaries":{"muse-wonder-full-xhigh":"concise, well documented invariant, strong reader-overlap tests","deepseek-v4.1-flash-max":"Correct, clean and well tested, with libopus names kept; nothing to fix.","gpt-6.1-terra-max":"Strong coverage, with a costly carry path and a misleading guard comment","opus-5.5-xhigh":"Bit-exact, every public method documents its precondition, strong tests; one false comment","gpt-6.1-sol-max":"Strong compatibility and misuse coverage; discarded carry writes remain unnecessarily expensive."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"finish merges leftover bits outside the no-error block and skips the masked partial write on the busted path","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Failed carry writes are replayed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Malformed-table comment describes an unreachable zero range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False zero-width claim that contradicts C's own comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Carry flush cost grows after capacity is exhausted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0465,"lines":16,"significant":344,"window":6},"complexity":{"functions":39,"mean":2.69,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.6,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":1.2,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.25,"tokens":{"uncached_input":13058,"cache_read":248448,"cache_write":0,"output":6505,"reasoning":239},"cost_usd":0.058,"cost_breakdown":{"input":0.013058,"cache_read":0.012422,"cache_write":0.0,"output":0.032525},"cost_usd_as_run":0.058,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":618,"code":582,"comment":31,"blank":5,"role":"impl+test"}],"impl_lines":489,"impl_code":453,"impl_comment":31,"test_lines":129,"comment_ratio":0.064,"tests_declared":4},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.43,"reference":10.21,"speedup":0.979,"baselines":{"oracle":10.21}},{"id":"dec","candidate":12.69,"reference":12.29,"speedup":0.968,"baselines":{"oracle":12.29}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"A very long valid sequence diverges from the oracle, and the tests and public contracts need expansion.","deepseek-v4.1-flash-max":"functionally correct and compact, but the docs and tests under-deliver the contract.","opus-5.5-xhigh":"Behaviour is correct and fast, but the tests and docs are too thin to merge","gpt-6.1-sol-max":"Bit-exact in probing; tests and API documentation need strengthening.","muse-wonder-full-xhigh":"The core passes differential probes, but tests and API documentation are thin"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Carry counter does not wrap like the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Encoder rejection and overflow outcomes are not asserted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method preconditions are largely omitted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"No public item documents its precondition or the error contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Size sweep is masked","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Overflow/`InvalidArgument` reporting is essentially untested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The tests don't verify most of the contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Public methods don't state their contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Several guards can never fire","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Hard to read","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Size sweep permits universal failure","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public preconditions are largely undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unreachable range fallbacks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Inline test suite is too thin for the contract","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Public docs omit the preconditions and protocol","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Hot public methods lack inlining and `decode` takes a defensive division path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Compressed layout and magic constants impede maintenance","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0465,"lines":16,"significant":344,"window":6},"complexity":{"functions":35,"mean":3.06,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.0,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5}},"defect_points":4.2,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-gpt-6.1-sol-max-r1-b2","task":"range-coder","lead":"gpt-6.1-sol-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":8.94,"tokens":{"cache_read":2890240,"cache_write":0,"output":26157,"reasoning":14140,"uncached_input":105079},"cost_usd":0.3804,"cost_breakdown":{"cache_read":0.144512,"cache_write":0.0,"input":0.105079,"output":0.130785},"cost_usd_as_run":0.3804,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":9.82,"tokens":{"cache_read":9314304,"cache_write":0,"output":95361,"reasoning":67578,"uncached_input":116843},"cost_usd":0.1027,"cost_breakdown":{"cache_read":0.027943,"cache_write":0.0,"input":0.017526,"output":0.057217},"cost_usd_as_run":0.1027,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":10.34,"tokens":{"cache_read":3924964,"cache_write":213633,"output":65231,"reasoning":39893,"uncached_input":50},"cost_usd":3.7989,"cost_breakdown":{"cache_read":0.784993,"cache_write":1.709064,"input":0.0002,"output":1.30462},"cost_usd_as_run":3.7989,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":10.95,"tokens":{"cache_read":4458172,"cache_write":0,"output":44213,"reasoning":22819,"uncached_input":235894},"cost_usd":2.7195,"cost_breakdown":{"cache_read":0.891634,"cache_write":0.0,"input":0.943576,"output":0.88426},"cost_usd_as_run":2.7195,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":14.22,"tokens":{"cache_read":2648320,"cache_write":0,"output":24717,"reasoning":11456,"uncached_input":133942},"cost_usd":0.7799,"cost_breakdown":{"cache_read":0.264832,"cache_write":0.0,"input":0.267884,"output":0.24717},"cost_usd_as_run":0.7799,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"L","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"T","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"M","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"L","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"T","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"M","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"L","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"T","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"M","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"L","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"T","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"M","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"L","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"T","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"M","score":10.0}],"panel":["gpt-6.1-terra-max","deepseek-v4.1-flash-max","opus-5.5-xhigh","muse-wonder-full-xhigh","gpt-6.1-sol-max"]},"entries":[{"letter":"X","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":6.02,"tokens":{"uncached_input":76388,"cache_read":465509,"cache_write":0,"output":35768,"reasoning":0},"cost_usd":0.557,"cost_breakdown":{"input":0.152776,"cache_read":0.046551,"cache_write":0.0,"output":0.35768},"cost_usd_as_run":0.557,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1355,"code":1172,"comment":71,"blank":112,"role":"impl+test"}],"impl_lines":659,"impl_code":568,"impl_comment":42,"test_lines":696,"comment_ratio":0.069,"tests_declared":9},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.18,"reference":9.93,"speedup":0.975,"baselines":{"oracle":9.93}},{"id":"dec","candidate":11.53,"reference":11.31,"speedup":0.981,"baselines":{"oracle":11.31}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Sampled behavior matches, but carry handling, API documentation and tests need work","deepseek-v4.1-flash-max":"correct and robust, but no public method documents its precondition or error behaviour, and several unreachable branches add noise.","opus-5.5-xhigh":"exact and robust; public methods don't document their contracts, and the dead guards silently reset the range","muse-wonder-full-xhigh":"bit-exact core, but contract docs are missing and tests lean on proxies","gpt-6.1-sol-max":"Correct in probes; minor performance and API documentation gaps."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred-byte flush continues after overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Deferred counter differs from the oracle on very long streams","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Pending cancellation is untested for two mutators","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method docs omit argument and error rules","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Small-integer corruption branch cannot execute","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"src/range_coder.rs:204","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"src/range_coder.rs:33","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"src/range_coder.rs:472","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"src/range_coder.rs:1011","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public methods don't document preconditions or error behaviour","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Zero-range guards that silently set `rng = TOP` without an error, and other unreachable branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicated logic","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin coverage of bad tables and garbage input","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Public docs omit preconditions and the pending lifecycle","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Pending/misuse tests are thin and proxy-based","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Dead zero-after-shift guard in `normalize`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Dead `CorruptStream` check in `decode_uint` small path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"`decode` duplicates the head of `decode_and_update_unit`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Redundant second break condition in `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Sixteen `#[inline]` on private functions and unused `const fn` helpers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Processes carry bytes that cannot fit","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public argument domains are missing","status":"valid","defect":"RC-016","detail":"","location":""}],"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":1.6,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":4.0,"gpt-6.1-sol-max":0.0}},"defect_points":4.5,"score_cross_vendor":7.6,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","task":"range-coder","lead":"gpt-6.1-sol-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":27.75,"tokens":{"cache_read":15652096,"cache_write":0,"output":89291,"reasoning":46469,"uncached_input":521539},"cost_usd":1.7505,"cost_breakdown":{"cache_read":0.782605,"cache_write":0.0,"input":0.521539,"output":0.446455},"cost_usd_as_run":1.7505,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":40.63,"tokens":{"cache_read":55106304,"cache_write":0,"output":412366,"reasoning":296229,"uncached_input":702999},"cost_usd":0.5182,"cost_breakdown":{"cache_read":0.165319,"cache_write":0.0,"input":0.10545,"output":0.24742},"cost_usd_as_run":0.5182,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":51.78,"tokens":{"cache_read":24503129,"cache_write":1137976,"output":250753,"reasoning":146380,"uncached_input":226},"cost_usd":19.0204,"cost_breakdown":{"cache_read":4.900626,"cache_write":9.103808,"input":0.000904,"output":5.01506},"cost_usd_as_run":19.0204,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":75.29,"tokens":{"cache_read":21687040,"cache_write":0,"output":117093,"reasoning":69948,"uncached_input":954871},"cost_usd":5.2494,"cost_breakdown":{"cache_read":2.168705,"cache_write":0.0,"input":1.909742,"output":1.17093},"cost_usd_as_run":5.2494,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":50.78,"tokens":{"cache_read":20101588,"cache_write":0,"output":179171,"reasoning":102258,"uncached_input":841089},"cost_usd":10.9681,"cost_breakdown":{"cache_read":4.020318,"cache_write":0.0,"input":3.364356,"output":3.58342},"cost_usd_as_run":10.9681,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":10.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"B","score":9.0}],"panel":["gpt-6.1-terra-max","deepseek-v4.1-flash-max","opus-5.5-xhigh","gpt-6.1-sol-max","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":4.93,"tokens":{"uncached_input":16,"cache_read":445451,"cache_write":67198,"output":35280,"reasoning":12908},"cost_usd":1.3323,"cost_breakdown":{"input":6.4e-05,"cache_read":0.08909,"cache_write":0.537584,"output":0.7056},"cost_usd_as_run":1.3323,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1297,"code":1076,"comment":141,"blank":80,"role":"impl+test"}],"impl_lines":702,"impl_code":524,"impl_comment":128,"test_lines":595,"comment_ratio":0.196,"tests_declared":15},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.53,"reference":10.48,"speedup":0.995,"baselines":{"oracle":10.48}},{"id":"dec","candidate":12.23,"reference":12.29,"speedup":1.005,"baselines":{"oracle":12.29}}]},"scores":{"gpt-6.1-terra-max":9.5,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and well tested, with a small fractional-count documentation issue.","deepseek-v4.1-flash-max":"Correct with good docs; a duplicated validation and a silent zero-range repair.","opus-5.5-xhigh":"exact and well tested; one false comment on a dead branch","gpt-6.1-sol-max":"Exact behavior, strong tests and bounded overflow handling.","muse-wonder-full-xhigh":"Oracle-exact, defensive, well documented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Fractional cost is an approximation, not always the exact ceiling","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"src/range_coder.rs:286 and :296-298","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"src/range_coder.rs:495-499","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`Cargo.lock` extra file.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False reachability claim","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Precondition missing from `encode_icdf` doc","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Unreachable clamps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unreachable range repair","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"`encode_icdf` re-reads table entries","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0399,"lines":16,"significant":401,"window":6},"complexity":{"functions":42,"mean":2.81,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.0,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":1.3,"score_cross_vendor":9.125,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":337.23,"tokens":{"uncached_input":257623,"cache_read":1877768,"cache_write":0,"output":40732,"reasoning":18781},"cost_usd":2.2207,"cost_breakdown":{"input":1.030492,"cache_read":0.375554,"cache_write":0.0,"output":0.81464},"cost_usd_as_run":2.2207,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1447,"code":1221,"comment":133,"blank":93,"role":"impl+test"}],"impl_lines":675,"impl_code":529,"impl_comment":97,"test_lines":772,"comment_ratio":0.155,"tests_declared":15},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.11,"reference":9.85,"speedup":0.974,"baselines":{"oracle":9.85}},{"id":"dec","candidate":11.36,"reference":11.65,"speedup":1.026,"baselines":{"oracle":11.65}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with an avoidable long carry flush.","deepseek-v4.1-flash-max":"Correct, well documented and heavily tested; one dead defensive clamp.","opus-5.5-xhigh":"exact and well tested; noisy wrapping arithmetic and unreachable arms","gpt-6.1-sol-max":"Correct and well covered, with minor avoidable work and unreachable logic.","muse-wonder-full-xhigh":"Correct with detailed preconditions, but wrapping shifts obscure bounds"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"A full buffer does not stop the carry loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"src/range_coder.rs:71","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Blanket `wrapping_*` and dead else-arms","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False reachability comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Discarded carry runs take linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Full-width raw-mask branch is unreachable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Pervasive `wrapping_shl`/`wrapping_shr` for bounded shifts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"`bits >= 32` raw-mask branch unreachable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Counters described as “approximate”","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0404,"lines":16,"significant":396,"window":6},"complexity":{"functions":40,"mean":2.95,"max":15,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.5,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":1.4,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":10.04,"tokens":{"uncached_input":22743,"cache_read":323328,"cache_write":0,"output":20330,"reasoning":3897},"cost_usd":1.5673,"cost_breakdown":{"input":0.22743,"cache_read":0.323328,"cache_write":0.0,"output":1.0165},"cost_usd_as_run":1.5673,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1300,"code":1157,"comment":66,"blank":77,"role":"impl+test"}],"impl_lines":583,"impl_code":473,"impl_comment":64,"test_lines":717,"comment_ratio":0.119,"tests_declared":15},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.67,"reference":11.24,"speedup":0.963,"baselines":{"oracle":11.24}},{"id":"dec","candidate":12.2,"reference":12.17,"speedup":0.998,"baselines":{"oracle":12.17}}]},"scores":{"gpt-6.1-terra-max":10.0,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, efficient and well covered in the reviewed cases.","deepseek-v4.1-flash-max":"Correct, well tested (including white-box zero-range tests); docs terse, one fragile invariant unstated.","opus-5.5-xhigh":"exact with strong tests; dead CorruptStream \"repair\" that is documented and tested as if real","gpt-6.1-sol-max":"Clear implementation with efficient carry flushing and strong state tests.","muse-wonder-full-xhigh":"Correct with a neat bulk carry flush, plus dead exceptional paths"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"src/range_coder.rs:61","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"src/range_coder.rs:501-503","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`Cargo.lock` extra file.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable zero-range repair latches CorruptStream","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Docs and tests describe an impossible case","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Recovery test manufactures an unreachable state","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Encoder zero-range repair latches `CorruptStream`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"`decode_icdf` fallthrough cannot happen","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Zero-range test fabricates private state","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0778,"lines":28,"significant":360,"window":6},"complexity":{"functions":38,"mean":2.79,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.6,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":2.0}},"defect_points":1.4,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":56.18,"tokens":{"uncached_input":318884,"cache_read":2860288,"cache_write":0,"output":214995,"reasoning":95814},"cost_usd":3.3579,"cost_breakdown":{"input":0.637768,"cache_read":1.430144,"cache_write":0.0,"output":1.28997},"cost_usd_as_run":3.3579,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1605,"code":1403,"comment":100,"blank":102,"role":"impl+test"}],"impl_lines":678,"impl_code":527,"impl_comment":100,"test_lines":927,"comment_ratio":0.159,"tests_declared":16},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.97,"reference":9.88,"speedup":0.991,"baselines":{"oracle":9.88}},{"id":"dec","candidate":11.49,"reference":11.62,"speedup":1.011,"baselines":{"oracle":11.62}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but carry flushing and two public descriptions need attention.","deepseek-v4.1-flash-max":"Strong, carefully documented, well tested; only nits.","opus-5.5-xhigh":"exact, strong snapshot-based tests; mostly defensive noise","gpt-6.1-sol-max":"Correct and well tested, with inaccurate ICDF documentation and redundant guards.","muse-wonder-full-xhigh":"Correct and well tested, with some obscuring dead guards"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"A full buffer does not stop the carry loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The ICDF description omits a required boundary check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Fractional tell is not whole-bit tell rescaled","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"src/range_coder.rs:197","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"src/range_coder.rs:602","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`Cargo.lock` extra file.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Unreachable defensive code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"`encode_icdf` doc imprecise","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Discarded carry runs take linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"ICDF width condition understates the restriction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Private ICDF core repeats public checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"`encode_icdf` condition stated as bin width","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"`decode_icdf_core` rechecks impossible states","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Negative-shift helper semantics are surprising","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0558,"lines":24,"significant":430,"window":6},"complexity":{"functions":47,"mean":2.66,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":43,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.3,"slop":{"score":1.5,"per_seat":{"gpt-6.1-terra-max":2.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.5,"muse-wonder-full-xhigh":2.0}},"defect_points":2.4,"score_cross_vendor":8.3,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":11.64,"tokens":{"uncached_input":38266,"cache_read":6812032,"cache_write":0,"output":129351,"reasoning":96142},"cost_usd":0.1038,"cost_breakdown":{"input":0.00574,"cache_read":0.020436,"cache_write":0.0,"output":0.077611},"cost_usd_as_run":0.1038,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1038,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1664,"code":1309,"comment":242,"blank":113,"role":"impl+test"}],"impl_lines":732,"impl_code":485,"impl_comment":197,"test_lines":932,"comment_ratio":0.289,"tests_declared":24},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.93,"reference":10.83,"speedup":0.991,"baselines":{"oracle":10.83}},{"id":"dec","candidate":13.04,"reference":13.0,"speedup":0.997,"baselines":{"oracle":13.0}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but pending-state tests miss regressions and carry flushing is costly.","deepseek-v4.1-flash-max":"Very good: precise docs with a doctest, strong tests; two nits and one extra file.","opus-5.5-xhigh":"exact and well documented overall, but false comments and vacuous pending tests","gpt-6.1-sol-max":"Correct implementation, but pending-state regressions escape its tests.","muse-wonder-full-xhigh":"Oracle-exact on specified behaviour with strong tests and docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"A full buffer does not stop the carry loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Pending-cancellation checks use the wrong interval","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The fractional-cost formula is false","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"src/range_coder.rs:567","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"src/range_coder.rs:344-348","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`Cargo.lock` extra file.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False reachability comments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Pending tests vacuous","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Inaccurate docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Unreachable `.max(1)` on `ext`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Pending-state assertions miss broken behavior","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Discarded carry runs take linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Fractional-accounting formula is wrong","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Tautological assertion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"`tell_frac_of` formula called exact","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Zero-range comment misattributes cause","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Double `ext.max(1)`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0434,"lines":16,"significant":369,"window":6},"complexity":{"functions":38,"mean":2.87,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":54,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":1.6,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":1.5,"muse-wonder-full-xhigh":2.5}},"defect_points":3.2,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.75,"tokens":{"uncached_input":28759,"cache_read":155904,"cache_write":0,"output":9488,"reasoning":370},"cost_usd":0.9179,"cost_breakdown":{"input":0.28759,"cache_read":0.155904,"cache_write":0.0,"output":0.4744},"cost_usd_as_run":0.9179,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":927,"code":892,"comment":30,"blank":5,"role":"impl+test"}],"impl_lines":498,"impl_code":463,"impl_comment":30,"test_lines":429,"comment_ratio":0.061,"tests_declared":7},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.25,"reference":11.25,"speedup":1.0,"baselines":{"oracle":11.25}},{"id":"dec","candidate":12.96,"reference":12.91,"speedup":0.996,"baselines":{"oracle":12.91}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":7.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with a costly carry flush and an incomplete API description.","deepseek-v4.1-flash-max":"Correct and tidy, but docs omit the contract and tests are thinner than the others.","opus-5.5-xhigh":"exact and compact with good state-snapshot tests; dead CorruptStream code and terse docs","gpt-6.1-sol-max":"Correct and compact, with efficient overflow handling and incomplete API docs.","muse-wonder-full-xhigh":"Compact and correct, with strong state tests but terse docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Carry runs are written byte by byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF encoder preconditions are incomplete","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"src/range_coder.rs:141 (and `decode` :371, `update` :381, `encode_bits` :218 …)","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"src/range_coder.rs:118-121","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"src/range_coder.rs:430","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"src/range_coder.rs:695","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"src/range_coder.rs:45","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`Cargo.lock` extra file.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable zero-range \"repair\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Terse, incomplete docs","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Boolean-flag helpers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Essential preconditions are missing","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unreachable encoder recovery","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Several public preconditions omitted","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Encoder zero-range repair latches `CorruptStream`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0456,"lines":16,"significant":351,"window":6},"complexity":{"functions":39,"mean":2.77,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":2.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":3.1,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":3.84,"tokens":{"uncached_input":18,"cache_read":489020,"cache_write":60038,"output":27688,"reasoning":7395},"cost_usd":1.1319,"cost_breakdown":{"input":7.2e-05,"cache_read":0.097804,"cache_write":0.480304,"output":0.55376},"cost_usd_as_run":1.1319,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1217,"code":1045,"comment":90,"blank":82,"role":"impl+test"}],"impl_lines":641,"impl_code":511,"impl_comment":79,"test_lines":576,"comment_ratio":0.134,"tests_declared":16},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.84,"reference":10.88,"speedup":1.004,"baselines":{"oracle":10.88}},{"id":"dec","candidate":12.76,"reference":12.11,"speedup":0.949,"baselines":{"oracle":12.11}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with an avoidable long carry flush.","deepseek-v4.1-flash-max":"Correct and readable; pending bookkeeping is a hidden side effect and tests are mid-weight.","opus-5.5-xhigh":"exact with solid sweeps; minor overclaims and test padding","gpt-6.1-sol-max":"Correct behavior and broad tests, with minor performance and cleanup issues.","muse-wonder-full-xhigh":"Oracle-exact including finish merging, clean and well tested"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"A full buffer does not stop the carry loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"src/range_coder.rs:466","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"src/range_coder.rs:568","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`Cargo.lock` extra file.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Module doc overclaims","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Unreachable defensive code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Test padding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Discarded carry runs take linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Random-symbol normalization is a no-op","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Exceptional `ext == 0` result differs from oracle","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Byte-by-byte zero fill in `finish`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0416,"lines":16,"significant":385,"window":6},"complexity":{"functions":43,"mean":2.74,"max":15,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.4,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":1.7,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":50.63,"tokens":{"uncached_input":167272,"cache_read":6073024,"cache_write":0,"output":105341,"reasoning":76813},"cost_usd":0.26,"cost_breakdown":{"input":0.025091,"cache_read":0.182191,"cache_write":0.0,"output":0.052671},"cost_usd_as_run":0.26,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1630,"code":1278,"comment":235,"blank":117,"role":"impl+test"}],"impl_lines":744,"impl_code":504,"impl_comment":189,"test_lines":886,"comment_ratio":0.273,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.99,"reference":11.46,"speedup":0.956,"baselines":{"oracle":11.46}},{"id":"dec","candidate":13.53,"reference":13.38,"speedup":0.989,"baselines":{"oracle":13.38}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but pending tests are masked and probability documentation is wrong.","deepseek-v4.1-flash-max":"Strong code and tests, but the decoder-bit doc is inverted and one \"determinism\" test compares nothing.","opus-5.5-xhigh":"exact, but two false public docs and vacuous pending-decode tests","gpt-6.1-sol-max":"Correct outputs, but several tests and public probability descriptions are misleading.","muse-wonder-full-xhigh":"Behaviourally correct, but with two inverted documentation claims"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"A full buffer does not stop the carry loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Pending-state assertions are masked by an earlier error","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"`BITRES` does not set `logp` units","status":"valid","defect":"RC-018","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The rare decoded bit is true","status":"valid","defect":"RC-017","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"src/range_coder.rs:601","status":"valid","defect":"RC-017","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"src/range_coder.rs:1582-1592","status":"valid","defect":"RC-021","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"src/range_coder.rs:54-57","status":"valid","defect":"RC-018","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`Cargo.lock` is present in the entry","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"BITRES doc wrong","status":"valid","defect":"RC-018","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Decoder bit doc inverted","status":"valid","defect":"RC-017","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Pending-decode tests are vacuous","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Determinism check asserts nothing","status":"valid","defect":"RC-021","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Extra division in `decode_uint`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`decode` duplicates `decode_core`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Pending-rule violations escape all tests","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Determinism replay never compares its observations","status":"valid","defect":"RC-021","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Discarded carry runs take linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"BITRES is incorrectly tied to logp units","status":"valid","defect":"RC-018","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Decoder rare-bit probability is reversed","status":"valid","defect":"RC-017","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"`BITRES` documentation is false","status":"valid","defect":"RC-018","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"`decode_bit_logp` probability inverted","status":"valid","defect":"RC-017","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Decode quotient duplicated and recomputed","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.042,"lines":16,"significant":381,"window":6},"complexity":{"functions":42,"mean":2.79,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":54,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":2.9,"per_seat":{"gpt-6.1-terra-max":2.0,"deepseek-v4.1-flash-max":3.0,"opus-5.5-xhigh":2.5,"gpt-6.1-sol-max":2.0,"muse-wonder-full-xhigh":5.0}},"defect_points":5.3,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.56,"tokens":{"uncached_input":7804,"cache_read":109952,"cache_write":0,"output":6871,"reasoning":9},"cost_usd":0.0953,"cost_breakdown":{"input":0.015608,"cache_read":0.010995,"cache_write":0.0,"output":0.06871},"cost_usd_as_run":0.0953,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":718,"code":685,"comment":30,"blank":3,"role":"impl+test"}],"impl_lines":489,"impl_code":456,"impl_comment":30,"test_lines":229,"comment_ratio":0.062,"tests_declared":6},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.67,"reference":11.02,"speedup":0.944,"baselines":{"oracle":11.02}},{"id":"dec","candidate":12.44,"reference":12.28,"speedup":0.987,"baselines":{"oracle":12.28}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but public preconditions and pending-state tests are thin.","deepseek-v4.1-flash-max":"Behaviourally correct but the thinnest docs and tests; no method documents its domain.","opus-5.5-xhigh":"exact but under-documented, with dead CorruptStream \"repairs\" and thin, partly masked tests","gpt-6.1-sol-max":"Compact and correct, with weak pending-state tests and sparse API documentation.","muse-wonder-full-xhigh":"Correct but under-documented with the thinnest tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"A full buffer does not stop the carry loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Several pending-state regressions pass the suite","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public coding methods omit their accepted domains","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"src/range_coder.rs:132 (and every other public method, e.g. :366 `decode`, :376 `update`)","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"src/range_coder.rs:110","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"src/range_coder.rs:421","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"src/range_coder.rs:534","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"`Cargo.lock` extra file (see the task contract §5).","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public API docs are bare labels","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable zero-range \"repair\" latches CorruptStream","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Pending test masked and suite thin","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Boolean-flag helpers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Pending-state regressions survive the suite","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Discarded carry runs take linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public domains are incompletely documented","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Encoder cannot reach its corruption repair","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Public preconditions largely absent","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Malformed/rejected coverage is shallow","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unreachable zero-range repairs use the wrong error","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0462,"lines":16,"significant":346,"window":6},"complexity":{"functions":37,"mean":2.84,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.2,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":4.4,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":11.96,"tokens":{"uncached_input":32,"cache_read":1562823,"cache_write":122929,"output":82403,"reasoning":47052},"cost_usd":2.9442,"cost_breakdown":{"input":0.000128,"cache_read":0.312565,"cache_write":0.983432,"output":1.64806},"cost_usd_as_run":2.9442,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1747,"code":1397,"comment":239,"blank":111,"role":"impl+test"}],"impl_lines":796,"impl_code":539,"impl_comment":203,"test_lines":951,"comment_ratio":0.274,"tests_declared":27},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.74,"reference":10.82,"speedup":1.007,"baselines":{"oracle":10.82}},{"id":"dec","candidate":13.14,"reference":12.17,"speedup":0.926,"baselines":{"oracle":12.17}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":9.5,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Broad tests and good API documentation, apart from carry cost and fractional-rounding wording.","deepseek-v4.1-flash-max":"The most complete doc/comment set and the broadest test suite; no defects found.","opus-5.5-xhigh":"exact, thorough tests and accurate docs; over-defensive types on hot paths","gpt-6.1-sol-max":"Thorough tests and documentation, with minor unnecessary work.","muse-wonder-full-xhigh":"Oracle-exact and exhaustively tested, but verbose"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"A full buffer does not stop the carry loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Fractional cost is an approximation, not always the exact ceiling","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Widened types and checks with no reachable effect","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Padding test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Discarded carry runs take linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Core arithmetic repeats established validation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"`INITIAL_NBITS` said to cover both coders","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Boilerplate constant/field commentary","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Zero-scale decode sentinel is arbitrary","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0387,"lines":16,"significant":413,"window":6},"complexity":{"functions":45,"mean":2.69,"max":10,"over_10":0,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.8,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":2.5}},"defect_points":1.5,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.82,"tokens":{"uncached_input":17796,"cache_read":121856,"cache_write":0,"output":4192,"reasoning":29},"cost_usd":0.0448,"cost_breakdown":{"input":0.017796,"cache_read":0.006093,"cache_write":0.0,"output":0.02096},"cost_usd_as_run":0.0448,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":500,"code":469,"comment":30,"blank":1,"role":"impl+test"}],"impl_lines":477,"impl_code":446,"impl_comment":30,"test_lines":23,"comment_ratio":0.063,"tests_declared":1},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.17,"reference":10.66,"speedup":0.954,"baselines":{"oracle":10.66}},{"id":"dec","candidate":12.73,"reference":12.57,"speedup":0.987,"baselines":{"oracle":12.57}}]},"scores":{"gpt-6.1-terra-max":6.0,"deepseek-v4.1-flash-max":6.0,"opus-5.5-xhigh":5.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":6.0},"council_summaries":{"gpt-6.1-terra-max":"Ordinary behavior matches, but counter wrapping, tests, and docs need work","deepseek-v4.1-flash-max":"Correct code, but one test and one-line docs: not mergeable on evidence.","opus-5.5-xhigh":"Exact code, but effectively untested and barely documented","gpt-6.1-sol-max":"Arithmetic matches, but most API behavior lacks regression coverage.","muse-wonder-full-xhigh":"Code proved correct under probing, but essentially untested and undocumented as delivered"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred carry count saturates","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Only one shared-byte golden case is tested","status":"valid","defect":"RC-020","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Flush continues after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method preconditions are largely absent","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"one test with 8 assertions is the entire suite","status":"valid","defect":"RC-020","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"public docs are one-liners with no contract text","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"trailing `0` after the `decode_icdf` loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`dec()` returns `(0, 0)` for an impossible zero scale","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"One test in total","status":"valid","defect":"RC-020","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"One-line public docs with no preconditions or error behaviour","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable guards","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Linear work for dropped carry bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"One fixture leaves major behavior untested","status":"valid","defect":"RC-020","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public contracts and invariants are largely absent","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Single test for the whole contract","status":"valid","defect":"RC-020","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No preconditions anywhere","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Golfed presentation hurts maintainability","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `decode_icdf` trailing return","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0476,"lines":16,"significant":336,"window":6},"complexity":{"functions":38,"mean":2.87,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.0,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":5.5,"score_cross_vendor":5.667,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":48.64,"tokens":{"uncached_input":177854,"cache_read":1164696,"cache_write":0,"output":29984,"reasoning":11837},"cost_usd":1.544,"cost_breakdown":{"input":0.711416,"cache_read":0.232939,"cache_write":0.0,"output":0.59968},"cost_usd_as_run":1.544,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1353,"code":1188,"comment":90,"blank":75,"role":"impl+test"}],"impl_lines":680,"impl_code":561,"impl_comment":66,"test_lines":673,"comment_ratio":0.105,"tests_declared":14},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.38,"reference":11.19,"speedup":0.983,"baselines":{"oracle":11.19}},{"id":"dec","candidate":12.37,"reference":12.41,"speedup":1.003,"baselines":{"oracle":12.41}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Broad coverage, with incomplete ICDF docs and a costly flush","deepseek-v4.1-flash-max":"Strong tests; redundant table guard, dead branch and a muddled misuse test leave polish on the table.","opus-5.5-xhigh":"Exact, but verbose defensive code throughout and sloppy, self-confirming tests","gpt-6.1-sol-max":"Correct implementation; carry handling and finish-budget coverage need improvement.","muse-wonder-full-xhigh":"Solid and thorough; small redundancies and a byte-at-a-time finish"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Flush continues after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF preconditions omit the selected upper bound","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"`tell_frac_of` computes the same expression twice and guards a lookup that cannot fail","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable `else { MASK }` arm in `finish`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"a *valid* call is mixed into the rejected-call block with a self-questioning comment","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"`every_buffer_size_around_finish` asserts nothing for sizes 9..=23","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Needless wrapping arithmetic and manual loops","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable fallbacks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Self-confirming and indeterminate tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Private constants, fields and helpers are undocumented.","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Linear work for dropped carry bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Finish sweep leaves the useful boundary unchecked","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"`finish` zero-fills byte-by-byte","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"`encode_icdf_core` re-reads validated entries","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"`ft==0` guard in `decode_core`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0376,"lines":16,"significant":425,"window":6},"complexity":{"functions":44,"mean":2.82,"max":19,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":2.7,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":14.01,"tokens":{"uncached_input":71474,"cache_read":533504,"cache_write":0,"output":34410,"reasoning":15522},"cost_usd":0.5404,"cost_breakdown":{"input":0.142948,"cache_read":0.05335,"cache_write":0.0,"output":0.3441},"cost_usd_as_run":0.5404,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1484,"code":1328,"comment":64,"blank":92,"role":"impl+test"}],"impl_lines":592,"impl_code":484,"impl_comment":60,"test_lines":892,"comment_ratio":0.11,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":12.18,"reference":11.17,"speedup":0.917,"baselines":{"oracle":11.17}},{"id":"dec","candidate":12.39,"reference":12.09,"speedup":0.976,"baselines":{"oracle":12.09}}]},"scores":{"gpt-6.1-terra-max":10.0,"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Clear contract documentation, bounded carry handling, and strong tests","deepseek-v4.1-flash-max":"Excellent oracle-exact coder with twin/state tests; one dead rescue path.","opus-5.5-xhigh":"Exact, bulk carry-run flush, strongest test suite; some dead CorruptStream paths","gpt-6.1-sol-max":"Bit-exact with efficient carry flushing and thorough state-preservation tests.","muse-wonder-full-xhigh":"Excellent test depth incl. exceptional states; slowest hot path of the set"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`decode_core` substitutes `unit = 1` on an impossible zero scale","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable CorruptStream paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Slowest encode/decode measured","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `CorruptStream` tail in `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"Zero-range `CorruptStream`+reset on both sides","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0435,"lines":16,"significant":368,"window":6},"complexity":{"functions":39,"mean":2.77,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.1,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":0.7,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":18.86,"tokens":{"uncached_input":38,"cache_read":2011576,"cache_write":186358,"output":280991,"reasoning":256712},"cost_usd":0.5527,"cost_breakdown":{"input":1.2e-05,"cache_read":0.091139,"cache_write":0.158755,"output":0.302777},"cost_usd_as_run":0.5527,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1267,"code":1067,"comment":112,"blank":88,"role":"impl+test"}],"impl_lines":646,"impl_code":496,"impl_comment":101,"test_lines":621,"comment_ratio":0.169,"tests_declared":20},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.02,"reference":10.88,"speedup":0.987,"baselines":{"oracle":10.88}},{"id":"dec","candidate":12.39,"reference":12.55,"speedup":1.013,"baselines":{"oracle":12.55}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Broad tests and correct observed behavior, with a costly overflow path","deepseek-v4.1-flash-max":"Correct and heavily tested; two dead defensive branches and an awkward cast.","opus-5.5-xhigh":"Exact, good docs and tests; unreachable branches that would mislabel the error","gpt-6.1-sol-max":"Correct implementation with weak regression assertions and avoidable carry work.","muse-wonder-full-xhigh":"Clean helpers, strongest test count among top entries; unreachable ext==0 path breaks its own pending contract"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Flush continues after the buffer is full","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`decode`'s zero-scale rescue silently drops the decode","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`u32::try_from(-l).unwrap_or(0)` in `finish`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable `ext == 0 → CorruptStream`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"decoded_value doc","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Golden B is encoded but never decoded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Linear work for dropped carry bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Important regression cases remain unasserted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Cumulative value is incorrectly called one-based","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"correctness","title":"`ext==0` decode path skips pending bookkeeping","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Dead defensive guards","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.042,"lines":16,"significant":381,"window":6},"complexity":{"functions":40,"mean":2.85,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":19,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.5,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":1.8,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.61,"tokens":{"uncached_input":9655,"cache_read":282112,"cache_write":0,"output":19806,"reasoning":5243},"cost_usd":0.2456,"cost_breakdown":{"input":0.01931,"cache_read":0.028211,"cache_write":0.0,"output":0.19806},"cost_usd_as_run":0.2456,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1306,"code":1166,"comment":61,"blank":79,"role":"impl+test"}],"impl_lines":581,"impl_code":473,"impl_comment":59,"test_lines":725,"comment_ratio":0.111,"tests_declared":15},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":12.07,"reference":11.2,"speedup":0.928,"baselines":{"oracle":11.2}},{"id":"dec","candidate":12.76,"reference":12.71,"speedup":0.996,"baselines":{"oracle":12.71}}]},"scores":{"gpt-6.1-terra-max":10.0,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Complete, well-documented implementation with strong tests","deepseek-v4.1-flash-max":"Very strong internal-state tests; one unreachable encoder error branch.","opus-5.5-xhigh":"Exact, bulk carry flush, very strong tests; dead code it labels as unreachable","gpt-6.1-sol-max":"Bit-exact with efficient carry handling and thorough boundary/misuse coverage.","muse-wonder-full-xhigh":"Well documented and correct; slowest-tier carry path and an asserted-not-proven division"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"encoder latches `CorruptStream` for an impossible zero range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Self-labelled unreachable code","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"`carry_out` does fill bookkeeping per emitted byte","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Plain division by `ext` rests on a comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `CorruptStream` tail in `decode_icdf`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0446,"lines":16,"significant":359,"window":6},"complexity":{"functions":41,"mean":2.66,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.0,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":0.8,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":28.03,"tokens":{"uncached_input":192,"cache_read":3755213,"cache_write":137480,"output":112756,"reasoning":0},"cost_usd":0.1406,"cost_breakdown":{"input":2.9e-05,"cache_read":0.060083,"cache_write":0.027496,"output":0.052995},"cost_usd_as_run":0.1406,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1051,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1657,"code":1309,"comment":261,"blank":87,"role":"impl+test"}],"impl_lines":765,"impl_code":493,"impl_comment":223,"test_lines":892,"comment_ratio":0.311,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.93,"reference":9.86,"speedup":0.993,"baselines":{"oracle":9.86}},{"id":"dec","candidate":11.39,"reference":11.49,"speedup":1.009,"baselines":{"oracle":11.49}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Comprehensive tests, with arithmetic documentation errors and a costly flush","deepseek-v4.1-flash-max":"Exhaustive tests and docs; two comments describe unreachable states imprecisely.","opus-5.5-xhigh":"Exact, extensive docs and tests, but several of those docs are false","gpt-6.1-sol-max":"Correct arithmetic; carry flushing, masked assertions, and inaccurate terminology need changes.","muse-wonder-full-xhigh":"Best explanatory docs and strong tests, marred by a false module descriptor"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Flush continues after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Module arithmetic description is inaccurate","status":"valid","defect":"RC-022","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"`tell_frac_core`'s comment says the degenerate fallback is for `rng == 0`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`finish` would shift by 32 if `rng` were ever 0","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Wrong headline docs","status":"valid","defect":"RC-022","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False reachability claims","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Hedged assertions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Linear work for dropped carry bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Pending recovery assertion is masked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Coder and precision are misidentified","status":"valid","defect":"RC-022","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"Module calls itself \"carry-less\"","status":"valid","defect":"RC-022","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Documentation and wrapper padding","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":43,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.2,"slop":{"score":1.5,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":3.0}},"defect_points":2.5,"score_cross_vendor":8.2,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.0,"tokens":{"uncached_input":31653,"cache_read":278016,"cache_write":0,"output":10596,"reasoning":702},"cost_usd":0.0985,"cost_breakdown":{"input":0.031653,"cache_read":0.013901,"cache_write":0.0,"output":0.05298},"cost_usd_as_run":0.0985,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":924,"code":884,"comment":31,"blank":9,"role":"impl+test"}],"impl_lines":507,"impl_code":468,"impl_comment":31,"test_lines":417,"comment_ratio":0.062,"tests_declared":8},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.91,"reference":10.81,"speedup":0.991,"baselines":{"oracle":10.81}},{"id":"dec","candidate":12.78,"reference":12.74,"speedup":0.997,"baselines":{"oracle":12.74}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct observed behavior, with limited API documentation and a costly flush","deepseek-v4.1-flash-max":"Good all-round entry; minor dead defensive code and thin pending-rule docs.","opus-5.5-xhigh":"Exact and compact, but public docs omit every precondition","gpt-6.1-sol-max":"Good functional coverage; carry flushing and public documentation need improvement.","muse-wonder-full-xhigh":"Correct and fast encoder, held back by missing precondition documentation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Flush continues after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Method descriptions omit essential preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`decode` records a pending decode with a zero scale on an impossible state","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"trailing `0` after `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No preconditions or error behaviour documented","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable CorruptStream branches in both normalize","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Redundant `|| index + 1 == icdf.len()` and an unreachable trailing `0`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Linear work for dropped carry bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public contracts are underspecified","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Most methods lack numeric preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `decode_icdf` trailing return","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"Unguarded finish shift relies on reset","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0793,"lines":28,"significant":353,"window":6},"complexity":{"functions":40,"mean":2.75,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":2.5,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.02,"tokens":{"uncached_input":37423,"cache_read":264832,"cache_write":0,"output":13270,"reasoning":1182},"cost_usd":0.234,"cost_breakdown":{"input":0.074846,"cache_read":0.026483,"cache_write":0.0,"output":0.1327},"cost_usd_as_run":0.234,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1067,"code":1017,"comment":42,"blank":8,"role":"impl+test"}],"impl_lines":511,"impl_code":464,"impl_comment":40,"test_lines":556,"comment_ratio":0.079,"tests_declared":11},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.76,"reference":10.93,"speedup":0.929,"baselines":{"oracle":10.93}},{"id":"dec","candidate":12.72,"reference":13.24,"speedup":1.041,"baselines":{"oracle":13.24}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Good tests and bounded flush, but deferred counter differs","deepseek-v4.1-flash-max":"Compact, oracle-exact coder with strong budget/state tests; only dead defensive code remains.","opus-5.5-xhigh":"Exact, bulk carry flush, good state-snapshot tests; recomputes a division","gpt-6.1-sol-max":"Strong coverage and carry handling; one extra division and incomplete ICDF documentation.","muse-wonder-full-xhigh":"Compact and correct; one misleading \"safe\" division, slower carry path"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred count does not wrap as `u32`","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF and raw-bit bounds are incomplete","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"encoder latches a decoder-only error on an impossible state","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable fallback in `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Second division per decoded symbol","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable `rng == 0 → CorruptStream; rng = TOP` in both normalize","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`self.val.wrapping_div(ext)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Update repeats the normalization division","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"ICDF scale and domains are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"`wrapping_div(ext)` still panics on zero","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Bulk-fill bookkeeping on every `carry_out`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `decode_icdf` fallthrough","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"`rng==0` reset latches `CorruptStream`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0457,"lines":16,"significant":350,"window":6},"complexity":{"functions":40,"mean":2.7,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.2,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":2.3,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.59,"tokens":{"uncached_input":8076,"cache_read":136320,"cache_write":0,"output":6147,"reasoning":8},"cost_usd":0.5244,"cost_breakdown":{"input":0.08076,"cache_read":0.13632,"cache_write":0.0,"output":0.30735},"cost_usd_as_run":0.5244,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":676,"code":642,"comment":31,"blank":3,"role":"impl+test"}],"impl_lines":472,"impl_code":439,"impl_comment":31,"test_lines":204,"comment_ratio":0.066,"tests_declared":5},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.34,"reference":11.42,"speedup":1.007,"baselines":{"oracle":11.42}},{"id":"dec","candidate":12.76,"reference":12.32,"speedup":0.966,"baselines":{"oracle":12.32}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Solid ordinary behavior, but deferred counter differs from the oracle","deepseek-v4.1-flash-max":"Correct, but terse docs, re-validating hot paths and dead branches keep it out of the top group.","opus-5.5-xhigh":"Exact, terse; docs lack preconditions and tests are thin","gpt-6.1-sol-max":"Correct and compact; finish assertions and API documentation need tightening.","muse-wonder-full-xhigh":"Dense private-state tests, but vague docs and fragile public-method layering"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred count does not wrap as `u32`","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF and update requirements are underspecified","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"composite coders call the public validating wrappers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"public docs are one-liners with no invariants","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"encoder latches `CorruptStream` on an impossible zero range","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"trailing `0` after `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No preconditions or error behaviour documented","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin assertions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable `rng == 0 → CorruptStream; rng = TOP` in both normalize","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"encode_uint calls the public, validating `encode` and `encode_bits`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Budget sweep leaves outcomes unpinned","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Important argument domains omitted","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Preconditions missing or vague","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Plain `val / ext` with no stated invariant","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Composite ops layered through public methods","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `decode_icdf` trailing return","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.048,"lines":16,"significant":333,"window":6},"complexity":{"functions":34,"mean":3.12,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":2.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":3.4,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.19,"tokens":{"uncached_input":22148,"cache_read":175872,"cache_write":0,"output":6162,"reasoning":236},"cost_usd":0.0618,"cost_breakdown":{"input":0.022148,"cache_read":0.008794,"cache_write":0.0,"output":0.03081},"cost_usd_as_run":0.0618,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":632,"code":596,"comment":31,"blank":5,"role":"impl+test"}],"impl_lines":509,"impl_code":473,"impl_comment":31,"test_lines":123,"comment_ratio":0.062,"tests_declared":5},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.12,"reference":10.71,"speedup":0.963,"baselines":{"oracle":10.71}},{"id":"dec","candidate":13.01,"reference":12.58,"speedup":0.967,"baselines":{"oracle":12.58}}]},"scores":{"gpt-6.1-terra-max":6.5,"deepseek-v4.1-flash-max":7.0,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Counter-wrap mismatch and masked pending-decode tests","deepseek-v4.1-flash-max":"Correct and readable, but its tests cover only part of the contract.","opus-5.5-xhigh":"Exact, but docs and tests are thin","gpt-6.1-sol-max":"Correct outputs, with weak regression assertions and unused decoder state.","muse-wonder-full-xhigh":"Correct but terse: missing precondition docs, dead state, thin misuse tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred carry count saturates","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Pending-decode assertions are masked","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method preconditions are incomplete","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"the shipped suite omits whole hidden-suite categories","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"module doc says the raw bits form \"a separate ... stream\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`decode`/`update` docs never state the pending-decode rules","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No preconditions or error behaviour documented","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin tests","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Dead decoder field and branches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"State and capacity regressions escape assertions","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Decoder quotient field is write-only","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public argument constraints are omitted","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Public methods omit preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Decoder `ext` field written, never read","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"`Display` delegated to `Debug`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Misuse coverage anecdotal","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"Encoder latches `CorruptStream` for unreachable `rng==0`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1114,"lines":40,"significant":359,"window":6},"complexity":{"functions":39,"mean":2.85,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.9,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":4.4,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.88,"tokens":{"uncached_input":19659,"cache_read":547968,"cache_write":0,"output":25021,"reasoning":8643},"cost_usd":0.1722,"cost_breakdown":{"input":0.019659,"cache_read":0.027398,"cache_write":0.0,"output":0.125105},"cost_usd_as_run":0.1722,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1355,"code":1197,"comment":73,"blank":85,"role":"impl+test"}],"impl_lines":607,"impl_code":487,"impl_comment":71,"test_lines":748,"comment_ratio":0.127,"tests_declared":15},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":12.16,"reference":11.25,"speedup":0.925,"baselines":{"oracle":11.25}},{"id":"dec","candidate":12.37,"reference":12.48,"speedup":1.009,"baselines":{"oracle":12.48}}]},"scores":{"gpt-6.1-terra-max":9.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Bit-exact in the sampled checks and handles long carry runs efficiently","deepseek-v4.1-flash-max":"Compact, correct and very well tested; one dead `rng == 0` branch that would report a decoder-only error from the encoder.","opus-5.5-xhigh":"Exact, with a neat O(1) bulk flush for long carry runs and full-state tests, but keeps an unreachable \"repair\" path","gpt-6.1-sol-max":"Efficient carry handling and thorough state checks, with one masked rejection test and unnecessary recovery paths.","muse-wonder-full-xhigh":"bit-exact, compact bulk-carry design; dead zero-range recovery latches the wrong error"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"The malformed-table fallback is unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"`normalize`'s `rng == 0` recovery is unreachable and latches `CorruptStream` on the encoder","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Post-loop failure in `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`finish`'s `MASK >> l` has no bound check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable `rng == 0` repair, and an ICDF fall-through, that latch `CorruptStream`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The `l == 0` and `l < 16` arms in `tell_frac`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Earlier errors mask uint lower-bound rejection","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Zero-range recovery requires private-state injection","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Zero-range `normalize` reset latches `CorruptStream` on the encoder","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"No `#[inline]`, joint-slowest encode despite bulk carry fill","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"robustness","title":"Buffer writes use direct indexing on an undocumented invariant","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0813,"lines":30,"significant":369,"window":6},"complexity":{"functions":40,"mean":2.67,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.0,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.5,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":1.6,"score_cross_vendor":8.667,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":347.45,"tokens":{"uncached_input":279435,"cache_read":3988756,"cache_write":0,"output":56078,"reasoning":28532},"cost_usd":3.0371,"cost_breakdown":{"input":1.11774,"cache_read":0.797751,"cache_write":0.0,"output":1.12156},"cost_usd_as_run":3.0371,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1483,"code":1315,"comment":77,"blank":91,"role":"impl+test"}],"impl_lines":677,"impl_code":557,"impl_comment":70,"test_lines":806,"comment_ratio":0.112,"tests_declared":17},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.06,"reference":9.8,"speedup":0.886,"baselines":{"oracle":9.8}},{"id":"dec","candidate":11.35,"reference":11.52,"speedup":1.015,"baselines":{"oracle":11.52}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Bit-exact in the sampled checks, with avoidable carry-flush and finalization costs","deepseek-v4.1-flash-max":"Correct and well documented with a strong test suite; a few unreachable guards and one avoidable full-buffer pass in `finish`.","opus-5.5-xhigh":"Matches the reference exactly, with strong tests, but cluttered with unreachable guards and needless wrapping/saturating arithmetic","gpt-6.1-sol-max":"Exact results, but finalization performs substantial unnecessary work and pending-consumption tests are masked.","muse-wonder-full-xhigh":"bit-exact; noisiest defensive arithmetic and joint-slowest encode"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Full-buffer carry runs are drained byte by byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Finalization scans the entire buffer","status":"valid","defect":"RC-024","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Zero-total guards cannot be reached","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable defensive guards in the coding helpers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`finish` zero-fills by scanning the whole buffer","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`TELL_FRAC_TABLE.get(index)` is a no-op test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable defensive guards","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Checked arithmetic on values already known to be in range hides the invariants","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"`finish` scans the whole buffer to zero the gap","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Finalization scans the entire buffer","status":"valid","defect":"RC-024","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Walks carry bytes after capacity is exhausted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Duplicate-update checks miss pending consumption","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Private cores recheck impossible zero totals","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"No `#[inline]`, joint-slowest encode","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"`finish` scans the whole buffer to zero the gap","status":"valid","defect":"RC-024","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Pervasive wrapping/saturating where invariants forbid wrap","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Rejected-call tests cannot see hidden state","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0368,"lines":16,"significant":435,"window":6},"complexity":{"functions":41,"mean":3.1,"max":21,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":44,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":3.3,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-gpt-6.1-terra-max-r1-b2","task":"range-coder","lead":"gpt-6.1-terra-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":10.35,"tokens":{"cache_read":4430920,"cache_write":0,"output":47615,"reasoning":25638,"uncached_input":259683},"cost_usd":2.8772,"cost_breakdown":{"cache_read":0.886184,"cache_write":0.0,"input":1.038732,"output":0.9523},"cost_usd_as_run":2.8772,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":11.96,"tokens":{"cache_read":3364366,"cache_write":202003,"output":74629,"reasoning":46169,"uncached_input":48},"cost_usd":3.7817,"cost_breakdown":{"cache_read":0.672873,"cache_write":1.616024,"input":0.000192,"output":1.49258},"cost_usd_as_run":3.7817,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":10.44,"tokens":{"cache_read":2781056,"cache_write":0,"output":27473,"reasoning":16591,"uncached_input":92244},"cost_usd":0.3687,"cost_breakdown":{"cache_read":0.139053,"cache_write":0.0,"input":0.092244,"output":0.137365},"cost_usd_as_run":0.3687,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":11.43,"tokens":{"cache_read":1890304,"cache_write":0,"output":26078,"reasoning":13149,"uncached_input":95014},"cost_usd":0.6398,"cost_breakdown":{"cache_read":0.18903,"cache_write":0.0,"input":0.190028,"output":0.26078},"cost_usd_as_run":0.6398,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":10.48,"tokens":{"cache_read":8489088,"cache_write":0,"output":109266,"reasoning":70789,"uncached_input":132374},"cost_usd":0.1109,"cost_breakdown":{"cache_read":0.025467,"cache_write":0.0,"input":0.019856,"output":0.06556},"cost_usd_as_run":0.1109,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"M","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"H","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"X","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"M","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"H","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"X","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"M","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"H","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"X","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"M","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"H","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"X","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"M","score":6.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"H","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"X","score":8.5}],"panel":["muse-wonder-full-xhigh","opus-5.5-xhigh","gpt-6.1-terra-max","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"Z","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":3.5,"tokens":{"uncached_input":61296,"cache_read":368698,"cache_write":0,"output":17231,"reasoning":0},"cost_usd":0.3318,"cost_breakdown":{"input":0.122592,"cache_read":0.03687,"cache_write":0.0,"output":0.17231},"cost_usd_as_run":0.3318,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1173,"code":1004,"comment":51,"blank":118,"role":"impl+test"}],"impl_lines":566,"impl_code":493,"impl_comment":30,"test_lines":607,"comment_ratio":0.057,"tests_declared":11},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.79,"reference":9.89,"speedup":1.01,"baselines":{"oracle":9.89}},{"id":"dec","candidate":11.69,"reference":11.39,"speedup":0.974,"baselines":{"oracle":11.39}}]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":8.0,"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"muse-wonder-full-xhigh":"Bit-exact in all probes; thinner docs and randomized tests than the leaders","opus-5.5-xhigh":"Bit-exact and robust; docs are thin and the tests are padded and miss two bug classes","gpt-6.1-terra-max":"Oracle comparisons pass, but carry flushing and coverage need work","gpt-6.1-sol-max":"Bit-exact implementation with a decoder regression-test gap and duplicated fixtures.","deepseek-v4.1-flash-max":"Bit-exact, compact and well documented; only unreachable branches and one unbounded shift in `finish`."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions and rejected-call returns undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No randomized sequence or garbage fuzz tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Dead clamp in `compute_tell_frac`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Golden sequence duplicated verbatim in tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Carry propagation and accounting after an overflow are untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public docs leave out preconditions and the error model","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Golden tests are hand-unrolled","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"A test comment claims a carry that never happens","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"`*_uint` calls back into the validating public methods","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Retries every deferred byte after overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Valid ICDF coverage is concentrated on one table","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public methods omit their argument constraints","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Decoder first-error priority is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public argument contracts are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Golden encoding fixture is duplicated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"Dead branches in `tell_frac`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`finish` shifts `(TOP - 1) >> (l as u32)` with no bound","status":"valid","detail":"","location":""}],"score":7.9,"slop":{"score":0.9,"per_seat":{"muse-wonder-full-xhigh":1.0,"opus-5.5-xhigh":2.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":1.0}},"defect_points":3.1,"score_cross_vendor":7.9,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","task":"range-coder","lead":"gpt-6.1-terra-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":35.99,"tokens":{"cache_read":47529728,"cache_write":0,"output":378765,"reasoning":291700,"uncached_input":614163},"cost_usd":0.4619,"cost_breakdown":{"cache_read":0.142589,"cache_write":0.0,"input":0.092125,"output":0.227259},"cost_usd_as_run":0.4619,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":39.7,"tokens":{"cache_read":23056257,"cache_write":1022529,"output":238802,"reasoning":138049,"uncached_input":220},"cost_usd":17.5684,"cost_breakdown":{"cache_read":4.611251,"cache_write":8.180232,"input":0.00088,"output":4.77604},"cost_usd_as_run":17.5684,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":37.93,"tokens":{"cache_read":16339968,"cache_write":0,"output":96729,"reasoning":48648,"uncached_input":426981},"cost_usd":1.7276,"cost_breakdown":{"cache_read":0.816998,"cache_write":0.0,"input":0.426981,"output":0.483645},"cost_usd_as_run":1.7276,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":45.21,"tokens":{"cache_read":18316800,"cache_write":0,"output":98260,"reasoning":57290,"uncached_input":738188},"cost_usd":4.2907,"cost_breakdown":{"cache_read":1.831681,"cache_write":0.0,"input":1.476376,"output":0.9826},"cost_usd_as_run":4.2907,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":69.62,"tokens":{"cache_read":22328628,"cache_write":0,"output":186073,"reasoning":103560,"uncached_input":915209},"cost_usd":11.848,"cost_breakdown":{"cache_read":4.465726,"cache_write":0.0,"input":3.660836,"output":3.72146},"cost_usd_as_run":11.848,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-gpt-6.1-terra-max-r1","letter":"B","score":9.0}],"panel":["deepseek-v4.1-flash-max","opus-5.5-xhigh","gpt-6.1-terra-max","gpt-6.1-sol-max","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":38.74,"tokens":{"uncached_input":58,"cache_read":3641460,"cache_write":224488,"output":446255,"reasoning":417392},"cost_usd":0.8404,"cost_breakdown":{"input":2e-05,"cache_read":0.170644,"cache_write":0.198099,"output":0.471623},"cost_usd_as_run":0.8404,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1462,"code":1226,"comment":144,"blank":92,"role":"impl+test"}],"impl_lines":629,"impl_code":461,"impl_comment":125,"test_lines":833,"comment_ratio":0.213,"tests_declared":21},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.22,"reference":10.12,"speedup":0.99,"baselines":{"oracle":10.12}},{"id":"dec","candidate":12.09,"reference":11.86,"speedup":0.981,"baselines":{"oracle":11.86}}]},"quality":{"duplication":{"ratio":0.0453,"lines":16,"significant":353,"window":6},"complexity":{"functions":34,"mean":2.94,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":31,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"deepseek-v4.1-flash-max":"bit-exact with defensive bounded loops and strong misuse tests","opus-5.5-xhigh":"clean, correct and well-documented; the bounded normalize loop is sound","gpt-6.1-terra-max":"Strong coverage and matching sampled behavior, with an avoidable long carry flush.","gpt-6.1-sol-max":"Complete contracts, meaningful tests, and no verified defects.","muse-wonder-full-xhigh":"Correct with bounded-loop idiosyncrasies"},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The `bits < 32 &&` guard in `encode_bits` can never be false after the `1..=25` check; `(self.rng / ft).max(1)` guards an impossible zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries cost linear work even with no space","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Bounded normalize obscures invariant","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Value register comment overstates","status":"valid","detail":"","location":""}],"score":8.8,"slop":{"score":0.2,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":0.7,"score_cross_vendor":8.875,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":54.67,"tokens":{"uncached_input":1140192,"cache_read":442624,"cache_write":0,"output":192427,"reasoning":88099},"cost_usd":3.6563,"cost_breakdown":{"input":2.280384,"cache_read":0.221312,"cache_write":0.0,"output":1.154562},"cost_usd_as_run":3.6563,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1288,"code":1110,"comment":90,"blank":88,"role":"impl+test"}],"impl_lines":602,"impl_code":479,"impl_comment":79,"test_lines":686,"comment_ratio":0.142,"tests_declared":12},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.18,"reference":10.03,"speedup":0.985,"baselines":{"oracle":10.03}},{"id":"dec","candidate":11.91,"reference":11.8,"speedup":0.991,"baselines":{"oracle":11.8}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":20,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":8.0,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"deepseek-v4.1-flash-max":"bit-exact, well documented, strong state-level tests","opus-5.5-xhigh":"correct and well-documented; some unreachable defensive branches","gpt-6.1-terra-max":"Detailed API guidance and matching sampled behavior, with a carry cost.","gpt-6.1-sol-max":"Correct with good contract coverage; one minor documentation error.","muse-wonder-full-xhigh":"Clean, defensive, well documented"},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The `|| i + 1 == icdf.len()` in the `decode_icdf` loop is redundant","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"`BITRES` is described as the denominator","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"BITRES is an exponent, not the denominator","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"BITRES mislabelled","status":"valid","detail":"","location":""}],"score":8.7,"slop":{"score":0.5,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":0.9,"score_cross_vendor":8.7,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":25.04,"tokens":{"uncached_input":438,"cache_read":10665315,"cache_write":192680,"output":157074,"reasoning":0},"cost_usd":0.2831,"cost_breakdown":{"input":6.6e-05,"cache_read":0.170645,"cache_write":0.038536,"output":0.073825},"cost_usd_as_run":0.2831,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2831,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":2065,"code":1705,"comment":250,"blank":110,"role":"impl+test"}],"impl_lines":728,"impl_code":516,"impl_comment":164,"test_lines":1337,"comment_ratio":0.241,"tests_declared":37},"hidden":{"passed":7,"total":11,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.23,"reference":10.18,"speedup":0.995,"baselines":{"oracle":10.18}},{"id":"dec","candidate":11.96,"reference":11.68,"speedup":0.977,"baselines":{"oracle":11.68}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":72,"flags":1,"audited":"2026-11-03","reviewed":true},"scores":{"deepseek-v4.1-flash-max":3.5,"opus-5.5-xhigh":3.5,"gpt-6.1-terra-max":3.0,"gpt-6.1-sol-max":3.5,"muse-wonder-full-xhigh":5.5},"council_summaries":{"deepseek-v4.1-flash-max":"rejects legal ICDF calls and accepts out-of-contract `update` calls that mutate state","opus-5.5-xhigh":"refuses to encode the last symbol of any ICDF table, and accepts `update` calls the contract forbids","gpt-6.1-terra-max":"Valid ICDFs and decoder sequencing fail, and long counters diverge.","gpt-6.1-sol-max":"Rejects valid ICDF symbols, accepts forbidden updates, and miscounts long streams.","muse-wonder-full-xhigh":"Bit-exact core, wrong update and icdf contracts"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`encode_icdf` rejects the legal final symbol and single-entry tables","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`update` accepts calls outside its precondition and mutates state","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"tests encode the wrong contract and systematically dodge the ICDF bug","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"docs state preconditions the SPEC does not have","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"`encode_icdf` rejects the last symbol of every table","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"correctness","title":"`update` without a matching pending decode is accepted and changes state","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The tests lock in both defects","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The comment says a degenerate range \"yields the top symbol, like libopus\", but `map_or(0, …)` returns 0","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The `decode_icdf` fallback is commented \"Unreachable for well-formed tables … keep the index in bounds for corrupt state\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Final ICDF symbol is rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Unpaired or mismatched updates mutate state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Fractional accounting saturates instead of wrapping","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred count differs from the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests endorse invalid update behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Update and ICDF docs state incorrect preconditions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Final ICDF symbol is excluded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Unmatched updates mutate decoder state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Fractional accounting saturates instead of wrapping","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Unpaired update accepted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Terminator icdf symbols rejected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Tests enshrine wrong update semantics","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Degenerate decode comment inverted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable icdf fallthrough","status":"valid","detail":"","location":""}],"score":3.8,"slop":{"score":0.9,"per_seat":{"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.0}},"defect_points":10.5,"score_cross_vendor":3.8,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.08,"tokens":{"uncached_input":11126,"cache_read":130304,"cache_write":0,"output":5597,"reasoning":35},"cost_usd":0.5214,"cost_breakdown":{"input":0.11126,"cache_read":0.130304,"cache_write":0.0,"output":0.27985},"cost_usd_as_run":0.5214,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":629,"code":627,"comment":1,"blank":1,"role":"impl+test"}],"impl_lines":425,"impl_code":423,"impl_comment":1,"test_lines":204,"comment_ratio":0.002,"tests_declared":5},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.17,"reference":10.15,"speedup":0.998,"baselines":{"oracle":10.15}},{"id":"dec","candidate":11.97,"reference":11.77,"speedup":0.983,"baselines":{"oracle":11.77}}]},"quality":{"duplication":{"ratio":0.0881,"lines":28,"significant":318,"window":6},"complexity":{"functions":34,"mean":2.94,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":6.0,"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"deepseek-v4.1-flash-max":"bit-exact, but the whole API is undocumented behind a silenced lint and the tests are thin","opus-5.5-xhigh":"correct, but nothing public is documented and the crate's missing-docs lint is switched off","gpt-6.1-terra-max":"Ordinary probes match, but long carry counts, documentation and a masked test need changes.","gpt-6.1-sol-max":"Behavior matches the oracle, but documentation is absent and sequencing tests mask regressions.","muse-wonder-full-xhigh":"Correct core golfed at the expense of docs"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"public API undocumented and the lint suppressed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"thin test suite","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`Display` prints the Debug repr","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No docs on any public item; `#![allow(missing_docs)]` silences the crate's `missing_docs = \"warn\"`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The tests are thin","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`Display` just prints the Debug variant name","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The decoder's zero-range guard can never fire","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`update` re-matches `if let Some(..) = p` after already validating `p.is_some_and(..)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred count diverges from the oracle after wrapping","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry resolution loops over dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Cancellation assertions are masked by an earlier sticky error","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"docs","title":"Entire public API is undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Pending-cancellation assertions are masked","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No public documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Display is Debug output","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Round-trip variety narrow","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable recovery and exit","status":"valid","detail":"","location":""}],"score":6.6,"slop":{"score":0.3,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":4.3,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.87,"tokens":{"uncached_input":18464,"cache_read":189440,"cache_write":0,"output":10584,"reasoning":650},"cost_usd":0.9033,"cost_breakdown":{"input":0.18464,"cache_read":0.18944,"cache_write":0.0,"output":0.5292},"cost_usd_as_run":0.9033,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":873,"code":833,"comment":33,"blank":7,"role":"impl+test"}],"impl_lines":468,"impl_code":430,"impl_comment":31,"test_lines":405,"comment_ratio":0.067,"tests_declared":8},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.45,"reference":10.18,"speedup":0.974,"baselines":{"oracle":10.18}},{"id":"dec","candidate":12.03,"reference":11.89,"speedup":0.988,"baselines":{"oracle":11.89}}]},"quality":{"duplication":{"ratio":0.0489,"lines":16,"significant":327,"window":6},"complexity":{"functions":33,"mean":2.97,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"deepseek-v4.1-flash-max":"bit-exact and clean; docs are one-liners and tests are thinner than the best","opus-5.5-xhigh":"correct, but thin public docs and a few misleading comments","gpt-6.1-terra-max":"Efficient handling of dropped carries, with an extreme counter mismatch and sparse API guidance.","gpt-6.1-sol-max":"Correct and compact, with incomplete API contracts.","muse-wonder-full-xhigh":"Correct, terse, small redundancies"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"public items carry only terse summaries","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The public docs are one-liners without preconditions or error behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Text describes a code path no input can reach","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`|| i == icdf.len() - 1` and the trailing `0` in `decode_icdf` are unreachable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Dense layout and single-letter fields `r`/`v`/`n`.","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred count does not wrap like the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Fitting carry bytes are emitted through a checked loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method contracts are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"ICDF and sequencing requirements are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unreachable zero-range recovery","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Pointless carry precomputation","status":"valid","detail":"","location":""}],"score":7.7,"slop":{"score":0.5,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":2.2,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":18.68,"tokens":{"uncached_input":35449,"cache_read":8075776,"cache_write":0,"output":188502,"reasoning":146026},"cost_usd":0.1426,"cost_breakdown":{"input":0.005317,"cache_read":0.024227,"cache_write":0.0,"output":0.113101},"cost_usd_as_run":0.1426,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1426,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":2021,"code":1771,"comment":125,"blank":125,"role":"impl+test"}],"impl_lines":628,"impl_code":480,"impl_comment":105,"test_lines":1393,"comment_ratio":0.179,"tests_declared":30},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.23,"reference":10.38,"speedup":1.015,"baselines":{"oracle":10.38}},{"id":"dec","candidate":11.68,"reference":11.53,"speedup":0.987,"baselines":{"oracle":11.53}}]},"quality":{"duplication":{"ratio":0.0437,"lines":16,"significant":366,"window":6},"complexity":{"functions":34,"mean":3.12,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":52,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":8.0,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"deepseek-v4.1-flash-max":"bit-exact with the widest test suite; only two small blemishes","opus-5.5-xhigh":"correct, well-documented and well-tested; some dead defensive code and one false test comment","gpt-6.1-terra-max":"Broad coverage and correct sampled behavior, with an avoidable carry-chain cost.","gpt-6.1-sol-max":"Correct and broadly tested, with minor misleading and redundant test code.","muse-wonder-full-xhigh":"Solid and oracle-clean, dead recovery branches"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"dead branch's rationale is wrong","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"redundant `front == 0` test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The normalize branch for a zero range (latches CorruptStream) is unreachable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Redundant checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"A tautological assertion, `assert_eq!(frac, tell*8 - (tell*8 - frac))`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries are flushed byte by byte after the budget is exhausted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Bit probability is underspecified","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Tautological assertion","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Duplicate ICDF entries do not collapse the range","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Tautological assertion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unreachable zero-range recovery latches wrong error","status":"valid","detail":"","location":""}],"score":8.5,"slop":{"score":0.9,"per_seat":{"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":1.6,"score_cross_vendor":8.375,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.63,"tokens":{"uncached_input":27082,"cache_read":492160,"cache_write":0,"output":22281,"reasoning":8096},"cost_usd":0.1631,"cost_breakdown":{"input":0.027082,"cache_read":0.024608,"cache_write":0.0,"output":0.111405},"cost_usd_as_run":0.1631,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1131,"code":1010,"comment":41,"blank":80,"role":"impl+test"}],"impl_lines":567,"impl_code":482,"impl_comment":37,"test_lines":564,"comment_ratio":0.071,"tests_declared":12},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.21,"reference":10.15,"speedup":0.905,"baselines":{"oracle":10.15}},{"id":"dec","candidate":11.8,"reference":11.7,"speedup":0.992,"baselines":{"oracle":11.7}}]},"quality":{"duplication":{"ratio":0.0438,"lines":16,"significant":365,"window":6},"complexity":{"functions":39,"mean":2.74,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":7.0,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"deepseek-v4.1-flash-max":"bit-exact, bulk carry flush, careful state-level tests","opus-5.5-xhigh":"correct; encoder measurably slower than the oracle, plus a false comment and thin docs","gpt-6.1-terra-max":"Strong ordinary behavior and bulk carry handling, with an extreme counter mismatch.","gpt-6.1-sol-max":"Correct implementation with efficient carry flushing; public contracts need fuller documentation.","muse-wonder-full-xhigh":"Correct with a good carry optimisation, thin docs"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The encoder is 16–22% slower than the oracle","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The comment \"Malformed ICDFs can produce a zero range\" is false","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The public docs leave out preconditions and error behaviour","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred count does not wrap like the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Argument and sequencing requirements are omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public preconditions are largely omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Malformed tables cannot produce the claimed zero range","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable icdf exit","status":"valid","detail":"","location":""}],"score":8.1,"slop":{"score":0.4,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":1.8,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":54.61,"tokens":{"uncached_input":88065,"cache_read":1108168,"cache_write":0,"output":36416,"reasoning":18144},"cost_usd":1.3022,"cost_breakdown":{"input":0.35226,"cache_read":0.221634,"cache_write":0.0,"output":0.72832},"cost_usd_as_run":1.3022,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1401,"code":1247,"comment":72,"blank":82,"role":"impl+test"}],"impl_lines":610,"impl_code":519,"impl_comment":41,"test_lines":791,"comment_ratio":0.073,"tests_declared":25},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.36,"reference":10.18,"speedup":0.983,"baselines":{"oracle":10.18}},{"id":"dec","candidate":11.86,"reference":11.66,"speedup":0.983,"baselines":{"oracle":11.66}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"deepseek-v4.1-flash-max":"bit-exact and well tested, but the public API is barely documented","opus-5.5-xhigh":"correct, but cluttered with unreachable defensive branches; docs are minimal","gpt-6.1-terra-max":"Correct sampled behavior and extensive tests, with a carry cost and incomplete API guidance.","gpt-6.1-sol-max":"Correct with substantial edge coverage; documentation needs clarification.","muse-wonder-full-xhigh":"Broad tests, correct, minor verbosity"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"public API is essentially undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable mask branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unreachable defensive branches throughout, one with a false comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The public docs are one-liners without preconditions or error behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`finish` zero-fills with a manual while loop instead of `fill(0)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are handled individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method preconditions are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Unreachable mask branch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public methods omit their accepted domains","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Duplicate thresholds cannot create the claimed zero range","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable raw-bits guards","status":"valid","detail":"","location":""}],"score":8.1,"slop":{"score":0.6,"per_seat":{"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":2.1,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":131.25,"tokens":{"uncached_input":1183447,"cache_read":18231936,"cache_write":0,"output":219242,"reasoning":169426},"cost_usd":0.8341,"cost_breakdown":{"input":0.177517,"cache_read":0.546958,"cache_write":0.0,"output":0.109621},"cost_usd_as_run":0.8341,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1287,"code":1028,"comment":181,"blank":78,"role":"impl+test"}],"impl_lines":681,"impl_code":474,"impl_comment":164,"test_lines":606,"comment_ratio":0.257,"tests_declared":19},"hidden":{"passed":5,"total":11,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.05,"reference":9.93,"speedup":0.988,"baselines":{"oracle":9.93}},{"id":"dec","candidate":11.51,"reference":11.34,"speedup":0.985,"baselines":{"oracle":11.34}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":84,"flags":1,"audited":"2026-11-03","reviewed":true},"scores":{"deepseek-v4.1-flash-max":3.0,"opus-5.5-xhigh":2.5,"gpt-6.1-terra-max":2.5,"gpt-6.1-sol-max":2.5,"muse-wonder-full-xhigh":4.0},"council_summaries":{"deepseek-v4.1-flash-max":"rejects legal ICDF symbols, panics on a huge symbol, and a decoder that never advances for small `decode_uint`","opus-5.5-xhigh":"decoder desyncs after any small-alphabet `decode_uint`, and the ICDF encoder rejects valid symbols","gpt-6.1-terra-max":"Common valid decodes and ICDF encodes fail, and valid or rejected calls can panic.","gpt-6.1-sol-max":"Basic alphabet and integer-decoding defects, plus a reproducible misuse panic.","muse-wonder-full-xhigh":"Small-uint decode desyncs, plus a panic"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`encode_icdf` rejects the legal final symbol","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"robustness","title":"`symbol + 1` overflows and panics","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`decode_uint` never advances the decoder for `ft <= 256`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"tests enshrine two of the bugs and never reach the third","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"`decode_uint` with ft ≤ 256 never advances the decoder","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"`encode_icdf` rejects the last symbol (`symbol + 1 < icdf.len()`)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The tests miss one bug and lock in the other","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"`self.ext += 1` on a `u32` can overflow and panic","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The BITRES doc says it is \"carried for API compatibility with … `ec_enc_bit_logp` style consumers\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The `&& self.rng != 0` loop guards can never fire","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`tell_frac` is duplicated verbatim in the encoder and decoder","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Small unsigned decodes do not advance the range","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Final ICDF symbol is rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Extreme symbol index panics","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Deferred-byte counter overflows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry resolution revisits every dropped byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Tests enforce the wrong ICDF boundary and miss small uints","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"ICDF documentation excludes a valid symbol","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Incorrect ICDF bound rejects valid symbols and can panic","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Small uniform integers never advance the decoder","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Random generator excludes important paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"Small decode_uint never updates state","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"robustness","title":"Huge icdf symbol panics","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"correctness","title":"Terminator icdf symbols rejected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Random uints mask the boundary bug","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Duplicated tell_frac","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"BITRES attributed to wrong API","status":"valid","detail":"","location":""}],"score":2.9,"slop":{"score":0.9,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":2.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":12.7,"score_cross_vendor":2.9,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.61,"tokens":{"uncached_input":36023,"cache_read":182912,"cache_write":0,"output":10744,"reasoning":1057},"cost_usd":0.1978,"cost_breakdown":{"input":0.072046,"cache_read":0.018291,"cache_write":0.0,"output":0.10744},"cost_usd_as_run":0.1978,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":898,"code":860,"comment":32,"blank":6,"role":"impl+test"}],"impl_lines":471,"impl_code":435,"impl_comment":30,"test_lines":427,"comment_ratio":0.065,"tests_declared":9},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.21,"reference":10.34,"speedup":1.013,"baselines":{"oracle":10.34}},{"id":"dec","candidate":12.17,"reference":11.77,"speedup":0.967,"baselines":{"oracle":11.77}}]},"quality":{"duplication":{"ratio":0.0864,"lines":28,"significant":324,"window":6},"complexity":{"functions":34,"mean":2.94,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"deepseek-v4.1-flash-max":"bit-exact on every probe with thorough tests and accurate docs","opus-5.5-xhigh":"correct, but public docs are thin, plus some dead code and dense layout","gpt-6.1-terra-max":"Ordinary behavior is well tested; an extreme carry count differs from the oracle.","gpt-6.1-sol-max":"Correct and well checked, with terse API documentation.","muse-wonder-full-xhigh":"Correct and compact, thin docs"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The public docs leave out preconditions and failure behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The decoder's zero-range guard (latches CorruptStream) is unreachable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"In `decode_icdf`, `|| i == icdf.len() - 1` and the trailing `0` are unreachable; `update` re-matches `if let Some` after validating","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Dense layout with no blank lines between items","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Deferred count does not wrap like the oracle","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Full-buffer carry resolution is linear in dropped bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Caller preconditions are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Important preconditions require reading the implementation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unreachable zero-range recovery","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Dead icdf exits","status":"valid","detail":"","location":""}],"score":8.0,"slop":{"score":0.5,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":2.2,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.4,"tokens":{"uncached_input":9286,"cache_read":274688,"cache_write":0,"output":6106,"reasoning":294},"cost_usd":0.0536,"cost_breakdown":{"input":0.009286,"cache_read":0.013734,"cache_write":0.0,"output":0.03053},"cost_usd_as_run":0.0536,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":555,"code":551,"comment":0,"blank":4,"role":"impl+test"}],"impl_lines":436,"impl_code":432,"impl_comment":0,"test_lines":119,"comment_ratio":0.0,"tests_declared":4},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.46,"reference":10.23,"speedup":0.978,"baselines":{"oracle":10.23}},{"id":"dec","candidate":11.79,"reference":11.77,"speedup":0.998,"baselines":{"oracle":11.77}}]},"quality":{"duplication":{"ratio":0.0488,"lines":16,"significant":328,"window":6},"complexity":{"functions":32,"mean":3.09,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":6.0,"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":5.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"deepseek-v4.1-flash-max":"behaviour is right, but the API is undocumented and tests are thin","gpt-6.1-terra-max":"The exercised coder behavior matches, but documentation and tests need substantial work.","opus-5.5-xhigh":"Bit-exact, but undocumented (lint suppressed) and thinly tested","gpt-6.1-sol-max":"Correct behavior with useful tests, but undocumented API and avoidable overflow work.","muse-wonder-full-xhigh":"Correct, but undocumented with sampled-only misuse tests"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"entire public API (30 items) has no documentation and the crate lint is suppressed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"only 4 tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"single-letter fields (`r`, `v`, `n`) and uncommented bit arithmetic","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"leftover raw bits are merged after an error was latched","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries are processed after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Four fixed tests leave important edges untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Empty ICDF is checked twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Public API undocumented and the scaffold lint suppressed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Four small tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Opaque names and bool-flag helpers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Redundant guards","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Resolving deferred bytes performs unnecessary failed writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is entirely suppressed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Robustness coverage is sampled, not systematic","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Debug output reused as user-facing error text","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Redundant emptiness check in `decode_icdf`","status":"valid","detail":"","location":""}],"score":6.6,"slop":{"score":0.3,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":4.7,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":8.6,"tokens":{"uncached_input":11789,"cache_read":363264,"cache_write":0,"output":20707,"reasoning":6610},"cost_usd":0.267,"cost_breakdown":{"input":0.023578,"cache_read":0.036326,"cache_write":0.0,"output":0.20707},"cost_usd_as_run":0.267,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1099,"code":996,"comment":35,"blank":68,"role":"impl+test"}],"impl_lines":531,"impl_code":455,"impl_comment":32,"test_lines":568,"comment_ratio":0.066,"tests_declared":13},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.19,"reference":10.01,"speedup":0.895,"baselines":{"oracle":10.01}},{"id":"dec","candidate":11.83,"reference":11.89,"speedup":1.005,"baselines":{"oracle":11.89}}]},"quality":{"duplication":{"ratio":0.0462,"lines":16,"significant":346,"window":6},"complexity":{"functions":35,"mean":2.94,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.0,"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"deepseek-v4.1-flash-max":"correct and well-tested; dead fallback and one doubtful comment","gpt-6.1-terra-max":"Efficient carry handling and strong tests, with incomplete API documentation.","opus-5.5-xhigh":"Correct with strong tests; same encoder hot-path cost as V, a false comment, dead error paths","gpt-6.1-sol-max":"Correct and compact, with efficient carry flushing and substantial edge coverage.","muse-wonder-full-xhigh":"Fast bulk carry handling and white-box tests; thin docs, dead Corrupt branches"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"unreachable `0` fallback after the ICDF search","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"`fractional_bits` comment claims a malformed table \"can produce zero … range\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public methods omit important preconditions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Empty ICDF is checked twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"The ICDF fallback cannot be reached","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Slice fill runs on every emitted byte","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False claim that malformed tables shrink the range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `CorruptStream` paths","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`icdf.is_empty() ||` is redundant","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions largely unstated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unreachable zero-range branches latch `CorruptStream`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Redundant emptiness check in `decode_icdf`","status":"valid","detail":"","location":""}],"score":8.4,"slop":{"score":1.2,"per_seat":{"deepseek-v4.1-flash-max":1.5,"opus-5.5-xhigh":2.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":1.8,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.01,"tokens":{"uncached_input":16641,"cache_read":126976,"cache_write":0,"output":6391,"reasoning":261},"cost_usd":0.1099,"cost_breakdown":{"input":0.033282,"cache_read":0.012698,"cache_write":0.0,"output":0.06391},"cost_usd_as_run":0.1099,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":701,"code":700,"comment":0,"blank":1,"role":"impl+test"}],"impl_lines":434,"impl_code":433,"impl_comment":0,"test_lines":267,"comment_ratio":0.0,"tests_declared":6},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.2,"reference":10.35,"speedup":1.015,"baselines":{"oracle":10.35}},{"id":"dec","candidate":12.14,"reference":11.76,"speedup":0.969,"baselines":{"oracle":11.76}}]},"quality":{"duplication":{"ratio":0.0926,"lines":30,"significant":324,"window":6},"complexity":{"functions":34,"mean":3.09,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":6.5,"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"deepseek-v4.1-flash-max":"correct and reasonably tested, but completely undocumented","gpt-6.1-terra-max":"Good exercised behavior and useful tests, but the API is undocumented and carry flushing is unnecessarily linear.","opus-5.5-xhigh":"Bit-exact, but undocumented (lint suppressed), terse code, moderate tests","gpt-6.1-sol-max":"Correct with useful coverage, but undocumented and unnecessarily expensive after overflow.","muse-wonder-full-xhigh":"Correct with strong white-box tests, but zero documentation"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no docs for any public item, lint suppressed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"6 tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"unguarded divisions/subtractions rely on undocumented invariants","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries are processed after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"The final-index ICDF check is unnecessary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"The ICDF fallback cannot be reached","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Public API undocumented and the scaffold lint suppressed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Rejection tests are batched","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Opaque names and bool-flag helpers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`if self.r == 0 { self.r = 1 }`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Resolving deferred bytes performs unnecessary failed writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is entirely suppressed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Cryptic one-letter state names","status":"valid","detail":"","location":""}],"score":7.1,"slop":{"score":0.3,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.8,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.02,"tokens":{"uncached_input":12563,"cache_read":319616,"cache_write":0,"output":9899,"reasoning":725},"cost_usd":0.078,"cost_breakdown":{"input":0.012563,"cache_read":0.015981,"cache_write":0.0,"output":0.049495},"cost_usd_as_run":0.078,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":782,"code":711,"comment":33,"blank":38,"role":"impl+test"}],"impl_lines":537,"impl_code":474,"impl_comment":33,"test_lines":245,"comment_ratio":0.065,"tests_declared":6},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.44,"reference":10.17,"speedup":0.974,"baselines":{"oracle":10.17}},{"id":"dec","candidate":12.05,"reference":11.69,"speedup":0.97,"baselines":{"oracle":11.69}}]},"quality":{"duplication":{"ratio":0.0443,"lines":16,"significant":361,"window":6},"complexity":{"functions":40,"mean":2.67,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":7.5,"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"deepseek-v4.1-flash-max":"correct, terse docs, moderate tests","gpt-6.1-terra-max":"Efficient and well exercised, but its public preconditions are underdocumented.","opus-5.5-xhigh":"Bit-exact and compact, but thin documentation and tests, a false comment, awkward internal plumbing","gpt-6.1-sol-max":"Correct and well tested; deferred-byte output could be cheaper.","muse-wonder-full-xhigh":"Compact, correct, dense tests; public docs never state the preconditions"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"6 tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`decode_uint` reads the scale back out of `self.pending`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Several method preconditions are omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"The final-index ICDF check is unnecessary","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Preconditions undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"`decode_uint` reads `ext` back out of `pending`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin tests (6)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Deferred bytes use repeated checked writes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions undocumented on every public method","status":"valid","detail":"","location":""}],"score":8.1,"slop":{"score":0.3,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.8,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":67.01,"tokens":{"uncached_input":314626,"cache_read":3529936,"cache_write":0,"output":58344,"reasoning":31377},"cost_usd":3.1314,"cost_breakdown":{"input":1.258504,"cache_read":0.705987,"cache_write":0.0,"output":1.16688},"cost_usd_as_run":3.1314,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1381,"code":1164,"comment":127,"blank":90,"role":"impl+test"}],"impl_lines":677,"impl_code":525,"impl_comment":101,"test_lines":704,"comment_ratio":0.161,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.33,"reference":10.23,"speedup":0.99,"baselines":{"oracle":10.23}},{"id":"dec","candidate":11.76,"reference":11.61,"speedup":0.987,"baselines":{"oracle":11.61}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":35,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.5,"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"deepseek-v4.1-flash-max":"excellent coverage and docs; two factually wrong sentences","gpt-6.1-terra-max":"Broad coverage, but public docs misstate two observable quantities and carry flushing is unnecessarily linear.","opus-5.5-xhigh":"Correct with good tests; two false public doc statements and dead defensive branches","gpt-6.1-sol-max":"Correct and well tested, with avoidable carry work and incorrect public explanations.","muse-wonder-full-xhigh":"Well structured with two doc typos that misstate formulas"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"`tell` is documented as \"rounded down\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries are processed after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The documented bit probability evaluates to zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"`tell` is described as rounded down","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"The zero-total branch in the private decoder cannot be reached","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"`tell` documented as \"rounded down\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Bit probability written as `1 >> logp`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`decode_priv` guards cannot fire","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Redundant mask on leftover bits","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Resolving deferred bytes performs unnecessary failed writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Probability expression evaluates to zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Whole-bit accounting is described as rounded down","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Nonsense probability formula","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"`tell` rounding direction wrong","status":"valid","detail":"","location":""}],"score":8.1,"slop":{"score":2.0,"per_seat":{"deepseek-v4.1-flash-max":1.5,"opus-5.5-xhigh":3.0,"gpt-6.1-terra-max":2.5,"gpt-6.1-sol-max":2.0,"muse-wonder-full-xhigh":1.0}},"defect_points":2.8,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.4,"tokens":{"uncached_input":14,"cache_read":384091,"cache_write":72915,"output":40095,"reasoning":17109},"cost_usd":1.4621,"cost_breakdown":{"input":5.6e-05,"cache_read":0.076818,"cache_write":0.58332,"output":0.8019},"cost_usd_as_run":1.4621,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1362,"code":1215,"comment":59,"blank":88,"role":"impl+test"}],"impl_lines":611,"impl_code":518,"impl_comment":42,"test_lines":751,"comment_ratio":0.075,"tests_declared":17},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.33,"reference":10.04,"speedup":0.972,"baselines":{"oracle":10.04}},{"id":"dec","candidate":11.7,"reference":11.75,"speedup":1.004,"baselines":{"oracle":11.75}}]},"quality":{"duplication":{"ratio":0.0754,"lines":30,"significant":398,"window":6},"complexity":{"functions":41,"mean":2.68,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.0,"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"deepseek-v4.1-flash-max":"correct with 17 tests; unreachable state mutation documented wrongly","gpt-6.1-terra-max":"Correct in the exercised cases and efficient on carry overflow, but several preconditions are undocumented.","opus-5.5-xhigh":"Correct with strong tests; a false internal comment and incomplete precondition docs","gpt-6.1-sol-max":"Thorough implementation and tests; one avoidable scalar output loop.","muse-wonder-full-xhigh":"Clean and correct; a few doc omissions and a no-op call"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"`normalize` sets `rng = 1` when the range is zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Some public method requirements are missing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Encoder initialization normalizes an already normalized range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Preconditions missing or incomplete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicate checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Deferred bytes use repeated checked writes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions missing or incomplete on three methods","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"No-op `normalize()` in constructor","status":"valid","detail":"","location":""}],"score":8.4,"slop":{"score":0.7,"per_seat":{"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":1.5,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.91,"tokens":{"uncached_input":18840,"cache_read":194304,"cache_write":0,"output":4291,"reasoning":121},"cost_usd":0.05,"cost_breakdown":{"input":0.01884,"cache_read":0.009715,"cache_write":0.0,"output":0.021455},"cost_usd_as_run":0.05,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":476,"code":476,"comment":0,"blank":0,"role":"impl+test"}],"impl_lines":439,"impl_code":439,"impl_comment":0,"test_lines":37,"comment_ratio":0.0,"tests_declared":1},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.24,"reference":10.14,"speedup":0.99,"baselines":{"oracle":10.14}},{"id":"dec","candidate":12.12,"reference":11.8,"speedup":0.974,"baselines":{"oracle":11.8}}]},"quality":{"duplication":{"ratio":0.0479,"lines":16,"significant":334,"window":6},"complexity":{"functions":32,"mean":3.06,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":5.5,"gpt-6.1-terra-max":6.0,"opus-5.5-xhigh":5.0,"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"deepseek-v4.1-flash-max":"correct core, but a single test and no docs at all","gpt-6.1-terra-max":"The exercised implementation matches the oracle, but it lacks contract documentation and meaningful edge coverage.","opus-5.5-xhigh":"Bit-exact, but no documentation at all (lint suppressed) and only one smoke test","gpt-6.1-sol-max":"Correct in probing, but documentation and regression coverage need substantial work.","muse-wonder-full-xhigh":"Correct code shipped with almost no tests and no documentation"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no documentation for any public item and the lint suppressed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"the whole test suite is one function","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`decode` uses `checked_div(ext).unwrap_or(ft)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries are processed after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"The only test covers a partial happy-path vector","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"The final-index ICDF check is unnecessary","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Public API undocumented and the scaffold lint suppressed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"One test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Opaque names and bool-flag helpers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `&& self.r != 0` loop guards","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Resolving deferred bytes performs unnecessary failed writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Only one partial happy-path test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public API documentation is entirely suppressed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Single partial test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Cryptic state naming, no invariant comments","status":"valid","detail":"","location":""}],"score":5.9,"slop":{"score":0.3,"per_seat":{"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":5.5,"score_cross_vendor":5.667,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":13.95,"tokens":{"uncached_input":34,"cache_read":1764050,"cache_write":133525,"output":94599,"reasoning":57216},"cost_usd":3.3131,"cost_breakdown":{"input":0.000136,"cache_read":0.35281,"cache_write":1.0682,"output":1.89198},"cost_usd_as_run":3.3131,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":2182,"code":1856,"comment":200,"blank":126,"role":"impl+test"}],"impl_lines":761,"impl_code":545,"impl_comment":164,"test_lines":1421,"comment_ratio":0.231,"tests_declared":32},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.11,"reference":10.16,"speedup":1.005,"baselines":{"oracle":10.16}},{"id":"dec","candidate":11.79,"reference":11.75,"speedup":0.997,"baselines":{"oracle":11.75}}]},"quality":{"duplication":{"ratio":0.0383,"lines":16,"significant":418,"window":6},"complexity":{"functions":43,"mean":2.67,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":9.5,"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"deepseek-v4.1-flash-max":"the most thorough entry; accurate docs and exhaustive tests","gpt-6.1-terra-max":"Extensive tests and strong documentation, with a slow carry path and several minor inaccuracies.","opus-5.5-xhigh":"Correct, the most thorough tests (32, including invariant fuzzing and tight-buffer monotonicity), accurate docs; over-defensive code","gpt-6.1-sol-max":"Extensive coverage and correct behavior, with avoidable carry work and one false initialization comment.","muse-wonder-full-xhigh":"Most thorough tests and docs; one unreachable error-inventing recovery branch"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"defensive `ft.max(1)` / `checked_shl` on hot paths","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries are processed after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Overflow need not drop a whole byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"The initial bit-count comment names both coders","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Private unchecked helpers revalidate positive totals","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Defensive re-checks in \"unchecked\" helpers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Resolving deferred bytes performs unnecessary failed writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Initial bit count is incorrectly attributed to both coders","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Zero-range recovery invents `CorruptStream` and resets state","status":"valid","detail":"","location":""}],"score":8.7,"slop":{"score":0.9,"per_seat":{"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":1.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":1.7,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.08,"tokens":{"uncached_input":34009,"cache_read":285952,"cache_write":0,"output":22226,"reasoning":7222},"cost_usd":1.7373,"cost_breakdown":{"input":0.34009,"cache_read":0.285952,"cache_write":0.0,"output":1.1113},"cost_usd_as_run":1.7373,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1247,"code":1134,"comment":37,"blank":76,"role":"impl+test"}],"impl_lines":532,"impl_code":452,"impl_comment":37,"test_lines":715,"comment_ratio":0.076,"tests_declared":16},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.19,"reference":10.3,"speedup":0.92,"baselines":{"oracle":10.3}},{"id":"dec","candidate":11.96,"reference":11.7,"speedup":0.978,"baselines":{"oracle":11.7}}]},"quality":{"duplication":{"ratio":0.0465,"lines":16,"significant":344,"window":6},"complexity":{"functions":34,"mean":2.94,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.0,"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"deepseek-v4.1-flash-max":"correct and well-tested; an unreachable guard mutates state","gpt-6.1-terra-max":"Efficient and thoroughly tested, with a small documentation gap.","opus-5.5-xhigh":"Bit-exact and robust with excellent tests; one measurable encoder hot-path cost and a dead guard with a false rationale","gpt-6.1-sol-max":"Correct, efficient, clearly documented, and comprehensively tested.","muse-wonder-full-xhigh":"Lean, well-tested; one unreachable error-inventing recovery branch"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"decoder `normalize` resets the coder on `rng == 0`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable `0` fallback after the ICDF search","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"The selected ICDF interval’s requirements are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"The ICDF fallback cannot be reached","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Slice fill runs on every emitted byte","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Dead zero-range guard with a false rationale","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Zero-range recovery invents `CorruptStream` and resets state","status":"valid","detail":"","location":""}],"score":8.7,"slop":{"score":0.6,"per_seat":{"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.0}},"defect_points":1.2,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":6.13,"tokens":{"uncached_input":18,"cache_read":568785,"cache_write":78302,"output":45091,"reasoning":20753},"cost_usd":1.6421,"cost_breakdown":{"input":7.2e-05,"cache_read":0.113757,"cache_write":0.626416,"output":0.90182},"cost_usd_as_run":1.6421,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1385,"code":1172,"comment":119,"blank":94,"role":"impl+test"}],"impl_lines":674,"impl_code":532,"impl_comment":90,"test_lines":711,"comment_ratio":0.145,"tests_declared":16},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.5,"reference":9.99,"speedup":0.951,"baselines":{"oracle":9.99}},{"id":"dec","candidate":11.71,"reference":11.75,"speedup":1.003,"baselines":{"oracle":11.75}}]},"quality":{"duplication":{"ratio":0.073,"lines":30,"significant":411,"window":6},"complexity":{"functions":43,"mean":2.6,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"deepseek-v4.1-flash-max":8.5,"gpt-6.1-terra-max":9.5,"opus-5.5-xhigh":8.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"deepseek-v4.1-flash-max":"thorough tests and honest docs; one no-op call","gpt-6.1-terra-max":"Strong coverage and an efficient carry path; only a trivial redundant operation found.","opus-5.5-xhigh":"Correct, well documented, thorough tests including tight-buffer and carry checks; minor redundancy","gpt-6.1-sol-max":"Thorough and correct, with a scalar carry loop and a masked coverage assertion.","muse-wonder-full-xhigh":"Clean and thoroughly specified in docs; one no-op call"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`RangeEncoder::new` calls `enc.normalize()`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Encoder initialization normalizes an already normalized range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"No-op `normalize()` in the constructor","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Deferred bytes use repeated checked writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Zero padding satisfies the carry-coverage predicate","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"No-op `normalize()` in constructor","status":"valid","detail":"","location":""}],"score":8.9,"slop":{"score":0.4,"per_seat":{"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":0.8,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":10.62,"tokens":{"uncached_input":38528,"cache_read":264832,"cache_write":0,"output":27363,"reasoning":11611},"cost_usd":0.3772,"cost_breakdown":{"input":0.077056,"cache_read":0.026483,"cache_write":0.0,"output":0.27363},"cost_usd_as_run":0.3772,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1295,"code":1162,"comment":39,"blank":94,"role":"impl+test"}],"impl_lines":554,"impl_code":469,"impl_comment":37,"test_lines":741,"comment_ratio":0.073,"tests_declared":16},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":11.14,"reference":10.1,"speedup":0.907,"baselines":{"oracle":10.1}},{"id":"dec","candidate":11.71,"reference":11.95,"speedup":1.02,"baselines":{"oracle":11.95}}]},"quality":{"duplication":{"ratio":0.0448,"lines":16,"significant":357,"window":6},"complexity":{"functions":39,"mean":2.74,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"opus-5.5-xhigh":8.0,"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"opus-5.5-xhigh":"correct, with good state-checking tests, but the hot path does needless work and the docs are thin","gpt-6.1-terra-max":"Bit-exact in the checks, with a small public documentation gap.","gpt-6.1-sol-max":"Strongest overflow handling and boundary tests, with a small API documentation gap.","muse-wonder-full-xhigh":"Bit-exact and compact with strong state-level tests; thin ICDF docs","deepseek-v4.1-flash-max":"Correct and the most defensively written of the six; only documentation nits."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"`carry_out` runs its bulk-fill path for every byte","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable guards","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Thin public docs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF preconditions are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Selected ICDF interval requirements are unstated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"`encode_icdf` doc never defines a valid call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`finish` doc omits its error contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`BITRES` doc names only the encoder's `tell_frac`","status":"valid","detail":"","location":""}],"score":8.7,"slop":{"score":0.1,"per_seat":{"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":62.62,"tokens":{"uncached_input":287758,"cache_read":4037676,"cache_write":0,"output":70220,"reasoning":42786},"cost_usd":3.363,"cost_breakdown":{"input":1.151032,"cache_read":0.807535,"cache_write":0.0,"output":1.4044},"cost_usd_as_run":3.363,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1593,"code":1448,"comment":51,"blank":94,"role":"impl+test"}],"impl_lines":633,"impl_code":550,"impl_comment":33,"test_lines":960,"comment_ratio":0.057,"tests_declared":21},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.72,"reference":9.99,"speedup":0.932,"baselines":{"oracle":9.99}},{"id":"dec","candidate":11.61,"reference":11.77,"speedup":1.014,"baselines":{"oracle":11.77}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":35,"flags":0,"audited":"2026-11-03","reviewed":false},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"opus-5.5-xhigh":"correct, but the public API is almost undocumented and the code is cluttered with unreachable defensive branches","gpt-6.1-terra-max":"Strong coverage and matching behavior, but deferred carries do unnecessary unbounded work.","gpt-6.1-sol-max":"Correct in probes, but weakened by incomplete documentation, ineffective testing and redundant paths.","muse-wonder-full-xhigh":"Bit-exact with strong tests, but the API contract is undocumented","deepseek-v4.1-flash-max":"Correct and very well tested, but the public method docs omit the preconditions that are the heart of this contract."},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public methods document no preconditions or misuse behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Many unreachable branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Zero-fill written as a per-index loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries replay dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public operation limits are omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Raw-bit helpers contain unreachable full-width branches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Deferred carries repeat dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Varied buffer sweep never reaches successful decoding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public methods omit important preconditions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Private helpers contain unreachable width cases","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Collision comment overstates spare capacity","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No public method documents its precondition or rejection behaviour","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"`encode_uint` passes the unmasked value to the raw-bit helper","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"public methods never state their preconditions or error semantics","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"`finish` merges leftover raw bits even with an error already latched","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"dead seeds and unreachable lookup arms","status":"valid","detail":"","location":""}],"score":7.7,"slop":{"score":0.9,"per_seat":{"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":3.1,"score_cross_vendor":7.75,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1-b2","task":"range-coder","lead":"muse-wonder-full-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":9.29,"tokens":{"cache_read":3560353,"cache_write":200760,"output":55341,"reasoning":31723,"uncached_input":52},"cost_usd":3.4252,"cost_breakdown":{"cache_read":0.712071,"cache_write":1.60608,"input":0.000208,"output":1.10682},"cost_usd_as_run":3.4252,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":7.36,"tokens":{"cache_read":2077312,"cache_write":0,"output":21002,"reasoning":10444,"uncached_input":93560},"cost_usd":0.3024,"cost_breakdown":{"cache_read":0.103866,"cache_write":0.0,"input":0.09356,"output":0.10501},"cost_usd_as_run":0.3024,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":8.88,"tokens":{"cache_read":5630336,"cache_write":0,"output":96133,"reasoning":74358,"uncached_input":110535},"cost_usd":0.0912,"cost_breakdown":{"cache_read":0.016891,"cache_write":0.0,"input":0.01658,"output":0.05768},"cost_usd_as_run":0.0912,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":13.21,"tokens":{"cache_read":2574208,"cache_write":0,"output":27401,"reasoning":14403,"uncached_input":99914},"cost_usd":0.7313,"cost_breakdown":{"cache_read":0.257421,"cache_write":0.0,"input":0.199828,"output":0.27401},"cost_usd_as_run":0.7313,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":15.23,"tokens":{"cache_read":4581656,"cache_write":0,"output":45839,"reasoning":28596,"uncached_input":227095},"cost_usd":2.7415,"cost_breakdown":{"cache_read":0.916331,"cache_write":0.0,"input":0.90838,"output":0.91678},"cost_usd_as_run":2.7415,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"K","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"V","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"K","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"V","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"K","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"V","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"V","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"V","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":10.0}],"panel":["opus-5.5-xhigh","gpt-6.1-terra-max","deepseek-v4.1-flash-max","gpt-6.1-sol-max","muse-wonder-full-xhigh"]},"entries":[{"letter":"Z","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":4.62,"tokens":{"uncached_input":59991,"cache_read":228114,"cache_write":0,"output":24575,"reasoning":8223},"cost_usd":0.3885,"cost_breakdown":{"input":0.119982,"cache_read":0.022811,"cache_write":0.0,"output":0.24575},"cost_usd_as_run":0.3885,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1206,"code":1053,"comment":56,"blank":97,"role":"impl+test"}],"impl_lines":664,"impl_code":577,"impl_comment":30,"test_lines":542,"comment_ratio":0.049,"tests_declared":7},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.88,"reference":9.72,"speedup":0.984,"baselines":{"oracle":9.72}},{"id":"dec","candidate":11.54,"reference":11.51,"speedup":0.997,"baselines":{"oracle":11.51}}]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":7.5,"gpt-6.1-terra-max":6.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"opus-5.5-xhigh":"Matches the reference exactly with clean structure, but its docs are one-liners and its rejection tests can't detect state leaks","gpt-6.1-terra-max":"Correct on ordinary-sized differential cases, with avoidable carry-flush work and incomplete docs.","deepseek-v4.1-flash-max":"correct and well-structured behind validated `_core` helpers, but the whole input contract is undocumented and misuse tests skip the bytes.","gpt-6.1-sol-max":"Bit-exact in probing; avoidable overflow work and documentation/test gaps.","muse-wonder-full-xhigh":"correct and robust with solid structure, but incomplete docs and fresh-only reject tests"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Same rejection blind spot as V","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public docs leave out preconditions, the InvalidArgument latch and the 0/false return on rejection","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`encode_icdf` (:285–295) reads and validates `prev`/`cur`, then `encode_icdf_core` (:209–222) reads both again through `get().unwrap_or(0)` fallbacks that can't be reached","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped at `u32::MAX`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Carry propagation retries every dropped write","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Valid ICDF precisions below 8 lack vector coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public method docs omit important preconditions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"ICDF bounds are read twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"No precondition is documented on any public method","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Decoder behaviour on short/corrupt input is undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Rejected-call tests check observers, not state or bytes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`overflowed: bool` duplicates the latched error","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Carry flushing repeats dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public preconditions are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ICDF round trips cover only one precision","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Public methods omit argument limits and reject behaviour","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Encoder rejects verified only on fresh coders via tell==1","status":"valid","detail":"","location":""}],"score":7.6,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":3.5,"score_cross_vendor":7.6,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","task":"range-coder","lead":"muse-wonder-full-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":38.95,"tokens":{"cache_read":27011800,"cache_write":994693,"output":227905,"reasoning":141697,"uncached_input":248},"cost_usd":17.919,"cost_breakdown":{"cache_read":5.40236,"cache_write":7.957544,"input":0.000992,"output":4.5581},"cost_usd_as_run":17.919,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":73.75,"tokens":{"cache_read":24923748,"cache_write":0,"output":142760,"reasoning":76646,"uncached_input":1121103},"cost_usd":12.3244,"cost_breakdown":{"cache_read":4.98475,"cache_write":0.0,"input":4.484412,"output":2.8552},"cost_usd_as_run":12.3244,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":36.26,"tokens":{"cache_read":16965376,"cache_write":0,"output":91095,"reasoning":47561,"uncached_input":484805},"cost_usd":1.7885,"cost_breakdown":{"cache_read":0.848269,"cache_write":0.0,"input":0.484805,"output":0.455475},"cost_usd_as_run":1.7885,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":63.08,"tokens":{"cache_read":43790592,"cache_write":0,"output":327480,"reasoning":230896,"uncached_input":654402},"cost_usd":0.426,"cost_breakdown":{"cache_read":0.131372,"cache_write":0.0,"input":0.09816,"output":0.196488},"cost_usd_as_run":0.426,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":46.95,"tokens":{"cache_read":15323136,"cache_write":0,"output":99258,"reasoning":53673,"uncached_input":668235},"cost_usd":3.8614,"cost_breakdown":{"cache_read":1.532314,"cache_write":0.0,"input":1.33647,"output":0.99258},"cost_usd_as_run":3.8614,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":10.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-muse-wonder-full-xhigh-r1","letter":"B","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"O","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"F","score":9.5}],"panel":["opus-5.5-xhigh","muse-wonder-full-xhigh","gpt-6.1-terra-max","deepseek-v4.1-flash-max","gpt-6.1-sol-max"],"guardrail_blocks":[{"date":"2026-11-03T19:48:30+00:00","provider":"openai","attempt":1,"message":"This content was flagged for possible cybersecurity risk (OpenAI cyber safeguards; suggests Daybreak access)","outcome":"retry-passed","role":"judge","seat":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2"}]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":12.65,"tokens":{"uncached_input":38,"cache_read":2129968,"cache_write":135848,"output":90836,"reasoning":54486},"cost_usd":3.3296,"cost_breakdown":{"input":0.000152,"cache_read":0.425994,"cache_write":1.086784,"output":1.81672},"cost_usd_as_run":3.3296,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1851,"code":1542,"comment":182,"blank":127,"role":"impl+test"}],"impl_lines":716,"impl_code":528,"impl_comment":135,"test_lines":1135,"comment_ratio":0.204,"tests_declared":27},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.99,"reference":9.93,"speedup":0.994,"baselines":{"oracle":9.93}},{"id":"dec","candidate":11.52,"reference":11.34,"speedup":0.984,"baselines":{"oracle":11.34}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":9.5,"muse-wonder-full-xhigh":9.5,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":9.5},"council_summaries":{"gpt-6.1-terra-max":"Thorough tests and documentation, with truncated storage","opus-5.5-xhigh":"Excellent: accurate module-level invariants, a complete precondition doc on every method, the strongest tests","muse-wonder-full-xhigh":"most complete contract documentation and independently-oracled tests","gpt-6.1-sol-max":"Exact implementation with particularly strong tests; deferred overflow work remains avoidable.","deepseek-v4.1-flash-max":"the most complete contract documentation and test suite in the field"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable `rng == 0` guards","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Deferred overflow writes remain linear","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge after `InvalidArgument`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0835,"lines":34,"significant":407,"window":6},"complexity":{"functions":44,"mean":2.68,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":19,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.0,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.2,"score_cross_vendor":8.875,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.0,"tokens":{"uncached_input":23598,"cache_read":3043456,"cache_write":0,"output":105786,"reasoning":85404},"cost_usd":0.0761,"cost_breakdown":{"input":0.00354,"cache_read":0.00913,"cache_write":0.0,"output":0.063472},"cost_usd_as_run":0.0761,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0761,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1369,"code":1145,"comment":123,"blank":101,"role":"impl+test"}],"impl_lines":710,"impl_code":553,"impl_comment":101,"test_lines":659,"comment_ratio":0.154,"tests_declared":15},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.91,"reference":9.84,"speedup":0.993,"baselines":{"oracle":9.84}},{"id":"dec","candidate":11.7,"reference":11.51,"speedup":0.984,"baselines":{"oracle":11.51}}]},"scores":{"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct ordinary paths, but large slices and API documentation need work","opus-5.5-xhigh":"Correct with good module docs; several method docs omit preconditions","muse-wonder-full-xhigh":"correct core; weakest docs of the wrapping-arithmetic entries","gpt-6.1-sol-max":"Exact arithmetic, with avoidable overflow work and substantial deterministic fuzz gaps.","deepseek-v4.1-flash-max":"well documented and well tested; only nits"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public argument limits are omitted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Missing preconditions on public methods","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`icdf.is_empty() ||` duplicates the trailing-zero check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"A manual zeroing loop instead of `fill(0)`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Misdescribed `encode` semantics","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Missing precondition ranges","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unchecked indexing after wrapping budget sum","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Observer-only state comparisons","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Hand-rolled zero-fill","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Deferred overflow writes remain linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Garbage-test branches are never exercised","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Numeric API boundaries are missing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"redundant emptiness check","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"direct slice in `finish`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge after `InvalidArgument`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1124,"lines":48,"significant":427,"window":6},"complexity":{"functions":48,"mean":2.46,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":26,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":3.2,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.35,"tokens":{"uncached_input":10733,"cache_read":224000,"cache_write":0,"output":9864,"reasoning":380},"cost_usd":0.8245,"cost_breakdown":{"input":0.10733,"cache_read":0.224,"cache_write":0.0,"output":0.4932},"cost_usd_as_run":0.8245,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":955,"code":917,"comment":31,"blank":7,"role":"impl+test"}],"impl_lines":525,"impl_code":487,"impl_comment":31,"test_lines":430,"comment_ratio":0.06,"tests_declared":8},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.26,"reference":10.02,"speedup":0.977,"baselines":{"oracle":10.02}},{"id":"dec","candidate":11.8,"reference":11.41,"speedup":0.967,"baselines":{"oracle":11.41}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Compact implementation with good carry handling, but truncated storage","opus-5.5-xhigh":"Compact and correct; minimal docs and dense, flag-driven helpers","muse-wonder-full-xhigh":"compact and correct; thinnest docs and test breadth","gpt-6.1-sol-max":"Compact implementation with efficient overflow handling; public contracts need fuller documentation.","deepseek-v4.1-flash-max":"clean core, but the contract's precondition table is undocumented and the suite is the thinnest"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public preconditions are incomplete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Thin docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Bool-flag multiplexing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions largely undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Narrow random coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Decoder patches encoder defaults","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Important method preconditions are omitted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no preconditions documented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"coverage stops short","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable trailing `0`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge after `InvalidArgument`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0438,"lines":16,"significant":365,"window":6},"complexity":{"functions":44,"mean":2.55,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":2.4,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.56,"tokens":{"uncached_input":31878,"cache_read":208256,"cache_write":0,"output":10852,"reasoning":799},"cost_usd":0.1931,"cost_breakdown":{"input":0.063756,"cache_read":0.020826,"cache_write":0.0,"output":0.10852},"cost_usd_as_run":0.1931,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":987,"code":946,"comment":34,"blank":7,"role":"impl+test"}],"impl_lines":527,"impl_code":488,"impl_comment":32,"test_lines":460,"comment_ratio":0.062,"tests_declared":8},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.03,"reference":9.94,"speedup":0.991,"baselines":{"oracle":9.94}},{"id":"dec","candidate":11.71,"reference":11.41,"speedup":0.974,"baselines":{"oracle":11.41}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Solid ordinary behavior, with truncated storage and an overconstraining test","opus-5.5-xhigh":"Compact and correct; minimal docs and a slightly misleading decoder doc","muse-wonder-full-xhigh":"clean twin of entry-I with better construction; docs/tests still lean","gpt-6.1-sol-max":"Correct shared core and bounded overflow processing, with sparse API documentation.","deepseek-v4.1-flash-max":"correct and compact, but the public API is essentially undocumented and the tests are thin"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public preconditions are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Test requires unspecified bytes after an error","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Thin docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"Creates a decoder over a complete packet.\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Same bool-flag `write`/`read` and dense packing as entry C","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Preconditions largely undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No garbage-encode coverage and fresh-coder rejection probe","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Inconsistent overflow bookkeeping","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public docs omit enforced constraints","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"one-line docs with no contract information","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"thin suite","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"encoder/decoder init behind a bool flag","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable final `0`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"direct slice in `finish`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge after `InvalidArgument`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0755,"lines":28,"significant":371,"window":6},"complexity":{"functions":43,"mean":2.6,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.7,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.9,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":49.48,"tokens":{"uncached_input":199378,"cache_read":3517452,"cache_write":0,"output":39087,"reasoning":16517},"cost_usd":2.2827,"cost_breakdown":{"input":0.797512,"cache_read":0.70349,"cache_write":0.0,"output":0.78174},"cost_usd_as_run":2.2827,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1259,"code":1126,"comment":46,"blank":87,"role":"impl+test"}],"impl_lines":650,"impl_code":560,"impl_comment":33,"test_lines":609,"comment_ratio":0.056,"tests_declared":15},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.08,"reference":10.07,"speedup":0.999,"baselines":{"oracle":10.07}},{"id":"dec","candidate":11.88,"reference":11.64,"speedup":0.98,"baselines":{"oracle":11.64}}]},"scores":{"gpt-6.1-terra-max":6.0,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":8.5,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Large slices and long ICDF tables produce wrong results","opus-5.5-xhigh":"Correct; public docs omit every precondition, plus noisy wrapping","muse-wonder-full-xhigh":"clean and correct; docs/tests a notch below the leaders","gpt-6.1-sol-max":"Ordinary coding is exact, but large-table bounds and overflow costs need correction.","deepseek-v4.1-flash-max":"solid implementation with the weakest public documentation of the contract"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"ICDF index wraps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public preconditions are largely absent","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public methods don't state their preconditions or rejection behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicate check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Wrapping operations on constants","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable fallbacks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Missing preconditions in method docs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Observer-only rejection checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead fallback in correction lookup","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Signed ICDF index escapes the table","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Deferred overflow writes remain linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public argument limits are missing","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"preconditions and error behaviour undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"back cursor advanced before the write is validated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge after `InvalidArgument`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":39,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.3,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":3.8,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":56.67,"tokens":{"uncached_input":212168,"cache_read":2350796,"cache_write":0,"output":38416,"reasoning":14373},"cost_usd":2.0872,"cost_breakdown":{"input":0.848672,"cache_read":0.470159,"cache_write":0.0,"output":0.76832},"cost_usd_as_run":2.0872,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1518,"code":1349,"comment":87,"blank":82,"role":"impl+test"}],"impl_lines":662,"impl_code":556,"impl_comment":50,"test_lines":856,"comment_ratio":0.083,"tests_declared":16},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.98,"reference":12.34,"speedup":1.236,"baselines":{"oracle":12.34}},{"id":"dec","candidate":11.86,"reference":11.69,"speedup":0.986,"baselines":{"oracle":11.69}}]},"scores":{"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct ordinary paths, with truncated storage and redundant work","opus-5.5-xhigh":"Correct; parts of the test suite are hedged and can't fail, plus redundant stores","muse-wonder-full-xhigh":"sound mid-pack entry; one masked sticky assertion","gpt-6.1-sol-max":"Exact core, but pending-rule tests mask failures and overflow work is avoidable.","deepseek-v4.1-flash-max":"good docs, but three test assertions are written so they cannot fail"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF preconditions are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Pending decode is cleared twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Hedged and dead assertions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`decode_uint` clears `pending` twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`icdf.is_empty() ||` duplicates the trailing-zero check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable fallbacks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Masked sticky-order assertion plus observer-only snapshots","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"No-op bookkeeping","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Deferred overflow writes remain linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Pending-preservation assertions are masked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Pending state is cleared twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"self-weakening assertions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"duplicate pending reset","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"back cursor advanced before the write is validated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"unwarranted \"approximately\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge after `InvalidArgument`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":32,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.7,"per_seat":{"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.5}},"defect_points":3.0,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":57.33,"tokens":{"uncached_input":162641,"cache_read":1476736,"cache_write":0,"output":229183,"reasoning":105595},"cost_usd":2.4387,"cost_breakdown":{"input":0.325282,"cache_read":0.738368,"cache_write":0.0,"output":1.375098},"cost_usd_as_run":2.4387,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1433,"code":1267,"comment":51,"blank":115,"role":"impl+test"}],"impl_lines":702,"impl_code":600,"impl_comment":42,"test_lines":731,"comment_ratio":0.065,"tests_declared":15},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.13,"reference":9.92,"speedup":0.979,"baselines":{"oracle":9.92}},{"id":"dec","candidate":11.87,"reference":11.81,"speedup":0.995,"baselines":{"oracle":11.81}}]},"scores":{"gpt-6.1-terra-max":6.5,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Large-buffer defect, avoidable carry work, and incomplete documentation","opus-5.5-xhigh":"Correct; thin public docs, a misleading comment and helper noise","muse-wonder-full-xhigh":"bit-exact but over-defended, with two comments the code contradicts","gpt-6.1-sol-max":"Exact core, but overflow cost, contract documentation, and randomized coverage need work.","deepseek-v4.1-flash-max":"careful pending-decode model and tests; docs state no preconditions and one comment contradicts SPEC §2"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public argument limits are omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Range invariant is inverted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public methods lack preconditions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The `finish` comment overstates its gate","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable guard","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"General-purpose shift and mask helpers for in-range values","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Vacuous malformed-table assertion","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Flush comment contradicted by check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Wrong helper contract","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Needless defensive layer on validated paths","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable guards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Malformed-table test passes by rejection","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Deferred overflow writes remain linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Garbage test never selects raw-bit decoding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public preconditions are largely undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Safety comments contradict the implementation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"rationale contradicts the contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no method documents its precondition","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable guard","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"branchy shift helpers on hot paths","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":1.5,"per_seat":{"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":3.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":0.5}},"defect_points":3.8,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":81.21,"tokens":{"uncached_input":129922,"cache_read":2796536,"cache_write":0,"output":36632,"reasoning":14338},"cost_usd":1.8116,"cost_breakdown":{"input":0.519688,"cache_read":0.559307,"cache_write":0.0,"output":0.73264},"cost_usd_as_run":1.8116,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1287,"code":1125,"comment":79,"blank":83,"role":"impl+test"}],"impl_lines":683,"impl_code":565,"impl_comment":61,"test_lines":604,"comment_ratio":0.097,"tests_declared":14},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.01,"reference":9.97,"speedup":0.996,"baselines":{"oracle":9.97}},{"id":"dec","candidate":11.87,"reference":11.77,"speedup":0.992,"baselines":{"oracle":11.77}}]},"scores":{"gpt-6.1-terra-max":6.5,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Two large-input correctness and safety defects","opus-5.5-xhigh":"Correct, well tested; internal invariants undocumented and some unreachable branches","muse-wonder-full-xhigh":"solid and readable; precondition tests cannot see register corruption","gpt-6.1-sol-max":"Ordinary coding is exact, but the malformed-table index guarantee fails.","deepseek-v4.1-flash-max":"correct and readable; dead defensive code and an unclear doc line"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"ICDF index wraps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF preconditions are incomplete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Invariants undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable fallback branches","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Wrapping arithmetic on values that cannot overflow","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Fresh-coder, observer-only rejection checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Fragmented impl and dead stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable decrement-back branch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Signed ICDF index becomes an invalid `usize`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Deferred overflow writes remain linear","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no malformed-table or finish-boundary tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable/dead guards","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"garbled probability line","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"needless rebinding","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge after `InvalidArgument`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":33,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":3.4,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":3.67,"tokens":{"uncached_input":14,"cache_read":345058,"cache_write":63499,"output":28957,"reasoning":6286},"cost_usd":1.1562,"cost_breakdown":{"input":5.6e-05,"cache_read":0.069012,"cache_write":0.507992,"output":0.57914},"cost_usd_as_run":1.1562,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1403,"code":1225,"comment":90,"blank":88,"role":"impl+test"}],"impl_lines":664,"impl_code":535,"impl_comment":78,"test_lines":739,"comment_ratio":0.127,"tests_declared":17},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.86,"reference":9.81,"speedup":0.995,"baselines":{"oracle":9.81}},{"id":"dec","candidate":11.52,"reference":11.52,"speedup":1.0,"baselines":{"oracle":11.52}}]},"scores":{"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":9.0,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Sound ordinary-path behavior, with a large-buffer defect and avoidable carry work","opus-5.5-xhigh":"Correct, with strong white-box tests; small dead guards in `decode_icdf`","muse-wonder-full-xhigh":"precise docs and strong randomized tests; only dead guards remain","gpt-6.1-sol-max":"Exact core, with avoidable overflow work and two test-quality defects.","deepseek-v4.1-flash-max":"good code and tests, but ships an extra file and carries dead decode logic"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF preconditions are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Unreachable test branch","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable termination guards","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable write fallbacks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Internal state undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable guards in ICDF search","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable test branch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Deferred overflow writes remain linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Alleged flush-overflow case never overflows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unreachable raw-width branch","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"extra file in the submission","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"dead conditions in `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"table read twice in `encode_icdf`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge after `InvalidArgument`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1205,"lines":50,"significant":415,"window":6},"complexity":{"functions":42,"mean":2.79,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.2,"slop":{"score":0.8,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.5}},"defect_points":2.7,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.01,"tokens":{"uncached_input":34300,"cache_read":406144,"cache_write":0,"output":21195,"reasoning":5464},"cost_usd":0.3212,"cost_breakdown":{"input":0.0686,"cache_read":0.040614,"cache_write":0.0,"output":0.21195},"cost_usd_as_run":0.3212,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1295,"code":1152,"comment":54,"blank":89,"role":"impl+test"}],"impl_lines":634,"impl_code":527,"impl_comment":52,"test_lines":661,"comment_ratio":0.09,"tests_declared":14},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.04,"reference":9.94,"speedup":0.99,"baselines":{"oracle":9.94}},{"id":"dec","candidate":11.67,"reference":11.31,"speedup":0.969,"baselines":{"oracle":11.31}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":9.0,"gpt-6.1-sol-max":10.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong tests and carry handling, with a large-buffer defect","opus-5.5-xhigh":"Correct, the only entry that bulk-fills 0xFF runs, a dedicated counter-wrap test; `State` undocumented","muse-wonder-full-xhigh":"excellent docs and white-box tests; unchecked indexing is the only blemish","gpt-6.1-sol-max":"Exact behavior, efficient deferred-run flushing, clear contracts, and strong misuse tests.","deepseek-v4.1-flash-max":"sound and well-tested; several guards for states that cannot occur"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is truncated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"`State` and its fields undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"Wrapping where values are bounded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"Unchecked indexing after wrapping budget sum","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable `rng == 0` handling","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"correctness","title":"padding/merge runs after a non-overflow error","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0397,"lines":16,"significant":403,"window":6},"complexity":{"functions":46,"mean":2.41,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.8,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":1.1,"score_cross_vendor":8.667,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":18.73,"tokens":{"uncached_input":42,"cache_read":2284639,"cache_write":191953,"output":278830,"reasoning":250294},"cost_usd":0.5645,"cost_breakdown":{"input":1.4e-05,"cache_read":0.10399,"cache_write":0.122378,"output":0.338091},"cost_usd_as_run":0.5645,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1336,"code":1143,"comment":93,"blank":100,"role":"impl+test"}],"impl_lines":689,"impl_code":558,"impl_comment":74,"test_lines":647,"comment_ratio":0.117,"tests_declared":14},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.99,"reference":9.86,"speedup":0.987,"baselines":{"oracle":9.86}},{"id":"dec","candidate":11.58,"reference":11.33,"speedup":0.978,"baselines":{"oracle":11.33}}]},"scores":{"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":8.5,"gpt-6.1-terra-max":6.0},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact; several false doc statements","deepseek-v4.1-flash-max":"correct and well tested; two public docs are false","opus-5.5-xhigh":"correct and well tested, but two public docs state wrong facts","gpt-6.1-sol-max":"Correct behavior, with misleading public documentation and avoidable carry work.","gpt-6.1-terra-max":"Ordinary coding works, but buffer handling, carry flushing, and public docs need changes"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"BITRES documented as a raw-bits precision","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"tell() rounding direction wrong","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"encoder impl split across three blocks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"pending decode cleared then immediately overwritten","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"`BITRES` documented as something else entirely","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"`tell` documented as rounding the wrong way","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`CORRECTION.get(..).unwrap_or(0)`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"`BITRES` is described wrongly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"`tell` is documented as \"rounded down\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Each coder's impl is split into three blocks mixed with free functions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Dead store","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dropped carry bytes still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Resolution and rounding are misstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are visited individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Observer documentation is wrong","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1174,"lines":50,"significant":426,"window":6},"complexity":{"functions":48,"mean":2.5,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":2.1,"per_seat":{"opus-5.5-xhigh":2.5,"muse-wonder-full-xhigh":3.5,"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":2.5,"gpt-6.1-sol-max":1.0}},"defect_points":3.6,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.67,"tokens":{"uncached_input":14686,"cache_read":400896,"cache_write":0,"output":31201,"reasoning":12748},"cost_usd":0.3815,"cost_breakdown":{"input":0.029372,"cache_read":0.04009,"cache_write":0.0,"output":0.31201},"cost_usd_as_run":0.3815,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1496,"code":1325,"comment":68,"blank":103,"role":"impl+test"}],"impl_lines":658,"impl_code":534,"impl_comment":67,"test_lines":838,"comment_ratio":0.111,"tests_declared":20},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.1,"reference":9.9,"speedup":0.98,"baselines":{"oracle":9.9}},{"id":"dec","candidate":11.67,"reference":11.33,"speedup":0.971,"baselines":{"oracle":11.33}}]},"scores":{"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":9.5,"gpt-6.1-sol-max":9.0,"gpt-6.1-terra-max":6.5},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact, defensive, register-level tests","deepseek-v4.1-flash-max":"clean and thorough; barely anything to fix","opus-5.5-xhigh":"correct, never panics, accurate docs, thorough tests (including counter wrap)","gpt-6.1-sol-max":"Robust implementation with useful boundary tests and one avoidable carry cost.","gpt-6.1-terra-max":"Sound ordinary-case behavior, with a large-buffer limit and a slow carry flush"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"pending decode cleared then immediately overwritten","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"two-phase validation split awkwardly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Dead store","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dropped carry bytes still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are visited individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1235,"lines":50,"significant":405,"window":6},"complexity":{"functions":49,"mean":2.43,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.4,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.5,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":39.92,"tokens":{"uncached_input":294,"cache_read":8539964,"cache_write":221024,"output":202644,"reasoning":0},"cost_usd":0.2761,"cost_breakdown":{"input":4.4e-05,"cache_read":0.136639,"cache_write":0.044205,"output":0.095243},"cost_usd_as_run":0.2761,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2761,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1526,"code":1252,"comment":171,"blank":103,"role":"impl+test"}],"impl_lines":715,"impl_code":537,"impl_comment":122,"test_lines":811,"comment_ratio":0.185,"tests_declared":21},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.2,"reference":10.17,"speedup":0.997,"baselines":{"oracle":10.17}},{"id":"dec","candidate":12.03,"reference":11.57,"speedup":0.962,"baselines":{"oracle":11.57}}]},"scores":{"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":4.0,"gpt-6.1-terra-max":3.5},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact, defensive, well documented","deepseek-v4.1-flash-max":"excellent docs and tests; one counter can overflow a debug build","opus-5.5-xhigh":"clean, well documented and well tested, but a bit counter can overflow and panic, against the spec and its own docs","gpt-6.1-sol-max":"Ordinary behavior matches, but unchecked counters violate the no-panic contract.","gpt-6.1-terra-max":"Long bit streams and large decoder tables can panic"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"plain `i32` additions on the bit counter","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"pending decode cleared then immediately overwritten","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"ftb bound expressed through an unrelated constant","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"i32 counter overflow panics after 2^31 bits","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"wrong constant used for the ICDF bound","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"`nbits_total` uses plain `+=`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`UINT_BITS` is reused as the ICDF `ftb` limit","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Dead store","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Bit counters overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dropped carry bytes still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Bit counters overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"ICDF search index overflows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are visited individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":54,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.4,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":4.3,"score_cross_vendor":6.4,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":54.55,"tokens":{"uncached_input":326573,"cache_read":11934272,"cache_write":0,"output":244436,"reasoning":197244},"cost_usd":0.5292,"cost_breakdown":{"input":0.048986,"cache_read":0.358028,"cache_write":0.0,"output":0.122218},"cost_usd_as_run":0.5292,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1599,"code":1195,"comment":294,"blank":110,"role":"impl+test"}],"impl_lines":821,"impl_code":538,"impl_comment":226,"test_lines":778,"comment_ratio":0.296,"tests_declared":16},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.04,"reference":9.9,"speedup":0.986,"baselines":{"oracle":9.9}},{"id":"dec","candidate":12.1,"reference":11.56,"speedup":0.955,"baselines":{"oracle":11.56}}]},"scores":{"muse-wonder-full-xhigh":8.0,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":4.0,"gpt-6.1-terra-max":4.0},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact; finish documentation contradicts the code","deepseek-v4.1-flash-max":"the most complete documentation, but \"total\" is not true for very long streams","opus-5.5-xhigh":"correct and heavily documented, but the counter can overflow and panic, which its own docs say cannot happen","gpt-6.1-sol-max":"Ordinary streams match, but long valid sequences panic.","gpt-6.1-terra-max":"Long sequences panic, and two large-input boundaries are incorrect"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"finish() doc claims overflow-ever gating that the code does not implement","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"robustness","title":"plain `i32` additions on the bit counter","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"one State struct overloads field meanings per side","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"robustness","title":"i32 counter overflow panics after 2^31 bits","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"the module doc promises what the line above breaks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`cancel_pending()` wrapper","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"robustness","title":"`nbits_total` uses plain `+=`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The `decode` doc contradicts itself","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Bit counters overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dropped carry bytes still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Incorrect padding comment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Bit counters overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Large ICDF symbol is truncated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are visited individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":62,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.3,"slop":{"score":1.0,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":2.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":0.5}},"defect_points":4.8,"score_cross_vendor":6.3,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":9.76,"tokens":{"uncached_input":11674,"cache_read":307712,"cache_write":0,"output":23457,"reasoning":5767},"cost_usd":1.5973,"cost_breakdown":{"input":0.11674,"cache_read":0.307712,"cache_write":0.0,"output":1.17285},"cost_usd_as_run":1.5973,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1415,"code":1243,"comment":80,"blank":92,"role":"impl+test"}],"impl_lines":634,"impl_code":508,"impl_comment":76,"test_lines":781,"comment_ratio":0.13,"tests_declared":20},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.96,"reference":9.9,"speedup":0.994,"baselines":{"oracle":9.9}},{"id":"dec","candidate":11.71,"reference":11.34,"speedup":0.968,"baselines":{"oracle":11.34}}]},"scores":{"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":8.5,"opus-5.5-xhigh":9.5,"gpt-6.1-sol-max":10.0,"gpt-6.1-terra-max":7.5},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact with the only batched carry flush","deepseek-v4.1-flash-max":"excellent, one dead branch","opus-5.5-xhigh":"correct and never panics, with accurate docs and thorough tests (including counter wrap); flushes runs of carried 0xFF bytes in one batch","gpt-6.1-sol-max":"Complete, well-tested implementation with bounded dropped-carry handling.","gpt-6.1-terra-max":"Strong ordinary-case implementation with a large-buffer limit and one overconstrained test"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"pending decode cleared then immediately overwritten","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable fallback after the ICDF search","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Dead store","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0967,"lines":38,"significant":393,"window":6},"complexity":{"functions":42,"mean":2.6,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.9,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":0.9,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.17,"tokens":{"uncached_input":8157,"cache_read":33792,"cache_write":0,"output":237,"reasoning":85},"cost_usd":0.011,"cost_breakdown":{"input":0.008157,"cache_read":0.00169,"cache_write":0.0,"output":0.001185},"cost_usd_as_run":0.011,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":0,"code":0,"comment":0,"blank":0,"role":"impl+test"}],"impl_lines":0,"impl_code":0,"impl_comment":0,"test_lines":0,"comment_ratio":0.0,"tests_declared":0},"hidden":{"passed":0,"total":11,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[]},"scores":{"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"gpt-6.1-terra-max":0.0},"council_summaries":{"muse-wonder-full-xhigh":"deliverable file missing, crate does not build","deepseek-v4.1-flash-max":"the deliverable is missing","opus-5.5-xhigh":"the deliverable is missing; the crate does not build","gpt-6.1-sol-max":"The required implementation is missing.","gpt-6.1-terra-max":"The implementation is absent"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"src/range_coder.rs missing entirely","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`src/range_coder.rs` does not exist","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"No `src/range_coder.rs`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"critical","category":"correctness","title":"Missing deliverable prevents compilation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"Missing deliverable","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":0,"window":6},"complexity":null,"files":[]},"isolation":{"tool_calls":1,"flags":0,"audited":"2026-11-02","reviewed":false},"score":0.0,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":4.0,"score_cross_vendor":0.0,"check":{"lint":{"diagnostics":4}}},{"letter":"U","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.52,"tokens":{"uncached_input":9790,"cache_read":115584,"cache_write":0,"output":6447,"reasoning":23},"cost_usd":0.5358,"cost_breakdown":{"input":0.0979,"cache_read":0.115584,"cache_write":0.0,"output":0.32235},"cost_usd_as_run":0.5358,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":823,"code":821,"comment":2,"blank":0,"role":"impl+test"}],"impl_lines":458,"impl_code":456,"impl_comment":2,"test_lines":365,"comment_ratio":0.004,"tests_declared":6},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.86,"reference":10.0,"speedup":1.014,"baselines":{"oracle":10.0}},{"id":"dec","candidate":11.7,"reference":11.42,"speedup":0.976,"baselines":{"oracle":11.42}}]},"scores":{"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":7.5,"gpt-6.1-terra-max":5.5},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact; documentation opted out entirely","deepseek-v4.1-flash-max":"correct code with the documentation requirement switched off","opus-5.5-xhigh":"matches the oracle exactly and never panics, but the public API has no docs at all and the code is very terse","gpt-6.1-sol-max":"Behavior is sound; documentation is largely absent.","gpt-6.1-terra-max":"Ordinary coding works, but large buffers, carry flushing, and documentation need work"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"missing-docs lint suppressed and public API undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"cryptic naming and Debug-as-Display","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"unused Clone derive on the shared state","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"all documentation suppressed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`Display` delegates to `Debug`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No docs on any public item, and the docs lint is switched off","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"Single-letter names make the code hard to maintain","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`Display` prints the variant name","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The malformed-table test never checks the returned index","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dropped carry bytes still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are visited individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0469,"lines":16,"significant":341,"window":6},"complexity":{"functions":41,"mean":2.66,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.6,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":3.8,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.78,"tokens":{"uncached_input":11015,"cache_read":285568,"cache_write":0,"output":11117,"reasoning":849},"cost_usd":0.0809,"cost_breakdown":{"input":0.011015,"cache_read":0.014278,"cache_write":0.0,"output":0.055585},"cost_usd_as_run":0.0809,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":940,"code":901,"comment":32,"blank":7,"role":"impl+test"}],"impl_lines":540,"impl_code":501,"impl_comment":32,"test_lines":400,"comment_ratio":0.06,"tests_declared":7},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.05,"reference":9.84,"speedup":0.979,"baselines":{"oracle":9.84}},{"id":"dec","candidate":11.78,"reference":11.6,"speedup":0.985,"baselines":{"oracle":11.6}}]},"scores":{"muse-wonder-full-xhigh":7.5,"deepseek-v4.1-flash-max":7.0,"opus-5.5-xhigh":8.0,"gpt-6.1-sol-max":9.0,"gpt-6.1-terra-max":7.0},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact but under-documented","deepseek-v4.1-flash-max":"correct but under-documented and lightly tested","opus-5.5-xhigh":"correct, never panics, adequate tests; thin docs and some redundancy","gpt-6.1-sol-max":"Correct, compact implementation with efficient dropped-carry handling.","gpt-6.1-terra-max":"Handles exhausted carry runs efficiently, but limits large buffers and underspecifies the API"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"public methods state no preconditions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"thin edge coverage beyond the goldens","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"magic numbers in place of named constants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"pending decode cleared then immediately overwritten","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"public method docs omit the preconditions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable fallback after the ICDF loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`overflowed()` accessor for a private field","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"only 7 tests, the thinnest of the well-formed suites","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public docs omit preconditions and error behaviour","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Always-true guard and trivial wrapper","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Dead store","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public method contracts are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Preconditions are omitted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0992,"lines":38,"significant":383,"window":6},"complexity":{"functions":42,"mean":2.69,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.7,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":2.7,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.27,"tokens":{"uncached_input":16,"cache_read":493728,"cache_write":77684,"output":40991,"reasoning":15937},"cost_usd":1.5401,"cost_breakdown":{"input":6.4e-05,"cache_read":0.098746,"cache_write":0.621472,"output":0.81982},"cost_usd_as_run":1.5401,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1535,"code":1335,"comment":98,"blank":102,"role":"impl+test"}],"impl_lines":694,"impl_code":559,"impl_comment":79,"test_lines":841,"comment_ratio":0.124,"tests_declared":20},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.78,"reference":9.83,"speedup":1.005,"baselines":{"oracle":9.83}},{"id":"dec","candidate":11.61,"reference":11.43,"speedup":0.984,"baselines":{"oracle":11.43}}]},"scores":{"muse-wonder-full-xhigh":9.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":9.0,"gpt-6.1-terra-max":6.5},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact, clean state split, thorough misuse tests","deepseek-v4.1-flash-max":"solid and correct; a couple of redundant guards","opus-5.5-xhigh":"correct, never panics, well documented and tested; two trivial redundancies","gpt-6.1-sol-max":"Comprehensive validation and tests; deferred overflow work can be bounded.","gpt-6.1-terra-max":"Correct on ordinary inputs, with a large-buffer limit and a slow carry flush"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"pending decode cleared then immediately overwritten","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"unreachable extra break condition in the ICDF search","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"double bounds check on the front read","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"redundant loop bound in `decode_icdf`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable loop guard","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Dead store","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dropped carry bytes still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are visited individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1106,"lines":48,"significant":434,"window":6},"complexity":{"functions":46,"mean":2.57,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.3,"slop":{"score":0.6,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0}},"defect_points":1.8,"score_cross_vendor":8.125,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.7,"tokens":{"uncached_input":13559,"cache_read":206720,"cache_write":0,"output":6687,"reasoning":91},"cost_usd":0.1147,"cost_breakdown":{"input":0.027118,"cache_read":0.020672,"cache_write":0.0,"output":0.06687},"cost_usd_as_run":0.1147,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":762,"code":760,"comment":1,"blank":1,"role":"impl+test"}],"impl_lines":459,"impl_code":457,"impl_comment":1,"test_lines":303,"comment_ratio":0.002,"tests_declared":5},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.0,"reference":9.88,"speedup":0.988,"baselines":{"oracle":9.88}},{"id":"dec","candidate":11.65,"reference":11.52,"speedup":0.989,"baselines":{"oracle":11.52}}]},"scores":{"muse-wonder-full-xhigh":6.5,"deepseek-v4.1-flash-max":6.0,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":7.5,"gpt-6.1-terra-max":5.5},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact; thinnest tests and no documentation","deepseek-v4.1-flash-max":"correct but undocumented and the thinnest test suite","opus-5.5-xhigh":"matches the oracle exactly and never panics, but the public API is undocumented and the tests are the thinnest","gpt-6.1-sol-max":"Correct behavior, but almost no API documentation.","gpt-6.1-terra-max":"Ordinary coding works, but large buffers, carry flushing, and documentation need work"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"missing-docs lint suppressed and public API undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"random round-trip covers only degenerate shapes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"cryptic naming and Debug-as-Display","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"all documentation suppressed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`Display` delegates to `Debug`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"only 5 tests, and rejection tests check observers but never the output buffer","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No docs on any public item, and the docs lint is switched off","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin random coverage","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`Display` prints the variant name","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dropped carry bytes still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"correctness","title":"Buffer length is capped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are visited individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Overflow bytes are asserted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0468,"lines":16,"significant":342,"window":6},"complexity":{"functions":37,"mean":2.76,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.3,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0}},"defect_points":4.1,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.16,"tokens":{"uncached_input":26786,"cache_read":243456,"cache_write":0,"output":5780,"reasoning":251},"cost_usd":0.0679,"cost_breakdown":{"input":0.026786,"cache_read":0.012173,"cache_write":0.0,"output":0.0289},"cost_usd_as_run":0.0679,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":606,"code":605,"comment":1,"blank":0,"role":"impl+test"}],"impl_lines":481,"impl_code":480,"impl_comment":1,"test_lines":125,"comment_ratio":0.002,"tests_declared":4},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.15,"reference":10.11,"speedup":0.996,"baselines":{"oracle":10.11}},{"id":"dec","candidate":11.75,"reference":11.39,"speedup":0.969,"baselines":{"oracle":11.39}}]},"scores":{"opus-5.5-xhigh":5.5,"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":6.5},"council_summaries":{"opus-5.5-xhigh":"behaviourally correct, but undocumented with the docs lint suppressed, thin tests, and cryptic code","gpt-6.1-terra-max":"Bit-exact in the probes, but underdocumented and lightly tested","deepseek-v4.1-flash-max":"Bit-exact and hang-free, but the public API is undocumented and the tests cover only four happy paths.","gpt-6.1-sol-max":"Correct in probes, but documentation and regression coverage are substantially incomplete.","muse-wonder-full-xhigh":"Bit-exact and safe, but undocumented and barely tested"},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"no public item is documented, and the scaffold's `missing_docs` lint is switched off","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"the tests are thin","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"terse naming and packed layout","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"unreachable guards","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"redundant code","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Carry propagation repeats dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Important contract cases lack tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"docs","title":"Public API documentation is suppressed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Pending state is cleared immediately before replacement","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"`#![allow(missing_docs)]` with no doc comments anywhere","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"only 4 tests, all happy-path vectors","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"unreachable break condition","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Discarded carry runs still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Four tests leave major contract areas uncovered","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"docs","title":"Missing API documentation is suppressed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"No documentation, lint suppressed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Only 4 tests, whole areas uncovered","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Cryptic names and magic numbers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead disjunct in `decode_icdf`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.124,"lines":46,"significant":371,"window":6},"complexity":{"functions":38,"mean":2.76,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.5,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0}},"defect_points":5.5,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.92,"tokens":{"uncached_input":11768,"cache_read":390144,"cache_write":0,"output":20732,"reasoning":6026},"cost_usd":0.1349,"cost_breakdown":{"input":0.011768,"cache_read":0.019507,"cache_write":0.0,"output":0.10366},"cost_usd_as_run":0.1349,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1253,"code":1097,"comment":71,"blank":85,"role":"impl+test"}],"impl_lines":629,"impl_code":509,"impl_comment":71,"test_lines":624,"comment_ratio":0.122,"tests_declared":13},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.11,"reference":10.08,"speedup":0.997,"baselines":{"oracle":10.08}},{"id":"dec","candidate":11.56,"reference":11.45,"speedup":0.99,"baselines":{"oracle":11.45}}]},"scores":{"opus-5.5-xhigh":8.5,"gpt-6.1-terra-max":9.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":9.5},"council_summaries":{"opus-5.5-xhigh":"oracle-exact with a correct batched 0xFF-run flush and good tests; minor defensive noise","gpt-6.1-terra-max":"Strong coverage and efficient carry handling, with one minor redundancy","deepseek-v4.1-flash-max":"Bit-exact, safe and well documented; tests are strong and I found no material defect.","gpt-6.1-sol-max":"Correct and efficient carry handling; randomized coverage has a verified blind spot.","muse-wonder-full-xhigh":"Bit-exact, full-state misuse tests, one unreachable return"},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"unreachable `rng == 0` guards and zero-divisor fallbacks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`pending = None` is immediately overwritten","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"magic numbers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Pending state is cleared immediately before replacement","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Garbage test repeatedly selects a restricted operation pattern","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"ICDF fallback cannot execute","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unreachable fallback in `decode_icdf`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0964,"lines":38,"significant":394,"window":6},"complexity":{"functions":41,"mean":2.68,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.1,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.5}},"defect_points":0.8,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-opus-5.5-xhigh-r1-b2","task":"range-coder","lead":"opus-5.5-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":6.51,"tokens":{"cache_read":2353792,"cache_write":0,"output":24560,"reasoning":13751,"uncached_input":99834},"cost_usd":0.3403,"cost_breakdown":{"cache_read":0.11769,"cache_write":0.0,"input":0.099834,"output":0.1228},"cost_usd_as_run":0.3403,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6-astra-xhigh","judge":"gpt-6-astra-xhigh","model":"gpt-6-astra","effort":"xhigh","label":"GPT‑6 Astra xhigh","usage":{"wall_minutes":6.75,"tokens":{"cache_read":1571456,"cache_write":0,"output":14137,"reasoning":4981,"uncached_input":99120},"cost_usd":3.2695,"cost_breakdown":{"cache_read":1.571456,"cache_write":0.0,"input":0.9912,"output":0.70685},"cost_usd_as_run":3.2695,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":11.06,"tokens":{"cache_read":8382720,"cache_write":0,"output":129046,"reasoning":86926,"uncached_input":106825},"cost_usd":0.1186,"cost_breakdown":{"cache_read":0.025148,"cache_write":0.0,"input":0.016024,"output":0.077428},"cost_usd_as_run":0.1186,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":42.26,"tokens":{"cache_read":3708712,"cache_write":0,"output":47691,"reasoning":28309,"uncached_input":212650},"cost_usd":2.5462,"cost_breakdown":{"cache_read":0.741742,"cache_write":0.0,"input":0.8506,"output":0.95382},"cost_usd_as_run":2.5462,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":12.99,"tokens":{"cache_read":3108736,"cache_write":0,"output":26317,"reasoning":14856,"uncached_input":126623},"cost_usd":0.8273,"cost_breakdown":{"cache_read":0.310874,"cache_write":0.0,"input":0.253246,"output":0.26317},"cost_usd_as_run":0.8273,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":24.59,"tokens":{"cache_read":5725160,"cache_write":205687,"output":56005,"reasoning":29612,"uncached_input":76},"cost_usd":3.9109,"cost_breakdown":{"cache_read":1.145032,"cache_write":1.645496,"input":0.000304,"output":1.1201},"cost_usd_as_run":3.9109,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"J","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Z","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"J","score":8.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Z","score":8.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"J","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Z","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"J","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Z","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"J","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Z","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":10.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"J","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Z","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":9.5}],"panel":["gpt-6.1-terra-max","deepseek-v4.1-flash-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","opus-5.5-xhigh"]},"entries":[{"letter":"W","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":8.82,"tokens":{"uncached_input":71307,"cache_read":371173,"cache_write":0,"output":36588,"reasoning":16871},"cost_usd":0.5456,"cost_breakdown":{"input":0.142614,"cache_read":0.037117,"cache_write":0.0,"output":0.36588},"cost_usd_as_run":0.5456,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1293,"code":1114,"comment":63,"blank":116,"role":"impl+test"}],"impl_lines":611,"impl_code":528,"impl_comment":34,"test_lines":682,"comment_ratio":0.06,"tests_declared":12},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":7.85,"reference":10.17,"speedup":1.296,"baselines":{"oracle":10.17}},{"id":"dec","candidate":8.8,"reference":11.64,"speedup":1.323,"baselines":{"oracle":11.64}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6-astra-xhigh":9.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":7.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Good ordinary-case coverage, but a long-run oracle mismatch and incomplete API documentation remain","gpt-6-astra-xhigh":"Correct implementation with useful tests, but incomplete API documentation and avoidable overflow work.","deepseek-v4.1-flash-max":"bit-exact and well structured, but its public contract is undocumented","muse-wonder-full-xhigh":"bit-exact code held back by missing contract docs and the thinnest tests","gpt-6.1-sol-max":"Oracle-compatible with useful edge tests, but incomplete API documentation and avoidable overflow work","opus-5.5-xhigh":"clean, correct code, but public docs leave out the contract and the tests are shallower"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"correctness","title":"Carry counter has a different width from the oracle","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Dropped carry bytes are processed one at a time","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public methods omit essential preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Predecessor lookup cannot fail","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Repeats writes after the buffer fills","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Public methods omit essential preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Public methods document no preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Decoder is never tested on arbitrary input","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`finish` does not say the buffer is unspecified on error","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"`let mut t = s;` / `let mut idx = 0usize;` are dead initial values","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Unreachable `get_mut` fallbacks in the byte writers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Unreachable latch in `encode_icdf`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Public docs omit all numeric preconditions and rejected-call semantics","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Plain arithmetic relies on undocumented invariants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Encoder reject harness compares only metrics and 4 buffer bytes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Malformed-table test never covers `ftb` 6..=8","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Delayed carries are flushed individually into a full buffer","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Public methods omit important contract requirements","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public methods don't document their preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Shallow rejection and random tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Plain subtraction relies on uncommented invariants","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable guards","status":"valid","detail":"","location":""}],"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5}},"defect_points":3.8,"score_cross_vendor":7.8,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-range-coder-opus-5.5-xhigh-r1","task":"range-coder","lead":"opus-5.5-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":2,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":51.51,"tokens":{"cache_read":31838337,"cache_write":1283320,"output":292747,"reasoning":182377,"uncached_input":302},"cost_usd":22.4905,"cost_breakdown":{"cache_read":6.367667,"cache_write":10.26656,"input":0.001208,"output":5.85494},"cost_usd_as_run":22.4905,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316","sessions":4}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":53.0,"tokens":{"cache_read":26439936,"cache_write":0,"output":127947,"reasoning":61334,"uncached_input":584154},"cost_usd":2.5459,"cost_breakdown":{"cache_read":1.321996,"cache_write":0.0,"input":0.584154,"output":0.639735},"cost_usd_as_run":2.5459,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":59.15,"tokens":{"cache_read":69803776,"cache_write":0,"output":576263,"reasoning":417697,"uncached_input":753066},"cost_usd":0.6682,"cost_breakdown":{"cache_read":0.209411,"cache_write":0.0,"input":0.11296,"output":0.345757},"cost_usd_as_run":0.6682,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":4}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":85.73,"tokens":{"cache_read":20642432,"cache_write":0,"output":152801,"reasoning":89268,"uncached_input":714249},"cost_usd":5.0209,"cost_breakdown":{"cache_read":2.064242,"cache_write":0.0,"input":1.428498,"output":1.52801},"cost_usd_as_run":5.0209,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"gpt-6-astra-xhigh","judge":"gpt-6-astra-xhigh","model":"gpt-6-astra","effort":"xhigh","label":"GPT‑6 Astra xhigh","usage":{"wall_minutes":83.54,"tokens":{"cache_read":18962560,"cache_write":0,"output":98045,"reasoning":39208,"uncached_input":796667},"cost_usd":31.8315,"cost_breakdown":{"cache_read":18.96256,"cache_write":0.0,"input":7.96667,"output":4.90225},"cost_usd_as_run":31.8315,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":143.18,"tokens":{"cache_read":21407704,"cache_write":0,"output":164931,"reasoning":82466,"uncached_input":872407},"cost_usd":11.0698,"cost_breakdown":{"cache_read":4.281541,"cache_write":0.0,"input":3.489628,"output":3.29862},"cost_usd_as_run":11.0698,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":10.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":8.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":9.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"V","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":10.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"V","score":8.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"V","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Y","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"V","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":3.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"V","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reT","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"Y","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reT","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":10.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"J","score":4.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-10-31-range-coder-opus-5.5-xhigh","letter":"C","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"B","score":8.0}],"panel":["opus-5.5-xhigh","gpt-6.1-terra-max","deepseek-v4.1-flash-max","gpt-6.1-sol-max","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":16.48,"tokens":{"uncached_input":47811,"cache_read":461824,"cache_write":0,"output":33013,"reasoning":15021},"cost_usd":0.4719,"cost_breakdown":{"input":0.095622,"cache_read":0.046182,"cache_write":0.0,"output":0.33013},"cost_usd_as_run":0.4719,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1409,"code":1252,"comment":57,"blank":100,"role":"impl+test"}],"impl_lines":602,"impl_code":504,"impl_comment":51,"test_lines":807,"comment_ratio":0.092,"tests_declared":26},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.46,"reference":11.07,"speedup":1.309,"baselines":{"oracle":11.07}},{"id":"dec","candidate":9.43,"reference":11.69,"speedup":1.24,"baselines":{"oracle":11.69}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":9.5,"gpt-6-astra-xhigh":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with avoidable carry work and minor documentation issues","opus-5.5-xhigh":"Bit-exact and robust; the most exhaustive tests (raw-window boundary sweep, full register and byte state compares after every rejection); clear docs","deepseek-v4.1-flash-max":"Bit-exact, with excellent docs and tests; one comment justifies a subtraction with an impossible overflow.","gpt-6.1-sol-max":"Correct, clearly documented, and extensively tested; carry overflow handling can improve.","gpt-6-astra-xhigh":"Correct with particularly good boundary tests, but avoidable carry work remains.","muse-wonder-full-xhigh":"bit-exact with the broadest tests; icdf precondition under-specified in docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries retry dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF upper bound is omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Test comment asserts unspecified error behavior","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"comment claims offset addition could overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Carry flushing performs redundant failed writes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"performance","title":"Full-buffer carry flushing remains linear","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"misleading-docs","title":"Offset-overflow explanation contradicts the invariant","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"encode_icdf doc incomplete","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0921,"lines":36,"significant":391,"window":6},"complexity":{"functions":39,"mean":2.67,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.8,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.9,"tokens":{"uncached_input":13204,"cache_read":179584,"cache_write":0,"output":8781,"reasoning":194},"cost_usd":0.1322,"cost_breakdown":{"input":0.026408,"cache_read":0.017958,"cache_write":0.0,"output":0.08781},"cost_usd_as_run":0.1322,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":951,"code":913,"comment":32,"blank":6,"role":"impl+test"}],"impl_lines":528,"impl_code":491,"impl_comment":31,"test_lines":423,"comment_ratio":0.059,"tests_declared":20},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.18,"reference":10.21,"speedup":1.248,"baselines":{"oracle":10.21}},{"id":"dec","candidate":9.06,"reference":11.9,"speedup":1.313,"baselines":{"oracle":11.9}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":9.5,"gpt-6-astra-xhigh":8.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with a slow carry loop and incomplete API documentation","opus-5.5-xhigh":"Bit-exact and robust; reasonable tests; minimal docs","deepseek-v4.1-flash-max":"Bit-exact and strongly tested; docs are thin and the carry loop is more intricate than it needs to be.","gpt-6.1-sol-max":"Correct and well tested, with incomplete public contract documentation.","gpt-6-astra-xhigh":"Correct, with modest documentation and carry-flush improvements needed.","muse-wonder-full-xhigh":"bit-exact and safe; terse docs and one masked random test"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Carry runs are emitted byte by byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF preconditions are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Rejection tests require bytes after an error","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Minimal docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Random round trips skip overflowing buffers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"preconditions are absent from the public docs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"robustness","title":"`carry`'s early exit duplicates the overflow check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"ICDF requirements are unexplained","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Carry runs are emitted through repeated checked writes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Public methods omit enforced contracts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Public docs omit preconditions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Masked random round-trip","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0635,"lines":24,"significant":378,"window":6},"complexity":{"functions":40,"mean":2.65,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.8,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.62,"tokens":{"uncached_input":19965,"cache_read":256896,"cache_write":0,"output":12447,"reasoning":709},"cost_usd":1.0789,"cost_breakdown":{"input":0.19965,"cache_read":0.256896,"cache_write":0.0,"output":0.62235},"cost_usd_as_run":1.0789,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1151,"code":1044,"comment":38,"blank":69,"role":"impl+test"}],"impl_lines":576,"impl_code":493,"impl_comment":36,"test_lines":575,"comment_ratio":0.068,"tests_declared":22},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.28,"reference":10.18,"speedup":1.229,"baselines":{"oracle":10.18}},{"id":"dec","candidate":9.28,"reference":12.01,"speedup":1.294,"baselines":{"oracle":12.01}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":9.5,"gpt-6-astra-xhigh":8.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with an inefficient carry fill and an incomplete precondition","opus-5.5-xhigh":"Bit-exact and robust; thorough register-level tests; public docs state argument ranges","deepseek-v4.1-flash-max":"Bit-exact, excellent tests, clean code; the rejection rules for two methods are undocumented.","gpt-6.1-sol-max":"Correct and well tested, with a small documentation omission.","gpt-6-astra-xhigh":"Correct and well tested, with minor performance and documentation gaps.","muse-wonder-full-xhigh":"bit-exact with strong seeded tests; one incomplete precondition doc"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Carry runs are written one byte at a time","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF interval requirements are incomplete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Register invariants are not documented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`encode_icdf` and `update` document neither the table nor the interval rule","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Selected-interval ICDF rules are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Carry flushing rechecks an already bounded count","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Sequencing and ICDF requirements are incomplete","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"encode_icdf doc incomplete","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0942,"lines":36,"significant":382,"window":6},"complexity":{"functions":40,"mean":2.58,"max":11,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.6,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":22.96,"tokens":{"uncached_input":80,"cache_read":7379552,"cache_write":244245,"output":330619,"reasoning":296368},"cost_usd":0.8881,"cost_breakdown":{"input":3.4e-05,"cache_read":0.361199,"cache_write":0.213595,"output":0.313309},"cost_usd_as_run":0.8881,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1420,"code":1172,"comment":150,"blank":98,"role":"impl+test"}],"impl_lines":707,"impl_code":525,"impl_comment":129,"test_lines":713,"comment_ratio":0.197,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.8,"reference":10.27,"speedup":1.167,"baselines":{"oracle":10.27}},{"id":"dec","candidate":10.13,"reference":11.85,"speedup":1.17,"baselines":{"oracle":11.85}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.5,"gpt-6.1-sol-max":9.0,"gpt-6-astra-xhigh":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Well tested and correct in the probes, with a costly carry path and documentation errors","opus-5.5-xhigh":"Bit-exact and robust; table-driven tests of every precondition row; thorough docs on private fields and helpers","deepseek-v4.1-flash-max":"Complete and accurate documentation, oracle-exact behaviour and an exhaustive misuse suite; no findings.","gpt-6.1-sol-max":"Strong behavior and tests, with wasted carry work and a false invariant.","gpt-6-astra-xhigh":"Correct and well tested, with avoidable carry work and incorrect invariant comments.","muse-wonder-full-xhigh":"bit-exact, full Requires docs, most explicit pending-decode tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries retry dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Encoder ICDF contract omits a bound","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Range invariant excludes a permitted state","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The `val` field doc is wrong about where the carry sits","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Carry flushing retries dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Documented decoder range bound is false","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"performance","title":"Carry flushing repeats writes after exhaustion","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Register bounds are overstated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Dead bounds re-check in write_front","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"tell_frac range claim too narrow","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0695,"lines":28,"significant":403,"window":6},"complexity":{"functions":45,"mean":2.47,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":46,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.8,"slop":{"score":0.6,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":1.4,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.69,"tokens":{"uncached_input":17569,"cache_read":513024,"cache_write":0,"output":26072,"reasoning":11083},"cost_usd":0.1736,"cost_breakdown":{"input":0.017569,"cache_read":0.025651,"cache_write":0.0,"output":0.13036},"cost_usd_as_run":0.1736,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1248,"code":1118,"comment":39,"blank":91,"role":"impl+test"}],"impl_lines":599,"impl_code":518,"impl_comment":34,"test_lines":649,"comment_ratio":0.062,"tests_declared":25},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.84,"reference":10.96,"speedup":1.24,"baselines":{"oracle":10.96}},{"id":"dec","candidate":9.2,"reference":11.69,"speedup":1.271,"baselines":{"oracle":11.69}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":9.5,"gpt-6-astra-xhigh":9.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct and efficient in the probes, with a test assumption and minor cleanup needed","opus-5.5-xhigh":"Bit-exact and robust in every differential and misuse check; strong tests; terse docs and one dead branch","deepseek-v4.1-flash-max":"Bit-exact with 25 strong tests, but one unreachable branch and a public API that never states its preconditions.","gpt-6.1-sol-max":"Correct, efficient, and well tested; public ICDF documentation needs detail.","gpt-6-astra-xhigh":"Correct, efficient and thoroughly tested; public contracts need more detail.","muse-wonder-full-xhigh":"bit-exact with field-level reject tests; docs omit preconditions"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Rejection tests compare unspecified buffers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF method omits its accepted domain","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Impossible previous-entry failure branch","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public docs leave out preconditions and register invariants","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Rejection branch in encode_icdf can never run","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"unreachable `else` in `encode_icdf`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"public methods never document their preconditions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"`finish` doc omits the zero-fill/leftover-bit merge and the \"buffer unspecified on error\" rule","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"ICDF requirements are unexplained","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Important rejection conditions are omitted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Public docs omit preconditions","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0948,"lines":38,"significant":401,"window":6},"complexity":{"functions":39,"mean":2.74,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.6,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.7,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":6.28,"tokens":{"uncached_input":24,"cache_read":887312,"cache_write":84639,"output":46267,"reasoning":19530},"cost_usd":1.78,"cost_breakdown":{"input":9.6e-05,"cache_read":0.177462,"cache_write":0.677112,"output":0.92534},"cost_usd_as_run":1.78,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1440,"code":1210,"comment":137,"blank":93,"role":"impl+test"}],"impl_lines":686,"impl_code":523,"impl_comment":116,"test_lines":754,"comment_ratio":0.182,"tests_declared":19},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.41,"reference":10.58,"speedup":1.258,"baselines":{"oracle":10.58}},{"id":"dec","candidate":9.45,"reference":12.11,"speedup":1.281,"baselines":{"oracle":12.11}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":9.5,"gpt-6-astra-xhigh":8.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with a costly carry loop and an overconstrained test","opus-5.5-xhigh":"Bit-exact and robust; strong tests, including an arbitrary-call fuzz test and per-size finish sweeps; well-documented constants","deepseek-v4.1-flash-max":"Bit-exact, well documented and very thoroughly tested; only two harmless defensive no-ops.","gpt-6.1-sol-max":"Thorough implementation and tests, with avoidable overflow-path work.","gpt-6-astra-xhigh":"Correct with broad tests, but carry flushing and one invariant comment need attention.","muse-wonder-full-xhigh":"bit-exact, fully documented preconditions, strong misuse/fuzz tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries retry dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Rejection tests compare unspecified buffers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Stale sentinel in a field doc","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`tell_frac` masks values that are already in range","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`write_byte` re-checks a bound it already knows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Carry flushing visits every dropped byte","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"performance","title":"Carry flushing processes every dropped byte","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"First-byte consumption is reversed","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0587,"lines":24,"significant":409,"window":6},"complexity":{"functions":39,"mean":2.67,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":16,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.9,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.1,"score_cross_vendor":8.875,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.17,"tokens":{"uncached_input":48445,"cache_read":432768,"cache_write":0,"output":11525,"reasoning":1334},"cost_usd":0.1277,"cost_breakdown":{"input":0.048445,"cache_read":0.021638,"cache_write":0.0,"output":0.057625},"cost_usd_as_run":0.1277,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":936,"code":899,"comment":30,"blank":7,"role":"impl+test"}],"impl_lines":527,"impl_code":490,"impl_comment":30,"test_lines":409,"comment_ratio":0.058,"tests_declared":17},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.21,"reference":10.43,"speedup":1.27,"baselines":{"oracle":10.43}},{"id":"dec","candidate":9.4,"reference":12.12,"speedup":1.289,"baselines":{"oracle":12.12}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":9.5,"gpt-6-astra-xhigh":9.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and efficient in the probes, with incomplete public documentation","opus-5.5-xhigh":"Bit-exact and robust; thin docs and narrow misuse tests","deepseek-v4.1-flash-max":"Bit-exact and competently built, but the public contract is undocumented and misuse tests cover one case per method.","gpt-6.1-sol-max":"Correct and efficient, with terse public contract documentation.","gpt-6-astra-xhigh":"Correct and efficient, with incomplete API documentation.","muse-wonder-full-xhigh":"bit-exact; single-variant misuse tests and terse docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF requirements are not documented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Misuse tests check one case per method","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"One-line docs with no preconditions or invariants","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"preconditions are absent from every public doc","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"rejection coverage is one case per method","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`s.saturating_add(1)` cannot saturate","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"ICDF requirements are unexplained","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Preconditions and sequencing are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Terse docs omit preconditions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"One bad variant per method","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0947,"lines":36,"significant":380,"window":6},"complexity":{"functions":38,"mean":2.71,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.7,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.27,"tokens":{"uncached_input":12572,"cache_read":171776,"cache_write":0,"output":6984,"reasoning":21},"cost_usd":0.6467,"cost_breakdown":{"input":0.12572,"cache_read":0.171776,"cache_write":0.0,"output":0.3492},"cost_usd_as_run":0.6467,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":853,"code":821,"comment":30,"blank":2,"role":"impl+test"}],"impl_lines":509,"impl_code":477,"impl_comment":30,"test_lines":344,"comment_ratio":0.059,"tests_declared":12},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.1,"reference":10.23,"speedup":1.124,"baselines":{"oracle":10.23}},{"id":"dec","candidate":10.31,"reference":11.93,"speedup":1.157,"baselines":{"oracle":11.93}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":9.0,"gpt-6-astra-xhigh":8.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but deferred carries can cause avoidable linear work","opus-5.5-xhigh":"Bit-exact and robust; minimal docs and the thinnest test suite among the passing entries","deepseek-v4.1-flash-max":"Bit-exact with decent tests, but preconditions are undocumented and the misuse/sweep coverage is partial.","gpt-6.1-sol-max":"Correct implementation with minor performance and documentation shortcomings.","gpt-6-astra-xhigh":"Correct, but carry handling and sparse documentation need improvement.","muse-wonder-full-xhigh":"bit-exact; thinnest tests and docs among the solid entries"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries retry dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF preconditions are omitted","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Few misuse cases","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Minimal docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Zero-fill uses an index loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"preconditions are absent from the public docs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the finish sweep never checks the size threshold","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Carry flushing retries dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"ICDF requirements are unexplained","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"Public documentation omits essential limits","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Terse docs omit preconditions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin random coverage","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0929,"lines":34,"significant":366,"window":6},"complexity":{"functions":40,"mean":2.55,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.3,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.22,"tokens":{"uncached_input":7976,"cache_read":80256,"cache_write":0,"output":690,"reasoning":131},"cost_usd":0.0154,"cost_breakdown":{"input":0.007976,"cache_read":0.004013,"cache_write":0.0,"output":0.00345},"cost_usd_as_run":0.0154,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":27,"code":17,"comment":6,"blank":4,"role":"impl+test"}],"impl_lines":27,"impl_code":17,"impl_comment":6,"test_lines":0,"comment_ratio":0.261,"tests_declared":0},"hidden":{"passed":0,"total":11,"failed":[]},"perf":{"correct":false,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[]},"scores":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.0,"gpt-6-astra-xhigh":0.0,"muse-wonder-full-xhigh":0.5},"council_summaries":{"gpt-6.1-terra-max":"The encoder and decoder are missing","opus-5.5-xhigh":"Stub: no RangeEncoder or RangeDecoder","deepseek-v4.1-flash-max":"Only the error type exists; the coder itself is missing.","gpt-6.1-sol-max":"The required encoder and decoder are absent.","gpt-6-astra-xhigh":"The required component is missing.","muse-wonder-full-xhigh":"stub: error type only, both coders missing"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"Required coder API is absent","status":"valid","defect":"RC-012","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No tests exercise the required interface","status":"valid","defect":"RC-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Encoder and decoder are missing","status":"valid","defect":"RC-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"no `RangeEncoder` and no `RangeDecoder`","status":"valid","defect":"RC-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"no tests exist","status":"valid","defect":"RC-013","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Required API is missing","status":"valid","defect":"RC-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"critical","category":"correctness","title":"Neither coder exists","status":"valid","defect":"RC-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"RangeEncoder/RangeDecoder absent","status":"valid","defect":"RC-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"tests","title":"No tests","status":"valid","defect":"RC-013","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":{"functions":1,"mean":3.0,"max":3,"over_10":0,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":3,"flags":0,"audited":"2026-11-02","reviewed":false},"score":0.4,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":5.4,"score_cross_vendor":0.667,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":38.24,"tokens":{"uncached_input":171398,"cache_read":2017544,"cache_write":0,"output":36477,"reasoning":15294},"cost_usd":1.8186,"cost_breakdown":{"input":0.685592,"cache_read":0.403509,"cache_write":0.0,"output":0.72954},"cost_usd_as_run":1.8186,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1362,"code":1143,"comment":125,"blank":94,"role":"impl+test"}],"impl_lines":697,"impl_code":540,"impl_comment":109,"test_lines":665,"comment_ratio":0.168,"tests_declared":21},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.94,"reference":10.35,"speedup":1.158,"baselines":{"oracle":10.35}},{"id":"dec","candidate":9.1,"reference":12.31,"speedup":1.353,"baselines":{"oracle":12.31}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.5,"gpt-6-astra-xhigh":7.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with avoidable carry work and misleading tests and documentation","opus-5.5-xhigh":"Bit-exact and robust; good docs and precondition tests; defensive dead code","deepseek-v4.1-flash-max":"Bit-exact, complete precondition docs, strong tests; duplicated interval computation and dead defensive branches.","gpt-6.1-sol-max":"Correct behavior, but redundant work and misleading comments weaken maintainability.","gpt-6-astra-xhigh":"Correct behavior, but unnecessary work and misleading defensive code reduce quality.","muse-wonder-full-xhigh":"bit-exact, full docs, most assertions; unreachable defensive branches"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred carries retry dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Decode documentation forbids accepted calls","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Rejection tests compare unspecified output","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Guards for states that cannot occur","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"encode_icdf fetches the same entries twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Small random coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"`encode_icdf` computes `prev`/`cur` twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"unreachable defensive branches","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"`decode`'s doc contradicts the pending-decode contract","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Carry flushing retries dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Sequencing documentation contradicts accepted behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Recovery branches handle unreachable states","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"performance","title":"Exhausted carry runs still take linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Decoder sequencing documentation contradicts acceptance behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"dead-code","title":"Recovery branches handle unreachable register states","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Dead invariant guards","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0574,"lines":24,"significant":418,"window":6},"complexity":{"functions":40,"mean":2.83,"max":18,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":22,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":1.6,"per_seat":{"opus-5.5-xhigh":1.5,"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":3.0,"gpt-6.1-sol-max":1.5,"muse-wonder-full-xhigh":1.0}},"defect_points":2.5,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.89,"tokens":{"uncached_input":50889,"cache_read":314496,"cache_write":0,"output":20269,"reasoning":4814},"cost_usd":0.3359,"cost_breakdown":{"input":0.101778,"cache_read":0.03145,"cache_write":0.0,"output":0.20269},"cost_usd_as_run":0.3359,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1318,"code":1172,"comment":53,"blank":93,"role":"impl+test"}],"impl_lines":600,"impl_code":504,"impl_comment":50,"test_lines":718,"comment_ratio":0.09,"tests_declared":24},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.38,"reference":10.41,"speedup":1.242,"baselines":{"oracle":10.41}},{"id":"dec","candidate":9.2,"reference":12.11,"speedup":1.316,"baselines":{"oracle":12.11}}]},"scores":{"gpt-6.1-terra-max":10.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":10.0,"gpt-6-astra-xhigh":10.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Concise implementation with strong contract and misuse tests","opus-5.5-xhigh":"exact, accurate concise docs, strong full-state tests","deepseek-v4.1-flash-max":"Bit-exact and very well tested; one inaccurate `tell()` doc line.","gpt-6.1-sol-max":"Exact, efficient, and supported by strong state-sensitive tests.","gpt-6-astra-xhigh":"Strongest overall verification, including the long-stream counter boundary.","muse-wonder-full-xhigh":"bit-exact and well tested; public docs omit rejection behaviour"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"`tell()` claims termination overhead is included","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Rejected-call behaviour undocumented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0923,"lines":36,"significant":390,"window":6},"complexity":{"functions":38,"mean":2.68,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.3,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.3,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.26,"tokens":{"uncached_input":39157,"cache_read":203264,"cache_write":0,"output":12695,"reasoning":1149},"cost_usd":0.2256,"cost_breakdown":{"input":0.078314,"cache_read":0.020326,"cache_write":0.0,"output":0.12695},"cost_usd_as_run":0.2256,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1083,"code":1034,"comment":38,"blank":11,"role":"impl+test"}],"impl_lines":544,"impl_code":502,"impl_comment":35,"test_lines":539,"comment_ratio":0.065,"tests_declared":20},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.48,"reference":10.41,"speedup":1.228,"baselines":{"oracle":10.41}},{"id":"dec","candidate":9.19,"reference":12.25,"speedup":1.333,"baselines":{"oracle":12.25}}]},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":10.0,"gpt-6-astra-xhigh":10.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Sound sampled behavior; ICDF and corruption documentation is incomplete","opus-5.5-xhigh":"exact, compact, good private-state tests; terse docs","deepseek-v4.1-flash-max":"Bit-exact with strong twin-based tests; precondition docs are thin and one comparison widens to `u64` pointlessly.","gpt-6.1-sol-max":"Compact, exact, and efficient, with strong private-state regression checks.","gpt-6-astra-xhigh":"Correct, concise implementation with bounded overflow handling and effective tests.","muse-wonder-full-xhigh":"bit-exact and well tested; public docs omit rejection behaviour"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public methods omit material contract details","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"The finish doc says it \"zeroes padding\" unconditionally and doesn't say the buffer is unspecified on error; the module doc is one line","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`encode_icdf` and `update` do not state their preconditions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"pointless widening in `carry_out`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Rejected-call behaviour undocumented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0928,"lines":36,"significant":388,"window":6},"complexity":{"functions":40,"mean":2.62,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.9,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.8,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":25.3,"tokens":{"uncached_input":60,"cache_read":3666010,"cache_write":150191,"output":96410,"reasoning":60204},"cost_usd":3.8632,"cost_breakdown":{"input":0.00024,"cache_read":0.733202,"cache_write":1.201528,"output":1.9282},"cost_usd_as_run":3.8632,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1588,"code":1268,"comment":204,"blank":116,"role":"impl+test"}],"impl_lines":727,"impl_code":512,"impl_comment":164,"test_lines":861,"comment_ratio":0.243,"tests_declared":25},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.26,"reference":10.31,"speedup":1.248,"baselines":{"oracle":10.31}},{"id":"dec","candidate":9.13,"reference":11.81,"speedup":1.294,"baselines":{"oracle":11.81}}]},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":9.5,"deepseek-v4.1-flash-max":9.5,"gpt-6.1-sol-max":9.0,"gpt-6-astra-xhigh":9.5,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Well-documented and broadly tested, with avoidable carry-run latency","opus-5.5-xhigh":"exact, precise invariant docs, clean structure, tests that catch real bugs","deepseek-v4.1-flash-max":"Bit-exact with an unusually rigorous suite (blinded register and byte twins, `tell` bounds, invariant fuzz); nothing found.","gpt-6.1-sol-max":"Exact behavior and strong tests; unnecessary work when carry bytes cannot fit.","gpt-6-astra-xhigh":"Thorough validation and tests; discarded carry bytes need not be processed individually.","muse-wonder-full-xhigh":"bit-exact, precise docs and invariants, cold reject paths, strong tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Retries every dropped carry byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Dropped carry bytes still incur linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Deferred carry loop scales with discarded output","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0602,"lines":24,"significant":399,"window":6},"complexity":{"functions":42,"mean":2.48,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":31,"flags":2,"audited":"2026-11-02","reviewed":true},"score":9.3,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.4,"score_cross_vendor":9.25,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":13.01,"tokens":{"uncached_input":28041,"cache_read":595968,"cache_write":0,"output":27413,"reasoning":8724},"cost_usd":2.247,"cost_breakdown":{"input":0.28041,"cache_read":0.595968,"cache_write":0.0,"output":1.37065},"cost_usd_as_run":2.247,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1481,"code":1323,"comment":58,"blank":100,"role":"impl+test"}],"impl_lines":593,"impl_code":493,"impl_comment":55,"test_lines":888,"comment_ratio":0.1,"tests_declared":27},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.07,"reference":10.37,"speedup":1.285,"baselines":{"oracle":10.37}},{"id":"dec","candidate":9.26,"reference":11.73,"speedup":1.267,"baselines":{"oracle":11.73}}]},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.5,"gpt-6.1-sol-max":9.0,"gpt-6-astra-xhigh":9.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong contract coverage, with avoidable carry-run latency","opus-5.5-xhigh":"exact, accurate docs, thorough state-level tests","deepseek-v4.1-flash-max":"Bit-exact, the most test coverage (27 tests, real register twins), precise docs; nothing found.","gpt-6.1-sol-max":"Correct and thoroughly tested, with one avoidable overflow-path cost.","gpt-6-astra-xhigh":"Correct, extensively tested implementation with one avoidable overflow-path loop.","muse-wonder-full-xhigh":"bit-exact and well tested; public docs omit rejection behaviour"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Retries every dropped carry byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Carry flushing continues after exhaustion","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Exhausted buffers still trigger per-byte carry flushing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Rejected-call behaviour undocumented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.094,"lines":36,"significant":383,"window":6},"complexity":{"functions":38,"mean":2.68,"max":13,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.0,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.6,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":15.37,"tokens":{"uncached_input":32445,"cache_read":7264384,"cache_write":0,"output":132073,"reasoning":102677},"cost_usd":0.1059,"cost_breakdown":{"input":0.004867,"cache_read":0.021793,"cache_write":0.0,"output":0.079244},"cost_usd_as_run":0.1059,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1059,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1570,"code":1275,"comment":190,"blank":105,"role":"impl+test"}],"impl_lines":740,"impl_code":548,"impl_comment":144,"test_lines":830,"comment_ratio":0.208,"tests_declared":22},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.7,"reference":10.27,"speedup":1.18,"baselines":{"oracle":10.27}},{"id":"dec","candidate":10.21,"reference":12.0,"speedup":1.175,"baselines":{"oracle":12.0}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":8.0,"gpt-6-astra-xhigh":9.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Sampled behavior matches, but several comments misstate the contract","opus-5.5-xhigh":"exact and thoroughly tested, but several public docs are false","deepseek-v4.1-flash-max":"Bit-exact and heavily tested; two inverted bit-probability doc sentences and a dead `finish` branch.","gpt-6.1-sol-max":"Correct implementation weakened by misleading docs, duplicated logic, and a misleading test.","gpt-6-astra-xhigh":"Strong correctness and tests, with inaccurate probability docs and avoidable overflow work.","muse-wonder-full-xhigh":"bit-exact and well tested, but bit_logp probability is documented inverted"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Retries every dropped carry byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Module documentation contradicts rejected-call behavior","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Bit probability is assigned to the wrong value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Decoder value is described from the wrong end","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Fractional accounting is duplicated","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The bit_logp docs invert the probability","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Module and decoder docs contradict the contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The finish comment names the wrong merge byte","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"tell_frac is duplicated verbatim in both coders, with dead masking","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"bit probability is inverted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable clamp in `finish`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Failed carry writes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Bit probabilities are reversed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Fractional-bit arithmetic is duplicated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Empty-input test rejects its own updates","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Reversed bit probability","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Repeated writes after capacity is exhausted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"bit_logp probability stated backwards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"tell_frac body duplicated verbatim in encoder and decoder","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1302,"lines":56,"significant":430,"window":6},"complexity":{"functions":40,"mean":2.92,"max":15,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":55,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.9,"slop":{"score":2.3,"per_seat":{"opus-5.5-xhigh":3.0,"gpt-6.1-terra-max":3.0,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":2.0,"muse-wonder-full-xhigh":2.0}},"defect_points":2.9,"score_cross_vendor":7.75,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":65.08,"tokens":{"uncached_input":209023,"cache_read":4141312,"cache_write":0,"output":283095,"reasoning":124580},"cost_usd":4.1873,"cost_breakdown":{"input":0.418046,"cache_read":2.070656,"cache_write":0.0,"output":1.69857},"cost_usd_as_run":4.1873,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1815,"code":1576,"comment":122,"blank":117,"role":"impl+test"}],"impl_lines":739,"impl_code":574,"impl_comment":111,"test_lines":1076,"comment_ratio":0.162,"tests_declared":21},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.86,"reference":13.35,"speedup":1.229,"baselines":{"oracle":13.35}},{"id":"dec","candidate":11.67,"reference":15.47,"speedup":1.326,"baselines":{"oracle":15.47}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.5,"gpt-6-astra-xhigh":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Broad coverage and correct probe results, with a slow carry flush and masked cases","opus-5.5-xhigh":"exact match to the reference, robust, good tests; minor dead code and verbose tests","deepseek-v4.1-flash-max":"equally bit-exact and well tested, with one unreachable branch and one wrong comment.","gpt-6.1-sol-max":"Correct and well tested, with inefficient carry flushing and duplicated test logic.","gpt-6-astra-xhigh":"Correct and well tested, with avoidable carry work and an unreachable recovery path.","muse-wonder-full-xhigh":"bit-exact with strong reject tests; outsized hand-unrolled test module"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Grouped invalid calls mask missing error latches","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Missing-terminator fallback cannot run","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"The stated ICDF loop invariant is false during a search","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Carry flushing repeatedly attempts impossible writes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"dead-code","title":"Missing-terminator recovery is unreachable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Missing-terminator branch can never run","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Golden A and C tests repeat the op list by hand","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"Unreachable \"missing terminator\" branch","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"Wrong shift range","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Carry runs keep attempting dropped writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Golden-A operations have three definitions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Test module is ~1070 lines with golden sequences spelled out repeatedly","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"encode_icdf validates then re-reads the same entries","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0541,"lines":24,"significant":444,"window":6},"complexity":{"functions":47,"mean":2.49,"max":8,"over_10":0,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":40,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.3,"slop":{"score":1.2,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.5}},"defect_points":2.2,"score_cross_vendor":8.3,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":50.97,"tokens":{"uncached_input":219003,"cache_read":4055260,"cache_write":0,"output":61502,"reasoning":34023},"cost_usd":2.9171,"cost_breakdown":{"input":0.876012,"cache_read":0.811052,"cache_write":0.0,"output":1.23004},"cost_usd_as_run":2.9171,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1604,"code":1304,"comment":194,"blank":106,"role":"impl+test"}],"impl_lines":784,"impl_code":570,"impl_comment":167,"test_lines":820,"comment_ratio":0.227,"tests_declared":24},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.37,"reference":10.53,"speedup":1.258,"baselines":{"oracle":10.53}},{"id":"dec","candidate":9.07,"reference":11.92,"speedup":1.314,"baselines":{"oracle":11.92}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":8.5,"gpt-6-astra-xhigh":9.0,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Strong coverage, with avoidable carry-run work and unreachable guards","opus-5.5-xhigh":"bit-exact and well tested; a little defensive padding","deepseek-v4.1-flash-max":"Bit-exact with excellent documentation and tests; a lot of documented can't-happen defensive code.","gpt-6.1-sol-max":"Behavior is exact; defensive dead paths obscure otherwise sound invariants.","gpt-6-astra-xhigh":"Correct implementation with a small performance issue and a narrow test gap.","muse-wonder-full-xhigh":"bit-exact, fully documented contract, thorough twin-style tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Retries every dropped carry byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Fractional-count fallback handles unreachable ranges","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Guards the code itself calls unreachable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"defensive branches that cannot be reached","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Carry runs waste work after overflow","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Unreachable fallback paths obscure invariants","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Carry flushing keeps visiting discarded bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Bit-decoder cancellation regression escapes the suite","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0545,"lines":24,"significant":440,"window":6},"complexity":{"functions":39,"mean":2.92,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":38,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.9,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.9,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":39.69,"tokens":{"uncached_input":192,"cache_read":4861953,"cache_write":186796,"output":150843,"reasoning":0},"cost_usd":0.1861,"cost_breakdown":{"input":2.9e-05,"cache_read":0.077791,"cache_write":0.037359,"output":0.070896},"cost_usd_as_run":0.1861,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0947,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1535,"code":1207,"comment":227,"blank":101,"role":"impl+test"}],"impl_lines":775,"impl_code":547,"impl_comment":174,"test_lines":760,"comment_ratio":0.241,"tests_declared":22},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.15,"reference":10.38,"speedup":1.274,"baselines":{"oracle":10.38}},{"id":"dec","candidate":9.6,"reference":11.94,"speedup":1.244,"baselines":{"oracle":11.94}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":6.5,"gpt-6-astra-xhigh":7.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Strong coverage, with an extreme counter overflow and avoidable carry-run work","opus-5.5-xhigh":"behaviour is exact, but the docs cite a different codec throughout and some tests check nothing","deepseek-v4.1-flash-max":"Bit-exact and well tested; the decoder divides `rng/ft` twice per symbol, and the docs cite `od_ec_*` names that are absent from the cited libopus sources.","gpt-6.1-sol-max":"Ordinary behavior matches, but counter exactness, misleading docs, and test blind spots need changes.","gpt-6-astra-xhigh":"Normal coding works, but counter behavior, masked tests, and misleading invariants need attention.","muse-wonder-full-xhigh":"bit-exact and well documented, but libopus symbol names in docs are invented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Carry counter can panic","status":"valid","defect":"RC-014","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Retries every dropped carry byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Pending-decode survival is not asserted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Decoder value is described as an interval bottom","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"misleading-docs","title":"Attributes Daala/AV1 `od_ec_*`/`OD_EC_*` names to libopus","status":"valid","defect":"RC-023","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Wrong explanations of a register and of the flush","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The pending-survival assertions check nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"update_raw recomputes `rng / ft`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unreachable else-branches in write_front, write_back and finish","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"redundant integer division on every range-coded symbol","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"many comments attribute the code to `od_ec_*`/`OD_EC_*` identifiers","status":"valid","defect":"RC-023","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Widened carry counter changes finalization","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Failed carry writes remain linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Sticky errors conceal lost pending state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Decoder invariant and upstream references are inaccurate","status":"valid","defect":"RC-023","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Carry count uses the wrong width","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Dropped carry bytes are processed individually","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Pending-rule checks reuse an already-failed decoder","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Decoder value invariant is described incorrectly","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Docs cite non-existent libopus names","status":"valid","defect":"RC-023","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"encode_icdf validates then re-reads the same entries","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0567,"lines":24,"significant":423,"window":6},"complexity":{"functions":46,"mean":2.46,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":38,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":1.5,"per_seat":{"opus-5.5-xhigh":3.5,"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.5}},"defect_points":3.6,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.27,"tokens":{"uncached_input":36418,"cache_read":180736,"cache_write":0,"output":8100,"reasoning":180},"cost_usd":0.086,"cost_breakdown":{"input":0.036418,"cache_read":0.009037,"cache_write":0.0,"output":0.0405},"cost_usd_as_run":0.086,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":808,"code":770,"comment":31,"blank":7,"role":"impl+test"}],"impl_lines":526,"impl_code":490,"impl_comment":31,"test_lines":282,"comment_ratio":0.06,"tests_declared":8},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.47,"reference":10.62,"speedup":1.254,"baselines":{"oracle":10.62}},{"id":"dec","candidate":9.43,"reference":12.11,"speedup":1.284,"baselines":{"oracle":12.11}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":7.0,"gpt-6-astra-xhigh":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Sound sampled behavior, with incomplete API docs and a masked sequencing test","opus-5.5-xhigh":"exact, but tests are thin and partly vacuous, and docs are bare","deepseek-v4.1-flash-max":"Bit-exact, but no precondition docs and the thinnest useful test suite among the correct entries.","gpt-6.1-sol-max":"Correct ordinary operation, but saturation breaks exactness and pending-state tests are masked.","gpt-6-astra-xhigh":"Ordinary behavior is correct, but counter saturation and weak contract documentation reduce confidence.","muse-wonder-full-xhigh":"functionally bit-exact, but undocumented preconditions and thin tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Public methods omit important preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Sticky error masks pending-decode survival","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Thin and partly vacuous tests","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Public docs state no preconditions and no rejection behaviour","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"frac's `l < 16` branch and `.min(7)` are unreachable","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Tests are split across three ad-hoc modules (tests, more_tests, validation_tests)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"preconditions and error semantics undocumented","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"`pending_and_invalid` cannot detect a broken pending rule","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"random test never exercises the range coder's own paths","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"unexplained indexing invariant","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"unreachable `frac` path","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"correctness","title":"Saturating carry count changes `finish()`","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Pending preservation is not actually asserted","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"correctness","title":"Saturation changes the required counter arithmetic","status":"valid","defect":"RC-015","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Sticky error masks pending-decode loss","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"API descriptions omit essential restrictions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Public methods document no preconditions","status":"valid","defect":"RC-016","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Thin, partially masked test coverage","status":"valid","defect":"RC-019","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Display prints the Debug variant name","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0955,"lines":36,"significant":377,"window":6},"complexity":{"functions":39,"mean":2.64,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":0.2,"per_seat":{"opus-5.5-xhigh":0.5,"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.2,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":53.29,"tokens":{"uncached_input":247782,"cache_read":9171968,"cache_write":0,"output":161039,"reasoning":136573},"cost_usd":0.3928,"cost_breakdown":{"input":0.037167,"cache_read":0.275159,"cache_write":0.0,"output":0.080519},"cost_usd_as_run":0.3928,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1651,"code":1304,"comment":242,"blank":105,"role":"impl+test"}],"impl_lines":699,"impl_code":473,"impl_comment":179,"test_lines":952,"comment_ratio":0.275,"tests_declared":21},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":10.2,"reference":10.02,"speedup":0.982,"baselines":{"oracle":10.02}},{"id":"dec","candidate":11.84,"reference":11.71,"speedup":0.989,"baselines":{"oracle":11.71}}]},"scores":{"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":6.5,"gpt-6-astra-xhigh":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Broad tests and documentation, but an extreme valid sequence panics","opus-5.5-xhigh":"passes every oracle check, but has wrong tell docs, vacuous pending tests and a counter that can panic","deepseek-v4.1-flash-max":"Bit-exact with excellent docs and tests, but the public hot methods are not `#[inline]`, so cross-crate callers are ~2x slower (7x on a tight loop).","gpt-6.1-sol-max":"Ordinary behavior is exact, but a reachable counter overflow violates robustness.","gpt-6-astra-xhigh":"Correct ordinary packets, but a reachable counter panic and ineffective tests need correction.","muse-wonder-full-xhigh":"bit-exact and well documented, except tell() rounding is stated backwards and hot methods are not inlinable"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"robustness","title":"Deferred carry counter can overflow","status":"valid","defect":"RC-014","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Retries every dropped carry byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Whole-bit usage is described as rounded down","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Sticky error masks pending-decode failures","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"robustness","title":"The 0xFF-run counter uses a checked `+=`","status":"valid","defect":"RC-014","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"tell is documented as \"rounded down\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The tests that a pending decode survives a rejected call are vacuous","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"no `#[inline]` on any public hot method","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"robustness","title":"Deferred-byte counter panics on overflow","status":"valid","defect":"RC-014","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Full-buffer carry flushing remains linear","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Sticky errors mask pending-decode loss","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"`tell()` rounding is documented backwards","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"robustness","title":"Deferred-carry increment can panic","status":"valid","defect":"RC-014","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Overflow flushing traverses discarded bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Pending-rule assertions can pass for unrelated reasons","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"`tell()` rounding is described backwards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"tell() documented as \"rounded down\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"No #[inline] on any public hot method","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0437,"lines":16,"significant":366,"window":6},"complexity":{"functions":39,"mean":2.62,"max":12,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":54,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":0.8,"per_seat":{"opus-5.5-xhigh":1.0,"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":3.2,"score_cross_vendor":7.4,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":4.41,"tokens":{"uncached_input":20,"cache_read":592687,"cache_write":65640,"output":30913,"reasoning":8446},"cost_usd":1.262,"cost_breakdown":{"input":8e-05,"cache_read":0.118537,"cache_write":0.52512,"output":0.61826},"cost_usd_as_run":1.262,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1257,"code":1060,"comment":118,"blank":79,"role":"impl+test"}],"impl_lines":660,"impl_code":517,"impl_comment":95,"test_lines":597,"comment_ratio":0.155,"tests_declared":14},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":8.44,"reference":10.86,"speedup":1.287,"baselines":{"oracle":10.86}},{"id":"dec","candidate":9.45,"reference":12.18,"speedup":1.289,"baselines":{"oracle":12.18}}]},"scores":{"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.5,"gpt-6-astra-xhigh":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"opus-5.5-xhigh":"Correct, well documented and well tested, but cluttered with defensive no-ops","deepseek-v4.1-flash-max":"bit-exact, panic-free, good libopus-referenced docs and broad tests; a cluster of dead defensive masks and one inconsistent unreachable arm.","gpt-6.1-terra-max":"Bit-exact in the probes, with an avoidable long carry flush and minor documentation issues.","gpt-6.1-sol-max":"Oracle-exact with broad tests; minor performance and documentation issues.","gpt-6-astra-xhigh":"Correct and thoroughly tested, with minor performance, documentation, and redundancy issues.","muse-wonder-full-xhigh":"Correct and robust, with minor docs and dead-defensiveness gaps"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Defensive operations that do nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"An unreachable `None` arm that would drop data silently","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"An `Option` field documented as \"-1 = none\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Masks that cannot trigger","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"More masked shifts","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Silent drop in `write_front`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`self.ext.max(1)`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred writes are retried after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"ICDF preconditions are incomplete","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Fractional accounting is an approximation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"The correction-table fallback cannot execute","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Overflowed carry runs still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"ICDF precision bounds are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Comment promises whole-table rejection","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Unbounded replay of dropped carry bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"docs","title":"ICDF argument limits are incompletely documented","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"redundancy","title":"Masks and fallbacks duplicate proven bounds","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Incomplete `encode_icdf` preconditions","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Unreachable defensive fallbacks","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0594,"lines":24,"significant":404,"window":6},"complexity":{"functions":40,"mean":2.62,"max":14,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.4,"slop":{"score":1.2,"per_seat":{"opus-5.5-xhigh":2.0,"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":2.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":2.4,"score_cross_vendor":8.375,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":40.15,"tokens":{"uncached_input":136090,"cache_read":2717188,"cache_write":0,"output":41502,"reasoning":19498},"cost_usd":1.9178,"cost_breakdown":{"input":0.54436,"cache_read":0.543438,"cache_write":0.0,"output":0.83004},"cost_usd_as_run":1.9178,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/range_coder.rs","lines":1349,"code":1178,"comment":79,"blank":92,"role":"impl+test"}],"impl_lines":649,"impl_code":539,"impl_comment":63,"test_lines":700,"comment_ratio":0.105,"tests_declared":17},"hidden":{"passed":11,"total":11,"failed":[]},"perf":{"correct":true,"host":"ai-agent (AMD Ryzen 9 9950X 16-Core Processor)","proxy":false,"metrics":[{"id":"enc","candidate":9.45,"reference":10.81,"speedup":1.144,"baselines":{"oracle":10.81}},{"id":"dec","candidate":9.21,"reference":12.16,"speedup":1.32,"baselines":{"oracle":12.16}}]},"scores":{"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":9.0,"gpt-6-astra-xhigh":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"opus-5.5-xhigh":"Correct, with good docs and tests, but the most dead code and blanket wrapping slop in the session","deepseek-v4.1-flash-max":"bit-exact and panic-free with a strong suite and explicit docs; only small dead defensive code.","gpt-6.1-terra-max":"Reference behavior held in the probes, but carry flushing and public ICDF documentation need attention.","gpt-6.1-sol-max":"Oracle-exact with strong tests; minor carry-flushing cost and redundant safeguards.","gpt-6-astra-xhigh":"Strong functional coverage, with avoidable carry work and redundant fallback logic.","muse-wonder-full-xhigh":"Sound and well tested, marred by layered unreachable defensiveness"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Several unreachable fallbacks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Blanket wrapping and checked arithmetic where invariants already bound the values","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"`decode` docs say it \"Returns the symbol about to be decoded\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"Unreachable index clamp","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"No-op saturating subtraction","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Defensive guard in `tell_frac_of`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Deferred writes are retried after the buffer is full","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The documented ICDF condition bounds the wrong quantity","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Corrupt integer behavior is omitted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"The ICDF overshoot guards are unreachable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Overflowed carry runs still require linear work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Unreachable ICDF overshoot handling","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Flushes every dropped carry byte","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"redundancy","title":"Impossible-state fallbacks obscure invariants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Random ICDF coverage skips cases","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Ext-zero fallback returns out-of-range value","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Admittedly unreachable branches","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Byte-at-a-time zero fill in `finish`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0574,"lines":24,"significant":418,"window":6},"complexity":{"functions":40,"mean":2.8,"max":15,"over_10":1,"tool":"lizard"},"files":["src/range_coder.rs"]},"isolation":{"tool_calls":34,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.5,"slop":{"score":1.4,"per_seat":{"opus-5.5-xhigh":2.5,"gpt-6.1-terra-max":1.5,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":2.5,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1-b2","task":"vp8lf-neon","lead":"gpt-6-astra-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":12.74,"tokens":{"cache_read":2314076,"cache_write":0,"output":24217,"reasoning":13008,"uncached_input":169768},"cost_usd":1.6262,"cost_breakdown":{"cache_read":0.462815,"cache_write":0.0,"input":0.679072,"output":0.48434},"cost_usd_as_run":1.6262,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":11.12,"tokens":{"cache_read":1677440,"cache_write":0,"output":23718,"reasoning":14874,"uncached_input":110797},"cost_usd":0.6265,"cost_breakdown":{"cache_read":0.167744,"cache_write":0.0,"input":0.221594,"output":0.23718},"cost_usd_as_run":0.6265,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":9.21,"tokens":{"cache_read":3370496,"cache_write":0,"output":23546,"reasoning":13462,"uncached_input":95046},"cost_usd":0.3813,"cost_breakdown":{"cache_read":0.168525,"cache_write":0.0,"input":0.095046,"output":0.11773},"cost_usd_as_run":0.3813,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":19.35,"tokens":{"cache_read":2583814,"cache_write":175184,"output":53832,"reasoning":38747,"uncached_input":42},"cost_usd":2.995,"cost_breakdown":{"cache_read":0.516763,"cache_write":1.401472,"input":0.000168,"output":1.07664},"cost_usd_as_run":2.995,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":11.97,"tokens":{"cache_read":6520704,"cache_write":0,"output":114656,"reasoning":84695,"uncached_input":106088},"cost_usd":0.1043,"cost_breakdown":{"cache_read":0.019562,"cache_write":0.0,"input":0.015913,"output":0.068794},"cost_usd_as_run":0.1043,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"V","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"C","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"O","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"V","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"C","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"O","score":10.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"V","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"C","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"O","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"V","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"C","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"O","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"V","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"C","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"O","score":9.5}],"panel":["muse-wonder-full-xhigh","gpt-6.1-sol-max","gpt-6.1-terra-max","opus-5.5-xhigh","deepseek-v4.1-flash-max"]},"entries":[{"letter":"K","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":24.27,"tokens":{"uncached_input":136097,"cache_read":1259493,"cache_write":0,"output":85325,"reasoning":49013},"cost_usd":1.2514,"cost_breakdown":{"input":0.272194,"cache_read":0.125949,"cache_write":0.0,"output":0.85325},"cost_usd_as_run":1.2514,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":441,"code":276,"comment":121,"blank":44},{"path":"tests/selftest.c","role":"test","lines":767,"code":653,"comment":69,"blank":45}],"impl_lines":441,"impl_code":276,"impl_comment":121,"test_lines":767,"comment_ratio":0.305,"tests_declared":4,"impl_trailing_comments":111},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.266,"reference":133.516,"speedup":18.374,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":133.516}},{"id":"h_mb","candidate":20.854,"reference":116.894,"speedup":5.605,"baselines":{"wpd_neon":20.77,"c_ref_apple_clang_O2":116.894}},{"id":"v_inner","candidate":12.553,"reference":129.974,"speedup":10.354,"baselines":{"wpd_neon":12.675,"c_ref_apple_clang_O2":129.974}},{"id":"h_inner","candidate":25.16,"reference":110.23,"speedup":4.381,"baselines":{"wpd_neon":26.215,"c_ref_apple_clang_O2":110.23}},{"id":"geomean","candidate":14.791,"reference":122.284,"speedup":8.268,"baselines":{"wpd_neon":14.606,"c_ref_apple_clang_O2":122.284}}]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":9.0,"gpt-6.1-sol-max":6.0,"gpt-6.1-terra-max":6.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"muse-wonder-full-xhigh":"bit-exact, well documented and tested; minor kernel/test polish only","gpt-6.1-sol-max":"Correct assembly, but the test wrapper cannot compile for Darwin.","gpt-6.1-terra-max":"Filter passes the shared tests, but its self-test fails the macOS build gate","opus-5.5-xhigh":"Same lean transpose design and accurate docs; slightly heavier MB arithmetic and minor test-harness gaps","deepseek-v4.1-flash-max":"Correct, well-structured and well-documented; the 6-tap delta computation is avoidably expensive"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"three independent smull pairs for 27/18/9","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"OR-merge invariant unstated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"ABI check covers x19–x28 but not x29","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"tests","title":"Inline assembly exhausts Darwin’s available registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Deterministic coverage claims exceed generated coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"The ABI test wrapper cannot allocate its input register on macOS","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Failed memory protection is reported as a passing guard test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The claimed exhaustive filter-value sweep is incomplete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard-page test can fail silently or without explanation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False comment on the guard allocation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"Exhaustive sweep of all 256 values of a … HEV (hev_thresh=0)\" overclaims","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The MB kernel converts all six taps to signed and back (12 `eor`) and runs three full smull/add/shrn chains","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"macroblock deltas cost 24 instructions where 14 suffice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"guard-page test only exercises two strides","status":"valid","detail":"","location":""}],"score":7.7,"slop":{"score":0.6,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":1.0,"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.3,"score_cross_vendor":7.7,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","task":"vp8lf-neon","lead":"gpt-6-astra-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":33.26,"tokens":{"cache_read":26453888,"cache_write":0,"output":84683,"reasoning":46879,"uncached_input":612143},"cost_usd":2.3583,"cost_breakdown":{"cache_read":1.322694,"cache_write":0.0,"input":0.612143,"output":0.423415},"cost_usd_as_run":2.3583,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":79.12,"tokens":{"cache_read":31854851,"cache_write":994133,"output":276511,"reasoning":171831,"uncached_input":334},"cost_usd":19.8556,"cost_breakdown":{"cache_read":6.37097,"cache_write":7.953064,"input":0.001336,"output":5.53022},"cost_usd_as_run":19.8556,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":103.69,"tokens":{"cache_read":12345800,"cache_write":0,"output":150033,"reasoning":85812,"uncached_input":894388},"cost_usd":9.0473,"cost_breakdown":{"cache_read":2.46916,"cache_write":0.0,"input":3.577552,"output":3.00066},"cost_usd_as_run":9.0473,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":54.77,"tokens":{"cache_read":21996032,"cache_write":0,"output":116842,"reasoning":74409,"uncached_input":733204},"cost_usd":4.8344,"cost_breakdown":{"cache_read":2.199603,"cache_write":0.0,"input":1.466408,"output":1.16842},"cost_usd_as_run":4.8344,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":64.7,"tokens":{"cache_read":38642048,"cache_write":0,"output":395878,"reasoning":305913,"uncached_input":473477},"cost_usd":0.4245,"cost_breakdown":{"cache_read":0.115926,"cache_write":0.0,"input":0.071022,"output":0.237526},"cost_usd_as_run":0.4245,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":10.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":10.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"A","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6-astra-xhigh-r1","letter":"B","score":7.5}],"panel":["gpt-6.1-terra-max","opus-5.5-xhigh","muse-wonder-full-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"A","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":12.5,"tokens":{"uncached_input":36,"cache_read":1557468,"cache_write":107627,"output":64238,"reasoning":38272},"cost_usd":2.4574,"cost_breakdown":{"input":0.000144,"cache_read":0.311494,"cache_write":0.861016,"output":1.28476},"cost_usd_as_run":2.4574,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":340,"code":222,"comment":104,"blank":14},{"path":"tests/selftest.c","role":"test","lines":819,"code":681,"comment":84,"blank":54}],"impl_lines":340,"impl_code":222,"impl_comment":104,"test_lines":819,"comment_ratio":0.319,"tests_declared":4,"impl_trailing_comments":74},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":6.71,"reference":133.558,"speedup":19.905,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.558}},{"id":"h_mb","candidate":18.082,"reference":116.694,"speedup":6.454,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":116.694}},{"id":"v_inner","candidate":12.831,"reference":129.584,"speedup":10.099,"baselines":{"wpd_neon":12.543,"c_ref_apple_clang_O2":129.584}},{"id":"h_inner","candidate":24.611,"reference":109.612,"speedup":4.454,"baselines":{"wpd_neon":26.164,"c_ref_apple_clang_O2":109.612}},{"id":"geomean","candidate":13.991,"reference":121.978,"speedup":8.719,"baselines":{"wpd_neon":14.522,"c_ref_apple_clang_O2":121.978}}]},"scores":{"gpt-6.1-terra-max":9.5,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":9.5,"opus-5.5-xhigh":10.0,"deepseek-v4.1-flash-max":9.5},"council_summaries":{"gpt-6.1-terra-max":"Compact kernel and output packing, with a small grid-test omission","gpt-6.1-sol-max":"Efficient SIMD structure with strong, effective tests.","muse-wonder-full-xhigh":"Leanest kernel and strongest directed tests; merge after fixing one stale macro comment","opus-5.5-xhigh":"Fewest instructions (fewer than wpd's), excellent docs and the most thorough tests","deepseek-v4.1-flash-max":"fastest on all four functions, lean code, thorough tests; nothing worth changing"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The final partial grid batch is discarded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Comment names nonexistent parameters","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Scheduling interleave splits one computation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"`st6row`'s comment names parameters \\p and \\q that the macro doesn't have; it actually stores through the fixed registers x9/x10","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"scratch reuse unstated","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":222,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.7,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.6,"score_cross_vendor":9.625,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":38.58,"tokens":{"uncached_input":70,"cache_read":6689076,"cache_write":312719,"output":526217,"reasoning":490906},"cost_usd":1.1377,"cost_breakdown":{"input":2.6e-05,"cache_read":0.321942,"cache_write":0.275963,"output":0.539782},"cost_usd_as_run":1.1377,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":452,"code":341,"comment":101,"blank":10},{"path":"tests/selftest.c","role":"test","lines":767,"code":656,"comment":58,"blank":53}],"impl_lines":452,"impl_code":341,"impl_comment":101,"test_lines":767,"comment_ratio":0.229,"tests_declared":2,"impl_trailing_comments":74},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.487,"reference":134.02,"speedup":17.9,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.02}},{"id":"h_mb","candidate":25.202,"reference":117.082,"speedup":4.646,"baselines":{"wpd_neon":20.644,"c_ref_apple_clang_O2":117.082}},{"id":"v_inner","candidate":13.882,"reference":129.629,"speedup":9.338,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.629}},{"id":"h_inner","candidate":28.344,"reference":109.877,"speedup":3.877,"baselines":{"wpd_neon":26.215,"c_ref_apple_clang_O2":109.877}},{"id":"geomean","candidate":16.507,"reference":122.269,"speedup":7.407,"baselines":{"wpd_neon":14.542,"c_ref_apple_clang_O2":122.269}}]},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Sound kernel, with extra transpose work and a horizontal guard-test gap","gpt-6.1-sol-max":"Correct arithmetic, but substantial shuffle overhead and incomplete memory tests.","muse-wonder-full-xhigh":"Correct with excellent tests, but a wrong kernel contract comment and the heaviest lean-group h path","opus-5.5-xhigh":"Readable and documented, with good structured tests, but inaccurate register docs, half-width h shuffles, and untested h write windows","deepseek-v4.1-flash-max":"strongest test suite, good code, but the register map goes stale and the h store shuffles per row"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Horizontal tests do not protect forbidden columns","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal filtering transposes each half twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Four separate transpose networks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Horizontal forbidden stores remain invisible","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Inner calls load unused coefficient constants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Kernel header states wrong clobbers and precondition","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Four fully inlined transpose networks plus per-row ext stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard tests skipped on non-4096 pages","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Inner kernel biases taps it never uses","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The header's register map (v16–v18 = thresholds, v20/v21 = 4/3, v26 = M, v27 = H) and LF_KERNEL's \"Clobbers v26..v31 and v1..v6\" are wrong for the macroblock path, which overwrites v16–v18, v20, v21, v26 and v27 with products","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The h path does 16 single-row loads and two separate 8×8 trn networks in each direction (96 trn), plus 16 ins and per-row ext","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"`guard_h` says the column window and store width \"are checked by the disassembly (see the notes in src/vp8_lf_neon.S)\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The ABI trampoline passes zero-extended thresholds and doesn't check x18, x29 or sp. `guard_v` and `guard_h` hard-code 4096-byte pages, so mprotect fails silently on 16 KiB hosts","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"A stray comment, \"LF_H_BODY is defined just above them\", sits above the definition","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"register map contradicts the kernel","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"one `ext` per row","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1349,"lines":46,"significant":341,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":34,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":1.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":2.5,"muse-wonder-full-xhigh":1.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":3.6,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":36.26,"tokens":{"uncached_input":98071,"cache_read":1616488,"cache_write":0,"output":27524,"reasoning":12663},"cost_usd":1.2661,"cost_breakdown":{"input":0.392284,"cache_read":0.323298,"cache_write":0.0,"output":0.55048},"cost_usd_as_run":1.2661,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":308,"code":249,"comment":46,"blank":13},{"path":"tests/selftest.c","role":"test","lines":299,"code":254,"comment":26,"blank":19}],"impl_lines":308,"impl_code":249,"impl_comment":46,"test_lines":299,"comment_ratio":0.156,"tests_declared":2,"impl_trailing_comments":35},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.947,"reference":133.506,"speedup":14.922,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.506}},{"id":"h_mb","candidate":23.102,"reference":117.093,"speedup":5.069,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":117.093}},{"id":"v_inner","candidate":14.791,"reference":129.632,"speedup":8.764,"baselines":{"wpd_neon":12.546,"c_ref_apple_clang_O2":129.632}},{"id":"h_inner","candidate":30.28,"reference":109.911,"speedup":3.63,"baselines":{"wpd_neon":26.215,"c_ref_apple_clang_O2":109.911}},{"id":"geomean","candidate":17.443,"reference":122.165,"speedup":7.004,"baselines":{"wpd_neon":14.531,"c_ref_apple_clang_O2":122.165}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the shared checks, with costly horizontal handling and incomplete safety tests","gpt-6.1-sol-max":"Correct filters, with expensive horizontal packing and missing safety tests.","muse-wonder-full-xhigh":"Clean documented kernel; its own tests never check guards or register preservation","opus-5.5-xhigh":"Correct, clean and well-commented code, but the tests don't check the memory or ABI contract, and the h path is about twice the expert's instruction count","deepseek-v4.1-flash-max":"clean code, but the self-test never checks the memory contract or the ABI"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Forbidden writes and ABI preservation are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal paths transpose and extract both halves separately","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Full half-wise reconstruction precedes scalar extraction","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Memory permissions are never tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI preservation is never checked","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"No write-window or ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"GPR-round-trip h stores and ins-built transposes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Coefficient broadcast via GPR","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page, read-only-page or register-preservation tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The 6-tap a-clamp and flim_e equality are effectively untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The h path does two half-width zip transposes, 16 `ins` in each direction, and per-row stores through GPRs (fmov/lsr/str/strh)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"only arithmetic is verified","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`fmov`-based row extraction","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0643,"lines":16,"significant":249,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":24,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.1,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.9,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.68,"tokens":{"uncached_input":12332,"cache_read":113536,"cache_write":0,"output":4119,"reasoning":260},"cost_usd":0.4428,"cost_breakdown":{"input":0.12332,"cache_read":0.113536,"cache_write":0.0,"output":0.20595},"cost_usd_as_run":0.4428,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":587,"code":585,"comment":2,"blank":0},{"path":"tests/selftest.c","role":"test","lines":283,"code":242,"comment":22,"blank":19}],"impl_lines":587,"impl_code":585,"impl_comment":2,"test_lines":283,"comment_ratio":0.003,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.707,"reference":133.653,"speedup":17.341,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":133.653}},{"id":"h_mb","candidate":33.13,"reference":116.799,"speedup":3.526,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":116.799}},{"id":"v_inner","candidate":13.153,"reference":129.676,"speedup":9.859,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.676}},{"id":"h_inner","candidate":33.742,"reference":109.853,"speedup":3.256,"baselines":{"wpd_neon":26.198,"c_ref_apple_clang_O2":109.853}},{"id":"geomean","candidate":18.348,"reference":122.116,"speedup":6.656,"baselines":{"wpd_neon":14.54,"c_ref_apple_clang_O2":122.116}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.0,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":5.0,"deepseek-v4.1-flash-max":5.5},"council_summaries":{"gpt-6.1-terra-max":"Correct output in the shared checks, but the horizontal path and its tests need work","gpt-6.1-sol-max":"Correct output, but excessive data movement, repetition and weak safety coverage.","muse-wonder-full-xhigh":"Functionally correct, but duplicated, undocumented and the slowest by a wide margin","opus-5.5-xhigh":"Correct, but unindented, undocumented, heavily duplicated and by far the slowest","deepseek-v4.1-flash-max":"correct but the lowest-quality artefact: unindented, copy-pasted, undocumented, slowest"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Transposes contain extensive avoidable moves","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Horizontal forbidden writes and ABI state are not tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"The stated scratch-register set omits `x6`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"The shared kernel lacks its arithmetic and mask invariants","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Copying every transpose stage doubles its work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI preservation is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Horizontal memory protection is absent","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Horizontal bodies and row stores are manually repeated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"redundancy","title":"Inverse transpose duplicated and copy-heavy","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Thinnest safety net","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Duplication is also the hot path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No rationale documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Column-0 instructions throughout macros","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Each transpose stage is copied back into v0–v7 (24 movs per transpose, 4 transposes per h call), plus `ins`+`mov` pairs and per-row umov/lsr/stur stores","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Beyond a 2-line header there is no documentation and no indentation. Nothing explains the lane layout, the 0x80 bias and saturation trick, the masks, or what each macro expects in and leaves in registers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No test for callee-saved registers, x18 or dirty arguments. `test_pages` covers only the v functions (`f += 2`) and silently returns success when the page size isn't 4096","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The h and h_inner bodies are near-verbatim copies with hand-unrolled per-row stores, and `movi v26.8h,#63` appears in each of the three coefficient blocks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"A trailing `add x5, x5, x1` follows the last load or store, and its result is overwritten or thrown away","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The header says \"All macros clobber … x5\", but the macros use no GPRs, and the h functions also clobber x6","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"redundancy","title":"four copies of the same store block","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"the only comment is wrong about the clobbers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"the entire file is written at column 0 with no indentation, and `kernel` (104-198) has no comment: the control flow and the saturating-arithmetic argument are unreadable.","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"slowest of the twelve on every function","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no ABI/callee-saved check, and `test_pages` (tests/selftest.c:191) covers only the two `v` functions and silently returns success when the page size is not 4096.","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.8496,"lines":497,"significant":585,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.8,"slop":{"score":1.9,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":2.5,"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":3.0}},"defect_points":6.8,"score_cross_vendor":5.5,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":12.55,"tokens":{"uncached_input":31680,"cache_read":353664,"cache_write":0,"output":24823,"reasoning":12521},"cost_usd":0.347,"cost_breakdown":{"input":0.06336,"cache_read":0.035366,"cache_write":0.0,"output":0.24823},"cost_usd_as_run":0.347,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":266,"code":219,"comment":31,"blank":16},{"path":"tests/selftest.c","role":"test","lines":518,"code":455,"comment":40,"blank":23}],"impl_lines":266,"impl_code":219,"impl_comment":31,"test_lines":518,"comment_ratio":0.124,"tests_declared":3,"impl_trailing_comments":3},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.497,"reference":133.516,"speedup":17.808,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.516}},{"id":"h_mb","candidate":20.035,"reference":116.978,"speedup":5.839,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.978}},{"id":"v_inner","candidate":13.482,"reference":129.588,"speedup":9.612,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.588}},{"id":"h_inner","candidate":26.618,"reference":109.809,"speedup":4.125,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.809}},{"id":"geomean","candidate":15.237,"reference":122.098,"speedup":8.013,"baselines":{"wpd_neon":14.53,"c_ref_apple_clang_O2":122.098}}]},"scores":{"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Well-documented and broadly tested, with a small output-packing optimization available","gpt-6.1-sol-max":"Compact and correct, with avoidable inverse-transpose work and weak directed mask cases.","muse-wonder-full-xhigh":"Lean, accurate and well tested; only cosmetic blemishes","opus-5.5-xhigh":"Lean expert-style code, good docs, and strong tests with one arithmetic gap","deepseek-v4.1-flash-max":"tight, well-factored, with the most complete self-test of the set"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"The horizontal inverse transpose reconstructs unused taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Full inverse transpose reconstructs discarded taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Directed interior comparisons can be removed undetected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Rounding constant rematerialised per weight","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"No file-level summary","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No case reaches the 6-tap a-clamp. The edge-equality lines with large d are hev, the extreme lines are either rejected or hev, and the random lines keep |q0−p0| ≤ e/4","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`movi v27.8h,#63` is reloaded in all three `weighted_adjustment` expansions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"rounding constant rebuilt per call","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0639,"lines":14,"significant":219,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.8,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":1.2,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.9,"tokens":{"uncached_input":17330,"cache_read":258432,"cache_write":0,"output":10375,"reasoning":2035},"cost_usd":0.9505,"cost_breakdown":{"input":0.1733,"cache_read":0.258432,"cache_write":0.0,"output":0.51875},"cost_usd_as_run":0.9505,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":220,"code":206,"comment":7,"blank":7},{"path":"tests/selftest.c","role":"test","lines":484,"code":435,"comment":27,"blank":22}],"impl_lines":220,"impl_code":206,"impl_comment":7,"test_lines":484,"comment_ratio":0.033,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.697,"reference":133.296,"speedup":17.318,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.296}},{"id":"h_mb","candidate":26.808,"reference":116.862,"speedup":4.359,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":116.862}},{"id":"v_inner","candidate":13.614,"reference":129.903,"speedup":9.542,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.903}},{"id":"h_inner","candidate":28.446,"reference":109.796,"speedup":3.86,"baselines":{"wpd_neon":26.208,"c_ref_apple_clang_O2":109.796}},{"id":"geomean","candidate":16.813,"reference":122.088,"speedup":7.261,"baselines":{"wpd_neon":14.541,"c_ref_apple_clang_O2":122.088}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the shared suite, with expensive transposes and a specific protection-test hole","gpt-6.1-sol-max":"Correct implementation with costly horizontal data movement and a partial write-window check.","muse-wonder-full-xhigh":"Correct with strong tests; the assembly itself is nearly comment-free","opus-5.5-xhigh":"Correct and short, but almost undocumented, with avoidable shuffles and gaps in the tests","deepseek-v4.1-flash-max":"the leanest kernel, but it documents nothing, and the h store is slow"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal paths repeatedly transpose eight-lane halves","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Inner horizontal protection leaves `p2` and `q2` writable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Weighted-adjustment register is documented incorrectly","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Transpose copy-backs inflate the hot path","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Inner horizontal p2 writes are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Wrong weighted-result register","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Kernel arithmetic is undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Copy-back transpose and per-row ext stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong result register in macro comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The only header is a 2-line clobber note. The kernel (masks, bias, hev selection, 6-tap) carries a single comment, and no macro documents its full contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"`transpose` ends with 8 copy-back movs and runs 4 times per h call. The h path uses two half-width transposes, 8 ins and 8 dup, and ext-based row stores","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"For h, the read-only pages protect only p3 and q3 (modes 2 and 3 each make one page read-only, holding col −4 or col 3), so the inner h function can write back cols −3 and 2 undetected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The .irp loops end with an `add x5, x5, x1` whose result is never used, and `movi v26.8h,#63` is reloaded on every `weighted` expansion","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"The inner-edge path is split into two `.if \\inner` blocks on either side of the macroblock `.else`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"kernel has no comments at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`str s24`/`str h24` per row with an `ext`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0583,"lines":12,"significant":206,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.3,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.7,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":13.46,"tokens":{"uncached_input":18624,"cache_read":368768,"cache_write":0,"output":22106,"reasoning":10308},"cost_usd":1.6603,"cost_breakdown":{"input":0.18624,"cache_read":0.368768,"cache_write":0.0,"output":1.1053},"cost_usd_as_run":1.6603,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":266,"code":214,"comment":36,"blank":16},{"path":"tests/selftest.c","role":"test","lines":531,"code":468,"comment":37,"blank":26}],"impl_lines":266,"impl_code":214,"impl_comment":36,"test_lines":531,"comment_ratio":0.144,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.445,"reference":133.758,"speedup":17.966,"baselines":{"wpd_neon":6.605,"c_ref_apple_clang_O2":133.758}},{"id":"h_mb","candidate":20.13,"reference":117.04,"speedup":5.814,"baselines":{"wpd_neon":20.655,"c_ref_apple_clang_O2":117.04}},{"id":"v_inner","candidate":14.228,"reference":129.554,"speedup":9.106,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.554}},{"id":"h_inner","candidate":26.957,"reference":109.846,"speedup":4.075,"baselines":{"wpd_neon":26.215,"c_ref_apple_clang_O2":109.846}},{"id":"geomean","candidate":15.484,"reference":122.172,"speedup":7.89,"baselines":{"wpd_neon":14.559,"c_ref_apple_clang_O2":122.172}}]},"scores":{"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":9.0,"opus-5.5-xhigh":9.5,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Compact implementation with broad tests and a small horizontal optimization opportunity","gpt-6.1-sol-max":"Strong, maintainable implementation with thorough tests and one modest optimization opportunity.","muse-wonder-full-xhigh":"Lean, correct, cleanly documented; no substantive defects found","opus-5.5-xhigh":"Compact, expert-style single 16-lane transpose, accurate comments, and tests that caught every mutant","deepseek-v4.1-flash-max":"small, elegant shift-then-lane-store h path, accurate docs, strong tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"The reverse transpose reconstructs unused taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Inverse transpose includes unwritten taps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Header understates register preservation","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0654,"lines":14,"significant":214,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.2,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.5,"score_cross_vendor":9.167,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":73.66,"tokens":{"uncached_input":810,"cache_read":40438043,"cache_write":480449,"output":420143,"reasoning":0},"cost_usd":0.9407,"cost_breakdown":{"input":0.000121,"cache_read":0.647009,"cache_write":0.09609,"output":0.197467},"cost_usd_as_run":0.9407,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.9407,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":789,"code":625,"comment":149,"blank":15},{"path":"tests/selftest.c","role":"test","lines":801,"code":677,"comment":93,"blank":31}],"impl_lines":789,"impl_code":625,"impl_comment":149,"test_lines":801,"comment_ratio":0.193,"tests_declared":7,"impl_trailing_comments":71},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":10.522,"reference":134.136,"speedup":12.748,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.136}},{"id":"h_mb","candidate":25.496,"reference":116.788,"speedup":4.581,"baselines":{"wpd_neon":20.571,"c_ref_apple_clang_O2":116.788}},{"id":"v_inner","candidate":15.438,"reference":129.768,"speedup":8.405,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.768}},{"id":"h_inner","candidate":29.534,"reference":110.016,"speedup":3.725,"baselines":{"wpd_neon":26.164,"c_ref_apple_clang_O2":110.016}},{"id":"geomean","candidate":18.701,"reference":122.29,"speedup":6.539,"baselines":{"wpd_neon":14.523,"c_ref_apple_clang_O2":122.29}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct output in the shared suite, but its test harness has false failures and missed contract violations","gpt-6.1-sol-max":"Correct filters, but the ABI test fails under Clang and several coverage claims are unreliable.","muse-wonder-full-xhigh":"The most thoroughly verified entry, delivered as the most over-engineered file","opus-5.5-xhigh":"Correct but over-engineered and slow, and its ABI test rejects valid code","deepseek-v4.1-flash-max":"the most engineered and best-tested file, but 789 lines and three names per register"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"The kernel and horizontal paths do considerable avoidable work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"The ABI probe rejects legal caller-saved changes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Forbidden unchanged writes are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The claimed edge-equality generator misses its target","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Poison bytes cannot reveal unchanged stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Random sign selection is multiplied by zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI result overwritten during restoration","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Unnecessary widening for common corrections","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Equality cases miss the edge boundary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Poisoning does not detect every extra store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"verbosity","title":"789 lines where peers need ~260","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Belt-and-braces kernel arithmetic","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"One macro text, two register meanings","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong register list in header","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Unused register aliases","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The ABI test stores its sp reference in x13, which is caller-saved, across `blr` and compares it afterwards. The asm also leaves x5–x10, x15, x17, x30, v0–v7, v16–v18 and v22–v31 out of the clobber list","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"`T_EDGE_EQ` and `T_EDGE_M1` are documented as producing an edge sum of exactly flim_e or flim_e−1","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The header claims load widths are \"audited by disassembling the shipped object (see README notes in report)\", and `test_window` claims to catch \"any store outside the permitted window\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"f1, f2 and a' are computed by widening to 16 bits (sshll/add/smin/sshr/sqxtn), although 8-bit sqadd+sshr is exact. Each tap is biased and unbiased around its own correction, and the h path uses two 3-pass uzp transposes with filler copies","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"A `.req` alias layer defines 33 names for each of the two layouts, plus an UNLAYOUT macro. Some aliases exist only so UNLAYOUT has something to unreq","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The header's \"Uses x0-x3, x10, x11\" omits w4. `test_algebra` item B claims to check the f1/f2 forms, but it computes f2 and discards it with `(void)f2`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"verbosity","title":"aliasing scheme dominates the file","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"duplicated row-store blocks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"slowest v paths","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":143,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.8,"slop":{"score":2.4,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":3.5,"muse-wonder-full-xhigh":3.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":3.0}},"defect_points":6.0,"score_cross_vendor":6.8,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":350.49,"tokens":{"uncached_input":399432,"cache_read":5138692,"cache_write":0,"output":65943,"reasoning":33480},"cost_usd":3.9443,"cost_breakdown":{"input":1.597728,"cache_read":1.027738,"cache_write":0.0,"output":1.31886},"cost_usd_as_run":3.9443,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":569,"code":402,"comment":131,"blank":36},{"path":"tests/selftest.c","role":"test","lines":670,"code":557,"comment":73,"blank":40}],"impl_lines":569,"impl_code":402,"impl_comment":131,"test_lines":670,"comment_ratio":0.246,"tests_declared":5,"impl_trailing_comments":40},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.842,"reference":134.398,"speedup":15.201,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.398}},{"id":"h_mb","candidate":25.107,"reference":117.24,"speedup":4.67,"baselines":{"wpd_neon":20.571,"c_ref_apple_clang_O2":117.24}},{"id":"v_inner","candidate":14.642,"reference":129.964,"speedup":8.876,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.964}},{"id":"h_inner","candidate":30.345,"reference":109.778,"speedup":3.618,"baselines":{"wpd_neon":26.174,"c_ref_apple_clang_O2":109.778}},{"id":"geomean","candidate":17.721,"reference":122.448,"speedup":6.91,"baselines":{"wpd_neon":14.517,"c_ref_apple_clang_O2":122.448}}]},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and well tested, but horizontal filtering does substantial avoidable work","gpt-6.1-sol-max":"Correct and well tested, with excessive horizontal-path work.","muse-wonder-full-xhigh":"Correct, but GPR-round-trip stores and select-per-output make it the second heaviest kernel","opus-5.5-xhigh":"Exemplary header docs and solid tests, but a heavy h path and mov-bsl-mov blends","deepseek-v4.1-flash-max":"correct with strong tests, but the h store goes through GPRs and two instructions are dead"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Four separate transpose networks and full result selection inflate the horizontal path","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Repeated half-width transposes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Duplicated horizontal wrappers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Scalarised h stores and BSL output selection","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Inner output contract hides biased leftovers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Inner biases unused taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Every blend is `mov vT,hev; bsl vT,old,new; mov vX,vT` (16 extra movs), where a single `bif vX,new,hev` would do. The h path runs two half-width transposes with 8 mov + 8 ins each way and stores rows through GPRs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The bodies of `cb_vp8_h_loop_filter16_neon` and `_inner_neon` are verbatim copies apart from the 0/1 argument","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The register check passes zero-extended thresholds (`ldr w2..w4`) and runs once per function, and no case reaches the 6-tap a-clamp","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"per-row vector→GPR store path","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"p2/q2 biased for nothing on inner edges","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"LOAD_H_8/STORE_H_8 (276-315) restate each instruction in prose","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.3085,"lines":124,"significant":402,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":45,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":2.7,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":13.06,"tokens":{"uncached_input":18287,"cache_read":433280,"cache_write":0,"output":30106,"reasoning":17034},"cost_usd":0.381,"cost_breakdown":{"input":0.036574,"cache_read":0.043328,"cache_write":0.0,"output":0.30106},"cost_usd_as_run":0.381,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":308,"code":247,"comment":44,"blank":17},{"path":"tests/selftest.c","role":"test","lines":564,"code":499,"comment":38,"blank":27}],"impl_lines":308,"impl_code":247,"impl_comment":44,"test_lines":564,"comment_ratio":0.151,"tests_declared":5,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.34,"reference":133.936,"speedup":18.247,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.936}},{"id":"h_mb","candidate":18.807,"reference":117.24,"speedup":6.234,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":117.24}},{"id":"v_inner","candidate":13.567,"reference":129.947,"speedup":9.578,"baselines":{"wpd_neon":12.543,"c_ref_apple_clang_O2":129.947}},{"id":"h_inner","candidate":25.35,"reference":109.639,"speedup":4.325,"baselines":{"wpd_neon":26.201,"c_ref_apple_clang_O2":109.639}},{"id":"geomean","candidate":14.761,"reference":122.3,"speedup":8.285,"baselines":{"wpd_neon":14.529,"c_ref_apple_clang_O2":122.3}}]},"scores":{"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":9.5,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"gpt-6.1-terra-max":"Strong tests and compact output packing, with extra work in the forward transpose","gpt-6.1-sol-max":"Clear, correct implementation with comprehensive safety tests and minor inefficiencies.","muse-wonder-full-xhigh":"Correct and well tested; h transpose path is heavier than the best entries","opus-5.5-xhigh":"Lean kernel, a zip-based partial inverse transpose for h, and thorough tests","deepseek-v4.1-flash-max":"second-fastest, clean two-bank transpose, thorough tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"The forward horizontal transpose handles the two halves separately","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Forward transpose handles halves separately","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Several interior-boundary cases are unobservable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Two separate 8x8 transposes plus pack-back","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicated pointer setup in both store arms","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"duplicated address computation","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0648,"lines":16,"significant":247,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.0,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":1.0,"score_cross_vendor":9.0,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.52,"tokens":{"uncached_input":14645,"cache_read":677120,"cache_write":0,"output":29101,"reasoning":15279},"cost_usd":0.194,"cost_breakdown":{"input":0.014645,"cache_read":0.033856,"cache_write":0.0,"output":0.145505},"cost_usd_as_run":0.194,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":320,"code":276,"comment":30,"blank":14},{"path":"tests/selftest.c","role":"test","lines":545,"code":477,"comment":38,"blank":30}],"impl_lines":320,"impl_code":276,"impl_comment":30,"test_lines":545,"comment_ratio":0.098,"tests_declared":6,"impl_trailing_comments":5},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.508,"reference":134.514,"speedup":17.916,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.514}},{"id":"h_mb","candidate":25.034,"reference":117.397,"speedup":4.69,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":117.397}},{"id":"v_inner","candidate":13.967,"reference":130.164,"speedup":9.319,"baselines":{"wpd_neon":12.583,"c_ref_apple_clang_O2":130.164}},{"id":"h_inner","candidate":27.835,"reference":110.073,"speedup":3.954,"baselines":{"wpd_neon":26.218,"c_ref_apple_clang_O2":110.073}},{"id":"geomean","candidate":16.441,"reference":122.645,"speedup":7.46,"baselines":{"wpd_neon":14.537,"c_ref_apple_clang_O2":122.645}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":7.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct, but its boundary generator misses its stated target and h does excess work","opus-5.5-xhigh":"correct with decent tests, but a heavy h path and sparse docs","muse-wonder-full-xhigh":"correct; h half-shuffling and scalar stores waste instructions","gpt-6.1-sol-max":"Correct, safety-conscious code with excess shuffling and a faulty boundary generator.","deepseek-v4.1-flash-max":"Correct and robust, but the h path pays for per-row GPR extraction and four full transposes."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal paths perform four full transposes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Edge-boundary generator constructs the wrong metric","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Inner filtering converts unused outer taps twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"h path does two 8-byte transposes, a full 8×8 inverse transpose of every tap, and per-row ext+umov+strh","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Macro names are backwards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Register-copy half shuffling","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar extraction for the q1/q2 store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Triplicated coefficient blocks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal path repeatedly rearranges complete blocks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Edge generator misses its advertised boundaries","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"q1/q2 pushed through a GPR","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Four complete 8x8 transpose networks","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.2174,"lines":60,"significant":276,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.2,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.34,"tokens":{"uncached_input":54179,"cache_read":337152,"cache_write":0,"output":10555,"reasoning":2962},"cost_usd":0.2476,"cost_breakdown":{"input":0.108358,"cache_read":0.033715,"cache_write":0.0,"output":0.10555},"cost_usd_as_run":0.2476,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":438,"code":415,"comment":17,"blank":6},{"path":"tests/selftest.c","role":"test","lines":375,"code":325,"comment":29,"blank":21}],"impl_lines":438,"impl_code":415,"impl_comment":17,"test_lines":375,"comment_ratio":0.039,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.603,"reference":133.506,"speedup":17.561,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.506}},{"id":"h_mb","candidate":27.428,"reference":117.103,"speedup":4.27,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":117.103}},{"id":"v_inner","candidate":13.855,"reference":129.54,"speedup":9.35,"baselines":{"wpd_neon":12.533,"c_ref_apple_clang_O2":129.54}},{"id":"h_inner","candidate":28.954,"reference":109.85,"speedup":3.794,"baselines":{"wpd_neon":26.171,"c_ref_apple_clang_O2":109.85}},{"id":"geomean","candidate":17.007,"reference":122.129,"speedup":7.181,"baselines":{"wpd_neon":14.527,"c_ref_apple_clang_O2":122.129}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct, with substantial avoidable horizontal-filter work","opus-5.5-xhigh":"correct, but a heavy h path, 175 lines of copy-paste, dead adds, sparse docs","muse-wonder-full-xhigh":"correct, but the h store is 16 hand-copied blocks and the kernel is undocumented","gpt-6.1-sol-max":"Correct kernels burdened by repetitive horizontal stores and avoidable address work.","deepseek-v4.1-flash-max":"Correct and reasonably clear, but the h store path is a 200-line copy-paste and is instruction-heavy."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Full inverse transposes and per-row address repairs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Callee-saved state is not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Inner filtering converts unused outer taps twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"16 copy-pasted `.if \\inner` store blocks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Two 8-byte transposes, each followed by 8 movs, then full inverse transposes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Trailing `add x5,x5,x1` after the final load or store","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Upper-half garbage is tested, but callee-saved registers, x18 and sp are not","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Sparse docs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"verbosity","title":"Hand-unrolled row stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No register-preservation test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Undocumented kernel arithmetic","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Per-row address round trips","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Sixteen duplicated store blocks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Dirty arguments are checked without ABI preservation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"Sixteen hand-copied store blocks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Six instructions per stored row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Useless signed-domain round trip","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"ABI untested and the protection test only proves absence of faults","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0675,"lines":28,"significant":415,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.0,"slop":{"score":1.4,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":2.0,"muse-wonder-full-xhigh":2.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.5}},"defect_points":4.0,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":351.12,"tokens":{"uncached_input":349630,"cache_read":2730256,"cache_write":0,"output":57926,"reasoning":32978},"cost_usd":3.1031,"cost_breakdown":{"input":1.39852,"cache_read":0.546051,"cache_write":0.0,"output":1.15852},"cost_usd_as_run":3.1031,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":473,"code":329,"comment":115,"blank":29},{"path":"tests/selftest.c","role":"test","lines":587,"code":518,"comment":42,"blank":27}],"impl_lines":473,"impl_code":329,"impl_comment":115,"test_lines":587,"comment_ratio":0.259,"tests_declared":8,"impl_trailing_comments":35},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":9.262,"reference":136.099,"speedup":14.695,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":136.099}},{"id":"h_mb","candidate":40.68,"reference":118.259,"speedup":2.907,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":118.259}},{"id":"v_inner","candidate":15.384,"reference":130.978,"speedup":8.514,"baselines":{"wpd_neon":12.706,"c_ref_apple_clang_O2":130.978}},{"id":"h_inner","candidate":37.652,"reference":110.606,"speedup":2.938,"baselines":{"wpd_neon":26.469,"c_ref_apple_clang_O2":110.606}},{"id":"geomean","candidate":21.614,"reference":123.571,"speedup":5.717,"baselines":{"wpd_neon":14.612,"c_ref_apple_clang_O2":123.571}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct, but the transpose network is unusually expensive","opus-5.5-xhigh":"correct and documented, but the least efficient implementation, plus a false \"verified exhaustively\" claim","muse-wonder-full-xhigh":"correct and well tested, but by far the largest and slowest implementation","gpt-6.1-sol-max":"Correct and documented, but horizontal register shuffling is unnecessarily costly.","deepseek-v4.1-flash-max":"Best documentation and tests in the set, but the slowest kernel by a wide margin."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Transpose pairs copy both results after every step","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal forbidden writes are not protected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Inflated transposes and selects","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"False claim","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Stale comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Move-heavy transposes and selects","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"False exhaustiveness claim","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Transposes copy every pair back","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Horizontal forbidden writes are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Exhaustive-test claim is inaccurate","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"Every conditional tap update costs three instructions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Coefficient constants recomputed","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.2006,"lines":66,"significant":329,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":29,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.1,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":3.2,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":23.67,"tokens":{"uncached_input":57724,"cache_read":18845184,"cache_write":0,"output":229077,"reasoning":167200},"cost_usd":0.2026,"cost_breakdown":{"input":0.008659,"cache_read":0.056536,"cache_write":0.0,"output":0.137446},"cost_usd_as_run":0.2026,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2026,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":408,"code":271,"comment":117,"blank":20},{"path":"tests/selftest.c","role":"test","lines":920,"code":776,"comment":88,"blank":56}],"impl_lines":408,"impl_code":271,"impl_comment":117,"test_lines":920,"comment_ratio":0.302,"tests_declared":9,"impl_trailing_comments":77},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.768,"reference":134.976,"speedup":15.394,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":134.976}},{"id":"h_mb","candidate":26.714,"reference":118.773,"speedup":4.446,"baselines":{"wpd_neon":20.781,"c_ref_apple_clang_O2":118.773}},{"id":"v_inner","candidate":8.636,"reference":130.402,"speedup":15.099,"baselines":{"wpd_neon":12.587,"c_ref_apple_clang_O2":130.402}},{"id":"h_inner","candidate":28.924,"reference":110.999,"speedup":3.838,"baselines":{"wpd_neon":26.465,"c_ref_apple_clang_O2":110.999}},{"id":"geomean","candidate":15.553,"reference":123.422,"speedup":7.936,"baselines":{"wpd_neon":14.617,"c_ref_apple_clang_O2":123.422}}]},"scores":{"gpt-6.1-terra-max":6.5,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct kernel, but several contract tests can pass or report misleading results","opus-5.5-xhigh":"clean and well documented; guard test steps outside the stride domain","muse-wonder-full-xhigh":"correct, well documented and well tested; kernel does redundant work","gpt-6.1-sol-max":"Correct kernels, but two added test groups can falsely succeed.","deepseek-v4.1-flash-max":"Correct, very well documented, but the 27/18/9 coefficients are rebuilt from scratch three times."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guarded output mismatches count as success","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Dirty-argument test is vacuous","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard cases use strides outside the contract","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal output transposes all eight taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Out-of-domain stride","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Test comment says x18 \"is only checked elsewhere\" on Apple (tests/selftest.c:638), but no other test checks it","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Both filters computed for every tap","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Full transpose in both directions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard mismatches count as success","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Dirty-argument test performs no effective filtering","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Three independent widening chains","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0664,"lines":18,"significant":271,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":96,"flags":3,"audited":"2026-11-02","reviewed":true},"score":7.9,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.9,"score_cross_vendor":7.75,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":35.78,"tokens":{"uncached_input":90,"cache_read":7354417,"cache_write":226113,"output":152173,"reasoning":95632},"cost_usd":6.3236,"cost_breakdown":{"input":0.00036,"cache_read":1.470883,"cache_write":1.808904,"output":3.04346},"cost_usd_as_run":6.3236,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":338,"code":234,"comment":95,"blank":9},{"path":"tests/selftest.c","role":"test","lines":1207,"code":1047,"comment":92,"blank":68}],"impl_lines":338,"impl_code":234,"impl_comment":95,"test_lines":1207,"comment_ratio":0.289,"tests_declared":7,"impl_trailing_comments":160},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":6.006,"reference":133.863,"speedup":22.287,"baselines":{"wpd_neon":6.605,"c_ref_apple_clang_O2":133.863}},{"id":"h_mb","candidate":17.988,"reference":117.251,"speedup":6.518,"baselines":{"wpd_neon":20.634,"c_ref_apple_clang_O2":117.251}},{"id":"v_inner","candidate":12.061,"reference":129.649,"speedup":10.749,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.649}},{"id":"h_inner","candidate":24.523,"reference":110.202,"speedup":4.494,"baselines":{"wpd_neon":26.198,"c_ref_apple_clang_O2":110.202}},{"id":"geomean","candidate":13.37,"reference":122.372,"speedup":9.153,"baselines":{"wpd_neon":14.552,"c_ref_apple_clang_O2":122.372}}]},"scores":{"gpt-6.1-terra-max":10.0,"opus-5.5-xhigh":9.5,"muse-wonder-full-xhigh":10.0,"gpt-6.1-sol-max":10.0,"deepseek-v4.1-flash-max":9.5},"council_summaries":{"gpt-6.1-terra-max":"Correct, compact kernel with strong contract-focused tests","opus-5.5-xhigh":"lean FFmpeg-style kernel (usqadd, addhn), best-in-class speed, excellent docs and tests","muse-wonder-full-xhigh":"merge as is; fastest, best documented and best tested","gpt-6.1-sol-max":"Efficient register allocation and arithmetic backed by comprehensive tests.","deepseek-v4.1-flash-max":"Correct, fastest overall, best-documented arithmetic (the addhn trick) and the most thorough test suite; no findings."},"findings":[{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Stage-2 comment in h_load","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":234,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":42,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.8,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.1,"score_cross_vendor":9.875,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":69.48,"tokens":{"uncached_input":296458,"cache_read":12486720,"cache_write":0,"output":176711,"reasoning":141520},"cost_usd":0.5074,"cost_breakdown":{"input":0.044469,"cache_read":0.374602,"cache_write":0.0,"output":0.088356},"cost_usd_as_run":0.5074,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":425,"code":298,"comment":118,"blank":9},{"path":"tests/selftest.c","role":"test","lines":719,"code":608,"comment":69,"blank":42}],"impl_lines":425,"impl_code":298,"impl_comment":118,"test_lines":719,"comment_ratio":0.284,"tests_declared":6,"impl_trailing_comments":97},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.443,"reference":134.545,"speedup":15.937,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":134.545}},{"id":"h_mb","candidate":19.626,"reference":117.376,"speedup":5.981,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":117.376}},{"id":"v_inner","candidate":13.255,"reference":129.608,"speedup":9.778,"baselines":{"wpd_neon":12.556,"c_ref_apple_clang_O2":129.608}},{"id":"h_inner","candidate":24.387,"reference":110.077,"speedup":4.514,"baselines":{"wpd_neon":26.177,"c_ref_apple_clang_O2":110.077}},{"id":"geomean","candidate":15.213,"reference":122.516,"speedup":8.054,"baselines":{"wpd_neon":14.541,"c_ref_apple_clang_O2":122.516}}]},"scores":{"gpt-6.1-terra-max":6.0,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct kernel, but its ABI test corrupts memory and fails on the Apple branch","opus-5.5-xhigh":"correct, well documented, early exit; one wrong transpose comment, thinner calling-convention tests","muse-wonder-full-xhigh":"correct with a useful all-rejected early exit; one contradictory transpose comment","gpt-6.1-sol-max":"Correct production kernels, but the preservation tests contain substantive defects.","deepseek-v4.1-flash-max":"Correct and well structured with a cheap early-out; one wrong doc sentence and a branch that rarely pays."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"ABI test keeps pointers in caller-saved registers across a call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Apple ABI check compares an uninitialized slot","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Unused inner-edge tap conversions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Transpose mapping contradicts the register list","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Transpose output described wrongly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Kernel output note","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Calling-convention check incomplete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The h load does two separate 8-byte transposes plus 16 mov/ins (src:292–328) instead of one 16-byte transpose: 113/221 instructions vs 92/160 for H.","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Self-contradictory transpose documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Early exit taxes the passing path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Macro secretly branches to a caller label","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"tests","title":"Apple path compares uninitialized state","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Trampoline relies on caller-saved registers surviving","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guards cannot detect unchanged forbidden stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Transpose mapping contradicts the implementation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"Generic mapping sentence contradicts the real one","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"Early-out costs every call","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":73,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":4.1,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":68.84,"tokens":{"uncached_input":214998,"cache_read":2548864,"cache_write":0,"output":286475,"reasoning":130340},"cost_usd":3.4233,"cost_breakdown":{"input":0.429996,"cache_read":1.274432,"cache_write":0.0,"output":1.71885},"cost_usd_as_run":3.4233,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":396,"code":274,"comment":98,"blank":24},{"path":"tests/selftest.c","role":"test","lines":1010,"code":903,"comment":56,"blank":51}],"impl_lines":396,"impl_code":274,"impl_comment":98,"test_lines":1010,"comment_ratio":0.263,"tests_declared":6,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.844,"reference":133.884,"speedup":17.068,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.884}},{"id":"h_mb","candidate":28.467,"reference":117.219,"speedup":4.118,"baselines":{"wpd_neon":20.665,"c_ref_apple_clang_O2":117.219}},{"id":"v_inner","candidate":14.008,"reference":130.466,"speedup":9.314,"baselines":{"wpd_neon":12.607,"c_ref_apple_clang_O2":130.466}},{"id":"h_inner","candidate":29.161,"reference":110.379,"speedup":3.785,"baselines":{"wpd_neon":26.32,"c_ref_apple_clang_O2":110.379}},{"id":"geomean","candidate":17.379,"reference":122.61,"speedup":7.055,"baselines":{"wpd_neon":14.571,"c_ref_apple_clang_O2":122.61}}]},"scores":{"gpt-6.1-terra-max":6.5,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct kernel, but its guard and Apple ABI tests violate their own platform assumptions","opus-5.5-xhigh":"correct and well documented; stack spills on the h path, two test hazards","muse-wonder-full-xhigh":"correct with strong tests, but the h path is stack-heavy and slow","gpt-6.1-sol-max":"Correct and thoroughly tested, with avoidable horizontal stack traffic.","deepseek-v4.1-flash-max":"Correct with excellent documentation and tests; the h path transposes all eight taps twice."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guard tests require behavior for invalid strides","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"abi","title":"Apple test overwrites the platform register","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal paths spill both halves and fully transpose them back","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Claimed edge-equality case is rejected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Test data is initialized and immediately overwritten","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Avoidable stack spills","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Out-of-domain strides","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Writes x18 on Apple","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"zip (not trn) … which is what an 8x8 byte transpose needs\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Stack-spilled h path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Three independent coefficient products","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong product range","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Unnecessary spills","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Equality case actually rejects","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Immediately overwritten initialization","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Full 8-tap inverse transpose for a 4- or 6-byte store","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":20,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.5,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.8,"score_cross_vendor":7.6,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.51,"tokens":{"uncached_input":38132,"cache_read":223360,"cache_write":0,"output":6664,"reasoning":1147},"cost_usd":0.1652,"cost_breakdown":{"input":0.076264,"cache_read":0.022336,"cache_write":0.0,"output":0.06664},"cost_usd_as_run":0.1652,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":464,"code":451,"comment":6,"blank":7},{"path":"tests/selftest.c","role":"test","lines":301,"code":260,"comment":22,"blank":19}],"impl_lines":464,"impl_code":451,"impl_comment":6,"test_lines":301,"comment_ratio":0.013,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.655,"reference":134.094,"speedup":17.517,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.094}},{"id":"h_mb","candidate":24.75,"reference":116.967,"speedup":4.726,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.967}},{"id":"v_inner","candidate":13.821,"reference":129.74,"speedup":9.387,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.74}},{"id":"h_inner","candidate":27.71,"reference":110.073,"speedup":3.972,"baselines":{"wpd_neon":26.191,"c_ref_apple_clang_O2":110.073}},{"id":"geomean","candidate":16.413,"reference":122.337,"speedup":7.454,"baselines":{"wpd_neon":14.533,"c_ref_apple_clang_O2":122.337}}]},"scores":{"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":5.5,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct output, with expensive duplicated horizontal paths and limited contract documentation","opus-5.5-xhigh":"correct, but undocumented, copy-pasted, with dead instructions","muse-wonder-full-xhigh":"correct, but essentially undocumented with duplicated function bodies","gpt-6.1-sol-max":"Correct implementation with expensive transposes and incomplete safety tests.","deepseek-v4.1-flash-max":"Correct and mid-pack on speed, but essentially undocumented and littered with dead instructions."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal paths reconstruct every tap for every row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI and unchanged forbidden writes are not tested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Arithmetic and memory invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Inner filtering converts unused outer taps twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"No documentation at all","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Copy-pasted h functions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Dead pointer increments","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Full inverse transposes plus per-row ushr/str/st1","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No calling-convention test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No file or contract documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Duplicated function bodies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thinnest test additions of the correct entries","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Four complete horizontal transposes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Horizontal forbidden stores remain undetected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"No register-preservation assertions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"Four entry points with no comments at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"Dead and no-op address arithmetic","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No ABI test and a one-lane boundary generator","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.7273,"lines":328,"significant":451,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.8,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":2.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":4.3,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.32,"tokens":{"uncached_input":27518,"cache_read":539136,"cache_write":0,"output":11393,"reasoning":2884},"cost_usd":0.1114,"cost_breakdown":{"input":0.027518,"cache_read":0.026957,"cache_write":0.0,"output":0.056965},"cost_usd_as_run":0.1114,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":285,"code":268,"comment":11,"blank":6},{"path":"tests/selftest.c","role":"test","lines":332,"code":291,"comment":21,"blank":20}],"impl_lines":285,"impl_code":268,"impl_comment":11,"test_lines":332,"comment_ratio":0.039,"tests_declared":3,"impl_trailing_comments":4},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.981,"reference":134.209,"speedup":16.817,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.209}},{"id":"h_mb","candidate":32.426,"reference":118.038,"speedup":3.64,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":118.038}},{"id":"v_inner","candidate":13.553,"reference":130.547,"speedup":9.632,"baselines":{"wpd_neon":12.587,"c_ref_apple_clang_O2":130.547}},{"id":"h_inner","candidate":31.986,"reference":110.375,"speedup":3.451,"baselines":{"wpd_neon":26.459,"c_ref_apple_clang_O2":110.375}},{"id":"geomean","candidate":18.301,"reference":122.917,"speedup":6.716,"baselines":{"wpd_neon":14.577,"c_ref_apple_clang_O2":122.917}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.0,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, but the h paths spill and transpose more data than needed","opus-5.5-xhigh":"correct, but the h path round-trips through the stack, docs are minimal, no calling-convention test","muse-wonder-full-xhigh":"correct; h path spills everything to the stack and documentation/tests are thin","gpt-6.1-sol-max":"Correct filtering with substantial avoidable stack traffic.","deepseek-v4.1-flash-max":"Correct and compact, but the least documented real entry and the slowest h path after L."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal paths spill and reload the full transpose","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal forbidden writes are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Inner filtering converts unused outer taps twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Avoidable stack spills","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"One-line file header","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No calling-convention test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Stack-based h path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI-preservation test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No contract-level documentation","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal path spills complete tap sets","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Horizontal guards permit excluded stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"No ABI sentinel checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"Thin documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Whole tap set spilled and reloaded","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No ABI test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0448,"lines":12,"significant":268,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.0,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.7,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.27,"tokens":{"uncached_input":8661,"cache_read":76800,"cache_write":0,"output":440,"reasoning":71},"cost_usd":0.0147,"cost_breakdown":{"input":0.008661,"cache_read":0.00384,"cache_write":0.0,"output":0.0022},"cost_usd_as_run":0.0147,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":23,"code":13,"comment":6,"blank":4},{"path":"tests/selftest.c","role":"test","lines":216,"code":181,"comment":18,"blank":17}],"impl_lines":23,"impl_code":13,"impl_comment":6,"test_lines":216,"comment_ratio":0.316,"tests_declared":1,"impl_trailing_comments":0},"hidden":{"passed":3,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.945,"reference":134.293,"speedup":142.094,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.293}},{"id":"h_mb","candidate":0.945,"reference":117.272,"speedup":124.084,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":117.272}},{"id":"v_inner","candidate":0.939,"reference":129.418,"speedup":137.782,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.418}},{"id":"h_inner","candidate":0.939,"reference":110.66,"speedup":117.811,"baselines":{"wpd_neon":26.303,"c_ref_apple_clang_O2":110.66}},{"id":"geomean","candidate":0.942,"reference":122.549,"speedup":130.067,"baselines":{"wpd_neon":14.54,"c_ref_apple_clang_O2":122.549}}]},"scores":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0},"council_summaries":{"gpt-6.1-terra-max":"All four entry points are unimplemented stubs","opus-5.5-xhigh":"unmodified scaffold stub: nothing is implemented","muse-wonder-full-xhigh":"untouched stub; no filtering is implemented","gpt-6.1-sol-max":"All four filters remain unimplemented stubs.","deepseek-v4.1-flash-max":"The scaffold stub: no implementation, every call is a no-op and its own tests fail."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"Every filter returns without filtering","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"No filter at all","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"tests/selftest.c is the untouched scaffold","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"All four entry points are no-ops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Every entry point returns without filtering","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"No filter at all","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"Tests are the unmodified scaffold and are red","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":3,"flags":0,"audited":"2026-11-02","reviewed":false},"score":0.6,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.8,"score_cross_vendor":0.667,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.27,"tokens":{"uncached_input":29238,"cache_read":217984,"cache_write":0,"output":4485,"reasoning":368},"cost_usd":0.0626,"cost_breakdown":{"input":0.029238,"cache_read":0.010899,"cache_write":0.0,"output":0.022425},"cost_usd_as_run":0.0626,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":632,"code":628,"comment":4,"blank":0},{"path":"tests/selftest.c","role":"test","lines":250,"code":212,"comment":20,"blank":18}],"impl_lines":632,"impl_code":628,"impl_comment":4,"test_lines":250,"comment_ratio":0.006,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.169,"reference":136.687,"speedup":16.731,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":136.687}},{"id":"h_mb","candidate":62.563,"reference":119.13,"speedup":1.904,"baselines":{"wpd_neon":21.526,"c_ref_apple_clang_O2":119.13}},{"id":"v_inner","candidate":14.489,"reference":131.965,"speedup":9.108,"baselines":{"wpd_neon":12.848,"c_ref_apple_clang_O2":131.965}},{"id":"h_inner","candidate":51.893,"reference":112.061,"speedup":2.159,"baselines":{"wpd_neon":26.794,"c_ref_apple_clang_O2":112.061}},{"id":"geomean","candidate":24.898,"reference":124.57,"speedup":5.003,"baselines":{"wpd_neon":14.862,"c_ref_apple_clang_O2":124.57}}]},"scores":{"gpt-6.1-terra-max":6.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":5.0,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":5.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, but the horizontal path needs a substantial performance rewrite","gpt-6.1-sol-max":"Correct filtering, but the stack transpose needs substantial rework","opus-5.5-xhigh":"correct, but the h path goes through a byte-by-byte stack transpose that costs 3–4× and is ~500 lines of unrolled code; tests are thin","deepseek-v4.1-flash-max":"bit-exact and lean on the v functions, but the two h functions transpose through the stack one byte at a time (~700 executed instructions per call), and both documentation and tests are thin.","muse-wonder-full-xhigh":"Correct and safe, but the h implementation, docs and tests need real rework"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"performance","title":"Horizontal transpose spills and gathers individual bytes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"No guard-page or ABI checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Arithmetic invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Bytewise stack transpose dominates h calls","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Memory and ABI constraints lack dedicated checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Inner filtering scatters unused outer taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"The h path transposes through a 128-byte stack buffer","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page, write-window or register tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"verbosity","title":"Hand-unrolled lane code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Nearly undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The inner path converts p2/q2 and writes them back to the scratch buffer although it never stores them out","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h entry points transpose via a 128-byte stack scratch, one byte per instruction","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"~250 lines of hand-unrolled, near-identical lane-shuffle boilerplate","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no file, entry-point, memory-window or ABI documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"the only clobber note is wrong","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"no memory-contract and no ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"h functions transpose byte-at-a-time through a stack buffer","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"No guard-page or ABI tests; boundary sweep does not target boundaries exactly","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Essentially undocumented assembly","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"verbosity","title":"224 hand-enumerated address+byte-transfer pairs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"\"horizontal\" naming inverted versus the contract","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Clobber comment names registers never used","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0413,"lines":26,"significant":629,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":5.9,"slop":{"score":1.7,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":2.5,"muse-wonder-full-xhigh":3.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.5}},"defect_points":7.1,"score_cross_vendor":5.667,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.83,"tokens":{"uncached_input":18,"cache_read":545894,"cache_write":75007,"output":33838,"reasoning":14675},"cost_usd":1.3861,"cost_breakdown":{"input":7.2e-05,"cache_read":0.109179,"cache_write":0.600056,"output":0.67676},"cost_usd_as_run":1.3861,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":284,"code":215,"comment":58,"blank":11},{"path":"tests/selftest.c","role":"test","lines":634,"code":547,"comment":50,"blank":37}],"impl_lines":284,"impl_code":215,"impl_comment":58,"test_lines":634,"comment_ratio":0.212,"tests_declared":7,"impl_trailing_comments":56},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":6.615,"reference":135.196,"speedup":20.437,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":135.196}},{"id":"h_mb","candidate":21.589,"reference":117.975,"speedup":5.464,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":117.975}},{"id":"v_inner","candidate":13.109,"reference":130.69,"speedup":9.969,"baselines":{"wpd_neon":12.682,"c_ref_apple_clang_O2":130.69}},{"id":"h_inner","candidate":25.106,"reference":110.409,"speedup":4.398,"baselines":{"wpd_neon":26.242,"c_ref_apple_clang_O2":110.409}},{"id":"geomean","candidate":14.724,"reference":123.169,"speedup":8.365,"baselines":{"wpd_neon":14.578,"c_ref_apple_clang_O2":123.169}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the exercised cases, with strong tests that both miss and overconstrain some ABI behavior","gpt-6.1-sol-max":"Efficient implementation with small test and documentation gaps","opus-5.5-xhigh":"tight kernel, good docs, thorough tests","deepseek-v4.1-flash-max":"smallest, cleanest code, excellent documentation and strong tests; one narrow invariant in the 27/18/9 narrowing is undocumented.","muse-wonder-full-xhigh":"Correct, well documented and tested; extra permute/copy work on mbedge paths"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Full inverse transpose for six output columns","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Horizontal forbidden writes are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Test rejects an ABI-compliant vector-register change","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Sampled step pairs are called exhaustive","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Horizontal forbidden writes are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Pair coverage is described as exhaustive","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"`test_steps` is called \"Exhaustive pairs over (p0, q0) … every clamp and every rounding\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Unchanged rewrites of forbidden h columns are not tested, only v rows","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"the a0/a1/a2 narrowing relies on an undocumented fit","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Full 8-tap reverse transpose for h stores that write only 6 taps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Non-obvious `a` invariant unjustified","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Skips counted as passes","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":215,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.7,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.1,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1-b2","task":"vp8lf-neon","lead":"gpt-6.1-sol-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":9.41,"tokens":{"cache_read":4580864,"cache_write":0,"output":21081,"reasoning":11987,"uncached_input":113032},"cost_usd":0.4475,"cost_breakdown":{"cache_read":0.229043,"cache_write":0.0,"input":0.113032,"output":0.105405},"cost_usd_as_run":0.4475,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":11.25,"tokens":{"cache_read":4417200,"cache_write":0,"output":45261,"reasoning":26343,"uncached_input":216301},"cost_usd":2.6539,"cost_breakdown":{"cache_read":0.88344,"cache_write":0.0,"input":0.865204,"output":0.90522},"cost_usd_as_run":2.6539,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":16.68,"tokens":{"cache_read":2951040,"cache_write":0,"output":28413,"reasoning":17588,"uncached_input":118715},"cost_usd":0.8167,"cost_breakdown":{"cache_read":0.295104,"cache_write":0.0,"input":0.23743,"output":0.28413},"cost_usd_as_run":0.8167,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":16.92,"tokens":{"cache_read":12694144,"cache_write":0,"output":143603,"reasoning":101492,"uncached_input":128392},"cost_usd":0.1435,"cost_breakdown":{"cache_read":0.038082,"cache_write":0.0,"input":0.019259,"output":0.086162},"cost_usd_as_run":0.1435,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":23.99,"tokens":{"cache_read":4662154,"cache_write":206872,"output":68222,"reasoning":50336,"uncached_input":64},"cost_usd":3.9521,"cost_breakdown":{"cache_read":0.932431,"cache_write":1.654976,"input":0.000256,"output":1.36444},"cost_usd_as_run":3.9521,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"D","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"G","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":10.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"D","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"G","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"D","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"G","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"D","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"G","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"D","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"G","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max","opus-5.5-xhigh"]},"entries":[{"letter":"W","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":4.75,"tokens":{"uncached_input":75231,"cache_read":463142,"cache_write":0,"output":26595,"reasoning":0},"cost_usd":0.4627,"cost_breakdown":{"input":0.150462,"cache_read":0.046314,"cache_write":0.0,"output":0.26595},"cost_usd_as_run":0.4627,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":339,"code":256,"comment":45,"blank":38},{"path":"tests/selftest.c","role":"test","lines":543,"code":430,"comment":74,"blank":39}],"impl_lines":339,"impl_code":256,"impl_comment":45,"test_lines":543,"comment_ratio":0.15,"tests_declared":3,"impl_trailing_comments":90},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.002,"reference":139.68,"speedup":17.457,"baselines":{"wpd_neon":6.867,"c_ref_apple_clang_O2":139.68}},{"id":"h_mb","candidate":24.383,"reference":121.976,"speedup":5.003,"baselines":{"wpd_neon":21.558,"c_ref_apple_clang_O2":121.976}},{"id":"v_inner","candidate":14.109,"reference":134.816,"speedup":9.555,"baselines":{"wpd_neon":13.017,"c_ref_apple_clang_O2":134.816}},{"id":"h_inner","candidate":28.924,"reference":113.892,"speedup":3.938,"baselines":{"wpd_neon":27.425,"c_ref_apple_clang_O2":113.892}},{"id":"geomean","candidate":16.798,"reference":127.178,"speedup":7.571,"baselines":{"wpd_neon":15.162,"c_ref_apple_clang_O2":127.178}}]},"scores":{"gpt-6.1-terra-max":6.5,"muse-wonder-full-xhigh":6.5,"gpt-6.1-sol-max":6.0,"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct filter in the checks run, but its guard test accesses outside its mapping","muse-wonder-full-xhigh":"solid fast kernel, but its own test suite intermittently segfaults","gpt-6.1-sol-max":"Correct kernels, but the guard harness can corrupt memory and crash.","deepseek-v4.1-flash-max":"correct and quick on the v paths, but its read-only guard test never runs on macOS, the kernel carries five no-op masks, and the docs are thin.","opus-5.5-xhigh":"Lean, correct kernel and transposes, but its own `make test` crashes intermittently and the source is barely documented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"robustness","title":"Guard test protects and clears memory beyond its allocation","status":"valid","defect":"VP8-021","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Corrections are masked after their input was already masked","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"guard test overruns its mapping and crashes intermittently","status":"valid","defect":"VP8-021","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"guard cases never compare output","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"file-level documentation missing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"full back-transpose of unstored taps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"redundant mask ANDs after zeroing filter value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard mapping is undersized","status":"valid","defect":"VP8-021","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Corrections are unnecessarily masked again","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"read-only outer-row guard is dead on macOS","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"five no-op mask operations","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"src/vp8_lf_neon.S:1-3","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"second 8×8 transpose on the h result path","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"tests","title":"The guard test writes past its own 12-page mapping","status":"valid","defect":"VP8-021","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The guard test can only fail by crashing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The h path runs a full 8×8 inverse transpose plus 16 register moves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The file header is one line","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Masks are applied after `a` is already zero","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The last store pair is written out by hand","status":"valid","detail":"","location":""}],"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.4,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":1.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0,"opus-5.5-xhigh":1.0}},"defect_points":5.0,"score_cross_vendor":6.4,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","task":"vp8lf-neon","lead":"gpt-6.1-sol-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":29.14,"tokens":{"cache_read":21819904,"cache_write":0,"output":87828,"reasoning":43644,"uncached_input":586810},"cost_usd":2.117,"cost_breakdown":{"cache_read":1.090995,"cache_write":0.0,"input":0.58681,"output":0.43914},"cost_usd_as_run":2.117,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":78.97,"tokens":{"cache_read":16250624,"cache_write":0,"output":112552,"reasoning":69588,"uncached_input":648174},"cost_usd":4.0469,"cost_breakdown":{"cache_read":1.625063,"cache_write":0.0,"input":1.296348,"output":1.12552},"cost_usd_as_run":4.0469,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":76.38,"tokens":{"cache_read":33280789,"cache_write":946559,"output":272490,"reasoning":172386,"uncached_input":338},"cost_usd":19.6798,"cost_breakdown":{"cache_read":6.656158,"cache_write":7.572472,"input":0.001352,"output":5.4498},"cost_usd_as_run":19.6798,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":63.15,"tokens":{"cache_read":44825344,"cache_write":0,"output":452997,"reasoning":365627,"uncached_input":469839},"cost_usd":0.4767,"cost_breakdown":{"cache_read":0.134476,"cache_write":0.0,"input":0.070476,"output":0.271798},"cost_usd_as_run":0.4767,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":97.44,"tokens":{"cache_read":23270484,"cache_write":0,"output":163903,"reasoning":88238,"uncached_input":801240},"cost_usd":11.1371,"cost_breakdown":{"cache_read":4.654097,"cache_write":0.0,"input":3.20496,"output":3.27806},"cost_usd_as_run":11.1371,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":10.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":10.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"B","score":9.5}],"panel":["gpt-6.1-terra-max","gpt-6.1-sol-max","opus-5.5-xhigh","deepseek-v4.1-flash-max","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":83.8,"tokens":{"uncached_input":295482,"cache_read":3871360,"cache_write":0,"output":349092,"reasoning":160890},"cost_usd":4.6212,"cost_breakdown":{"input":0.590964,"cache_read":1.93568,"cache_write":0.0,"output":2.094552},"cost_usd_as_run":4.6212,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":353,"code":308,"comment":25,"blank":20},{"path":"tests/selftest.c","role":"test","lines":1006,"code":916,"comment":42,"blank":48}],"impl_lines":353,"impl_code":308,"impl_comment":25,"test_lines":1006,"comment_ratio":0.075,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":9.85,"reference":134.157,"speedup":13.621,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.157}},{"id":"h_mb","candidate":29.003,"reference":116.999,"speedup":4.034,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":116.999}},{"id":"v_inner","candidate":12.875,"reference":129.595,"speedup":10.066,"baselines":{"wpd_neon":12.546,"c_ref_apple_clang_O2":129.595}},{"id":"h_inner","candidate":30.901,"reference":109.568,"speedup":3.546,"baselines":{"wpd_neon":26.177,"c_ref_apple_clang_O2":109.568}},{"id":"geomean","candidate":18.361,"reference":122.184,"speedup":6.655,"baselines":{"wpd_neon":14.53,"c_ref_apple_clang_O2":122.184}}]},"scores":{"gpt-6.1-terra-max":6.5,"deepseek-v4.1-flash-max":6.5,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the common probes, but its self-test fails on 16 KB pages","deepseek-v4.1-flash-max":"correct filter, but its own test suite fails on the SPEC's Apple platform","opus-5.5-xhigh":"Correct, but the selftest fails on the Mac and this is the slowest h path","gpt-6.1-sol-max":"Correct arithmetic, but the page-size restriction breaks platform testing and horizontal filtering does unnecessary work.","muse-wonder-full-xhigh":"Bit-exact with strong tests; h path spills through the stack"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Guard test rejects the M1 page size","status":"valid","defect":"VP8-016","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal loads spill the first transpose to the stack","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"`test_guards` hard-fails whenever the page size exceeds 4096, which is always on Apple Silicon","status":"valid","defect":"VP8-016","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"`load_horizontal` spills eight transposed vectors to the stack and reloads them between the two 8x8 transposes","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"the `main` guard bookkeeping (`if (failed == 0 && test_guards()) passed++","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"the guard test rejects 16 KiB pages","status":"valid","defect":"VP8-016","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"the first h transpose is parked on the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"gaps in coverage","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"v0/v7 are XORed in and out but never used","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"tests","title":"Guard suite rejects 16 KB pages","status":"valid","defect":"VP8-016","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Avoidable stack traffic and repeated transposes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"h load parks the first transpose on the stack","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Outer taps biased and restored but never stored","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Header misdescribes the base accumulation","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":24,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.6,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":4.5,"score_cross_vendor":6.6,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":19.37,"tokens":{"uncached_input":40,"cache_read":2190897,"cache_write":154661,"output":110460,"reasoning":78534},"cost_usd":3.8848,"cost_breakdown":{"input":0.00016,"cache_read":0.438179,"cache_write":1.237288,"output":2.2092},"cost_usd_as_run":3.8848,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":326,"code":267,"comment":51,"blank":8},{"path":"tests/selftest.c","role":"test","lines":973,"code":814,"comment":90,"blank":69}],"impl_lines":326,"impl_code":267,"impl_comment":51,"test_lines":973,"comment_ratio":0.16,"tests_declared":7,"impl_trailing_comments":82},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":6.752,"reference":134.125,"speedup":19.865,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.125}},{"id":"h_mb","candidate":18.723,"reference":116.862,"speedup":6.242,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":116.862}},{"id":"v_inner","candidate":12.899,"reference":129.523,"speedup":10.042,"baselines":{"wpd_neon":12.539,"c_ref_apple_clang_O2":129.523}},{"id":"h_inner","candidate":24.706,"reference":109.945,"speedup":4.45,"baselines":{"wpd_neon":26.201,"c_ref_apple_clang_O2":109.945}},{"id":"geomean","candidate":14.167,"reference":122.23,"speedup":8.628,"baselines":{"wpd_neon":14.533,"c_ref_apple_clang_O2":122.23}}]},"scores":{"gpt-6.1-terra-max":9.5,"deepseek-v4.1-flash-max":9.5,"opus-5.5-xhigh":9.5,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Strong implementation and unusually thorough differential and guard tests","deepseek-v4.1-flash-max":"no defect found: fastest h/`hi`, smallest functions, and the strongest test suite in the set","opus-5.5-xhigh":"Leanest code, accurate docs, best tests","gpt-6.1-sol-max":"Compact, correct implementation with strong, self-validating tests.","muse-wonder-full-xhigh":"Leanest bit-exact kernel with self-verifying test coverage"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Page-layout comment assumes 4 KB pages","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved/x18/d8–d15 check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"No range statement for saturating narrow","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0524,"lines":14,"significant":267,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":23,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.6,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.3,"score_cross_vendor":9.625,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.77,"tokens":{"uncached_input":18,"cache_read":517581,"cache_write":67855,"output":29961,"reasoning":13090},"cost_usd":1.2456,"cost_breakdown":{"input":7.2e-05,"cache_read":0.103516,"cache_write":0.54284,"output":0.59922},"cost_usd_as_run":1.2456,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":312,"code":265,"comment":37,"blank":10},{"path":"tests/selftest.c","role":"test","lines":595,"code":487,"comment":63,"blank":45}],"impl_lines":312,"impl_code":265,"impl_comment":37,"test_lines":595,"comment_ratio":0.123,"tests_declared":6,"impl_trailing_comments":54},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":6.647,"reference":133.39,"speedup":20.068,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.39}},{"id":"h_mb","candidate":19.752,"reference":116.788,"speedup":5.913,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.788}},{"id":"v_inner","candidate":12.916,"reference":129.618,"speedup":10.036,"baselines":{"wpd_neon":12.539,"c_ref_apple_clang_O2":129.618}},{"id":"h_inner","candidate":26.703,"reference":109.697,"speedup":4.108,"baselines":{"wpd_neon":26.181,"c_ref_apple_clang_O2":109.697}},{"id":"geomean","candidate":14.587,"reference":121.996,"speedup":8.363,"baselines":{"wpd_neon":14.519,"c_ref_apple_clang_O2":121.996}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":9.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct filter, but its protection tests use only out-of-domain strides","deepseek-v4.1-flash-max":"fastest or tied, compact, well documented and well tested; one test-only assumption","opus-5.5-xhigh":"Lean, clear and well documented; thorough tests","gpt-6.1-sol-max":"Correct shared kernel; guard tests omit the contractual stride range.","muse-wonder-full-xhigh":"Bit-exact and lean; only a missing range proof for the narrowing shift"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guard tests exercise strides greater than 4096","status":"valid","defect":"VP8-017","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"the guard test demands behaviour at strides the contract does not cover","status":"valid","defect":"VP8-017","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"the guard test calls outside the stride domain","status":"valid","defect":"VP8-017","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved/x18/d8 check.","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"All guard calls use out-of-domain strides","status":"valid","defect":"VP8-017","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"No range argument for truncating `shrn`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":265,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.7,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.1,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":65.44,"tokens":{"uncached_input":307194,"cache_read":20825728,"cache_write":0,"output":195353,"reasoning":141630},"cost_usd":0.7685,"cost_breakdown":{"input":0.046079,"cache_read":0.624772,"cache_write":0.0,"output":0.097676},"cost_usd_as_run":0.7685,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":561,"code":407,"comment":127,"blank":27},{"path":"tests/selftest.c","role":"test","lines":539,"code":453,"comment":50,"blank":36}],"impl_lines":561,"impl_code":407,"impl_comment":127,"test_lines":539,"comment_ratio":0.238,"tests_declared":5,"impl_trailing_comments":146},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":11.015,"reference":134.314,"speedup":12.194,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.314}},{"id":"h_mb","candidate":28.1,"reference":117.103,"speedup":4.167,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":117.103}},{"id":"v_inner","candidate":13.451,"reference":129.656,"speedup":9.639,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.656}},{"id":"h_inner","candidate":26.184,"reference":109.85,"speedup":4.195,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.85}},{"id":"geomean","candidate":18.171,"reference":122.341,"speedup":6.733,"baselines":{"wpd_neon":14.523,"c_ref_apple_clang_O2":122.341}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":5.5,"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct filter, but its guard test both misses violations and rejects permitted behavior","deepseek-v4.1-flash-max":"best-in-class docs, but the slowest implementation and a macroblock path that computes a filter base it never reads","opus-5.5-xhigh":"Correct and well documented, but its guard test contradicts the contract and the kernel is the heaviest","gpt-6.1-sol-max":"Correct filtering, but expensive processing and misleadingly weak memory tests need changes.","muse-wonder-full-xhigh":"Bit-exact with the heaviest tests; largest code and one wrong comment"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guard pages are too far from the window","status":"valid","defect":"VP8-018","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Read-only test rejects allowed unchanged stores","status":"valid","defect":"VP8-019","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Macroblock kernel computes unused inner-filter values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Mask comment names the minimum","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"`base4` and `f1_4`/`f2_4` are computed on every call but only the inner expansion reads them","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"slowest of the twelve on all four functions (17.0/36.2/15.3/29.0 ns vs 13.1/23.8/11.3/17.2; 146/379/110/297 static instructions)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"the read-only-page guard case requires that *no* store happens when every line fails the mask","status":"valid","defect":"VP8-019","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"the read-only guard demands zero stores","status":"valid","defect":"VP8-019","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"the most instructions of any entry","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"the guard comment says any outside access faults","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"wrong comment on the interior compare","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved/x18/d8 check.","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal path repeatedly transposes and repacks complete rows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard window is centered far from its boundaries","status":"valid","defect":"VP8-018","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Inner-only corrections run in macroblock kernels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Unused sweep axis repeats cases seven times","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Comment says \"min interior\" for a maximum test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Macroblock path computes then discards the 4-tap base","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard test passes when it cannot run","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Sixteen merge moves per h load","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Comment bulk repeats the contract","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":425,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":109,"flags":6,"audited":"2026-11-02","reviewed":true},"score":7.0,"slop":{"score":1.9,"per_seat":{"gpt-6.1-terra-max":1.5,"gpt-6.1-sol-max":2.0,"opus-5.5-xhigh":1.5,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":3.5}},"defect_points":4.9,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":356.36,"tokens":{"uncached_input":444590,"cache_read":3829420,"cache_write":0,"output":60200,"reasoning":32350},"cost_usd":3.7482,"cost_breakdown":{"input":1.77836,"cache_read":0.765884,"cache_write":0.0,"output":1.204},"cost_usd_as_run":3.7482,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":532,"code":461,"comment":55,"blank":16},{"path":"tests/selftest.c","role":"test","lines":508,"code":447,"comment":33,"blank":28}],"impl_lines":532,"impl_code":461,"impl_comment":55,"test_lines":508,"comment_ratio":0.107,"tests_declared":6,"impl_trailing_comments":31},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":9.472,"reference":134.02,"speedup":14.15,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.02}},{"id":"h_mb","candidate":23.878,"reference":117.198,"speedup":4.908,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":117.198}},{"id":"v_inner","candidate":12.533,"reference":129.591,"speedup":10.34,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.591}},{"id":"h_inner","candidate":28.066,"reference":109.772,"speedup":3.911,"baselines":{"wpd_neon":26.184,"c_ref_apple_clang_O2":109.772}},{"id":"geomean","candidate":16.794,"reference":122.261,"speedup":7.28,"baselines":{"wpd_neon":14.521,"c_ref_apple_clang_O2":122.261}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":7.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and well tested, with avoidable horizontal-store overhead","deepseek-v4.1-flash-max":"solid, correct, mid-pack; some instruction-level redundancy","opus-5.5-xhigh":"Correct, well commented, good tests; slow constant setup and store path","gpt-6.1-sol-max":"Correct and well tested, with an unnecessarily expensive horizontal path.","muse-wonder-full-xhigh":"Bit-exact and well commented step-by-step; slowest-shaped h code"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal stores transfer each row through general-purpose registers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"`lf_store_h_batch` spells out the same five-instruction store sequence eight times and is instantiated twice, giving 16 copies","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"h is 29.6/25.8 ns vs 23.8/17.2 best (361/289 static instructions)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"nine byte constants are materialised as `mov w9, #imm; dup vN.16b, w9`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"avoidable cost throughout","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved/x18/d8 check (d8 mutant passes).","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Gather/scatter expands excessive work","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Serial mask chain and scalarised stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Four near-duplicate transpose macros","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Directed coverage is an order of magnitude thinner than peers","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.2104,"lines":97,"significant":461,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":38,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.6,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":1.5,"muse-wonder-full-xhigh":1.0}},"defect_points":2.2,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":330.38,"tokens":{"uncached_input":213492,"cache_read":2277244,"cache_write":0,"output":37386,"reasoning":19397},"cost_usd":2.0571,"cost_breakdown":{"input":0.853968,"cache_read":0.455449,"cache_write":0.0,"output":0.74772},"cost_usd_as_run":2.0571,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":336,"code":282,"comment":45,"blank":9},{"path":"tests/selftest.c","role":"test","lines":507,"code":428,"comment":50,"blank":29}],"impl_lines":336,"impl_code":282,"impl_comment":45,"test_lines":507,"comment_ratio":0.138,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":9.157,"reference":134.083,"speedup":14.643,"baselines":{"wpd_neon":6.605,"c_ref_apple_clang_O2":134.083}},{"id":"h_mb","candidate":28.992,"reference":116.999,"speedup":4.035,"baselines":{"wpd_neon":20.634,"c_ref_apple_clang_O2":116.999}},{"id":"v_inner","candidate":15.333,"reference":129.737,"speedup":8.461,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.737}},{"id":"h_inner","candidate":31.081,"reference":109.558,"speedup":3.525,"baselines":{"wpd_neon":26.181,"c_ref_apple_clang_O2":109.558}},{"id":"geomean","candidate":18.86,"reference":122.198,"speedup":6.479,"baselines":{"wpd_neon":14.547,"c_ref_apple_clang_O2":122.198}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct filter with a gap in read-bound testing","deepseek-v4.1-flash-max":"bit-exact and clean, but the slowest h path of the set and a little dead arithmetic","opus-5.5-xhigh":"Correct and well commented, but the h path is heavy and the tests never check over-reads","gpt-6.1-sol-max":"Correct arithmetic and stores, with avoidable transpose work and incomplete read guards.","muse-wonder-full-xhigh":"Bit-exact with clean structure; serial mask and four-block transposes cost ~2×"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard cases do not protect against overreads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Unused outer taps are biased and restored","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"v0/v7 are biased into the signed domain and biased back although nothing reads them there","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"the h path is second-slowest of the twelve (34.9 ns `h`, 29.2 ns `hi`, vs 23.8/17.2 for the best; 367/302 static instructions vs 163/120)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"the h path rebuilds 16-lane vectors the slow way","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"padded mask code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"no PROT_NONE guard","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved/x18/d8 check.","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal filtering repeats complete half-block networks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guard pages remain readable","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Dependent mask chain and extracted-half transposes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"p3/q3 biased and restored for nothing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Directed suite reports no case count","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1773,"lines":50,"significant":282,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":26,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.3,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":2.5,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":21.61,"tokens":{"uncached_input":50140,"cache_read":9850368,"cache_write":0,"output":198700,"reasoning":156388},"cost_usd":0.1563,"cost_breakdown":{"input":0.007521,"cache_read":0.029551,"cache_write":0.0,"output":0.11922},"cost_usd_as_run":0.1563,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1563,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":612,"code":498,"comment":89,"blank":25},{"path":"tests/selftest.c","role":"test","lines":772,"code":654,"comment":62,"blank":56}],"impl_lines":612,"impl_code":498,"impl_comment":89,"test_lines":772,"comment_ratio":0.152,"tests_declared":7,"impl_trailing_comments":79},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":10.091,"reference":134.293,"speedup":13.308,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":134.293}},{"id":"h_mb","candidate":24.624,"reference":117.282,"speedup":4.763,"baselines":{"wpd_neon":20.634,"c_ref_apple_clang_O2":117.282}},{"id":"v_inner","candidate":10.115,"reference":129.639,"speedup":12.817,"baselines":{"wpd_neon":12.587,"c_ref_apple_clang_O2":129.639}},{"id":"h_inner","candidate":29.076,"reference":109.941,"speedup":3.781,"baselines":{"wpd_neon":26.198,"c_ref_apple_clang_O2":109.941}},{"id":"geomean","candidate":16.442,"reference":122.404,"speedup":7.445,"baselines":{"wpd_neon":14.554,"c_ref_apple_clang_O2":122.404}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct filter, but capped generators substantially weaken its directed tests","deepseek-v4.1-flash-max":"excellent documentation and good tests; the store paths are hand-unrolled to a fault","opus-5.5-xhigh":"Correct, but comments are wrong in places and there is heavy copy-paste","gpt-6.1-sol-max":"Correct outputs, weakened by ineffective arithmetic tests and excess instruction work.","muse-wonder-full-xhigh":"Bit-exact with broad tests; false \"untouched\" claim and scalar h stores"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Directed generators fill their lists before reaching later cases","status":"valid","defect":"VP8-022","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Guard comment claims `memcmp` detects unchanged stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Kernel output contract says outer registers are untouched","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Outer pixels are converted without being used","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"verbosity","title":"the two h store paths are ~100 lines each of `umov w5, vN.s[i]` + `str` per row","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"the `umov`→GPR→`str` round trip is one extra instruction per store and a GPR/SIMD transfer on the critical path (`h` 29.2 ns, `hi` 20.9 ns vs 23.8/17.2 best)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"the LF_FILTER contract is wrong","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"the guard test claims more than it can check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"blocks duplicated verbatim between the mbedge and inner entry points","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"h stores go through general registers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved/x18/d8 check (d8 mutant passes).","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Arithmetic corpus never passes the mask","status":"valid","defect":"VP8-022","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal paths repeat transposes and extract scalar stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Filter macro violates its documented register contract","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Header claims v0/v7 untouched; they are biased and left biased","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard and read-only tests pass when mmap/mprotect fails","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"h results stored via 32 scalar extracts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Double masking around `bit` selects","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.3614,"lines":180,"significant":498,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":55,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":2.0,"per_seat":{"gpt-6.1-terra-max":2.5,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":3.0,"deepseek-v4.1-flash-max":2.0,"muse-wonder-full-xhigh":1.5}},"defect_points":4.1,"score_cross_vendor":7.25,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.3,"tokens":{"uncached_input":44378,"cache_read":425088,"cache_write":0,"output":9938,"reasoning":2642},"cost_usd":0.1153,"cost_breakdown":{"input":0.044378,"cache_read":0.021254,"cache_write":0.0,"output":0.04969},"cost_usd_as_run":0.1153,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":280,"code":269,"comment":4,"blank":7},{"path":"tests/selftest.c","role":"test","lines":351,"code":304,"comment":26,"blank":21}],"impl_lines":280,"impl_code":269,"impl_comment":4,"test_lines":351,"comment_ratio":0.015,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.18,"reference":134.44,"speedup":16.435,"baselines":{"wpd_neon":6.605,"c_ref_apple_clang_O2":134.44}},{"id":"h_mb","candidate":29.15,"reference":117.019,"speedup":4.014,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":117.019}},{"id":"v_inner","candidate":13.919,"reference":129.578,"speedup":9.309,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.578}},{"id":"h_inner","candidate":31.23,"reference":109.843,"speedup":3.517,"baselines":{"wpd_neon":26.221,"c_ref_apple_clang_O2":109.843}},{"id":"geomean","candidate":17.943,"reference":122.327,"speedup":6.818,"baselines":{"wpd_neon":14.553,"c_ref_apple_clang_O2":122.327}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":7.0,"opus-5.5-xhigh":6.5,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct filter, with ineffective edge-boundary cases and skipped Mac protection checks","deepseek-v4.1-flash-max":"correct and compact, but undocumented and carrying pure register-shuffle overhead","opus-5.5-xhigh":"Correct with solid tests, but undocumented and move-heavy","gpt-6.1-sol-max":"Correct results, but horizontal processing is costly and some directed boundaries are unobservable.","muse-wonder-full-xhigh":"Bit-exact with thoughtful boundary tests; no header and heavy transposes"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Edge sweep does not construct the stated threshold","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Vertical protection tests are skipped on 16 KB pages","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"no file header at all and 4 comment lines in 280","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`transpose8` routes every result through an extra `mov` (8 per transpose) and `horizontal` adds 16 more `mov`s to park halves","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`product` re-materialises `#63` on each of its three expansions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"no file header at all","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"heavy shuffling in the h path","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"the \"round\" comment is inaccurate","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"hev_thresh is broadcast twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved/x18/d8 check.","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Four full transposes and repeated copies per horizontal call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Outer interior-boundary cases produce zero correction","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No file header; four comment lines total","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Transpose network expanded four times per h call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"v page test silently passes on 16K-page hosts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Weight macro mislabelled as rounding","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0967,"lines":26,"significant":269,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.0,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":3.5,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.03,"tokens":{"uncached_input":30825,"cache_read":257280,"cache_write":0,"output":5370,"reasoning":820},"cost_usd":0.1411,"cost_breakdown":{"input":0.06165,"cache_read":0.025728,"cache_write":0.0,"output":0.0537},"cost_usd_as_run":0.1411,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":386,"code":379,"comment":1,"blank":6},{"path":"tests/selftest.c","role":"test","lines":293,"code":252,"comment":22,"blank":19}],"impl_lines":386,"impl_code":379,"impl_comment":1,"test_lines":293,"comment_ratio":0.003,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.466,"reference":133.6,"speedup":17.894,"baselines":{"wpd_neon":6.626,"c_ref_apple_clang_O2":133.6}},{"id":"h_mb","candidate":22.892,"reference":116.862,"speedup":5.105,"baselines":{"wpd_neon":20.739,"c_ref_apple_clang_O2":116.862}},{"id":"v_inner","candidate":13.333,"reference":130.212,"speedup":9.766,"baselines":{"wpd_neon":12.617,"c_ref_apple_clang_O2":130.212}},{"id":"h_inner","candidate":26.754,"reference":110.206,"speedup":4.119,"baselines":{"wpd_neon":26.276,"c_ref_apple_clang_O2":110.206}},{"id":"geomean","candidate":15.713,"reference":122.344,"speedup":7.786,"baselines":{"wpd_neon":14.61,"c_ref_apple_clang_O2":122.344}}]},"scores":{"gpt-6.1-terra-max":6.0,"deepseek-v4.1-flash-max":7.5,"opus-5.5-xhigh":6.0,"gpt-6.1-sol-max":5.5,"muse-wonder-full-xhigh":6.0},"council_summaries":{"gpt-6.1-terra-max":"Correct filter in common probes, but its boundary test writes outside its buffers","deepseek-v4.1-flash-max":"very fast, very compact, effectively undocumented","opus-5.5-xhigh":"Correct kernel, but the test has a buffer overflow; padded transposes and almost no docs","gpt-6.1-sol-max":"Correct production kernel, but its self-test writes outside its buffers.","muse-wonder-full-xhigh":"Bit-exact, but uncommented code and no over-read test"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Horizontal boundary sweep overruns both static buffers","status":"valid","defect":"VP8-020","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Each transpose copies every intermediate vector","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"one comment line in 386","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`transpose` performs 24 `mov vN.16b` register copies per invocation (48 per `h` call)","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"the h store loop repeats the same 12-line block sixteen times","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the guard test cannot detect over-reads in the `v` functions","status":"valid","defect":"VP8-018","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"out-of-bounds writes in `test_boundaries`","status":"valid","defect":"VP8-020","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"effectively undocumented","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"wasted moves in the transpose","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"no PROT_NONE guards","status":"valid","defect":"VP8-018","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"rebuilt constants and dead updates","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"robustness","title":"Horizontal boundary cases overflow test buffers","status":"valid","defect":"VP8-020","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Protection tests cannot detect overreads","status":"valid","defect":"VP8-018","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Transpose copies every intermediate stage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Own suite never checks for over-reads","status":"valid","defect":"VP8-018","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"One comment line in the deliverable","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Transpose copies every register between stages","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Boundary thresholds vary only in lockstep","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.2929,"lines":111,"significant":379,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.2,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":5.1,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.59,"tokens":{"uncached_input":16468,"cache_read":248192,"cache_write":0,"output":7663,"reasoning":1866},"cost_usd":0.796,"cost_breakdown":{"input":0.16468,"cache_read":0.248192,"cache_write":0.0,"output":0.38315},"cost_usd_as_run":0.796,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":362,"code":343,"comment":8,"blank":11},{"path":"tests/selftest.c","role":"test","lines":340,"code":292,"comment":28,"blank":20}],"impl_lines":362,"impl_code":343,"impl_comment":8,"test_lines":340,"comment_ratio":0.023,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.55,"reference":134.325,"speedup":17.791,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.325}},{"id":"h_mb","candidate":24.845,"reference":117.009,"speedup":4.71,"baselines":{"wpd_neon":20.634,"c_ref_apple_clang_O2":117.009}},{"id":"v_inner","candidate":13.94,"reference":129.754,"speedup":9.308,"baselines":{"wpd_neon":12.587,"c_ref_apple_clang_O2":129.754}},{"id":"h_inner","candidate":27.405,"reference":109.907,"speedup":4.011,"baselines":{"wpd_neon":26.208,"c_ref_apple_clang_O2":109.907}},{"id":"geomean","candidate":16.361,"reference":122.358,"speedup":7.479,"baselines":{"wpd_neon":14.55,"c_ref_apple_clang_O2":122.358}}]},"scores":{"gpt-6.1-terra-max":9.0,"deepseek-v4.1-flash-max":7.0,"opus-5.5-xhigh":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation with strong directed and protection tests","deepseek-v4.1-flash-max":"correct and tidy arithmetic, but almost undocumented and mechanically duplicated","opus-5.5-xhigh":"Lean and correct, with strong tests; thin docs and a 160-line hand-copied store block","gpt-6.1-sol-max":"Correct SIMD implementation with minor guard-coverage and maintenance issues.","muse-wonder-full-xhigh":"Lean, bit-exact kernel; near-undocumented transpose/store tricks"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Coefficients are broadcast twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"the file carries 8 comment lines for 362","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"`horizontal` repeats the same 10-line per-row store block eight times with only the register number changed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`weighted` re-materialises `#63` and the coefficient on every expansion","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"h store sequence is copy-pasted 16 times","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"almost no documentation","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"constants rebuilt for every product","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved/x18/d8–d15 check","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Inner vertical guard leaves p3 writable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Row-store logic is copied sixteen times","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Eight comment lines for the whole file","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"h store extracts every row with `ext`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Horizontal store unrolled by copy-paste","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.035,"lines":12,"significant":343,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.8,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.5,"deepseek-v4.1-flash-max":1.5,"muse-wonder-full-xhigh":0.5}},"defect_points":2.3,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.77,"tokens":{"uncached_input":22886,"cache_read":216576,"cache_write":0,"output":10107,"reasoning":2162},"cost_usd":0.1685,"cost_breakdown":{"input":0.045772,"cache_read":0.021658,"cache_write":0.0,"output":0.10107},"cost_usd_as_run":0.1685,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":243,"code":232,"comment":4,"blank":7},{"path":"tests/selftest.c","role":"test","lines":376,"code":323,"comment":32,"blank":21}],"impl_lines":243,"impl_code":232,"impl_comment":4,"test_lines":376,"comment_ratio":0.017,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.424,"reference":133.516,"speedup":17.984,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":133.516}},{"id":"h_mb","candidate":30.515,"reference":116.936,"speedup":3.832,"baselines":{"wpd_neon":20.665,"c_ref_apple_clang_O2":116.936}},{"id":"v_inner","candidate":14.34,"reference":129.696,"speedup":9.044,"baselines":{"wpd_neon":12.573,"c_ref_apple_clang_O2":129.696}},{"id":"h_inner","candidate":30.25,"reference":109.887,"speedup":3.633,"baselines":{"wpd_neon":26.204,"c_ref_apple_clang_O2":109.887}},{"id":"geomean","candidate":17.705,"reference":122.135,"speedup":6.898,"baselines":{"wpd_neon":14.557,"c_ref_apple_clang_O2":122.135}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but the horizontal path does substantial redundant work.","opus-5.5-xhigh":"correct, but the h path is about twice as long as needed and docs are minimal","deepseek-v4.1-flash-max":"correct but pays ~140 instructions on every `h` call for its transpose scheme","gpt-6.1-sol-max":"Correct implementation with expensive horizontal shuffling and guards limited to rejected masks.","muse-wonder-full-xhigh":"correct but slow h transposes and almost no rationale"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Four low-half transposes are used for each horizontal call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Directed cases miss an omitted outer comparison","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The h path uses four half-width transposes, each ending in 8 `mov`s, plus 8 `ins` and 8 `dup`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The only documentation is a one-line header","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The inner edge flips p2/q2 for nothing, and `movi #1`+`add`+`sshr` stands in for `srshr`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The test comment \"16 KiB Apple pages: use the boundary tests at minimum stride\" describes handling the code doesn't have","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI register checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"the `h` path transposes the two 8-row halves separately and merges/splits them again","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`store_row` post-indexes by 2 and then subtracts the 2 back","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Half-width transpose machinery adds excessive work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Protected tests never exercise accepted filtering","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Four separate 64-bit 8x8 transposes plus ins/dup merging","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Saturation tricks unexplained","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0517,"lines":12,"significant":232,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.5,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.9,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.89,"tokens":{"uncached_input":15404,"cache_read":446080,"cache_write":0,"output":19064,"reasoning":8947},"cost_usd":1.5533,"cost_breakdown":{"input":0.15404,"cache_read":0.44608,"cache_write":0.0,"output":0.9532},"cost_usd_as_run":1.5533,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":242,"code":206,"comment":20,"blank":16},{"path":"tests/selftest.c","role":"test","lines":534,"code":462,"comment":43,"blank":29}],"impl_lines":242,"impl_code":206,"impl_comment":20,"test_lines":534,"comment_ratio":0.088,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.351,"reference":134.01,"speedup":18.231,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.01}},{"id":"h_mb","candidate":20.298,"reference":117.114,"speedup":5.77,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.114}},{"id":"v_inner","candidate":13.475,"reference":129.493,"speedup":9.61,"baselines":{"wpd_neon":12.543,"c_ref_apple_clang_O2":129.493}},{"id":"h_inner","candidate":26.852,"reference":109.972,"speedup":4.095,"baselines":{"wpd_neon":26.194,"c_ref_apple_clang_O2":109.972}},{"id":"geomean","candidate":15.243,"reference":122.269,"speedup":8.021,"baselines":{"wpd_neon":14.529,"c_ref_apple_clang_O2":122.269}}]},"scores":{"gpt-6.1-terra-max":9.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct and well tested, with a small horizontal-store optimization available.","opus-5.5-xhigh":"correct, compact, transposes with no moves; good guard tests","deepseek-v4.1-flash-max":"same design as T with the constant hoisted; nothing to fix","gpt-6.1-sol-max":"Correct, well-structured implementation with strong portable tests.","muse-wonder-full-xhigh":"correct, clean and efficient; only a documentation nit"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal output transposes all eight taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No callee-saved/x18/d8–d15/dirty-argument checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Inverse transpose processes unused taps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Header omits the memory contract","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0777,"lines":16,"significant":206,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.1,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.7,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":46.44,"tokens":{"uncached_input":140580,"cache_read":2386056,"cache_write":0,"output":54420,"reasoning":35088},"cost_usd":2.1279,"cost_breakdown":{"input":0.56232,"cache_read":0.477211,"cache_write":0.0,"output":1.0884},"cost_usd_as_run":2.1279,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":396,"code":343,"comment":27,"blank":26},{"path":"tests/selftest.c","role":"test","lines":626,"code":533,"comment":55,"blank":38}],"impl_lines":396,"impl_code":343,"impl_comment":27,"test_lines":626,"comment_ratio":0.073,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.537,"reference":134.02,"speedup":15.699,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":134.02}},{"id":"h_mb","candidate":26.829,"reference":117.135,"speedup":4.366,"baselines":{"wpd_neon":20.655,"c_ref_apple_clang_O2":117.135}},{"id":"v_inner","candidate":15.028,"reference":129.496,"speedup":8.617,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.496}},{"id":"h_inner","candidate":29.643,"reference":109.768,"speedup":3.703,"baselines":{"wpd_neon":26.215,"c_ref_apple_clang_O2":109.768}},{"id":"geomean","candidate":17.872,"reference":122.222,"speedup":6.839,"baselines":{"wpd_neon":14.553,"c_ref_apple_clang_O2":122.222}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with expensive transposes and test gaps.","opus-5.5-xhigh":"correct and reasonably documented, but slow h paths and duplicated bodies","deepseek-v4.1-flash-max":"correct and clever, but the `h` entry points are duplicated wholesale","gpt-6.1-sol-max":"Correct filtering, but horizontal shuffling is costly and some targeted tests are ineffective.","muse-wonder-full-xhigh":"correct; duplicated h functions and a heavy transpose path"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal calls use four separate 8×8 transposes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Isolated interior tests can pass without the comparison","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests skip 16 KiB page hosts","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The h paths run four half-width `tr8` passes with 16 `mov` and 16 `ins`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The kernel flips all eight taps, computes the weights through `sxtl`+`mul`+`sshr`+`sqxtn`, and does HEV with two compares plus `orr`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The h and h_inner bodies duplicate the whole load/transpose/store sequence","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`movi #1`+`add`+`sshr` stands in for `srshr`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI register checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"the two `h` entry points repeat the same load/transpose/merge/store sequence","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"both guard-page tests skip unless the page size is exactly 4096","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h`/`h_inner` are 313/261 instructions (second-highest)","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal paths duplicate half-width transpose work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Horizontal guards miss overreads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Outer-seam cases have flat filter inputs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"64-bit transpose pipeline with shuffling","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"h functions duplicated","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guards skip-as-pass on 16 KiB pages","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"False register claim","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.5627,"lines":193,"significant":343,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":27,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.3,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.5,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.5}},"defect_points":4.0,"score_cross_vendor":7.375,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.15,"tokens":{"uncached_input":7147,"cache_read":74880,"cache_write":0,"output":316,"reasoning":24},"cost_usd":0.0125,"cost_breakdown":{"input":0.007147,"cache_read":0.003744,"cache_write":0.0,"output":0.00158},"cost_usd_as_run":0.0125,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":23,"code":13,"comment":6,"blank":4},{"path":"tests/selftest.c","role":"test","lines":216,"code":181,"comment":18,"blank":17}],"impl_lines":23,"impl_code":13,"impl_comment":6,"test_lines":216,"comment_ratio":0.316,"tests_declared":1,"impl_trailing_comments":0},"hidden":{"passed":3,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.935,"reference":134.01,"speedup":143.387,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.01}},{"id":"h_mb","candidate":0.945,"reference":117.324,"speedup":124.139,"baselines":{"wpd_neon":20.655,"c_ref_apple_clang_O2":117.324}},{"id":"v_inner","candidate":0.939,"reference":129.72,"speedup":138.103,"baselines":{"wpd_neon":12.573,"c_ref_apple_clang_O2":129.72}},{"id":"h_inner","candidate":0.939,"reference":109.88,"speedup":116.981,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.88}},{"id":"geomean","candidate":0.94,"reference":122.352,"speedup":130.222,"baselines":{"wpd_neon":14.547,"c_ref_apple_clang_O2":122.352}}]},"scores":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.0},"council_summaries":{"gpt-6.1-terra-max":"All four entry points are stubs.","opus-5.5-xhigh":"unmodified stub: nothing is implemented","deepseek-v4.1-flash-max":"stub; the scaffold file was never replaced","gpt-6.1-sol-max":"All four filtering functions remain unimplemented.","muse-wonder-full-xhigh":"unmodified scaffold stub; no filtering at all"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"No filtering is performed","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"All four entry points are the scaffold stub","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The selftest is the unmodified scaffold","status":"valid","defect":"VP8-002","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"no implementation at all","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the selftest is the unextended scaffold","status":"valid","defect":"VP8-002","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Entry points immediately return","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"All four entry points are bare `ret`","status":"valid","defect":"VP8-001","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":3,"flags":0,"audited":"2026-11-02","reviewed":false},"score":0.2,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.0,"score_cross_vendor":0.0,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.05,"tokens":{"uncached_input":13948,"cache_read":142080,"cache_write":0,"output":4104,"reasoning":295},"cost_usd":0.4868,"cost_breakdown":{"input":0.13948,"cache_read":0.14208,"cache_write":0.0,"output":0.2052},"cost_usd_as_run":0.4868,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":370,"code":363,"comment":1,"blank":6},{"path":"tests/selftest.c","role":"test","lines":295,"code":251,"comment":25,"blank":19}],"impl_lines":370,"impl_code":363,"impl_comment":1,"test_lines":295,"comment_ratio":0.003,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.476,"reference":133.978,"speedup":17.92,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":133.978}},{"id":"h_mb","candidate":19.437,"reference":117.314,"speedup":6.036,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":117.314}},{"id":"v_inner","candidate":13.523,"reference":129.669,"speedup":9.589,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.669}},{"id":"h_inner","candidate":26.343,"reference":109.721,"speedup":4.165,"baselines":{"wpd_neon":26.215,"c_ref_apple_clang_O2":109.721}},{"id":"geomean","candidate":15.084,"reference":122.286,"speedup":8.107,"baselines":{"wpd_neon":14.545,"c_ref_apple_clang_O2":122.286}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but horizontal packing and documentation are weak.","opus-5.5-xhigh":"correct but undocumented, move-heavy transposes, hand-unrolled stores","deepseek-v4.1-flash-max":"correct, but a 370-line assembly kernel with a single comment and a wasteful transpose","gpt-6.1-sol-max":"Correct implementation with excessive register copying and weak memory probes.","muse-wonder-full-xhigh":"correct kernel let down by absent documentation and thin tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Transpose repeatedly copies entire register banks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests silently skip larger pages","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Kernel invariants are undocumented","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"No-op in the guard setup","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"There is no header and no description of the entry points, kernel, register use or memory windows; the file has one comment line in 370","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The transpose macro copies every stage back through v16..v23 with 24 `mov`s","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"The h store sequences are hand-unrolled 16 times (about 130 lines) instead of using a macro","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`add x5, x5, x1` follows the last row store","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The inner edge flips p2/q2 for nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Single-space instruction indentation, unlike the rest of the scaffold.","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"`test_pages` only checks that nothing faults (it never compares output), silently returns success on pages over 4 KiB, prints no result, and contains a no-op `d += 0;`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI register checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"the kernel is undocumented","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"the transpose shuffles every stage through `mov`s","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the guard-page test silently succeeds on page sizes above 4 KiB","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`movi v25.8h, #63` is re-executed for each weight","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Transpose repeatedly copies entire register banks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guard coverage misses horizontal overreads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"No-op pointer adjustment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Effectively uncommented","status":"valid","defect":"VP8-023","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Wasteful transpose and triplicated weights","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard test vacuous on Apple pages","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"No-op statement in guard test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.3058,"lines":111,"significant":363,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.0,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":2.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":4.7,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":12.37,"tokens":{"uncached_input":33892,"cache_read":404864,"cache_write":0,"output":22320,"reasoning":12805},"cost_usd":0.3315,"cost_breakdown":{"input":0.067784,"cache_read":0.040486,"cache_write":0.0,"output":0.2232},"cost_usd_as_run":0.3315,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":239,"code":203,"comment":20,"blank":16},{"path":"tests/selftest.c","role":"test","lines":509,"code":441,"comment":42,"blank":26}],"impl_lines":239,"impl_code":203,"impl_comment":20,"test_lines":509,"comment_ratio":0.09,"tests_declared":6,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.686,"reference":143.849,"speedup":18.714,"baselines":{"wpd_neon":6.983,"c_ref_apple_clang_O2":143.849}},{"id":"h_mb","candidate":23.07,"reference":119.382,"speedup":5.175,"baselines":{"wpd_neon":20.875,"c_ref_apple_clang_O2":119.382}},{"id":"v_inner","candidate":14.262,"reference":129.727,"speedup":9.096,"baselines":{"wpd_neon":12.607,"c_ref_apple_clang_O2":129.727}},{"id":"h_inner","candidate":27.903,"reference":114.261,"speedup":4.095,"baselines":{"wpd_neon":27.096,"c_ref_apple_clang_O2":114.261}},{"id":"geomean","candidate":16.299,"reference":126.312,"speedup":7.75,"baselines":{"wpd_neon":14.938,"c_ref_apple_clang_O2":126.312}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes; one threshold test and horizontal output packing need work.","opus-5.5-xhigh":"correct, branchless, tight full-width transposes; tests are strong but have no ABI checks","deepseek-v4.1-flash-max":"compact, well-documented and correct; nothing worth changing","gpt-6.1-sol-max":"Correct and maintainable, with minor performance and directed-test weaknesses.","muse-wonder-full-xhigh":"correct, compact fused-transpose kernel with page-size-agnostic tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Isolated outer comparisons have no visible adjustment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Reverse transpose includes unused taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The transpose's last stage copies results back through v16 with 4 `mov`s per transpose","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test of callee-saved registers, x18, d8–d15 or garbage in the upper bits of the int arguments","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Full inverse transpose exceeds output needs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Outer-seam tests produce no observable correction","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Lane mappings under-documented","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.069,"lines":14,"significant":203,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.8,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.2,"score_cross_vendor":8.833,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":47.78,"tokens":{"uncached_input":72,"cache_read":5961824,"cache_write":267557,"output":607425,"reasoning":577721},"cost_usd":1.0072,"cost_breakdown":{"input":2.4e-05,"cache_read":0.281344,"cache_write":0.230507,"output":0.495313},"cost_usd_as_run":1.0072,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":337,"code":265,"comment":51,"blank":21},{"path":"tests/selftest.c","role":"test","lines":546,"code":468,"comment":38,"blank":40}],"impl_lines":337,"impl_code":265,"impl_comment":51,"test_lines":546,"comment_ratio":0.161,"tests_declared":5,"impl_trailing_comments":68},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.466,"reference":133.611,"speedup":17.896,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.611}},{"id":"h_mb","candidate":27.795,"reference":116.799,"speedup":4.202,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.799}},{"id":"v_inner","candidate":11.237,"reference":129.673,"speedup":11.539,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.673}},{"id":"h_inner","candidate":27.781,"reference":109.697,"speedup":3.949,"baselines":{"wpd_neon":26.177,"c_ref_apple_clang_O2":109.697}},{"id":"geomean","candidate":15.954,"reference":122.062,"speedup":7.651,"baselines":{"wpd_neon":14.53,"c_ref_apple_clang_O2":122.062}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with avoidable work and a horizontal guard gap.","opus-5.5-xhigh":"correct and well documented; the h path uses half-width transposes","deepseek-v4.1-flash-max":"solid; guard coverage disappears off Linux","gpt-6.1-sol-max":"Correct kernel with costly horizontal transposes and incomplete horizontal memory tests.","muse-wonder-full-xhigh":"correct, well-documented; separate transposes and Mac-skipped guards"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal filtering performs four separate 8×8 transposes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner paths convert unused outer taps twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal guards do not detect unchanged outer stores","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The h path transposes in 64-bit halves: four `.8b` TRN8 passes plus 16 d-lane `mov`s and two `ext` per stored row","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The inner edge flips p2/q2 to signed and back although it never uses or stores them","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The h guard only makes rows −1 and 16 inaccessible; columns −4/3 (and −3/2 for inner) sit on writable pages","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No callee-saved/x18/d8–d15/dirty-argument checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the guard test skips on any page size other than 4096","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`movi v29.16b, #0x80` is emitted twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Transposes each vector half separately","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Horizontal guards protect rows rather than tap bounds","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Four separate 8x8 transposes with d-lane shuffling","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard suite does not run on 16 KiB pages","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Inner outputs misstated","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":265,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":37,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":2.7,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":14.93,"tokens":{"uncached_input":30941,"cache_read":509568,"cache_write":0,"output":26652,"reasoning":16215},"cost_usd":0.3794,"cost_breakdown":{"input":0.061882,"cache_read":0.050957,"cache_write":0.0,"output":0.26652},"cost_usd_as_run":0.3794,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":245,"code":200,"comment":31,"blank":14},{"path":"tests/selftest.c","role":"test","lines":537,"code":457,"comment":48,"blank":32}],"impl_lines":245,"impl_code":200,"impl_comment":31,"test_lines":537,"comment_ratio":0.134,"tests_declared":7,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.351,"reference":134.451,"speedup":18.291,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":134.451}},{"id":"h_mb","candidate":21.096,"reference":117.23,"speedup":5.557,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":117.23}},{"id":"v_inner","candidate":13.55,"reference":129.649,"speedup":9.568,"baselines":{"wpd_neon":12.57,"c_ref_apple_clang_O2":129.649}},{"id":"h_inner","candidate":26.876,"reference":109.86,"speedup":4.088,"baselines":{"wpd_neon":26.221,"c_ref_apple_clang_O2":109.86}},{"id":"geomean","candidate":15.415,"reference":122.406,"speedup":7.941,"baselines":{"wpd_neon":14.551,"c_ref_apple_clang_O2":122.406}}]},"scores":{"gpt-6.1-terra-max":9.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and well tested, with a small horizontal-store optimization available.","opus-5.5-xhigh":"correct, compact, efficient; solid tests that work with 16 KiB pages","deepseek-v4.1-flash-max":"clean, compact kernel; well-tested; one redundant constant","gpt-6.1-sol-max":"Correct, compact implementation with comprehensive tests and one minor hot-path inefficiency.","muse-wonder-full-xhigh":"correct and efficient; slightly cryptic documentation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Full reverse transpose reconstructs unused taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No callee-saved/x18/d8–d15/dirty-argument checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`cb_mb_adjust` re-materialises the +63 bias","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Reconstructs unused output columns","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Opaque kernel contract","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.08,"lines":16,"significant":200,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.0,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":0.8,"score_cross_vendor":8.667,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":61.05,"tokens":{"uncached_input":372,"cache_read":11816557,"cache_write":254850,"output":213071,"reasoning":0},"cost_usd":0.3402,"cost_breakdown":{"input":5.6e-05,"cache_read":0.189065,"cache_write":0.05097,"output":0.100143},"cost_usd_as_run":0.3402,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.3402,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":464,"code":387,"comment":69,"blank":8},{"path":"tests/selftest.c","role":"test","lines":557,"code":454,"comment":69,"blank":34}],"impl_lines":464,"impl_code":387,"impl_comment":69,"test_lines":557,"comment_ratio":0.151,"tests_declared":3,"impl_trailing_comments":54},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":11.561,"reference":138.63,"speedup":11.991,"baselines":{"wpd_neon":6.605,"c_ref_apple_clang_O2":138.63}},{"id":"h_mb","candidate":26.756,"reference":114.237,"speedup":4.27,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":114.237}},{"id":"v_inner","candidate":13.082,"reference":129.547,"speedup":9.903,"baselines":{"wpd_neon":12.546,"c_ref_apple_clang_O2":129.547}},{"id":"h_inner","candidate":30.487,"reference":109.806,"speedup":3.602,"baselines":{"wpd_neon":26.167,"c_ref_apple_clang_O2":109.806}},{"id":"geomean","candidate":18.741,"reference":122.512,"speedup":6.537,"baselines":{"wpd_neon":14.536,"c_ref_apple_clang_O2":122.512}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with avoidable work and test gaps.","opus-5.5-xhigh":"correct but the slowest kernel; duplicated h bodies and a false comment","deepseek-v4.1-flash-max":"correct and thoroughly described, but the most instructions for the least work","gpt-6.1-sol-max":"Correct kernel with substantial avoidable SIMD work and several test/documentation defects.","muse-wonder-full-xhigh":"correct and well-tested archetypes, but verbose and instruction-heavy"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Interior absolute differences use expanded sequences","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Outer threshold cases can be invisible","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal guard test misses unchanged forbidden stores","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Each |a−b| is built from `uqsub`+`uqsub`+`umax` with its own `cmhi`+`orr`, and the macroblock taps go through `ext`+`sxtl`+`mul`+`sshr`+`xtn`+`ins`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The h paths run four half-width CB_TRN8 passes, 8 `ins` plus 8 `ext`, and an extra `add x10` per row","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"h and h_inner (and v and v_inner) are separate hand-written bodies repeating the same load/transpose sequence","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The CB_TRN8 comment says the `.2s/.4h/.8b` writes \"only touch the low eight bytes\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Signed copies of p3/q3 (v16, v23) are never read, and `movi #1`+`add`+`sshr` stands in for `srshr`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The h guard protects only rows outside 0..15, and the comment claims column isolation is \"impossible at page granularity\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI register checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"the filter mask is assembled from six `LF_ABSCMP` expansions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the guard test prints SKIP and returns success for any page size other than 4096","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Expensive absolute differences and transposes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Horizontal guards accept forbidden accesses","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Several comparison cases cannot reveal mask errors","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Transpose aliasing guarantee is false","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"LF_ABSCMP spends 3 instructions per |a−b| and HEV diffs are recomputed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Duplicated h bodies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard failures reported as passes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Wrong mnemonic names and garbled kernel contract","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":61,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.2,"slop":{"score":1.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":2.5,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":4.4,"score_cross_vendor":7.2,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":5.56,"tokens":{"uncached_input":16005,"cache_read":725888,"cache_write":0,"output":24406,"reasoning":13495},"cost_usd":0.1743,"cost_breakdown":{"input":0.016005,"cache_read":0.036294,"cache_write":0.0,"output":0.12203},"cost_usd_as_run":0.1743,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":248,"code":216,"comment":18,"blank":14},{"path":"tests/selftest.c","role":"test","lines":513,"code":453,"comment":31,"blank":29}],"impl_lines":248,"impl_code":216,"impl_comment":18,"test_lines":513,"comment_ratio":0.077,"tests_declared":5,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.075,"reference":133.274,"speedup":16.505,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":133.274}},{"id":"h_mb","candidate":19.899,"reference":116.999,"speedup":5.88,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":116.999}},{"id":"v_inner","candidate":13.852,"reference":130.11,"speedup":9.393,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":130.11}},{"id":"h_inner","candidate":27.259,"reference":109.911,"speedup":4.032,"baselines":{"wpd_neon":26.208,"c_ref_apple_clang_O2":109.911}},{"id":"geomean","candidate":15.694,"reference":122.2,"speedup":7.786,"baselines":{"wpd_neon":14.545,"c_ref_apple_clang_O2":122.2}}]},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with extra scalar work in horizontal stores.","opus-5.5-xhigh":"correct, sensible kernel; GPR-based h stores and a thin header","deepseek-v4.1-flash-max":"correct, but every `h` row is routed through a GPR","gpt-6.1-sol-max":"Correct and well tested, but horizontal stores perform unnecessary scalar extraction.","muse-wonder-full-xhigh":"correct with strong tests; GPR-mediated h stores cost speed"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Each horizontal row is extracted to a general register","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Each stored h row goes through a GPR (`umov`, `lsr`, two scalar stores), and the forward transpose is copied back with 8 `mov`s","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The file header is one line","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"The inner edge flips p2/q2 for nothing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI register checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"each stored row is moved to a GPR and shifted","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"eight `mov v0..v7, v16..v23` copy the transpose result","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Scalar extraction inflates horizontal stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Per-row NEON→GPR store path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Inner results misstated","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0556,"lines":12,"significant":216,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.1,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":1.9,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.24,"tokens":{"uncached_input":28,"cache_read":1027125,"cache_write":85469,"output":43974,"reasoning":21658},"cost_usd":1.7688,"cost_breakdown":{"input":0.000112,"cache_read":0.205425,"cache_write":0.683752,"output":0.87948},"cost_usd_as_run":1.7688,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":312,"code":258,"comment":46,"blank":8},{"path":"tests/selftest.c","role":"test","lines":660,"code":574,"comment":45,"blank":41}],"impl_lines":312,"impl_code":258,"impl_comment":46,"test_lines":660,"comment_ratio":0.151,"tests_declared":6,"impl_trailing_comments":85},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":6.741,"reference":133.369,"speedup":19.784,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":133.369}},{"id":"h_mb","candidate":18.786,"reference":116.936,"speedup":6.225,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":116.936}},{"id":"v_inner","candidate":12.889,"reference":129.601,"speedup":10.056,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.601}},{"id":"h_inner","candidate":24.658,"reference":109.887,"speedup":4.456,"baselines":{"wpd_neon":26.194,"c_ref_apple_clang_O2":109.887}},{"id":"geomean","candidate":14.164,"reference":122.079,"speedup":8.619,"baselines":{"wpd_neon":14.542,"c_ref_apple_clang_O2":122.079}}]},"scores":{"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":9.0,"opus-5.5-xhigh":9.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Compact filter with strong differential and memory tests","gpt-6.1-sol-max":"Lean, correct kernel with broad arithmetic and memory coverage.","deepseek-v4.1-flash-max":"Correct, the leanest code of the six and the most systematic test set; only the ABI register contract is untested.","opus-5.5-xhigh":"Same quality as I: lowest instruction count, accurate docs, thorough guard tests","muse-wonder-full-xhigh":"Leanest kernel (444 insns) with strong guarded tests; randomized coverage gates"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No register-preservation test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI violations escape the self-test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No callee-saved/x18/dirty-argument test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"No callee-saved, x18 or d8–d15 check in the entry's own tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Pass/fail depends on randomized coverage search","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"No callee-saved/x18 canary test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":258,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-02","reviewed":false},"score":9.2,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":1.0,"score_cross_vendor":9.125,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.54,"tokens":{"uncached_input":22532,"cache_read":143104,"cache_write":0,"output":1883,"reasoning":239},"cost_usd":0.0391,"cost_breakdown":{"input":0.022532,"cache_read":0.007155,"cache_write":0.0,"output":0.009415},"cost_usd_as_run":0.0391,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":23,"code":13,"comment":6,"blank":4},{"path":"tests/selftest.c","role":"test","lines":275,"code":237,"comment":20,"blank":18}],"impl_lines":23,"impl_code":13,"impl_comment":6,"test_lines":275,"comment_ratio":0.316,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":3,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.945,"reference":133.558,"speedup":141.316,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.558}},{"id":"h_mb","candidate":0.945,"reference":117.744,"speedup":124.584,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":117.744}},{"id":"v_inner","candidate":0.939,"reference":130.269,"speedup":138.688,"baselines":{"wpd_neon":12.566,"c_ref_apple_clang_O2":130.269}},{"id":"h_inner","candidate":0.939,"reference":109.996,"speedup":117.104,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.996}},{"id":"geomean","candidate":0.942,"reference":122.52,"speedup":130.037,"baselines":{"wpd_neon":14.537,"c_ref_apple_clang_O2":122.52}}]},"scores":{"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.0},"council_summaries":{"gpt-6.1-terra-max":"All four filter entry points are still stubs","gpt-6.1-sol-max":"All four filters remain unimplemented.","deepseek-v4.1-flash-max":"The scaffold stub: all four entry points return without filtering, so every call is wrong.","opus-5.5-xhigh":"The deliverable is the unmodified scaffold stub","muse-wonder-full-xhigh":"Unmodified stub: no filtering in any function"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"No entry point filters pixels","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Edge-boundary construction uses the wrong measure","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Memory and ABI contracts are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Entry points return without filtering","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"All four entry points are unimplemented","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Test file is the scaffold's random/boundary pair and is not extended","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"All four entry points are a bare `ret`","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"All four entry points are bare `ret`","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Boundary test barely reaches the edge limit","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":6,"flags":0,"audited":"2026-11-02","reviewed":false},"score":0.9,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":4.4,"score_cross_vendor":0.833,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1-b2","task":"vp8lf-neon","lead":"gpt-6.1-terra-max","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":40.95,"tokens":{"cache_read":3483912,"cache_write":0,"output":25021,"reasoning":11738,"uncached_input":314019},"cost_usd":2.4533,"cost_breakdown":{"cache_read":0.696782,"cache_write":0.0,"input":1.256076,"output":0.50042},"cost_usd_as_run":2.4533,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":12.2,"tokens":{"cache_read":1784832,"cache_write":0,"output":23760,"reasoning":17115,"uncached_input":125962},"cost_usd":0.668,"cost_breakdown":{"cache_read":0.178483,"cache_write":0.0,"input":0.251924,"output":0.2376},"cost_usd_as_run":0.668,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":13.87,"tokens":{"cache_read":3880020,"cache_write":191260,"output":62118,"reasoning":47552,"uncached_input":56},"cost_usd":3.5487,"cost_breakdown":{"cache_read":0.776004,"cache_write":1.53008,"input":0.000224,"output":1.24236},"cost_usd_as_run":3.5487,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":10.31,"tokens":{"cache_read":5304832,"cache_write":0,"output":99544,"reasoning":82573,"uncached_input":94746},"cost_usd":0.0899,"cost_breakdown":{"cache_read":0.015914,"cache_write":0.0,"input":0.014212,"output":0.059726},"cost_usd_as_run":0.0899,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":45.55,"tokens":{"cache_read":3896064,"cache_write":0,"output":23957,"reasoning":14051,"uncached_input":100407},"cost_usd":0.415,"cost_breakdown":{"cache_read":0.194803,"cache_write":0.0,"input":0.100407,"output":0.119785},"cost_usd_as_run":0.415,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"Z","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"X","score":6.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"G","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"Z","score":6.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"X","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"G","score":10.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"Z","score":5.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"X","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"G","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"Z","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"X","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"G","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"Z","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"X","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"G","score":9.0}],"panel":["muse-wonder-full-xhigh","gpt-6.1-sol-max","opus-5.5-xhigh","deepseek-v4.1-flash-max","gpt-6.1-terra-max"]},"entries":[{"letter":"D","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":10.18,"tokens":{"uncached_input":137031,"cache_read":1182406,"cache_write":0,"output":61636,"reasoning":0},"cost_usd":1.0087,"cost_breakdown":{"input":0.274062,"cache_read":0.118241,"cache_write":0.0,"output":0.61636},"cost_usd_as_run":1.0087,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":441,"code":310,"comment":77,"blank":54},{"path":"tests/selftest.c","role":"test","lines":521,"code":443,"comment":37,"blank":41}],"impl_lines":441,"impl_code":310,"impl_comment":77,"test_lines":521,"comment_ratio":0.199,"tests_declared":3,"impl_trailing_comments":124},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.38,"reference":134.157,"speedup":16.01,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":134.157}},{"id":"h_mb","candidate":21.064,"reference":117.114,"speedup":5.56,"baselines":{"wpd_neon":20.634,"c_ref_apple_clang_O2":117.114}},{"id":"v_inner","candidate":12.648,"reference":129.574,"speedup":10.245,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.574}},{"id":"h_inner","candidate":26.289,"reference":109.677,"speedup":4.172,"baselines":{"wpd_neon":26.167,"c_ref_apple_clang_O2":109.677}},{"id":"geomean","candidate":15.565,"reference":122.24,"speedup":7.854,"baselines":{"wpd_neon":14.542,"c_ref_apple_clang_O2":122.24}}]},"isolation":{"tool_calls":26,"flags":11,"audited":"2026-11-02","reviewed":true},"scores":{"muse-wonder-full-xhigh":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-terra-max":8.0},"council_summaries":{"muse-wonder-full-xhigh":"Clean, fast, correct; guard test can pass vacuously","gpt-6.1-sol-max":"Correct byte-wide implementation with minor efficiency and test-harness issues","opus-5.5-xhigh":"correct and lean 8-bit design, close to the best; some avoidable instructions and gaps in the guard and ABI tests","deepseek-v4.1-flash-max":"correct, compact and near-best in instruction count with good documentation; its guard test only proves \"no fault\", and a couple of test comments overstate what is covered.","gpt-6.1-terra-max":"Correct, readable 16-lane implementation; its tests leave specific boundary and memory gaps."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Guard test masks its own failure","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Avoidable constant reloads and register copies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Core macro clobber list is incomplete","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Opaque accumulator seeding","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Transpose adds eight avoidable vector copies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guards miss forbidden unchanged writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Failed guard setup can report success","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI probe omits x18","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Coverage comments overstate generated cases","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Extra instructions in the macroblock a0/a1/a2 block","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Eight register-renaming `mov`s after the h transpose","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The guard and ABI checks miss whole bug classes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The \"exact edge threshold\" lines never test the top of the flim_e range","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"The `inner` field added to `FNS` is never read","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"guard-page test never compares results","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"\"every alignment and stride\" is not what the sweep does","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"a0/a1/a2 are built with three separate long-multiply sequences","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"case 2 comment says \"|p1-p0| == i\"","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"ABI checker omits x18 and a caller-frame canary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Maximum edge-limit equality is not exercised","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard cases miss unchanged stores to input-only taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard setup errors can pass silently","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Test coverage claims exceed the loops","status":"valid","detail":"","location":""}],"score":8.2,"slop":{"score":0.7,"per_seat":{"muse-wonder-full-xhigh":0.0,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.0,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-terra-max":0.5}},"defect_points":4.0,"score_cross_vendor":8.2,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","task":"vp8lf-neon","lead":"gpt-6.1-terra-max","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":58.77,"tokens":{"cache_read":37726402,"cache_write":970333,"output":248741,"reasoning":147885,"uncached_input":360},"cost_usd":20.2843,"cost_breakdown":{"cache_read":7.54528,"cache_write":7.762664,"input":0.00144,"output":4.97482},"cost_usd_as_run":20.2843,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":3}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":81.07,"tokens":{"cache_read":23709448,"cache_write":0,"output":181349,"reasoning":98895,"uncached_input":982780},"cost_usd":12.2999,"cost_breakdown":{"cache_read":4.741889,"cache_write":0.0,"input":3.93112,"output":3.62698},"cost_usd_as_run":12.2999,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":3}},{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":34.95,"tokens":{"cache_read":33026688,"cache_write":0,"output":80001,"reasoning":38190,"uncached_input":630445},"cost_usd":2.6817,"cost_breakdown":{"cache_read":1.651334,"cache_write":0.0,"input":0.630445,"output":0.400005},"cost_usd_as_run":2.6817,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":55.29,"tokens":{"cache_read":17705216,"cache_write":0,"output":122171,"reasoning":75911,"uncached_input":681123},"cost_usd":4.3545,"cost_breakdown":{"cache_read":1.770521,"cache_write":0.0,"input":1.362246,"output":1.22171},"cost_usd_as_run":4.3545,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":3}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":66.76,"tokens":{"cache_read":46851712,"cache_write":0,"output":502798,"reasoning":393788,"uncached_input":517384},"cost_usd":0.5199,"cost_breakdown":{"cache_read":0.140555,"cache_write":0.0,"input":0.077608,"output":0.301678},"cost_usd_as_run":0.5199,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":3}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":5.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":6.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":5.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"A","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-gpt-6.1-terra-max-r1","letter":"B","score":6.5}],"panel":["opus-5.5-xhigh","muse-wonder-full-xhigh","gpt-6.1-terra-max","gpt-6.1-sol-max","deepseek-v4.1-flash-max"]},"entries":[{"letter":"A","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":12.92,"tokens":{"uncached_input":38133,"cache_read":456192,"cache_write":0,"output":28551,"reasoning":15957},"cost_usd":0.4074,"cost_breakdown":{"input":0.076266,"cache_read":0.045619,"cache_write":0.0,"output":0.28551},"cost_usd_as_run":0.4074,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":252,"code":221,"comment":16,"blank":15},{"path":"tests/selftest.c","role":"test","lines":541,"code":484,"comment":32,"blank":25}],"impl_lines":252,"impl_code":221,"impl_comment":16,"test_lines":541,"comment_ratio":0.068,"tests_declared":5,"impl_trailing_comments":19},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.537,"reference":134.419,"speedup":15.745,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.419}},{"id":"h_mb","candidate":21.883,"reference":118.227,"speedup":5.403,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":118.227}},{"id":"v_inner","candidate":14.014,"reference":130.32,"speedup":9.299,"baselines":{"wpd_neon":12.58,"c_ref_apple_clang_O2":130.32}},{"id":"h_inner","candidate":28.039,"reference":109.996,"speedup":3.923,"baselines":{"wpd_neon":26.198,"c_ref_apple_clang_O2":109.996}},{"id":"geomean","candidate":16.46,"reference":122.855,"speedup":7.464,"baselines":{"wpd_neon":14.541,"c_ref_apple_clang_O2":122.855}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":221,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":10,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":8.5,"gpt-6.1-terra-max":9.0,"gpt-6.1-sol-max":9.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"opus-5.5-xhigh":"Compact and efficient. One false test comment and a fragile store-macro coupling","muse-wonder-full-xhigh":"Strong tests and compact macros; h pays for transposing twice","gpt-6.1-terra-max":"Correct and compact, with a misleading guard-coverage report.","gpt-6.1-sol-max":"Correct and well tested, with unnecessary horizontal output shuffling","deepseek-v4.1-flash-max":"solid and fast with good tests; one false comment and a hand-rolled per-row `h` store"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Claims a disassembly check that does not exist","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"style","title":"An undocumented ordering dependency in H_STORE","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Rounding constant loaded from rodata","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Full transpose on both h legs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Protected-row guard claim does not hold on macOS","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Protected-row coverage is reported even when skipped","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Full inverse transpose reconstructs discarded taps","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"claims a disassembly check that does not exist","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"`h` re-applies a byte `ext` per row and stores 4+2 bytes per row","status":"valid","detail":"","location":""}],"score":8.5,"slop":{"score":0.7,"per_seat":{"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":2.0}},"defect_points":1.7,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":27.47,"tokens":{"uncached_input":77799,"cache_read":22899200,"cache_write":0,"output":234174,"reasoning":179342},"cost_usd":0.2209,"cost_breakdown":{"input":0.01167,"cache_read":0.068698,"cache_write":0.0,"output":0.140504},"cost_usd_as_run":0.2209,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.2209,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":674,"code":517,"comment":104,"blank":53},{"path":"tests/selftest.c","role":"test","lines":792,"code":674,"comment":68,"blank":50}],"impl_lines":674,"impl_code":517,"impl_comment":104,"test_lines":792,"comment_ratio":0.167,"tests_declared":7,"impl_trailing_comments":25},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":18.208,"reference":133.831,"speedup":7.35,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.831}},{"id":"h_mb","candidate":26.84,"reference":116.946,"speedup":4.357,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.946}},{"id":"v_inner","candidate":14.123,"reference":129.615,"speedup":9.178,"baselines":{"wpd_neon":12.546,"c_ref_apple_clang_O2":129.615}},{"id":"h_inner","candidate":29.748,"reference":109.595,"speedup":3.684,"baselines":{"wpd_neon":26.181,"c_ref_apple_clang_O2":109.595}},{"id":"geomean","candidate":21.287,"reference":122.109,"speedup":5.736,"baselines":{"wpd_neon":14.521,"c_ref_apple_clang_O2":122.109}}]},"quality":{"duplication":{"ratio":0.3907,"lines":202,"significant":517,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":114,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.0,"gpt-6.1-terra-max":5.0,"gpt-6.1-sol-max":5.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"opus-5.5-xhigh":"The kernel is correct and well documented, but the selftest does not build on macOS and the filter math is all 16-bit","muse-wonder-full-xhigh":"Correct and well structured, but pays a frame, a 64-byte constant load and two halfword passes per call","gpt-6.1-terra-max":"The filter passed Linux probes, but its own Apple build is blocked.","gpt-6.1-sol-max":"Correct filters, but the macOS test harness fails to build","deepseek-v4.1-flash-max":"clean, best-documented of the widening designs and correct; ~2–2.5× the best speed and two loose test details"},"findings":[{"seat":"opus-5.5-xhigh","severity":"high","category":"robustness","title":"The selftest ABI macro fails to compile for Apple targets","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Widened arithmetic with re-widen after every clamp","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The test claims exhaustiveness it lacks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"`skips` is never used","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Garbled prototype block in the header","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Callee-saved frame plus narrow/widen round-trips","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Stride-test header overclaims coverage","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Single-call ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"\"One clip\" comment is wrong","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"ABI helper uses ELF relocations on Apple","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Each filter runs twice over widened half-vectors","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"tests","title":"ABI helper uses ELF-only addressing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Widen/narrow round trips inflate the core","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Coverage claims exceed implemented tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"every tap is widened twice (`ushll` + `ushll2`) and the arithmetic runs twice per function","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"dead-code","title":"vestigial variable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"ABI check stops at `x28`","status":"valid","detail":"","location":""}],"score":6.2,"slop":{"score":0.9,"per_seat":{"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.5,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":4.9,"score_cross_vendor":5.875,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":10.86,"tokens":{"uncached_input":21896,"cache_read":245504,"cache_write":0,"output":19944,"reasoning":9023},"cost_usd":1.4617,"cost_breakdown":{"input":0.21896,"cache_read":0.245504,"cache_write":0.0,"output":0.9972},"cost_usd_as_run":1.4617,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":264,"code":242,"comment":10,"blank":12},{"path":"tests/selftest.c","role":"test","lines":571,"code":515,"comment":32,"blank":24}],"impl_lines":264,"impl_code":242,"impl_comment":10,"test_lines":571,"comment_ratio":0.04,"tests_declared":7,"impl_trailing_comments":4},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":14.869,"reference":136.194,"speedup":9.16,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":136.194}},{"id":"h_mb","candidate":32.951,"reference":119.445,"speedup":3.625,"baselines":{"wpd_neon":21.243,"c_ref_apple_clang_O2":119.445}},{"id":"v_inner","candidate":23.695,"reference":131.483,"speedup":5.549,"baselines":{"wpd_neon":12.814,"c_ref_apple_clang_O2":131.483}},{"id":"h_inner","candidate":38.805,"reference":110.925,"speedup":2.859,"baselines":{"wpd_neon":26.489,"c_ref_apple_clang_O2":110.925}},{"id":"geomean","candidate":25.907,"reference":124.11,"speedup":4.791,"baselines":{"wpd_neon":14.761,"c_ref_apple_clang_O2":124.11}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":242,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":7.5,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"opus-5.5-xhigh":"Correct, with 8-bit masks and very thorough tests, but per-output widening adds cost","muse-wonder-full-xhigh":"Correct with strong layout/guard tests, but a two-half filter and cryptic macro encoding","gpt-6.1-terra-max":"Correct in the probes, with avoidable transpose work and an overstated guard result.","gpt-6.1-sol-max":"Correct and well tested, with avoidable arithmetic and output work","deepseek-v4.1-flash-max":"compact, correct, best register/FPCR test; mid-pack speed and magic lane numbers"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Per-output widen/narrow and per-half constant broadcasts","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Eight `mov`s before the back-transpose","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Two-half tap-adjust pipeline","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Opaque constant/parameter encoding","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Read-only-row guard claim does not hold on macOS","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal stores transpose the entire block back","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard result claims checks that may not run","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Repeated widening and full inverse transpose","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"style","title":"unexplained lane constants","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"no per-entry-point doc comments","status":"valid","detail":"","location":""}],"score":7.7,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.1,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":13.05,"tokens":{"uncached_input":32,"cache_read":1408326,"cache_write":110577,"output":64318,"reasoning":32880},"cost_usd":2.4528,"cost_breakdown":{"input":0.000128,"cache_read":0.281665,"cache_write":0.884616,"output":1.28636},"cost_usd_as_run":2.4528,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":343,"code":257,"comment":69,"blank":17},{"path":"tests/selftest.c","role":"test","lines":938,"code":800,"comment":87,"blank":51}],"impl_lines":343,"impl_code":257,"impl_comment":69,"test_lines":938,"comment_ratio":0.212,"tests_declared":7,"impl_trailing_comments":78},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.151,"reference":135.354,"speedup":18.928,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":135.354}},{"id":"h_mb","candidate":18.975,"reference":118.49,"speedup":6.245,"baselines":{"wpd_neon":20.676,"c_ref_apple_clang_O2":118.49}},{"id":"v_inner","candidate":12.848,"reference":132.06,"speedup":10.279,"baselines":{"wpd_neon":12.838,"c_ref_apple_clang_O2":132.06}},{"id":"h_inner","candidate":25.991,"reference":112.094,"speedup":4.313,"baselines":{"wpd_neon":26.794,"c_ref_apple_clang_O2":112.094}},{"id":"geomean","candidate":14.59,"reference":124.13,"speedup":8.508,"baselines":{"wpd_neon":14.704,"c_ref_apple_clang_O2":124.13}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":257,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":17,"flags":1,"audited":"2026-11-02","reviewed":true},"conduct":{"severity":"minor","summary":"While working it ran `pkill -f 'qemu-aarch64 build/selftest'`, a host-wide pattern that also matches other contestants' running self-tests, not just its own. No other process is known to have been hit; scores stand (owner decision, 2026-11-02)."},"scores":{"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":9.0,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":6.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"opus-5.5-xhigh":"Kernel quality is top-tier, but the selftest does not compile for macOS","muse-wonder-full-xhigh":"Tied-best instruction count and strong coverage-counted tests; one false contract claim","gpt-6.1-terra-max":"Strong SIMD implementation and coverage, with undefined behavior in a test generator.","gpt-6.1-sol-max":"Strong filter implementation undermined by test-harness defects","deepseek-v4.1-flash-max":"fast (100/168/76/125), excellent docs and the most complete tests; one false contract claim"},"findings":[{"seat":"opus-5.5-xhigh","severity":"high","category":"robustness","title":"The ABI inline asm exhausts registers on Apple","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Misstates the contract on constants","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"False claim that constant loads violate the contract","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Boundary generation can divide by zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Constant-table restriction is misstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"tests","title":"ABI helper exhausts Apple’s available registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Generator can divide by zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Read-only constants incorrectly described as forbidden","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"wrong justification for `movi` constants","status":"valid","detail":"","location":""}],"score":7.7,"slop":{"score":0.8,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":2.8,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.25,"tokens":{"uncached_input":7204,"cache_read":75392,"cache_write":0,"output":398,"reasoning":106},"cost_usd":0.013,"cost_breakdown":{"input":0.007204,"cache_read":0.00377,"cache_write":0.0,"output":0.00199},"cost_usd_as_run":0.013,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":23,"code":13,"comment":6,"blank":4},{"path":"tests/selftest.c","role":"test","lines":216,"code":181,"comment":18,"blank":17}],"impl_lines":23,"impl_code":13,"impl_comment":6,"test_lines":216,"comment_ratio":0.316,"tests_declared":1,"impl_trailing_comments":0},"hidden":{"passed":3,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.945,"reference":133.747,"speedup":141.516,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.747}},{"id":"h_mb","candidate":0.935,"reference":117.135,"speedup":125.332,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.135}},{"id":"v_inner","candidate":0.936,"reference":129.991,"speedup":138.894,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.991}},{"id":"h_inner","candidate":0.939,"reference":111.043,"speedup":118.219,"baselines":{"wpd_neon":26.449,"c_ref_apple_clang_O2":111.043}},{"id":"geomean","candidate":0.939,"reference":122.629,"speedup":130.635,"baselines":{"wpd_neon":14.561,"c_ref_apple_clang_O2":122.629}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":3,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5},"council_summaries":{"opus-5.5-xhigh":"An unmodified stub: none of the four functions filters anything","muse-wonder-full-xhigh":"Unmodified stub: no filtering at all","gpt-6.1-terra-max":"All four exported functions are unimplemented stubs.","gpt-6.1-sol-max":"All four filters remain unimplemented","deepseek-v4.1-flash-max":"untouched scaffold stub: every function returns without filtering and its own selftest fails"},"findings":[{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"No implementation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"All four functions are bare `ret`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"Every entry point returns without filtering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"critical","category":"correctness","title":"Every entry point immediately returns","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"all four entry points are `ret`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"tests/selftest.c is also the unmodified scaffold test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"the file still says the deliverable is missing","status":"valid","detail":"","location":""}],"score":0.4,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":5.0,"score_cross_vendor":0.333,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.5,"tokens":{"uncached_input":16857,"cache_read":406272,"cache_write":0,"output":11840,"reasoning":3105},"cost_usd":0.1927,"cost_breakdown":{"input":0.033714,"cache_read":0.040627,"cache_write":0.0,"output":0.1184},"cost_usd_as_run":0.1927,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":487,"code":468,"comment":11,"blank":8},{"path":"tests/selftest.c","role":"test","lines":514,"code":462,"comment":29,"blank":23}],"impl_lines":487,"impl_code":468,"impl_comment":11,"test_lines":514,"comment_ratio":0.023,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":16.917,"reference":134.262,"speedup":7.937,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.262}},{"id":"h_mb","candidate":44.082,"reference":116.799,"speedup":2.65,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":116.799}},{"id":"v_inner","candidate":26.838,"reference":129.662,"speedup":4.831,"baselines":{"wpd_neon":12.539,"c_ref_apple_clang_O2":129.662}},{"id":"h_inner","candidate":44.939,"reference":109.636,"speedup":2.44,"baselines":{"wpd_neon":26.174,"c_ref_apple_clang_O2":109.636}},{"id":"geomean","candidate":30.796,"reference":122.191,"speedup":3.968,"baselines":{"wpd_neon":14.522,"c_ref_apple_clang_O2":122.191}}]},"quality":{"duplication":{"ratio":0.4487,"lines":210,"significant":468,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":6.0,"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":7.0,"deepseek-v4.1-flash-max":6.5},"council_summaries":{"opus-5.5-xhigh":"Correct, but it widens everything to 16-bit, round-trips h through the stack, and is thinly documented","muse-wonder-full-xhigh":"Correct, but h path is stack-heavy and tests/docs are thin at the edges","gpt-6.1-terra-max":"Correct in the probes, but horizontal paths repeatedly transpose and spill.","gpt-6.1-sol-max":"Correct and thoroughly tested, with an expensive horizontal path","deepseek-v4.1-flash-max":"correct and heavily tested, but its `h` path is ~2.9× the field and `H_BODY` is 16 copies of one store block"},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Widened two-pass filter plus a stack round trip for h","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Two-line header only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Eight trailing `mov`s in every TRANSPOSE","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Final pointer bumps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Stack/scalar h path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard test silently passes on 16K-page hosts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Single-shot ABI test with identical sentinels, no x18","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Undocumented stack/transpose layout","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Dead pointer adds","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal path stages both halves on the stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner filtering narrows untouched outer taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Large pages skip every guard test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Avoidable stack transfers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Row-store template copied sixteen times","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h` costs 480 (mb) / 411 (inner) instructions per call vs 168/125 for the best entries","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"verbosity","title":"180 lines of mechanical duplication","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"entry points and macro parameters undocumented","status":"valid","detail":"","location":""}],"score":6.5,"slop":{"score":0.9,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":3.8,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":36.11,"tokens":{"uncached_input":96,"cache_read":8295934,"cache_write":222145,"output":154148,"reasoning":103592},"cost_usd":6.5197,"cost_breakdown":{"input":0.000384,"cache_read":1.659187,"cache_write":1.77716,"output":3.08296},"cost_usd_as_run":6.5197,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":361,"code":277,"comment":75,"blank":9},{"path":"tests/selftest.c","role":"test","lines":1083,"code":910,"comment":102,"blank":71}],"impl_lines":361,"impl_code":277,"impl_comment":75,"test_lines":1083,"comment_ratio":0.213,"tests_declared":7,"impl_trailing_comments":109},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.224,"reference":134.146,"speedup":18.568,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.146}},{"id":"h_mb","candidate":18.66,"reference":116.809,"speedup":6.26,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":116.809}},{"id":"v_inner","candidate":12.655,"reference":129.69,"speedup":10.248,"baselines":{"wpd_neon":12.566,"c_ref_apple_clang_O2":129.69}},{"id":"h_inner","candidate":24.56,"reference":110.145,"speedup":4.485,"baselines":{"wpd_neon":26.201,"c_ref_apple_clang_O2":110.145}},{"id":"geomean","candidate":14.307,"reference":122.315,"speedup":8.549,"baselines":{"wpd_neon":14.534,"c_ref_apple_clang_O2":122.315}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":277,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":56,"flags":2,"audited":"2026-11-02","reviewed":true},"scores":{"opus-5.5-xhigh":9.5,"muse-wonder-full-xhigh":9.5,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":10.0,"deepseek-v4.1-flash-max":9.5},"council_summaries":{"opus-5.5-xhigh":"Best of the set: tight, well-scheduled 8-bit kernel, accurate docs, the most thorough tests","muse-wonder-full-xhigh":"Tied-best performance with the most exhaustive test suite in the set","gpt-6.1-terra-max":"Efficient and well documented, with a guard test that can reject conforming code.","gpt-6.1-sol-max":"Correct, efficient, clearly documented, and comprehensively tested","deepseek-v4.1-flash-max":"top speed, the most thorough test suite and the most complete arithmetic rationale; no defects found"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Guard test requires behavior beyond the stride domain","status":"valid","detail":"","location":""}],"score":9.4,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.4,"score_cross_vendor":9.375,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":5.93,"tokens":{"uncached_input":20,"cache_read":612171,"cache_write":72501,"output":32855,"reasoning":15675},"cost_usd":1.3596,"cost_breakdown":{"input":8e-05,"cache_read":0.122434,"cache_write":0.580008,"output":0.6571},"cost_usd_as_run":1.3596,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":333,"code":266,"comment":50,"blank":17},{"path":"tests/selftest.c","role":"test","lines":585,"code":518,"comment":38,"blank":29}],"impl_lines":333,"impl_code":266,"impl_comment":50,"test_lines":585,"comment_ratio":0.158,"tests_declared":6,"impl_trailing_comments":40},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.33,"reference":133.632,"speedup":18.232,"baselines":{"wpd_neon":6.626,"c_ref_apple_clang_O2":133.632}},{"id":"h_mb","candidate":19.08,"reference":116.999,"speedup":6.132,"baselines":{"wpd_neon":20.76,"c_ref_apple_clang_O2":116.999}},{"id":"v_inner","candidate":12.624,"reference":130.28,"speedup":10.32,"baselines":{"wpd_neon":12.587,"c_ref_apple_clang_O2":130.28}},{"id":"h_inner","candidate":25.638,"reference":110.192,"speedup":4.298,"baselines":{"wpd_neon":26.33,"c_ref_apple_clang_O2":110.192}},{"id":"geomean","candidate":14.586,"reference":122.399,"speedup":8.392,"baselines":{"wpd_neon":14.612,"c_ref_apple_clang_O2":122.399}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":266,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":9.5,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.5,"deepseek-v4.1-flash-max":9.0},"council_summaries":{"opus-5.5-xhigh":"Lean kernel with a neat 2-stage tr4 store transpose and strong tests. One false comment","muse-wonder-full-xhigh":"Lean 16-lane core, exhaustive own tests, accurate exactness docs","gpt-6.1-terra-max":"Efficient and correct in the probes, with small ABI-test and documentation defects.","gpt-6.1-sol-max":"Efficient and correct, with one inaccurate pointer invariant","deepseek-v4.1-flash-max":"fastest tier, clean and accurate docs, thorough tests; no defects found"},"findings":[{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Wrong exit state in the v_load comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"Coverage report asserts nothing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"abi","title":"Apple test helper touches the reserved platform register","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Load helper’s exit pointer is wrong","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Incorrect load-macro exit address","status":"valid","detail":"","location":""}],"score":9.0,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":0.8,"score_cross_vendor":9.125,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":40.61,"tokens":{"uncached_input":112210,"cache_read":2071052,"cache_write":0,"output":41392,"reasoning":24456},"cost_usd":1.6909,"cost_breakdown":{"input":0.44884,"cache_read":0.41421,"cache_write":0.0,"output":0.82784},"cost_usd_as_run":1.6909,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":377,"code":298,"comment":63,"blank":16},{"path":"tests/selftest.c","role":"test","lines":600,"code":537,"comment":34,"blank":29}],"impl_lines":377,"impl_code":298,"impl_comment":63,"test_lines":600,"comment_ratio":0.175,"tests_declared":4,"impl_trailing_comments":32},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":20.844,"reference":133.506,"speedup":6.405,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.506}},{"id":"h_mb","candidate":26.84,"reference":116.936,"speedup":4.357,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.936}},{"id":"v_inner","candidate":27.028,"reference":129.635,"speedup":4.796,"baselines":{"wpd_neon":12.577,"c_ref_apple_clang_O2":129.635}},{"id":"h_inner","candidate":28.242,"reference":110.833,"speedup":3.924,"baselines":{"wpd_neon":26.476,"c_ref_apple_clang_O2":110.833}},{"id":"geomean","candidate":25.563,"reference":122.38,"speedup":4.787,"baselines":{"wpd_neon":14.57,"c_ref_apple_clang_O2":122.38}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":26,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":6.5,"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.5,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"opus-5.5-xhigh":"Correct and clearly documented, but the 16-bit core is verbose and roughly 2.5× heavier than needed","muse-wonder-full-xhigh":"Correct and readable 8-lane design, roughly 2.5× the instructions of the leaders","gpt-6.1-terra-max":"Correct in the probes, but its eight-lane implementation repeats substantial work.","gpt-6.1-sol-max":"Correct and safe, but substantially more arithmetic work than necessary","deepseek-v4.1-flash-max":"correct, well-documented and neatly factored, but the second-slowest complete version and its register test skips `x18`"},"findings":[{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Heavy widened core and GPR-based h stores","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Narrowing and moving data that gets discarded","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Halfword core run twice with per-tap masking","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Duplicate inner clip and dead narrows","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin ABI/guard coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Duplicate filter and transpose work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guards cover the enclosing span, not every allowed row","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Entire filter runs twice in widened lanes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Threshold sweeps lack corresponding boundary inputs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"two 8-lane widening passes per call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"ABI helper checks `x19–x29` and `v8–v15` but not `x18`","status":"valid","detail":"","location":""}],"score":6.9,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.8,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.82,"tokens":{"uncached_input":13384,"cache_read":232576,"cache_write":0,"output":4113,"reasoning":315},"cost_usd":0.5721,"cost_breakdown":{"input":0.13384,"cache_read":0.232576,"cache_write":0.0,"output":0.20565},"cost_usd_as_run":0.5721,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":1269,"code":1268,"comment":1,"blank":0},{"path":"tests/selftest.c","role":"test","lines":264,"code":225,"comment":21,"blank":18}],"impl_lines":1269,"impl_code":1268,"impl_comment":1,"test_lines":264,"comment_ratio":0.001,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":18.114,"reference":134.997,"speedup":7.453,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":134.997}},{"id":"h_mb","candidate":93.12,"reference":117.03,"speedup":1.257,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.03}},{"id":"v_inner","candidate":17.819,"reference":131.042,"speedup":7.354,"baselines":{"wpd_neon":12.566,"c_ref_apple_clang_O2":131.042}},{"id":"h_inner","candidate":75.362,"reference":110.48,"speedup":1.466,"baselines":{"wpd_neon":26.201,"c_ref_apple_clang_O2":110.48}},{"id":"geomean","candidate":38.794,"reference":122.979,"speedup":3.17,"baselines":{"wpd_neon":14.543,"c_ref_apple_clang_O2":122.979}}]},"quality":{"duplication":{"ratio":0.5331,"lines":676,"significant":1268,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"opus-5.5-xhigh":5.0,"muse-wonder-full-xhigh":5.0,"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":6.0,"deepseek-v4.1-flash-max":5.5},"council_summaries":{"opus-5.5-xhigh":"Output is correct and ABI-safe, but the h path transposes one byte at a time, and the code has no documentation","muse-wonder-full-xhigh":"Bit-exact, but per-byte scalar transpose, stack-resident taps and almost no documentation","gpt-6.1-terra-max":"Correct in the probes, but the transpose and stack traffic impose substantial avoidable work.","gpt-6.1-sol-max":"Correct output, but excessive data movement and weak maintainability","deepseek-v4.1-flash-max":"correct but 4.7–5.2× slower than the field, 1 100 lines of copy-paste and almost no documentation"},"findings":[{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"Scalar per-byte transposes in both h functions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"docs","title":"Almost no documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Thin ABI and guard coverage","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Recomputed and re-materialised values","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Pointer bumps nobody reads","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"Per-byte transpose and stack-resident filter","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"Essentially undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"No ABI test, crash-only guard test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Dead adjustment computation in FILTER","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"Manually unrolled repetition","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Transpose moves individual bytes through a GPR","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner filter writes unused stack taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No register-preservation probe","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Scalar byte transfers replace a vector transpose","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Every function stages taps through the stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Shared arithmetic lacks essential invariants","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Register preservation is untested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"performance","title":"element-by-element `h` transpose via GPRs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"verbosity","title":"~1 100 of 1 269 lines are the same 6-line sequence repeated with a different lane index","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"docs","title":"one line of documentation for the whole file","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"the \"guard\" test never compares output and there is no ABI test","status":"valid","detail":"","location":""}],"score":5.4,"slop":{"score":1.2,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":2.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":2.0}},"defect_points":7.0,"score_cross_vendor":5.167,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":67.36,"tokens":{"uncached_input":173716,"cache_read":1778816,"cache_write":0,"output":244497,"reasoning":109571},"cost_usd":2.7038,"cost_breakdown":{"input":0.347432,"cache_read":0.889408,"cache_write":0.0,"output":1.466982},"cost_usd_as_run":2.7038,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":474,"code":401,"comment":47,"blank":26},{"path":"tests/selftest.c","role":"test","lines":950,"code":866,"comment":46,"blank":38}],"impl_lines":474,"impl_code":401,"impl_comment":47,"test_lines":950,"comment_ratio":0.105,"tests_declared":6,"impl_trailing_comments":70},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":14.417,"reference":135.154,"speedup":9.374,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":135.154}},{"id":"h_mb","candidate":32.29,"reference":118.647,"speedup":3.674,"baselines":{"wpd_neon":20.665,"c_ref_apple_clang_O2":118.647}},{"id":"v_inner","candidate":11.963,"reference":131.449,"speedup":10.988,"baselines":{"wpd_neon":12.773,"c_ref_apple_clang_O2":131.449}},{"id":"h_inner","candidate":33.122,"reference":110.874,"speedup":3.347,"baselines":{"wpd_neon":26.459,"c_ref_apple_clang_O2":110.874}},{"id":"geomean","candidate":20.724,"reference":123.643,"speedup":5.966,"baselines":{"wpd_neon":14.644,"c_ref_apple_clang_O2":123.643}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":27,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":6.0,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct filter arithmetic, but the ABI selftest is unsafe","gpt-6.1-sol-max":"Correct filter undermined by an unsafe test caller and excess transpose work","muse-wonder-full-xhigh":"correct, strongest guard tests, computes every candidate in 16 bits","opus-5.5-xhigh":"Correct and well tested, but roughly twice the expert instruction count, and its selftest trips Apple clang warnings","deepseek-v4.1-flash-max":"correct, the most thorough test suite of the set (guard rows/widths, stride×align, ABI incl. x18) and precise comments; heavier than needed."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"ABI probe destroys the compiler’s frame pointer","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"abi","title":"ABI probe overwrites Apple’s reserved `x18`","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal guard tests miss unchanged overstores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Macroblock calls compute an unused inner-only value","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI probe assumes caller-saved pointers survive","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"abi","title":"Apple test path overwrites x18","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal packing repeats transposes and copies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Width guards miss unchanged forbidden writes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"both `a` variants and all candidate taps always computed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"16-bit candidate application throughout","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"transpose and store paths move dummy data","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"ABI probe breaks the warning-free macOS build","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Widened arithmetic and duplicated filter work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Register shuffling and dead zeroing","status":"valid","detail":"","location":""}],"score":6.6,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.9,"score_cross_vendor":6.6,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.18,"tokens":{"uncached_input":22887,"cache_read":257024,"cache_write":0,"output":8700,"reasoning":1642},"cost_usd":0.9209,"cost_breakdown":{"input":0.22887,"cache_read":0.257024,"cache_write":0.0,"output":0.435},"cost_usd_as_run":0.9209,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":371,"code":353,"comment":9,"blank":9},{"path":"tests/selftest.c","role":"test","lines":402,"code":355,"comment":25,"blank":22}],"impl_lines":371,"impl_code":353,"impl_comment":9,"test_lines":402,"comment_ratio":0.025,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":15.594,"reference":136.53,"speedup":8.756,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":136.53}},{"id":"h_mb","candidate":35.87,"reference":116.978,"speedup":3.261,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":116.978}},{"id":"v_inner","candidate":16.876,"reference":132.016,"speedup":7.823,"baselines":{"wpd_neon":12.811,"c_ref_apple_clang_O2":132.016}},{"id":"h_inner","candidate":33.61,"reference":112.04,"speedup":3.334,"baselines":{"wpd_neon":26.794,"c_ref_apple_clang_O2":112.04}},{"id":"geomean","candidate":23.733,"reference":123.974,"speedup":5.224,"baselines":{"wpd_neon":14.689,"c_ref_apple_clang_O2":123.974}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":353,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks, with substantial avoidable stack traffic","gpt-6.1-sol-max":"Correct arithmetic with unnecessary stack traffic on every call","muse-wonder-full-xhigh":"correct and well tested, but everything round-trips through the stack","opus-5.5-xhigh":"Correct with good tests, but round-trips taps through a stack frame and filters 8 lanes at a time","deepseek-v4.1-flash-max":"correct and conceptually neat, but every call spills the entire tap set to a 128-byte frame and reloads it twice; documentation is a two-line header."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"The vertical path stages all taps through a stack frame","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal write bounds are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Tap buffer round-trips through the stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Read-only guards omit horizontal forbidden taps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"all taps staged through a 128-byte stack frame","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"8-lane widened core executed twice, h does two 8×8 transposes in and out","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"filter narrows and stack-stores all six taps even for inner edges","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"three-line header, stack tap layout unexplained","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"guard test silently passes when page size isn't 4096","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Stack round trips and half-width filtering","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Minimal documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Extra moves","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"tap vectors round-trip through the stack","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"frame layout undocumented","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"hand-unrolled h writeback","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"no x18 check","status":"valid","detail":"","location":""}],"score":6.8,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":4.2,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.67,"tokens":{"uncached_input":19247,"cache_read":720896,"cache_write":0,"output":31332,"reasoning":17684},"cost_usd":0.212,"cost_breakdown":{"input":0.019247,"cache_read":0.036045,"cache_write":0.0,"output":0.15666},"cost_usd_as_run":0.212,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":370,"code":321,"comment":27,"blank":22},{"path":"tests/selftest.c","role":"test","lines":652,"code":578,"comment":36,"blank":38}],"impl_lines":370,"impl_code":321,"impl_comment":27,"test_lines":652,"comment_ratio":0.078,"tests_declared":5,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":10.039,"reference":135.91,"speedup":13.539,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":135.91}},{"id":"h_mb","candidate":24.782,"reference":118.059,"speedup":4.764,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":118.059}},{"id":"v_inner","candidate":11.817,"reference":129.859,"speedup":10.989,"baselines":{"wpd_neon":12.577,"c_ref_apple_clang_O2":129.859}},{"id":"h_inner","candidate":27.086,"reference":110.118,"speedup":4.065,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":110.118}},{"id":"geomean","candidate":16.798,"reference":123.075,"speedup":7.327,"baselines":{"wpd_neon":14.531,"c_ref_apple_clang_O2":123.075}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":321,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.0,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation with focused boundary and ABI tests","gpt-6.1-sol-max":"Correct byte-lane arithmetic; horizontal transposes and register saves add cost","muse-wonder-full-xhigh":"good byte-lane core undermined by needless callee-saved use","opus-5.5-xhigh":"Correct and well tested; avoidable d8–d15 spills and a heavy h store path","deepseek-v4.1-flash-max":"correct, 16-lane and well explained; pays an eight-register spill on every call and hides a non-obvious BSL idiom."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal protection covers only boundary rows","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal path processes halves separately","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Horizontal inner p2 remains writable in guards","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"callee-saved vectors used as scratch, forcing save/restore","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"weights via rodata lane broadcast and 16-bit multiply","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"h store transposes zero-padded dummy columns","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"six-line file header only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Unneeded callee-saved spills","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Expensive h stores","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Dead zeroing","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Gap writes not checked","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"SIMD save/restore on every call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"undocumented BSL direction","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"no x18 check","status":"valid","detail":"","location":""}],"score":7.8,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.9,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":4.99,"tokens":{"uncached_input":20232,"cache_read":562048,"cache_write":0,"output":7735,"reasoning":1652},"cost_usd":0.174,"cost_breakdown":{"input":0.040464,"cache_read":0.056205,"cache_write":0.0,"output":0.07735},"cost_usd_as_run":0.174,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":390,"code":380,"comment":4,"blank":6},{"path":"tests/selftest.c","role":"test","lines":332,"code":291,"comment":21,"blank":20}],"impl_lines":390,"impl_code":380,"impl_comment":4,"test_lines":332,"comment_ratio":0.01,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":15.037,"reference":133.789,"speedup":8.897,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.789}},{"id":"h_mb","candidate":30.683,"reference":117.335,"speedup":3.824,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.335}},{"id":"v_inner","candidate":12.217,"reference":130.71,"speedup":10.699,"baselines":{"wpd_neon":12.57,"c_ref_apple_clang_O2":130.71}},{"id":"h_inner","candidate":27.384,"reference":110.962,"speedup":4.052,"baselines":{"wpd_neon":26.472,"c_ref_apple_clang_O2":110.962}},{"id":"geomean","candidate":19.821,"reference":122.838,"speedup":6.197,"baselines":{"wpd_neon":14.57,"c_ref_apple_clang_O2":122.838}}]},"quality":{"duplication":{"ratio":0.4421,"lines":168,"significant":380,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":17,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":6.5,"opus-5.5-xhigh":5.0,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks, but processes only eight lines per iteration","gpt-6.1-sol-max":"Correct implementation with repeated half-width work and sparse kernel documentation","muse-wonder-full-xhigh":"simple and correct, undocumented and looped at half width","opus-5.5-xhigh":"Correct but slow (an 8-lane widened loop), undocumented, and its guard test misses v-function over-reads","deepseek-v4.1-flash-max":"correct and easy to follow, but the core runs 8 lanes at a time (twice per call) and the file carries no header."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Vertical and horizontal paths repeat the entire core for two halves","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard test misses unchanged writes to outer taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"The core’s input and register contract is undocumented","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Both loop iterations repeat the full kernel","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guard tests accept forbidden unchanged writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"docs","title":"Shared kernel invariants are undocumented","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"everything runs twice at 8-lane width","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"docs","title":"no file header at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"dead narrows and copy-pasted bodies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"guard far-side placements don't guard, ABI patterns can't distinguish registers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Guard test misses v-function tail over-reads","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Half-width loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Fragile ABI test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"no file header","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"8-lane, two-pass core","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"guard/ABI blind spots","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"four near-identical bodies","status":"valid","detail":"","location":""}],"score":6.6,"slop":{"score":0.3,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":4.7,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":85.56,"tokens":{"uncached_input":726,"cache_read":39915001,"cache_write":542638,"output":453233,"reasoning":0},"cost_usd":0.9603,"cost_breakdown":{"input":0.000109,"cache_read":0.63864,"cache_write":0.108528,"output":0.21302},"cost_usd_as_run":0.9603,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.9603,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":612,"code":493,"comment":105,"blank":14},{"path":"tests/selftest.c","role":"test","lines":624,"code":526,"comment":60,"blank":38}],"impl_lines":612,"impl_code":493,"impl_comment":105,"test_lines":624,"comment_ratio":0.176,"tests_declared":5,"impl_trailing_comments":67},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":18.45,"reference":134.073,"speedup":7.267,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.073}},{"id":"h_mb","candidate":34.222,"reference":117.796,"speedup":3.442,"baselines":{"wpd_neon":20.655,"c_ref_apple_clang_O2":117.796}},{"id":"v_inner","candidate":15.978,"reference":130.375,"speedup":8.16,"baselines":{"wpd_neon":12.6,"c_ref_apple_clang_O2":130.375}},{"id":"h_inner","candidate":35.692,"reference":110.413,"speedup":3.093,"baselines":{"wpd_neon":26.218,"c_ref_apple_clang_O2":110.413}},{"id":"geomean","candidate":24.496,"reference":122.792,"speedup":5.013,"baselines":{"wpd_neon":14.559,"c_ref_apple_clang_O2":122.792}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":120,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":5.5,"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":5.5,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"The filter checks out, but its ABI test crashes with Clang","gpt-6.1-sol-max":"Correct filter with excessive transpose work and substantial test defects","muse-wonder-full-xhigh":"correct, but the heaviest design: 16×16 transposes for 16×8 data","opus-5.5-xhigh":"Correct but very heavy (two 16×16 transposes with a mov per pair), with some wrong terminology and dead work","deepseek-v4.1-flash-max":"correct and heavily documented 16-lane implementation; the two `a` values are mislabelled and the code is more than twice the lean size."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"tests","title":"Inline ABI probe omits caller-saved clobbers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Full 16×16 transposes process unused columns","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"dead-code","title":"Generated boundary cases are overwritten before use","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Unchanged forbidden writes escape the guard test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Hidden call has incomplete register clobbers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"dead-code","title":"Boundary fixtures are overwritten before testing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Full square transpose computes unused columns","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guards do not enforce the narrower write window","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"performance","title":"full 16×16 transpose used for a 16×8 problem","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"limit/HEV masks built per half in 16-bit lanes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"redundancy","title":"28 no-op clamps before saturating narrows","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"inverted naming and wrong term","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"extra test groups collapse to one pass/fail and skips count as pass","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Oversized transposes and widened core","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Duplicate constants and dead work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Wrong terms and confusing names","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard span off for v","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"`a_hev` is the non-HEV value","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"244/556 instructions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"no x18 check","status":"valid","detail":"","location":""}],"score":6.1,"slop":{"score":1.8,"per_seat":{"opus-5.5-xhigh":2.0,"muse-wonder-full-xhigh":3.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":2.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":6.1,"score_cross_vendor":6.1,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":91.51,"tokens":{"uncached_input":468586,"cache_read":10572436,"cache_write":0,"output":82402,"reasoning":43758},"cost_usd":5.6369,"cost_breakdown":{"input":1.874344,"cache_read":2.114487,"cache_write":0.0,"output":1.64804},"cost_usd_as_run":5.6369,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":442,"code":369,"comment":63,"blank":10},{"path":"tests/selftest.c","role":"test","lines":673,"code":615,"comment":30,"blank":28}],"impl_lines":442,"impl_code":369,"impl_comment":63,"test_lines":673,"comment_ratio":0.146,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":19.92,"reference":134.188,"speedup":6.736,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.188}},{"id":"h_mb","candidate":36.553,"reference":116.757,"speedup":3.194,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":116.757}},{"id":"v_inner","candidate":24.943,"reference":129.795,"speedup":5.204,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.795}},{"id":"h_inner","candidate":41.165,"reference":109.863,"speedup":2.669,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.863}},{"id":"geomean","candidate":29.405,"reference":122.258,"speedup":4.158,"baselines":{"wpd_neon":14.53,"c_ref_apple_clang_O2":122.258}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":76,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":6.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct and well structured, with a write-boundary test gap","gpt-6.1-sol-max":"Correct and well documented, but the widened kernel repeats substantial work","muse-wonder-full-xhigh":"correct, but per-half scalar-style mask and GPR-extracted stores","opus-5.5-xhigh":"Correct and well documented, but the core is fully 16-bit and slow","deepseek-v4.1-flash-max":"correct, well documented, with a clear two-half core; two 8-lane passes and an odd empty-`.if` idiom in the row store."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests cannot detect unchanged stores to read-only taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Entire kernel runs separately for both halves","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Inner write boundary is unprotected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"mask built step-by-step per half","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"h row tails extracted through GPRs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"constants materialized via scalar+dup every half","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"store-order register reuse obscures dataflow","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"garbled V_BODY comment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"16-bit core per half","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"GPR round trip per row","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Limited layout coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"8-lane two-pass core","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"empty `.if \\inner` branches","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"no x18 check","status":"valid","detail":"","location":""}],"score":7.2,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":1.5,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":3.3,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":113.89,"tokens":{"uncached_input":634413,"cache_read":61043328,"cache_write":0,"output":464765,"reasoning":363367},"cost_usd":2.1588,"cost_breakdown":{"input":0.095162,"cache_read":1.8313,"cache_write":0.0,"output":0.232382},"cost_usd_as_run":2.1588,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":596,"code":442,"comment":104,"blank":50},{"path":"tests/selftest.c","role":"test","lines":758,"code":635,"comment":68,"blank":55}],"impl_lines":596,"impl_code":442,"impl_comment":104,"test_lines":758,"comment_ratio":0.19,"tests_declared":4,"impl_trailing_comments":129},"hidden":{"passed":4,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":9.808,"reference":138.378,"speedup":14.109,"baselines":{"wpd_neon":6.836,"c_ref_apple_clang_O2":138.378}},{"id":"h_mb","candidate":21.201,"reference":122.784,"speedup":5.791,"baselines":{"wpd_neon":21.6,"c_ref_apple_clang_O2":122.784}},{"id":"v_inner","candidate":11.746,"reference":136.603,"speedup":11.63,"baselines":{"wpd_neon":13.217,"c_ref_apple_clang_O2":136.603}},{"id":"h_inner","candidate":27.486,"reference":113.725,"speedup":4.138,"baselines":{"wpd_neon":27.117,"c_ref_apple_clang_O2":113.725}},{"id":"geomean","candidate":16.096,"reference":127.462,"speedup":7.919,"baselines":{"wpd_neon":15.167,"c_ref_apple_clang_O2":127.462}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":190,"flags":1,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":2.0,"gpt-6.1-sol-max":2.0,"muse-wonder-full-xhigh":2.0,"opus-5.5-xhigh":2.0,"deepseek-v4.1-flash-max":3.0},"council_summaries":{"gpt-6.1-terra-max":"All four filters produce wrong results on valid input","gpt-6.1-sol-max":"Missing common-filter arithmetic corrupts valid inputs","muse-wonder-full-xhigh":"HEV filter values never computed; wrong output on most filtered lines","opus-5.5-xhigh":"Wrong output on every HEV line and every inner non-HEV line; its own `make test` fails","deepseek-v4.1-flash-max":"wrong output on every HEV line plus an uninitialized \"+4\" constant; the submitted code fails its own `make test`."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"HEV corrections are never computed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Inner output depends on an uninitialized SIMD register","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"high","category":"abi","title":"The selftest overwrites Apple’s platform register","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"dead-code","title":"The constant table is unused","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"HEV adjustments use thresholds and masks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Inner rounding reads an uninitialized constant","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI caller retains unprotected caller-saved operands","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"abi","title":"Test caller overwrites Apple’s platform register","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Constant table is never referenced","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"f1h/f2h are never computed","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"inner-edge f1p adds an uninitialized register","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"dead-code","title":"constant table never referenced","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"header and inline comments state false facts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"24-parameter core macro with churning register roles","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"per-tap delta application is ~7 ops per tap","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"HEV deltas never computed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"Uninitialised register used as +4","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"Unused constants","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False claims in comments","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"HEV deltas come from stale registers","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`s6` used as \"+4\" but never initialised","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"misleading-docs","title":"the core comment describes code that is not there","status":"valid","detail":"","location":""}],"score":2.2,"slop":{"score":2.1,"per_seat":{"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":5.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":2.0}},"defect_points":10.9,"score_cross_vendor":2.2,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.1,"tokens":{"uncached_input":23181,"cache_read":485504,"cache_write":0,"output":23623,"reasoning":11363},"cost_usd":0.3311,"cost_breakdown":{"input":0.046362,"cache_read":0.04855,"cache_write":0.0,"output":0.23623},"cost_usd_as_run":0.3311,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":253,"code":215,"comment":22,"blank":16},{"path":"tests/selftest.c","role":"test","lines":606,"code":545,"comment":37,"blank":24}],"impl_lines":253,"impl_code":215,"impl_comment":22,"test_lines":606,"comment_ratio":0.093,"tests_declared":5,"impl_trailing_comments":3},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.149,"reference":135.574,"speedup":16.638,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":135.574}},{"id":"h_mb","candidate":20.928,"reference":118.91,"speedup":5.682,"baselines":{"wpd_neon":21.222,"c_ref_apple_clang_O2":118.91}},{"id":"v_inner","candidate":13.519,"reference":132.131,"speedup":9.774,"baselines":{"wpd_neon":12.865,"c_ref_apple_clang_O2":132.131}},{"id":"h_inner","candidate":26.876,"reference":111.382,"speedup":4.144,"baselines":{"wpd_neon":26.754,"c_ref_apple_clang_O2":111.382}},{"id":"geomean","candidate":15.777,"reference":124.109,"speedup":7.866,"baselines":{"wpd_neon":14.803,"c_ref_apple_clang_O2":124.109}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":215,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":8.5,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct, compact implementation with extensive differential tests","gpt-6.1-sol-max":"Correct, maintainable implementation with particularly broad boundary and layout tests","muse-wonder-full-xhigh":"compact and elegant, sparse documentation","opus-5.5-xhigh":"Correct, clean two-bank transposes, well-commented macros, strong tests","deepseek-v4.1-flash-max":"the smallest and leanest 16-lane implementation (112/204 instructions), tidy transpose, good tests; only an x18 test gap."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard comparisons cannot detect unchanged forbidden writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Write guards cover the read footprint","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"edge metric widened to halfwords","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"h transposes fully back before storing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"five-line file header only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Wasted bic","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Heavier h path than needed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"x18 not checked","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"no x18 check","status":"valid","detail":"","location":""}],"score":8.8,"slop":{"score":0.1,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.4,"score_cross_vendor":8.667,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":62.59,"tokens":{"uncached_input":96,"cache_read":7593941,"cache_write":276920,"output":861112,"reasoning":826158},"cost_usd":1.2848,"cost_breakdown":{"input":2.9e-05,"cache_read":0.352138,"cache_write":0.237911,"output":0.69471},"cost_usd_as_run":1.2848,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":403,"code":324,"comment":58,"blank":21},{"path":"tests/selftest.c","role":"test","lines":790,"code":673,"comment":70,"blank":47}],"impl_lines":403,"impl_code":324,"impl_comment":58,"test_lines":790,"comment_ratio":0.152,"tests_declared":6,"impl_trailing_comments":23},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":10.354,"reference":136.015,"speedup":13.137,"baselines":{"wpd_neon":6.605,"c_ref_apple_clang_O2":136.015}},{"id":"h_mb","candidate":31.45,"reference":119.025,"speedup":3.785,"baselines":{"wpd_neon":20.76,"c_ref_apple_clang_O2":119.025}},{"id":"v_inner","candidate":9.81,"reference":132.043,"speedup":13.46,"baselines":{"wpd_neon":12.865,"c_ref_apple_clang_O2":132.043}},{"id":"h_inner","candidate":31.016,"reference":111.905,"speedup":3.608,"baselines":{"wpd_neon":26.72,"c_ref_apple_clang_O2":111.905}},{"id":"geomean","candidate":17.741,"reference":124.365,"speedup":7.01,"baselines":{"wpd_neon":14.734,"c_ref_apple_clang_O2":124.365}}]},"quality":{"duplication":{"ratio":0.2006,"lines":65,"significant":324,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":43,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":7.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct filter with a guard-test gap and minor redundant work","gpt-6.1-sol-max":"Correct filter with avoidable transpose work and overclaimed memory testing","muse-wonder-full-xhigh":"good byte-lane v core, heavyweight h transpose machinery","opus-5.5-xhigh":"Correct and well documented with strong tests; transpose-back store path costs about 2× on h","deepseek-v4.1-flash-max":"clean 16-lane implementation, the clearest arithmetic documentation of the set, and a thorough six-suite test file; no defects found."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard test leaves forbidden taps writable","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Unstored taps are converted to signed and back","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal transposes handle halves independently","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Rounded mappings leave trailing over-read slack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Unchanged forbidden stores remain invisible","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"h path does four 8×8 transposes plus split/merge shuffles","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"edge metric computed in 16-bit lanes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"HEV differences recomputed and unused taps sign-converted","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"candidates merged with two rounds of `bit`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Heavy h path and unneeded conversions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Guard comment overstates","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`(f1+1)>>1` built with `movi`+`sqadd`+`sshr`","status":"valid","detail":"","location":""}],"score":7.8,"slop":{"score":0.5,"per_seat":{"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.6,"score_cross_vendor":7.875,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":77.18,"tokens":{"uncached_input":156752,"cache_read":2519396,"cache_write":0,"output":41652,"reasoning":22684},"cost_usd":1.9639,"cost_breakdown":{"input":0.627008,"cache_read":0.503879,"cache_write":0.0,"output":0.83304},"cost_usd_as_run":1.9639,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":479,"code":417,"comment":43,"blank":19},{"path":"tests/selftest.c","role":"test","lines":537,"code":485,"comment":29,"blank":23}],"impl_lines":479,"impl_code":417,"impl_comment":43,"test_lines":537,"comment_ratio":0.093,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":20.308,"reference":133.432,"speedup":6.57,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.432}},{"id":"h_mb","candidate":35.051,"reference":118.164,"speedup":3.371,"baselines":{"wpd_neon":20.697,"c_ref_apple_clang_O2":118.164}},{"id":"v_inner","candidate":16.9,"reference":131.687,"speedup":7.792,"baselines":{"wpd_neon":12.845,"c_ref_apple_clang_O2":131.687}},{"id":"h_inner","candidate":38.201,"reference":111.582,"speedup":2.921,"baselines":{"wpd_neon":26.784,"c_ref_apple_clang_O2":111.582}},{"id":"geomean","candidate":26.037,"reference":123.373,"speedup":4.738,"baselines":{"wpd_neon":14.715,"c_ref_apple_clang_O2":123.373}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":26,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":5.5,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct halfword implementation with repeated mask setup","gpt-6.1-sol-max":"Correct and readable, with an unnecessarily large hot path","muse-wonder-full-xhigh":"correct, most static instructions of any correct entry","opus-5.5-xhigh":"Correct but the heaviest kernel of the set, with redundant constant rebuilding and full transposes; moderate tests","deepseek-v4.1-flash-max":"correct with good docs and tests, but the heaviest instruction count of the set and a couple of small style issues."},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Interior threshold is broadcast six times per half","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Compact guard region permits unchanged outer-tap stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Fully widened halves inflate the kernel","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Forbidden inner writes are not protected","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"h stores extracted through a single GPR","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"halfword halves with duplicated threshold broadcasts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"oversized save frame for accumulator registers","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"directed coverage is thin and ABI sentinels weak","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"`orr v,v,v` written out as a register copy","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"16-bit two-pass core with repeated rebuilds","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Unneeded spills","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Narrow coverage","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`orr vX, vY, vY` used as a register move","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"two 8-lane passes plus six d-register saves","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`orr v18.16b, v30.16b, v30.16b`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"no x18 check","status":"valid","detail":"","location":""}],"score":6.7,"slop":{"score":0.4,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.6,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.99,"tokens":{"uncached_input":11763,"cache_read":188032,"cache_write":0,"output":2498,"reasoning":829},"cost_usd":0.0337,"cost_breakdown":{"input":0.011763,"cache_read":0.009402,"cache_write":0.0,"output":0.01249},"cost_usd_as_run":0.0337,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":23,"code":13,"comment":6,"blank":4},{"path":"tests/selftest.c","role":"test","lines":261,"code":224,"comment":19,"blank":18}],"impl_lines":23,"impl_code":13,"impl_comment":6,"test_lines":261,"comment_ratio":0.316,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":3,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.935,"reference":134.241,"speedup":143.634,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.241}},{"id":"h_mb","candidate":0.945,"reference":117.629,"speedup":124.461,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":117.629}},{"id":"v_inner","candidate":0.939,"reference":129.927,"speedup":138.323,"baselines":{"wpd_neon":12.566,"c_ref_apple_clang_O2":129.927}},{"id":"h_inner","candidate":0.939,"reference":109.979,"speedup":117.086,"baselines":{"wpd_neon":26.184,"c_ref_apple_clang_O2":109.979}},{"id":"geomean","candidate":0.94,"reference":122.561,"speedup":130.444,"baselines":{"wpd_neon":14.531,"c_ref_apple_clang_O2":122.561}}]},"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":1.0,"gpt-6.1-terra-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":1.0},"council_summaries":{"muse-wonder-full-xhigh":"scaffold stub: the component is not implemented at all","gpt-6.1-terra-max":"All four entry points remain no-op stubs.","gpt-6.1-sol-max":"All four filters remain unimplemented.","opus-5.5-xhigh":"Unmodified stub; nothing is filtered","deepseek-v4.1-flash-max":"untouched scaffold stub: no filtering at all, and its own selftest fails"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"all four entry points return without filtering","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"no guard-page or ABI tests","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"critical","category":"correctness","title":"Filtering is absent","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Memory and ABI checks are absent","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Every entry point immediately returns","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"No implementation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"`src/vp8_lf_neon.S:9-23`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"`tests/selftest.c:253-261`","status":"valid","detail":"","location":""}],"score":0.9,"slop":{"score":0.0,"per_seat":{"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0,"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.6,"score_cross_vendor":0.833,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.14,"tokens":{"uncached_input":14359,"cache_read":291840,"cache_write":0,"output":9916,"reasoning":2541},"cost_usd":0.0785,"cost_breakdown":{"input":0.014359,"cache_read":0.014592,"cache_write":0.0,"output":0.04958},"cost_usd_as_run":0.0785,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":393,"code":379,"comment":10,"blank":4},{"path":"tests/selftest.c","role":"test","lines":387,"code":338,"comment":26,"blank":23}],"impl_lines":393,"impl_code":379,"impl_comment":10,"test_lines":387,"comment_ratio":0.026,"tests_declared":5,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":15.803,"reference":136.246,"speedup":8.621,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":136.246}},{"id":"h_mb","candidate":73.883,"reference":119.057,"speedup":1.611,"baselines":{"wpd_neon":21.316,"c_ref_apple_clang_O2":119.057}},{"id":"v_inner","candidate":17.7,"reference":131.371,"speedup":7.422,"baselines":{"wpd_neon":12.824,"c_ref_apple_clang_O2":131.371}},{"id":"h_inner","candidate":67.501,"reference":110.097,"speedup":1.631,"baselines":{"wpd_neon":26.204,"c_ref_apple_clang_O2":110.097}},{"id":"geomean","candidate":34.367,"reference":123.762,"speedup":3.601,"baselines":{"wpd_neon":14.737,"c_ref_apple_clang_O2":123.762}}]},"quality":{"duplication":{"ratio":0.058,"lines":22,"significant":379,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-02","reviewed":false},"scores":{"muse-wonder-full-xhigh":6.5,"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":5.0},"council_summaries":{"muse-wonder-full-xhigh":"correct with honest exhaustive tests, but stack/scalar data movement dominates the hot path","gpt-6.1-terra-max":"Correct in the checks run, but the horizontal path has substantial avoidable work.","gpt-6.1-sol-max":"Correct filtering, with substantial avoidable work in the hot paths.","opus-5.5-xhigh":"Correct, but the h path is effectively scalar through the stack and the v path bounces through memory; needs a rewrite for NEON-level speed","deepseek-v4.1-flash-max":"correct, but the scalar byte transpose makes the h functions barely faster than C; tests also skip the memory contract on macOS"},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"all data round-trips through the stack and scalar loops","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"false mechanism comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"verbosity","title":"transpose unrolled as 128 literal load lines","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"guard test silently passes on 16 KiB-page hosts","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"ABI probe cannot detect register swaps and skips x18","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Stack-based transpose and scalar packing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests skip 16 KiB pages","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Alignment coverage is overstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Stack staging replaces register transposes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Inner filters compute unused outer outputs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guard coverage disappears on other page sizes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"high","category":"performance","title":"Byte-by-byte transpose and scalar write-back","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"v path round-trips through the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost no documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard and ABI tests don't check results","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Wasted work in inner mode","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Unneeded moves around selects","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"performance","title":"`src/vp8_lf_neon.S:200-334,345-377`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"`tests/selftest.c:221`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`src/vp8_lf_neon.S:98-103,345-377`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`src/vp8_lf_neon.S:80,109-126`","status":"valid","detail":"","location":""}],"score":6.0,"slop":{"score":1.0,"per_seat":{"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":2.0,"gpt-6.1-terra-max":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":5.5,"score_cross_vendor":5.667,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1-b2","task":"vp8lf-neon","lead":"muse-wonder-full-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":9.45,"tokens":{"cache_read":3655424,"cache_write":0,"output":24009,"reasoning":12864,"uncached_input":102419},"cost_usd":0.4052,"cost_breakdown":{"cache_read":0.182771,"cache_write":0.0,"input":0.102419,"output":0.120045},"cost_usd_as_run":0.4052,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":12.26,"tokens":{"cache_read":6877312,"cache_write":0,"output":120098,"reasoning":100244,"uncached_input":95068},"cost_usd":0.107,"cost_breakdown":{"cache_read":0.020632,"cache_write":0.0,"input":0.01426,"output":0.072059},"cost_usd_as_run":0.107,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":17.78,"tokens":{"cache_read":2861440,"cache_write":0,"output":33474,"reasoning":19995,"uncached_input":91930},"cost_usd":0.8047,"cost_breakdown":{"cache_read":0.286144,"cache_write":0.0,"input":0.18386,"output":0.33474},"cost_usd_as_run":0.8047,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":25.7,"tokens":{"cache_read":4328996,"cache_write":191718,"output":60139,"reasoning":36561,"uncached_input":60},"cost_usd":3.6026,"cost_breakdown":{"cache_read":0.865799,"cache_write":1.533744,"input":0.00024,"output":1.20278},"cost_usd_as_run":3.6026,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":19.55,"tokens":{"cache_read":2238940,"cache_write":0,"output":34829,"reasoning":19534,"uncached_input":226661},"cost_usd":2.051,"cost_breakdown":{"cache_read":0.447788,"cache_write":0.0,"input":0.906644,"output":0.69658},"cost_usd_as_run":2.051,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":6.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"V","score":5.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"V","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"V","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":5.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"V","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"V","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.5}],"panel":["gpt-6.1-terra-max","deepseek-v4.1-flash-max","gpt-6.1-sol-max","opus-5.5-xhigh","muse-wonder-full-xhigh"]},"entries":[{"letter":"U","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":19.1,"tokens":{"uncached_input":133215,"cache_read":807103,"cache_write":0,"output":85806,"reasoning":62493},"cost_usd":1.2052,"cost_breakdown":{"input":0.26643,"cache_read":0.08071,"cache_write":0.0,"output":0.85806},"cost_usd_as_run":1.2052,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":525,"code":324,"comment":150,"blank":51},{"path":"tests/selftest.c","role":"test","lines":505,"code":422,"comment":53,"blank":30}],"impl_lines":525,"impl_code":324,"impl_comment":150,"test_lines":505,"comment_ratio":0.316,"tests_declared":5,"impl_trailing_comments":185},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":13.441,"reference":133.432,"speedup":9.927,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.432}},{"id":"h_mb","candidate":39.545,"reference":116.894,"speedup":2.956,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.894}},{"id":"v_inner","candidate":9.569,"reference":129.561,"speedup":13.54,"baselines":{"wpd_neon":12.543,"c_ref_apple_clang_O2":129.561}},{"id":"h_inner","candidate":32.505,"reference":109.684,"speedup":3.374,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.684}},{"id":"geomean","candidate":20.164,"reference":122.016,"speedup":6.051,"baselines":{"wpd_neon":14.527,"c_ref_apple_clang_O2":122.016}}]},"isolation":{"tool_calls":13,"flags":2,"audited":"2026-11-02","reviewed":true},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks performed, but horizontal stores and unused macroblock calculations add substantial work.","deepseek-v4.1-flash-max":"bit-exact, well documented, second-fastest; one dead computation and a heavy h store","gpt-6.1-sol-max":"Correct, relatively lean arithmetic with wasteful horizontal writeback.","opus-5.5-xhigh":"Correct, well documented and the second-fastest, but uses the scalar byte-gather `h` store loop, wastes work in the macroblock kernel and has no guard or register tests","muse-wonder-full-xhigh":"Bit-exact and well documented; second-leanest, with dead common-filter work on the macroblock path and scalar h stores"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal stores serialize through a scalar loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"dead-code","title":"Macroblock calls calculate unused common-filter outer taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"No guard-page or register-preservation tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"dead-code","title":"src/vp8_lf_neon.S:172","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:450","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"tests/selftest.c:493","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal writeback serializes vector results","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Macroblock calls calculate unused inner-filter outputs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Safety invariants lack regression coverage","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"The `h` stores use a scalar byte-gather loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The macroblock kernel does dead and duplicate work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The common filter runs in 16-bit lanes and blends take 3 instructions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard pages, no register checks and no junk in the upper halves of x2–x4","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Scalar h-store loops and 3-op blends","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Outer adjustment computed but unused on macroblock path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page or ABI-junk tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Comment on nearly every instruction","status":"valid","detail":"","location":""}],"score":7.6,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":1.5}},"defect_points":4.7,"score_cross_vendor":7.6,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","task":"vp8lf-neon","lead":"muse-wonder-full-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":40.88,"tokens":{"cache_read":23922432,"cache_write":0,"output":100285,"reasoning":50625,"uncached_input":668311},"cost_usd":2.3659,"cost_breakdown":{"cache_read":1.196121,"cache_write":0.0,"input":0.668311,"output":0.501425},"cost_usd_as_run":2.3659,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":72.26,"tokens":{"cache_read":31698709,"cache_write":1079000,"output":316789,"reasoning":201931,"uncached_input":346},"cost_usd":21.3088,"cost_breakdown":{"cache_read":6.339742,"cache_write":8.632,"input":0.001384,"output":6.33578},"cost_usd_as_run":21.3088,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.317","sessions":4}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":64.98,"tokens":{"cache_read":17742720,"cache_write":0,"output":131531,"reasoning":82964,"uncached_input":966086},"cost_usd":5.0219,"cost_breakdown":{"cache_read":1.774271,"cache_write":0.0,"input":1.932172,"output":1.31531},"cost_usd_as_run":5.0219,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":4}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":203.32,"tokens":{"cache_read":23890624,"cache_write":0,"output":173999,"reasoning":88199,"uncached_input":1229656},"cost_usd":13.1767,"cost_breakdown":{"cache_read":4.778125,"cache_write":0.0,"input":4.918624,"output":3.47998},"cost_usd_as_run":13.1767,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":4}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":71.26,"tokens":{"cache_read":48545280,"cache_write":0,"output":562291,"reasoning":417417,"uncached_input":579886},"cost_usd":0.5701,"cost_breakdown":{"cache_read":0.145635,"cache_write":0.0,"input":0.086982,"output":0.337373},"cost_usd_as_run":0.5701,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":4}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":6.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":9.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"A","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"B","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K","score":8.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s1","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"G","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"G","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":7.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"G","score":6.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"G","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"L","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"G","score":7.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-bfW","source_run":"2026-11-02-vp8lf-neon-muse-wonder-full-xhigh-r1","letter":"O","score":9.0}],"panel":["gpt-6.1-terra-max","opus-5.5-xhigh","gpt-6.1-sol-max","muse-wonder-full-xhigh","deepseek-v4.1-flash-max"]},"entries":[{"letter":"A","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.17,"tokens":{"uncached_input":23313,"cache_read":748544,"cache_write":0,"output":29273,"reasoning":15490},"cost_usd":0.2071,"cost_breakdown":{"input":0.023313,"cache_read":0.037427,"cache_write":0.0,"output":0.146365},"cost_usd_as_run":0.2071,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":375,"code":322,"comment":36,"blank":17},{"path":"tests/selftest.c","role":"test","lines":572,"code":515,"comment":33,"blank":24}],"impl_lines":375,"impl_code":322,"impl_comment":36,"test_lines":572,"comment_ratio":0.101,"tests_declared":5,"impl_trailing_comments":19},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":16.381,"reference":133.632,"speedup":8.158,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.632}},{"id":"h_mb","candidate":50.193,"reference":117.082,"speedup":2.333,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":117.082}},{"id":"v_inner","candidate":24.638,"reference":130.483,"speedup":5.296,"baselines":{"wpd_neon":12.604,"c_ref_apple_clang_O2":130.483}},{"id":"h_inner","candidate":44.569,"reference":109.809,"speedup":2.464,"baselines":{"wpd_neon":26.184,"c_ref_apple_clang_O2":109.809}},{"id":"geomean","candidate":30.825,"reference":122.362,"speedup":3.97,"baselines":{"wpd_neon":14.544,"c_ref_apple_clang_O2":122.362}}]},"scores":{"gpt-6.1-terra-max":8.0,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with an avoidable horizontal-store loop.","deepseek-v4.1-flash-max":"bit-exact, clean and compact with the strongest test suite in the field; the h path still pays a stack round-trip and a scalar gather loop.","gpt-6.1-sol-max":"Correct with strong memory tests, but horizontal output is heavily scalarized.","opus-5.5-xhigh":"correct, with the strongest memory-contract tests, but the h path is slow and the mb kernel does dead work","muse-wonder-full-xhigh":"Correct with the strongest guard tests; scalar store, one dead computation, sparse file header"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Scalar gathering on every horizontal call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Some adjacent-limit cases cannot reveal a mask error","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"h path transposes twice and gathers rows with a byte loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`sshll` of the hev mask is emitted twice per mb half","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Scalar byte gathering dominates horizontal stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"dead-code","title":"Macroblock path computes a discarded adjustment","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"h output goes through a stack scratch and a 16-iteration scalar byte-gather loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"LOAD_H runs two separate 8×8 transposes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"COMMON computes (f1+1)>>1 even on macroblock edges","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"no callee-saved or x18 check","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Outer adjustment computed but unused on macroblock path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar byte-gather h store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Five-line file header","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":322,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":16,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.7,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":3.0,"score_cross_vendor":7.667,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":56.36,"tokens":{"uncached_input":54,"cache_read":3418829,"cache_write":224837,"output":566220,"reasoning":541196},"cost_usd":0.7315,"cost_breakdown":{"input":1.6e-05,"cache_read":0.156047,"cache_write":0.187974,"output":0.387488},"cost_usd_as_run":0.7315,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":494,"code":417,"comment":62,"blank":15},{"path":"tests/selftest.c","role":"test","lines":446,"code":373,"comment":40,"blank":33}],"impl_lines":494,"impl_code":417,"impl_comment":62,"test_lines":446,"comment_ratio":0.129,"tests_declared":3,"impl_trailing_comments":106},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":15.163,"reference":135.91,"speedup":8.963,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":135.91}},{"id":"h_mb","candidate":33.077,"reference":120.107,"speedup":3.631,"baselines":{"wpd_neon":21.316,"c_ref_apple_clang_O2":120.107}},{"id":"v_inner","candidate":29.507,"reference":132.314,"speedup":4.484,"baselines":{"wpd_neon":12.865,"c_ref_apple_clang_O2":132.314}},{"id":"h_inner","candidate":41.918,"reference":111.074,"speedup":2.65,"baselines":{"wpd_neon":26.483,"c_ref_apple_clang_O2":111.074}},{"id":"geomean","candidate":28.065,"reference":124.454,"speedup":4.435,"baselines":{"wpd_neon":14.787,"c_ref_apple_clang_O2":124.454}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Register-only geometry and correct results in the probes, with unreliable directed-case construction.","deepseek-v4.1-flash-max":"correct, well documented, but it transposes back after filtering, which is the one avoidable cost in an otherwise lean design.","gpt-6.1-sol-max":"Correct and readable, with excessive horizontal output shuffling.","opus-5.5-xhigh":"correct, works entirely in registers, good docs; middling speed","muse-wonder-full-xhigh":"Correct with a genuine transpose-back h path; shuffle-heavy stores and unverified boundary generator"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Generated lines are not guaranteed to reach their claimed boundaries","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard test leaves immutable taps writable","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Full inverse transpose for the stores","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"Store macros are 70 and 36 unrolled `ext`/`st1` pairs","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Full inverse transpose reconstructs unused taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"the filter runs as two .8h halves, and h output goes through an inverse transpose plus 32 `ext` before the lane stores","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"no register checks and no write-protected outer taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"the register map says v26 (mask & !hev) serves the inner p1/q1 adjustment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"One `ext` shuffle per stored row segment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Boundary lines are not re-measured","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0911,"lines":38,"significant":417,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":25,"flags":0,"audited":"2026-11-04","reviewed":false},"score":8.2,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.5}},"defect_points":1.8,"score_cross_vendor":8.375,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.08,"tokens":{"uncached_input":13877,"cache_read":209280,"cache_write":0,"output":4507,"reasoning":208},"cost_usd":0.5734,"cost_breakdown":{"input":0.13877,"cache_read":0.20928,"cache_write":0.0,"output":0.22535},"cost_usd_as_run":0.5734,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":630,"code":616,"comment":14,"blank":0},{"path":"tests/selftest.c","role":"test","lines":270,"code":233,"comment":19,"blank":18}],"impl_lines":630,"impl_code":616,"impl_comment":14,"test_lines":270,"comment_ratio":0.022,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":20.644,"reference":138.598,"speedup":6.714,"baselines":{"wpd_neon":6.825,"c_ref_apple_clang_O2":138.598}},{"id":"h_mb","candidate":54.992,"reference":119.529,"speedup":2.174,"baselines":{"wpd_neon":20.76,"c_ref_apple_clang_O2":119.529}},{"id":"v_inner","candidate":25.743,"reference":130.242,"speedup":5.059,"baselines":{"wpd_neon":12.607,"c_ref_apple_clang_O2":130.242}},{"id":"h_inner","candidate":49.211,"reference":110.65,"speedup":2.248,"baselines":{"wpd_neon":26.201,"c_ref_apple_clang_O2":110.65}},{"id":"geomean","candidate":34.63,"reference":124.304,"speedup":3.589,"baselines":{"wpd_neon":14.709,"c_ref_apple_clang_O2":124.304}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":5.0,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but duplicated kernels and scalar output gathering make it costly to maintain and run.","deepseek-v4.1-flash-max":"correct, but the most slop of any entry: a dead empty macro, a 45-line block duplicated between the two kernels, and 56 `mov` register shuffles in the h transposes.","gpt-6.1-sol-max":"Correct, but substantial copying, scalar gathering, and duplicated source need cleanup.","opus-5.5-xhigh":"bit-exact, but copy-pasted, full of redundant movs and undocumented, with a dead \"transpose\" macro and thin tests","muse-wonder-full-xhigh":"Correct, but machine-expanded duplication, thin tests and sparse docs need a real cleanup before merge"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal output is gathered byte by byte","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Mask construction is duplicated between kernels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"dead-code","title":"Empty transpose macro","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No protected-page coverage","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"dead-code","title":"Dead macro, with a comment describing work it does not do","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"redundancy","title":"Identical mask block duplicated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h path is the most expensive in the field","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Own tests are the thinnest of the non-stub entries","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal output is serialized through byte gathers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Geometry and mask logic are copied between entry points","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"dead-code","title":"Unused empty transpose macro","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Memory-access restrictions are untested","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"redundancy","title":"heavy copy-paste","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"every transpose stage writes v24–v31 and copies back with 8 movs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"dead-code","title":"an empty `transpose` macro","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"a two-line header and no per-macro or per-function docs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"a thin directed test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`put` takes an unused `half` parameter","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"redundancy","title":"Whole kernel and transpose duplicated verbatim","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"dead-code","title":"Empty transpose macro","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Register-copy chains between transpose stages","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Boundary test is thin and lane-uniform","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Two-line file header","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Inner store comment copied from macroblock path","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.7143,"lines":440,"significant":616,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":7,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.3,"slop":{"score":2.7,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":3.0,"gpt-6.1-sol-max":1.5,"muse-wonder-full-xhigh":3.5,"deepseek-v4.1-flash-max":4.0}},"defect_points":6.4,"score_cross_vendor":6.0,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":78.61,"tokens":{"uncached_input":274967,"cache_read":3907784,"cache_write":0,"output":55356,"reasoning":31624},"cost_usd":2.9885,"cost_breakdown":{"input":1.099868,"cache_read":0.781557,"cache_write":0.0,"output":1.10712},"cost_usd_as_run":2.9885,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":520,"code":378,"comment":114,"blank":28},{"path":"tests/selftest.c","role":"test","lines":413,"code":351,"comment":34,"blank":28}],"impl_lines":520,"impl_code":378,"impl_comment":114,"test_lines":413,"comment_ratio":0.232,"tests_declared":2,"impl_trailing_comments":48},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":20.77,"reference":136.488,"speedup":6.571,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":136.488}},{"id":"h_mb","candidate":42.759,"reference":118.626,"speedup":2.774,"baselines":{"wpd_neon":20.749,"c_ref_apple_clang_O2":118.626}},{"id":"v_inner","candidate":17.971,"reference":131.975,"speedup":7.344,"baselines":{"wpd_neon":12.861,"c_ref_apple_clang_O2":131.975}},{"id":"h_inner","candidate":36.292,"reference":111.898,"speedup":3.083,"baselines":{"wpd_neon":26.764,"c_ref_apple_clang_O2":111.898}},{"id":"geomean","candidate":27.588,"reference":124.35,"speedup":4.507,"baselines":{"wpd_neon":14.726,"c_ref_apple_clang_O2":124.35}}]},"scores":{"gpt-6.1-terra-max":7.0,"deepseek-v4.1-flash-max":6.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":5.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but repeated arithmetic and scratch traffic burden the hot paths.","deepseek-v4.1-flash-max":"correct and well documented, but every tap goes through a 96-byte stack draft, coefficients are computed twice, and f1 three times: the highest instruction count of the field.","gpt-6.1-sol-max":"Correct, but pervasive scratch traffic and repeated arithmetic weaken the hot path.","opus-5.5-xhigh":"correct, but needless memory round-trips and duplicated arithmetic make it the slowest; thin tests","muse-wonder-full-xhigh":"Correct, but recomputes filter coefficients and round-trips all results through stack scratch"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Vertical results unnecessarily pass through stack scratch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Macroblock weights are calculated twice per half","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Boundary and memory checks leave gaps","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"Needlessly copies all results to the stack, even for the v functions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"redundancy","title":"Coefficients and `f1` recomputed instead of reused","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No guard-page, callee-saved or x18 coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Every result passes through stack memory","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Identical adjustments are recomputed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Protected-memory behavior is not exercised","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"the v entries send every output tap through a stack scratch","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"redundancy","title":"each mbedge weight is computed twice, and f1 three times","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"h output uses a scalar byte-gather loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"few thresholds, and no memory or register checks","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"mbedge coefficients and f-values recomputed per tap","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scratch round-trip and scalar byte gather on stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page or ABI tests","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":37,"flags":1,"audited":"2026-11-04","reviewed":true},"score":6.6,"slop":{"score":1.2,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":2.0}},"defect_points":4.8,"score_cross_vendor":6.375,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":10.56,"tokens":{"uncached_input":17251,"cache_read":640896,"cache_write":0,"output":21166,"reasoning":9564},"cost_usd":1.8717,"cost_breakdown":{"input":0.17251,"cache_read":0.640896,"cache_write":0.0,"output":1.0583},"cost_usd_as_run":1.8717,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":384,"code":298,"comment":62,"blank":24},{"path":"tests/selftest.c","role":"test","lines":404,"code":333,"comment":44,"blank":27}],"impl_lines":384,"impl_code":298,"impl_comment":62,"test_lines":404,"comment_ratio":0.172,"tests_declared":3,"impl_trailing_comments":3},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":18.828,"reference":133.474,"speedup":7.089,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.474}},{"id":"h_mb","candidate":48.272,"reference":116.925,"speedup":2.422,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.925}},{"id":"v_inner","candidate":24.597,"reference":129.92,"speedup":5.282,"baselines":{"wpd_neon":12.539,"c_ref_apple_clang_O2":129.92}},{"id":"h_inner","candidate":43.467,"reference":109.599,"speedup":2.521,"baselines":{"wpd_neon":26.164,"c_ref_apple_clang_O2":109.599}},{"id":"geomean","candidate":31.397,"reference":122.095,"speedup":3.889,"baselines":{"wpd_neon":14.517,"c_ref_apple_clang_O2":122.095}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.5,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes and well documented, with a costly horizontal store and no guard tests.","deepseek-v4.1-flash-max":"correct and well documented, but the h path is a 16-iteration scalar byte-gather loop and the blend helper wastes two instructions per select.","gpt-6.1-sol-max":"Correct implementation with costly scalar output and incomplete memory testing.","opus-5.5-xhigh":"correct and robust, but the h path is a scalar stack gather, all the work is doubled into 16-bit halves, and the tests can't see over-reads","muse-wonder-full-xhigh":"Correct and well documented; scalar h store and missing guard/ABI tests hold it back"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal store serializes vector results through scalar loads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Memory access bounds are not protected","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h store is a per-row scalar gather","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"`SELECT_TAP` expands to three instructions where one suffices","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"memory contract and ABI are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal stores gather bytes from stack scratch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Buffer comparisons cannot enforce the memory-access contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"h output goes through a stack scratch and a 16-iteration scalar byte-gather loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"every blend takes 3 instructions, and all filter arithmetic runs as two .8h halves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"no guard pages and no register checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"constants are rebuilt for each half, and -128 is built as movi+neg","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar byte-gather h store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page or ABI tests","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":298,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":15,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.3,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":3.3,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":9.86,"tokens":{"uncached_input":38,"cache_read":1689151,"cache_write":101441,"output":56295,"reasoning":31326},"cost_usd":2.2754,"cost_breakdown":{"input":0.000152,"cache_read":0.33783,"cache_write":0.811528,"output":1.1259},"cost_usd_as_run":2.2754,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":425,"code":270,"comment":141,"blank":14},{"path":"tests/selftest.c","role":"test","lines":539,"code":460,"comment":49,"blank":30}],"impl_lines":425,"impl_code":270,"impl_comment":141,"test_lines":539,"comment_ratio":0.343,"tests_declared":7,"impl_trailing_comments":110},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":10.91,"reference":134.965,"speedup":12.371,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.965}},{"id":"h_mb","candidate":33.581,"reference":117.397,"speedup":3.496,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.397}},{"id":"v_inner","candidate":8.775,"reference":129.662,"speedup":14.775,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.662}},{"id":"h_inner","candidate":28.846,"reference":110.168,"speedup":3.819,"baselines":{"wpd_neon":26.174,"c_ref_apple_clang_O2":110.168}},{"id":"geomean","candidate":17.451,"reference":122.656,"speedup":7.029,"baselines":{"wpd_neon":14.532,"c_ref_apple_clang_O2":122.656}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Well-explained, correct in the probes, with a few test and documentation gaps.","deepseek-v4.1-flash-max":"bit-exact, excellent docs, second-fastest; header stack size is wrong and there is no ABI check.","gpt-6.1-sol-max":"Correct, carefully explained arithmetic with an avoidable output scratch round trip.","opus-5.5-xhigh":"correct, fast, clearly documented, good tests","muse-wonder-full-xhigh":"Correct, fast and well explained; one wrong stack-size comment and a skip-as-pass guard setup"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Outer adjacent-limit cases can be no-ops","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Inner p2/q2 writes are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Stack usage is misstated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"\"The h functions allocate 96 bytes of stack scratch\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal output unnecessarily spills and reloads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Stack description omits the inner size","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"no register checks, and the read-only pages cover only the p3/q3 rows","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"The h functions allocate 96 bytes of stack scratch\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Header overstates stack allocation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard test passes on mmap failure","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":270,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":19,"flags":1,"audited":"2026-11-04","reviewed":true},"score":8.7,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.5,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":80.79,"tokens":{"uncached_input":130761,"cache_read":2518536,"cache_write":0,"output":45325,"reasoning":24832},"cost_usd":1.9333,"cost_breakdown":{"input":0.523044,"cache_read":0.503707,"cache_write":0.0,"output":0.9065},"cost_usd_as_run":1.9333,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":537,"code":367,"comment":141,"blank":29},{"path":"tests/selftest.c","role":"test","lines":389,"code":326,"comment":35,"blank":28}],"impl_lines":537,"impl_code":367,"impl_comment":141,"test_lines":389,"comment_ratio":0.278,"tests_declared":2,"impl_trailing_comments":76},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":21.81,"reference":134.356,"speedup":6.16,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.356}},{"id":"h_mb","candidate":47.652,"reference":117.639,"speedup":2.469,"baselines":{"wpd_neon":20.634,"c_ref_apple_clang_O2":117.639}},{"id":"v_inner","candidate":17.588,"reference":130.341,"speedup":7.411,"baselines":{"wpd_neon":12.583,"c_ref_apple_clang_O2":130.341}},{"id":"h_inner","candidate":40.449,"reference":110.911,"speedup":2.742,"baselines":{"wpd_neon":26.472,"c_ref_apple_clang_O2":110.911}},{"id":"geomean","candidate":29.324,"reference":122.946,"speedup":4.193,"baselines":{"wpd_neon":14.585,"c_ref_apple_clang_O2":122.946}}]},"scores":{"gpt-6.1-terra-max":7.5,"deepseek-v4.1-flash-max":7.0,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with expensive horizontal staging and weak memory-boundary coverage.","deepseek-v4.1-flash-max":"correct and cleanly factored, but the mask is computed twice in 16-bit instead of once in byte lanes, making this the slowest v path.","gpt-6.1-sol-max":"Correct, but repeated widening and scratch-based output impose substantial overhead.","opus-5.5-xhigh":"correct and well documented, but about 3–4× the expert's instruction count, and thin tests","muse-wonder-full-xhigh":"Correct and documented, but the mask re-widens every tap for every difference"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Horizontal output takes an extra transpose and stack round trip","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Outer adjacent-limit cases have no observable filter adjustment","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No guard-page tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"Mask computed per half with two widenings per difference","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"h path pays a double transpose plus a stack round-trip and loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"\"widened with uxtl … to signed .8H lanes\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Masks repeatedly widen the same input taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Already-transposed rows are copied through scratch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Memory checks only compare writable buffers","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"masks and hev are computed in .8h, re-widening both taps for every difference","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"the h path transposes back with two more 8×8 networks, makes 16 st1 to a stack scratch, then runs a 16-iteration scalar copy loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"few thresholds, and no memory or register checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"a separate `add` after every row load instead of post-indexed addressing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Repeated widening throughout the kernel","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Deterministic tests mostly repeat one line on all lanes, no guard/ABI","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":32,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.9,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.0,"score_cross_vendor":6.75,"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.5,"tokens":{"uncached_input":16680,"cache_read":104832,"cache_write":0,"output":1956,"reasoning":262},"cost_usd":0.0317,"cost_breakdown":{"input":0.01668,"cache_read":0.005242,"cache_write":0.0,"output":0.00978},"cost_usd_as_run":0.0317,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":23,"code":13,"comment":6,"blank":4},{"path":"tests/selftest.c","role":"test","lines":283,"code":242,"comment":23,"blank":18}],"impl_lines":23,"impl_code":13,"impl_comment":6,"test_lines":283,"comment_ratio":0.316,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":3,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.945,"reference":133.6,"speedup":141.361,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.6}},{"id":"h_mb","candidate":0.945,"reference":117.166,"speedup":123.973,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.166}},{"id":"v_inner","candidate":0.936,"reference":129.662,"speedup":138.543,"baselines":{"wpd_neon":12.546,"c_ref_apple_clang_O2":129.662}},{"id":"h_inner","candidate":0.939,"reference":109.806,"speedup":116.902,"baselines":{"wpd_neon":26.191,"c_ref_apple_clang_O2":109.806}},{"id":"geomean","candidate":0.941,"reference":122.183,"speedup":129.797,"baselines":{"wpd_neon":14.53,"c_ref_apple_clang_O2":122.183}}]},"scores":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5},"council_summaries":{"gpt-6.1-terra-max":"All four entry points are unimplemented.","deepseek-v4.1-flash-max":"the deliverable is the untouched scaffold stub: all four symbols return without filtering, and the entry's own selftest fails 4/4.","gpt-6.1-sol-max":"All four required filters remain unimplemented.","opus-5.5-xhigh":"the unmodified scaffold stub; nothing is filtered","muse-wonder-full-xhigh":"Unchanged scaffold stub; no filtering is implemented"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Every function returns without filtering","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"No implementation at all","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Every entry point returns without filtering","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"all four entry points just `ret`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"All four entry points are bare `ret`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":4,"flags":0,"audited":"2026-11-04","reviewed":false},"score":0.7,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.6,"score_cross_vendor":0.5,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":8.37,"tokens":{"uncached_input":28,"cache_read":1079126,"cache_write":91838,"output":48619,"reasoning":26837},"cost_usd":1.923,"cost_breakdown":{"input":0.000112,"cache_read":0.215825,"cache_write":0.734704,"output":0.97238},"cost_usd_as_run":1.923,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":358,"code":242,"comment":101,"blank":15},{"path":"tests/selftest.c","role":"test","lines":593,"code":513,"comment":50,"blank":30}],"impl_lines":358,"impl_code":242,"impl_comment":101,"test_lines":593,"comment_ratio":0.294,"tests_declared":5,"impl_trailing_comments":110},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":11.089,"reference":134.965,"speedup":12.171,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.965}},{"id":"h_mb","candidate":30.41,"reference":117.072,"speedup":3.85,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":117.072}},{"id":"v_inner","candidate":8.789,"reference":129.934,"speedup":14.784,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.934}},{"id":"h_inner","candidate":27.964,"reference":109.985,"speedup":3.933,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.985}},{"id":"geomean","candidate":16.967,"reference":122.584,"speedup":7.225,"baselines":{"wpd_neon":14.532,"c_ref_apple_clang_O2":122.584}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":9.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Well-structured filters and substantial tests, with a horizontal write-boundary gap.","deepseek-v4.1-flash-max":"bit-exact, well documented and cleanly factored; only the stack-scratch sizes in the header are wrong.","gpt-6.1-sol-max":"Correct and well tested, with avoidable horizontal-store traffic.","opus-5.5-xhigh":"clean, fast and well tested; two small nits","muse-wonder-full-xhigh":"Correct, compact and well documented; boundary coverage is partly fallback random lines"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal outer taps are never write-protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Stack usage is misstated","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"\"96-byte stack scratch\" / \"sp is lowered by 96 for the h entries\"","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal results unnecessarily pass through memory","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Incorrect stack-size description","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"no callee-saved or x18 check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"the header says h entries lower sp by 96 and use \"a 96-byte stack scratch\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"mbtap copies the shifted weights (2 movs, three times) only because addclip overwrites its delta inputs","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Header overstates stack allocation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Failed boundary constructions silently become random lines","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Read-only-row guard silently skipped on 16 KiB pages","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":242,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":14,"flags":1,"audited":"2026-11-04","reviewed":true},"score":8.6,"slop":{"score":0.5,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.6,"score_cross_vendor":8.625,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":68.76,"tokens":{"uncached_input":522,"cache_read":23726566,"cache_write":366981,"output":303340,"reasoning":0},"cost_usd":0.5957,"cost_breakdown":{"input":7.8e-05,"cache_read":0.379625,"cache_write":0.073396,"output":0.14257},"cost_usd_as_run":0.5957,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.5957,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":610,"code":453,"comment":146,"blank":11},{"path":"tests/selftest.c","role":"test","lines":647,"code":539,"comment":64,"blank":44}],"impl_lines":610,"impl_code":453,"impl_comment":146,"test_lines":647,"comment_ratio":0.244,"tests_declared":6,"impl_trailing_comments":148},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":12.296,"reference":135.354,"speedup":11.008,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":135.354}},{"id":"h_mb","candidate":25.706,"reference":117.891,"speedup":4.586,"baselines":{"wpd_neon":20.665,"c_ref_apple_clang_O2":117.891}},{"id":"v_inner","candidate":16.998,"reference":131.714,"speedup":7.749,"baselines":{"wpd_neon":12.848,"c_ref_apple_clang_O2":131.714}},{"id":"h_inner","candidate":29.877,"reference":111.647,"speedup":3.737,"baselines":{"wpd_neon":26.73,"c_ref_apple_clang_O2":111.647}},{"id":"geomean","candidate":20.016,"reference":123.768,"speedup":6.183,"baselines":{"wpd_neon":14.697,"c_ref_apple_clang_O2":123.768}}]},"scores":{"gpt-6.1-terra-max":8.5,"deepseek-v4.1-flash-max":8.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":7.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Sound implementation and broad tests, but the guard and boundary coverage overstate what they check.","deepseek-v4.1-flash-max":"excellent documentation and factoring, but the guard-page test does not measure the window it claims to, and its v-path transposes cost ~55 % over the best entry.","gpt-6.1-sol-max":"Correct kernels; duplicated transpose work and a verified guard-test gap.","opus-5.5-xhigh":"correct, works entirely in registers, thoroughly documented; its guard test misses v-path over-reads","muse-wonder-full-xhigh":"Correct and exemplary docs/tests; only minor h-store and test-skip issues"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Vertical high guard is placed beyond the actual read window","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Boundary blocks repeat the same relation order","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Guard test's v window is wrong, so the comment is false","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"`m_h` is the *non*-hev mask and `m_nh` is the hev mask","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal halves are transposed separately","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Vertical high guard is beyond the legal read window","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Inner deltas are masked twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"the guard test takes the v read window to be 8·stride","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"the h path runs 4 TR8 networks and 16 `mov w10, v.s[1]` vector-to-GPR moves for the q1/q2 halves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"the mask names are inverted: m_h = mask & ~hev and m_nh = mask & hev","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"LF_IN ANDs f1/f2 with m_all even though a is already zero on masked-off lanes, as its own comment says","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Vector-to-GPR transfer per h row","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard test passes on setup failure","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":86,"flags":1,"audited":"2026-11-04","reviewed":true},"score":8.3,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":2.2,"score_cross_vendor":8.3,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":47.22,"tokens":{"uncached_input":159580,"cache_read":1585756,"cache_write":0,"output":32029,"reasoning":15957},"cost_usd":1.5961,"cost_breakdown":{"input":0.63832,"cache_read":0.317151,"cache_write":0.0,"output":0.64058},"cost_usd_as_run":1.5961,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":469,"code":370,"comment":77,"blank":22},{"path":"tests/selftest.c","role":"test","lines":361,"code":303,"comment":29,"blank":29}],"impl_lines":469,"impl_code":370,"impl_comment":77,"test_lines":361,"comment_ratio":0.172,"tests_declared":2,"impl_trailing_comments":72},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":18.187,"reference":135.259,"speedup":7.437,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":135.259}},{"id":"h_mb","candidate":52.797,"reference":117.681,"speedup":2.229,"baselines":{"wpd_neon":20.665,"c_ref_apple_clang_O2":117.681}},{"id":"v_inner","candidate":16.554,"reference":130.48,"speedup":7.882,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":130.48}},{"id":"h_inner","candidate":40.971,"reference":111.084,"speedup":2.711,"baselines":{"wpd_neon":26.483,"c_ref_apple_clang_O2":111.084}},{"id":"geomean","candidate":28.408,"reference":123.244,"speedup":4.338,"baselines":{"wpd_neon":14.583,"c_ref_apple_clang_O2":123.244}}]},"scores":{"gpt-6.1-terra-max":6.5,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but both geometries incur substantial avoidable scratch traffic.","gpt-6.1-sol-max":"Correct and documented, but the output path introduces especially heavy memory traffic.","opus-5.5-xhigh":"Correct, but the slowest h path (byte copies through the stack) and a test that misses the boundary it names","deepseek-v4.1-flash-max":"correct, but the h stores walk bytes individually and every result goes through the stack.","muse-wonder-full-xhigh":"correct with good docs, slowest store paths and sparsest directed tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Repeated stack round trips","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Forbidden unchanged writes go undetected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Rows pass through a second scratch buffer","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Forbidden unchanged writes escape the suite","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Each h row is copied byte by byte into a stack buffer and then reloaded as a word and a halfword (a store-to-load forwarding hazard on wider loads after narrower stores), in a 16-iteration loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Every result is narrowed to the stack (STORE_TAP), then reloaded to store, even for v","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The transpose macros copy back after every zip pair and then do 8 more `mov`s","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The \"Edge-term mask boundary … near flim_e\" test is not near the boundary","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page, callee-saved, x18 or upper-bit tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h store assembles each row with 6 `ldrb` + 6 `strb`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"all results are written to a tap-major stack scratch and reloaded even for the v geometry","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"H store round-trips every byte through row scratch","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Directed thresholds are sparse and misuse is untested","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"V store reloads scratch tap-by-tap with recomputed address","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":25,"flags":1,"audited":"2026-11-04","reviewed":true},"score":6.2,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.4,"score_cross_vendor":6.125,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.7,"tokens":{"uncached_input":42430,"cache_read":274688,"cache_write":0,"output":9251,"reasoning":3027},"cost_usd":0.2048,"cost_breakdown":{"input":0.08486,"cache_read":0.027469,"cache_write":0.0,"output":0.09251},"cost_usd_as_run":0.2048,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":398,"code":364,"comment":24,"blank":10},{"path":"tests/selftest.c","role":"test","lines":320,"code":273,"comment":27,"blank":20}],"impl_lines":398,"impl_code":364,"impl_comment":24,"test_lines":320,"comment_ratio":0.062,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":18.418,"reference":135.721,"speedup":7.369,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":135.721}},{"id":"h_mb","candidate":49.374,"reference":118.721,"speedup":2.405,"baselines":{"wpd_neon":20.644,"c_ref_apple_clang_O2":118.721}},{"id":"v_inner","candidate":24.614,"reference":131.677,"speedup":5.35,"baselines":{"wpd_neon":12.763,"c_ref_apple_clang_O2":131.677}},{"id":"h_inner","candidate":45.007,"reference":111.694,"speedup":2.482,"baselines":{"wpd_neon":26.767,"c_ref_apple_clang_O2":111.694}},{"id":"geomean","candidate":31.681,"reference":124.073,"speedup":3.916,"baselines":{"wpd_neon":14.679,"c_ref_apple_clang_O2":124.073}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with scalar gathering and unnecessary conditional machinery.","gpt-6.1-sol-max":"Correct filtering, but scalar output packing and generated conditions reduce quality.","opus-5.5-xhigh":"Correct 16-bit kernel; scalar h-store loop and machine-expanded conditional slop","deepseek-v4.1-flash-max":"correct, but the tap narrowing conditions are written as constant-folding gibberish and the h store is a scalar loop.","muse-wonder-full-xhigh":"correct, but duplicated transpose and obfuscated tap packing"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Scalar horizontal gathering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Whole-buffer comparison cannot observe unchanged writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Constant conditions obscure narrowing","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal output is serialized through scratch","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Buffer comparisons do not enforce access permissions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Constant conditions obscure the writable taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"h stores use a scalar byte-gather loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"verbosity","title":"Conditions are left as constant expressions (`.if !\\inner || (1 >= 2 && 1 <= 5)` …), and there are eight separate `.if \\high` blocks for the widening","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"SELECT is `mov`/`bsl`/`mov` (14 uses)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"H_STORE reserves 96 bytes for the inner edge, which uses 64","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page or ABI tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"style","title":"the per-tap narrow guards are unreadable constant expressions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h stores use a 16-iteration `ldrb`/`orr`/`str` loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"the file header is two lines","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"redundancy","title":"H_LOAD writes the 8×8 transpose twice inline","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"PACK gates are constant-folded arithmetic puzzles","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Two-line file header, no register map","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar H store gather loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page or ABI-register test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Inner H store allocates 96B, uses 64B","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":364,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.6,"slop":{"score":1.3,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":0.5,"muse-wonder-full-xhigh":2.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.8,"score_cross_vendor":6.333,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":20.22,"tokens":{"uncached_input":104773,"cache_read":18237312,"cache_write":0,"output":196676,"reasoning":153180},"cost_usd":0.1884,"cost_breakdown":{"input":0.015716,"cache_read":0.054712,"cache_write":0.0,"output":0.118006},"cost_usd_as_run":0.1884,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1884,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":568,"code":405,"comment":132,"blank":31},{"path":"tests/selftest.c","role":"test","lines":553,"code":472,"comment":42,"blank":39}],"impl_lines":568,"impl_code":405,"impl_comment":132,"test_lines":553,"comment_ratio":0.246,"tests_declared":7,"impl_trailing_comments":84},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":16.287,"reference":134.503,"speedup":8.259,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":134.503}},{"id":"h_mb","candidate":30.956,"reference":117.986,"speedup":3.811,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.986}},{"id":"v_inner","candidate":14.964,"reference":129.561,"speedup":8.658,"baselines":{"wpd_neon":12.556,"c_ref_apple_clang_O2":129.561}},{"id":"h_inner","candidate":28.188,"reference":110.172,"speedup":3.908,"baselines":{"wpd_neon":26.191,"c_ref_apple_clang_O2":110.172}},{"id":"geomean","candidate":21.474,"reference":122.681,"speedup":5.713,"baselines":{"wpd_neon":14.533,"c_ref_apple_clang_O2":122.681}}]},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with gaps in guard-test reliability and claims.","gpt-6.1-sol-max":"Correct and well explained, but register allocation creates substantial avoidable work.","opus-5.5-xhigh":"Correct and thoroughly commented; the v entries use callee-saved v8–v15 and the stack for no need","deepseek-v4.1-flash-max":"the best documentation of the set and fully vectorised stores, but it saves d8–d15 and stages the v results through the stack.","muse-wonder-full-xhigh":"correct and well-tested, but pays callee-saved traffic no peer needs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Guard coverage is overstated","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard setup failures count as skips","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Two-pass kernel spills intermediate results","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guards cannot detect unchanged outer-tap stores","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The v entries keep un-widened taps in v8–v15, which forces four `stp`/`ldp` of d8–d15 plus a 48-byte stack scratch for the low-half results","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The mask is built with six separate compare+`and` pairs in 16-bit, and INNER_BODY clips both a candidates before a `bif`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The comment says \"four written rows (-2..1 = p1 q0 q1)\", naming three and omitting p0","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Guard windows are filled with uniform random bytes (≈0.7% of lines pass the mask at 193/63), skip strides above 256, and there is no ABI test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"v entries save and restore the whole d8–d15 bank on every call","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"v results are staged through a 48-byte stack scratch and reloaded before storing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"v entries save/restore d8–d15 on every call","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Two v entries duplicate load/widen/join bodies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"Comment lists three taps for four rows","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"H_ROWS moves unused results on inner path","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.2617,"lines":106,"significant":405,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":101,"flags":4,"audited":"2026-11-04","reviewed":true},"score":7.5,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":0.5,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":2.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.1,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.07,"tokens":{"uncached_input":19536,"cache_read":476288,"cache_write":0,"output":9567,"reasoning":1996},"cost_usd":0.0912,"cost_breakdown":{"input":0.019536,"cache_read":0.023814,"cache_write":0.0,"output":0.047835},"cost_usd_as_run":0.0912,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":367,"code":338,"comment":18,"blank":11},{"path":"tests/selftest.c","role":"test","lines":312,"code":269,"comment":24,"blank":19}],"impl_lines":367,"impl_code":338,"impl_comment":18,"test_lines":312,"comment_ratio":0.051,"tests_declared":2,"impl_trailing_comments":1},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":24.855,"reference":134.293,"speedup":5.403,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.293}},{"id":"h_mb","candidate":56.063,"reference":118.5,"speedup":2.114,"baselines":{"wpd_neon":20.644,"c_ref_apple_clang_O2":118.5}},{"id":"v_inner","candidate":33.206,"reference":132.345,"speedup":3.986,"baselines":{"wpd_neon":12.861,"c_ref_apple_clang_O2":132.345}},{"id":"h_inner","candidate":51.642,"reference":111.179,"speedup":2.153,"baselines":{"wpd_neon":26.645,"c_ref_apple_clang_O2":111.179}},{"id":"geomean","candidate":39.317,"reference":123.702,"speedup":3.146,"baselines":{"wpd_neon":14.685,"c_ref_apple_clang_O2":123.702}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":6.0,"opus-5.5-xhigh":5.5,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with unnecessary stack work and a misleading transpose comment.","gpt-6.1-sol-max":"Correct results, but pervasive scratch traffic and a false register-layout comment need attention.","opus-5.5-xhigh":"Correct, but tap vectors round-trip through the stack, h stores are scalar, code is duplicated and the tests are thin","deepseek-v4.1-flash-max":"correct, but every tap is round-tripped through the stack and the h store is a scalar loop, leaving it the slowest of the halfword designs.","muse-wonder-full-xhigh":"correct, but stack-round-trip geometry and duplicated widen blocks"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Stack-based processing and gathering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Write-footprint gap","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Wrong transpose output registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Every call round-trips tap vectors through the stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Access permissions are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Transpose output registers are documented incorrectly","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Every row is stored to the stack, reloaded once per 8-lane half, results are written back as d-halves and reloaded before the real store","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"h stores use a scalar byte-gather loop with a compare-and-branch","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The 48-line widen block is duplicated verbatim in KERNEL_MB and KERNEL_INNER, and OUTPUT re-narrows the original tap on every call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The directed cases are thin (interior limit only at flim_i=10, hev only at 2, edge at a single metric), with no guard-page or ABI tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Docs are minimal: one header comment on register roles, no per-function contract","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"all eight tap vectors are stored to the stack on load and reloaded for each half","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h stores use the scalar per-row gather","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"the identical 48-line load-and-widen prologue is duplicated in `KERNEL_MB` and `KERNEL_INNER`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Stack round-trip for all geometry","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"KERNEL_MB/KERNEL_INNER duplicate the 8-tap load/widen block verbatim","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar H store gather loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page or ABI-register test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.2899,"lines":98,"significant":338,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.4,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":4.9,"score_cross_vendor":6.167,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":26.7,"tokens":{"uncached_input":66,"cache_read":5071982,"cache_write":199277,"output":145248,"reasoning":106111},"cost_usd":5.5138,"cost_breakdown":{"input":0.000264,"cache_read":1.014396,"cache_write":1.594216,"output":2.90496},"cost_usd_as_run":5.5138,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":398,"code":257,"comment":126,"blank":15},{"path":"tests/selftest.c","role":"test","lines":794,"code":657,"comment":88,"blank":49}],"impl_lines":398,"impl_code":257,"impl_comment":126,"test_lines":794,"comment_ratio":0.329,"tests_declared":7,"impl_trailing_comments":139},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":8.873,"reference":135.406,"speedup":15.26,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":135.406}},{"id":"h_mb","candidate":32.657,"reference":116.978,"speedup":3.582,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":116.978}},{"id":"v_inner","candidate":8.748,"reference":130.744,"speedup":14.945,"baselines":{"wpd_neon":12.746,"c_ref_apple_clang_O2":130.744}},{"id":"h_inner","candidate":26.672,"reference":110.219,"speedup":4.132,"baselines":{"wpd_neon":26.191,"c_ref_apple_clang_O2":110.219}},{"id":"geomean","candidate":16.125,"reference":122.915,"speedup":7.622,"baselines":{"wpd_neon":14.588,"c_ref_apple_clang_O2":122.915}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with strong directed tests and two localized issues.","gpt-6.1-sol-max":"Efficient arithmetic and broad tests, with modest copying overhead and a horizontal permission gap.","opus-5.5-xhigh":"Exceptionally documented, branch-free and exact, with the most thorough tests; slightly heavier tap updates","deepseek-v4.1-flash-max":"correct, fastest of the halfword designs, with best-in-class tests and documentation.","muse-wonder-full-xhigh":"exemplary proofs and deepest tests; only a store round-trip separates it from the top"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI test calls through an incompatible function type","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Horizontal results make a stack round trip","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal results take an avoidable stack detour","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Read-only-tap checks cover only the vertical functions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Every tap update widens the tap (uxtl, saddw/ssubw, sqxtun) instead of applying signed-byte `sqadd` after `eor #0x80`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Nothing checks that x19–x29, d8–d15 or x18 are preserved, although the header (line 43) makes register claims","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"H store round-trips through stack scratch","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":257,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":37,"flags":1,"audited":"2026-11-04","reviewed":true},"score":8.8,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":1.3,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.33,"tokens":{"uncached_input":16090,"cache_read":69632,"cache_write":0,"output":388,"reasoning":88},"cost_usd":0.0215,"cost_breakdown":{"input":0.01609,"cache_read":0.003482,"cache_write":0.0,"output":0.00194},"cost_usd_as_run":0.0215,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":23,"code":13,"comment":6,"blank":4},{"path":"tests/selftest.c","role":"test","lines":216,"code":181,"comment":18,"blank":17}],"impl_lines":23,"impl_code":13,"impl_comment":6,"test_lines":216,"comment_ratio":0.316,"tests_declared":1,"impl_trailing_comments":0},"hidden":{"passed":3,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.935,"reference":136.267,"speedup":145.803,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":136.267}},{"id":"h_mb","candidate":0.935,"reference":119.319,"speedup":127.669,"baselines":{"wpd_neon":21.211,"c_ref_apple_clang_O2":119.319}},{"id":"v_inner","candidate":0.939,"reference":131.883,"speedup":140.406,"baselines":{"wpd_neon":12.841,"c_ref_apple_clang_O2":131.883}},{"id":"h_inner","candidate":0.939,"reference":111.63,"speedup":118.844,"baselines":{"wpd_neon":26.686,"c_ref_apple_clang_O2":111.63}},{"id":"geomean","candidate":0.937,"reference":124.385,"speedup":132.756,"baselines":{"wpd_neon":14.791,"c_ref_apple_clang_O2":124.385}}]},"scores":{"gpt-6.1-terra-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.0,"deepseek-v4.1-flash-max":1.0,"muse-wonder-full-xhigh":1.0},"council_summaries":{"gpt-6.1-terra-max":"All four entry points are unimplemented.","gpt-6.1-sol-max":"All four entry points remain unimplemented stubs.","opus-5.5-xhigh":"Unmodified stub: no filtering at all","deepseek-v4.1-flash-max":"no implementation: the unmodified scaffold stub returns without filtering.","muse-wonder-full-xhigh":"unmodified stub, no filtering at all"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Filters return without modifying pixels","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"No function performs filtering","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"All four functions are a bare `ret`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"tests/selftest.c is the unmodified scaffold","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"all four entry points are a bare `ret`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"All four entry points return without filtering","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":3,"flags":0,"audited":"2026-11-04","reviewed":false},"score":0.6,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":0.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":4.0,"score_cross_vendor":0.667,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":11.98,"tokens":{"uncached_input":34022,"cache_read":500736,"cache_write":0,"output":24795,"reasoning":12471},"cost_usd":0.3661,"cost_breakdown":{"input":0.068044,"cache_read":0.050074,"cache_write":0.0,"output":0.24795},"cost_usd_as_run":0.3661,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":379,"code":296,"comment":56,"blank":27},{"path":"tests/selftest.c","role":"test","lines":442,"code":373,"comment":43,"blank":26}],"impl_lines":379,"impl_code":296,"impl_comment":56,"test_lines":442,"comment_ratio":0.159,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":17.956,"reference":133.526,"speedup":7.436,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.526}},{"id":"h_mb","candidate":47.442,"reference":116.862,"speedup":2.463,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":116.862}},{"id":"v_inner","candidate":23.18,"reference":129.764,"speedup":5.598,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.764}},{"id":"h_inner","candidate":42.372,"reference":109.89,"speedup":2.593,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.89}},{"id":"geomean","candidate":30.244,"reference":122.135,"speedup":4.038,"baselines":{"wpd_neon":14.526,"c_ref_apple_clang_O2":122.135}}]},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes and well tested arithmetically, with a footprint gap and scalar stores.","gpt-6.1-sol-max":"Strong arithmetic tests and clear structure, with a costly horizontal output path.","opus-5.5-xhigh":"Correct and well commented; 16-bit kernel with a scalar h-store loop and copy-heavy selects","deepseek-v4.1-flash-max":"correct and well documented; its only real cost is the byte-at-a-time h store loop.","muse-wonder-full-xhigh":"clean register-resident halfword design, only store/test gaps"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Scalar horizontal gathering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Forbidden unchanged writes go undetected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Scalar gathering follows vector filtering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Detailed numerical tests omit memory permissions","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"h stores use a scalar byte-gather loop","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"`mov`/`bsl`/`mov` triplets throughout","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Masks and filter run in two 16-bit halves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Strong arithmetic edge-case tests, but no guard pages and no register or upper-bit checks","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h stores gather rows one byte at a time in a scalar loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar H store gather loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page or ABI-register test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"mov/bsl/mov select triple repeated 8 times","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":296,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":11,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.4,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.1,"score_cross_vendor":7.333,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.28,"tokens":{"uncached_input":31096,"cache_read":271232,"cache_write":0,"output":8198,"reasoning":1654},"cost_usd":0.9921,"cost_breakdown":{"input":0.31096,"cache_read":0.271232,"cache_write":0.0,"output":0.4099},"cost_usd_as_run":0.9921,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":409,"code":371,"comment":21,"blank":17},{"path":"tests/selftest.c","role":"test","lines":311,"code":266,"comment":25,"blank":20}],"impl_lines":409,"impl_code":371,"impl_comment":21,"test_lines":311,"comment_ratio":0.054,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":19.111,"reference":136.561,"speedup":7.146,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":136.561}},{"id":"h_mb","candidate":54.751,"reference":118.815,"speedup":2.17,"baselines":{"wpd_neon":21.232,"c_ref_apple_clang_O2":118.815}},{"id":"v_inner","candidate":25.533,"reference":132.012,"speedup":5.17,"baselines":{"wpd_neon":12.773,"c_ref_apple_clang_O2":132.012}},{"id":"h_inner","candidate":51.771,"reference":111.657,"speedup":2.157,"baselines":{"wpd_neon":26.777,"c_ref_apple_clang_O2":111.657}},{"id":"geomean","candidate":34.294,"reference":124.358,"speedup":3.626,"baselines":{"wpd_neon":14.787,"c_ref_apple_clang_O2":124.358}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":6.5,"opus-5.5-xhigh":6.0,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with scalar gathering and incomplete footprint tests.","gpt-6.1-sol-max":"Correct and organized, with expensive horizontal packing and incomplete memory tests.","opus-5.5-xhigh":"Correct, but a 16-bit two-pass kernel, a scalar byte-gather loop for h stores and many redundant moves","deepseek-v4.1-flash-max":"correct and readable; scalar h store loop and tests without guard-page or ABI checks.","muse-wonder-full-xhigh":"correct, duplicated transpose macros and scalar stores"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Scalar horizontal stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No protection for unwritable taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Vector results are repacked with scalar byte loads","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Unchanged stores outside the write window pass","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"h stores go through a 16-iteration scalar loop of `ldrb`+`bfi` from a stack copy","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"All mask and filter arithmetic runs in two 8-lane 16-bit passes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"The two transposes contain 48 copy-back `mov`s","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"`mov`+`bsl`+`mov` triplets where one `bit`/`bif` would do","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page, callee-saved, x18 or upper-bit tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h stores use the 16-iteration byte gather","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"the added directed suite never leaves the buffer","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"redundancy","title":"TRANSPOSE_0/TRANSPOSE_16 duplicate ~50-line bodies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar H store gather loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page or ABI-register test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Inner H store allocates 96B, stores 64B","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1294,"lines":48,"significant":371,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":9,"flags":0,"audited":"2026-11-04","reviewed":false},"score":6.8,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.0,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":2.5,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.9,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":98.19,"tokens":{"uncached_input":432356,"cache_read":33806912,"cache_write":0,"output":308776,"reasoning":250518},"cost_usd":1.2334,"cost_breakdown":{"input":0.064853,"cache_read":1.014207,"cache_write":0.0,"output":0.154388},"cost_usd_as_run":1.2334,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":570,"code":403,"comment":153,"blank":14},{"path":"tests/selftest.c","role":"test","lines":514,"code":429,"comment":51,"blank":34}],"impl_lines":570,"impl_code":403,"impl_comment":153,"test_lines":514,"comment_ratio":0.275,"tests_declared":2,"impl_trailing_comments":150},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":13.913,"reference":133.4,"speedup":9.588,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.4}},{"id":"h_mb","candidate":32.941,"reference":116.652,"speedup":3.541,"baselines":{"wpd_neon":20.571,"c_ref_apple_clang_O2":116.652}},{"id":"v_inner","candidate":22.207,"reference":129.696,"speedup":5.84,"baselines":{"wpd_neon":12.556,"c_ref_apple_clang_O2":129.696}},{"id":"h_inner","candidate":39.92,"reference":109.738,"speedup":2.749,"baselines":{"wpd_neon":26.164,"c_ref_apple_clang_O2":109.738}},{"id":"geomean","candidate":25.247,"reference":121.993,"speedup":4.832,"baselines":{"wpd_neon":14.518,"c_ref_apple_clang_O2":121.993}}]},"scores":{"muse-wonder-full-xhigh":7.0,"opus-5.5-xhigh":6.5,"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":6.5,"deepseek-v4.1-flash-max":7.5},"council_summaries":{"muse-wonder-full-xhigh":"Correct kernel, but its headline boundary generator does not construct what it claims and every path round-trips through stack scratch","opus-5.5-xhigh":"correct and well commented, but the kernel round-trips through the stack, has redundant work, and its tests have a broken generator and no memory checks","gpt-6.1-terra-max":"Correct in the checks run, with avoidable stack traffic and a faulty boundary generator","gpt-6.1-sol-max":"Correct outputs, but unnecessary spills and unreliable directed tests need changes","deepseek-v4.1-flash-max":"Correct, robust and the second-fastest, with the most careful write-up; dragged down by a directed-test generator that never reaches the boundary it documents and by a heavier h transpose than necessary."},"findings":[{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"gen_main does not produce the claimed edge-sum boundary","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"All four entry points round-trip results through stack scratch and the h transpose is a 4-level zip network","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Kernel header p0'/q0' select formula does not match the code","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"gen_adj claims \"every other pair is 0\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Inner kernel recomputes f1 and the half adjustment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page, ABI-upper-bits or callee-saved tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Unexplained dead scratch headroom","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Per-half kernel writes results to the stack and reloads them","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page or register checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"Same values computed twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The flim_e boundary generator doesn't produce the sums its comment claims","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"Transpose spends many instructions on register moves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Some comments contradict the code","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Unused stack space","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Filtered halves make an unnecessary stack round trip","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Main-limit generator misses its intended boundary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Memory and register protections are untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Lane geometry is reversed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Masked deltas are recomputed unchanged","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Filter results unnecessarily round-trip through the stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Boundary generators miss their intended comparisons","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Memory and ABI violations escape the suite","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Lane geometry is reversed","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"\"Main-limit boundary\" generator does not sit at the boundary","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Own tests never check the memory contract or ABI","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"h path spends ~108 instructions on shuffles","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Duplicate `and`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":136,"flags":1,"audited":"2026-11-04","reviewed":false},"score":6.9,"slop":{"score":1.7,"per_seat":{"gpt-6.1-terra-max":1.5,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":3.5,"deepseek-v4.1-flash-max":0.5}},"defect_points":6.6,"score_cross_vendor":6.9,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":9.82,"tokens":{"uncached_input":25052,"cache_read":574720,"cache_write":0,"output":21354,"reasoning":11257},"cost_usd":0.3211,"cost_breakdown":{"input":0.050104,"cache_read":0.057472,"cache_write":0.0,"output":0.21354},"cost_usd_as_run":0.3211,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":368,"code":301,"comment":47,"blank":20},{"path":"tests/selftest.c","role":"test","lines":373,"code":323,"comment":27,"blank":23}],"impl_lines":368,"impl_code":301,"impl_comment":47,"test_lines":373,"comment_ratio":0.135,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":18.849,"reference":133.516,"speedup":7.084,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.516}},{"id":"h_mb","candidate":49.038,"reference":117.103,"speedup":2.388,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.103}},{"id":"v_inner","candidate":23.492,"reference":129.578,"speedup":5.516,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.578}},{"id":"h_inner","candidate":43.942,"reference":109.653,"speedup":2.495,"baselines":{"wpd_neon":26.164,"c_ref_apple_clang_O2":109.653}},{"id":"geomean","candidate":31.254,"reference":122.085,"speedup":3.906,"baselines":{"wpd_neon":14.528,"c_ref_apple_clang_O2":122.085}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6.1-sol-max":7.0,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, with avoidable horizontal-store traffic and an overstated test.","gpt-6.1-sol-max":"Correct, readable implementation whose horizontal stores need optimization.","opus-5.5-xhigh":"Correct 16-bit two-pass kernel with a scalar h-store loop and redundant selects","deepseek-v4.1-flash-max":"correct and tidy with the same scalar h store; its narrowing macro silently destroys the mask register.","muse-wonder-full-xhigh":"clean and well-documented, only store/test gaps"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Scalar horizontal gathering","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Whole-buffer comparison cannot detect every store","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Tap-major scratch feeds a scalar row loop","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Buffer equality does not enforce the write window","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"h stores use a scalar `ldrb`/`orr` gather loop through the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"On the macroblock edge, clip(c+3d) is computed twice (v23, then v26 after `c&hev`); the two are equal on the only lanes where v26 is used (mask&hev)","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"`mov`/`bsl`/`mov` triplets, 8 sites","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"The whole filter, masks included, runs in 16-bit halves","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page or ABI/upper-bit tests","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"h stores use the same per-row byte gather","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"`NARROW` overwrites v24, the normal-limit mask, without saying so","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Scalar H store gather loop","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No guard-page or ABI-register test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":301,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":14,"flags":0,"audited":"2026-11-04","reviewed":false},"score":7.2,"slop":{"score":0.6,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":2.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":0.0,"deepseek-v4.1-flash-max":0.0}},"defect_points":3.4,"score_cross_vendor":7.167,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.1,"tokens":{"uncached_input":37398,"cache_read":223360,"cache_write":0,"output":5223,"reasoning":632},"cost_usd":0.1494,"cost_breakdown":{"input":0.074796,"cache_read":0.022336,"cache_write":0.0,"output":0.05223},"cost_usd_as_run":0.1494,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":980,"code":958,"comment":17,"blank":5},{"path":"tests/selftest.c","role":"test","lines":284,"code":242,"comment":23,"blank":19}],"impl_lines":980,"impl_code":958,"impl_comment":17,"test_lines":284,"comment_ratio":0.017,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":17.389,"reference":134.062,"speedup":7.71,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.062}},{"id":"h_mb","candidate":43.63,"reference":117.03,"speedup":2.682,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":117.03}},{"id":"v_inner","candidate":16.561,"reference":129.852,"speedup":7.841,"baselines":{"wpd_neon":12.566,"c_ref_apple_clang_O2":129.852}},{"id":"h_inner","candidate":37.52,"reference":109.85,"speedup":2.928,"baselines":{"wpd_neon":26.181,"c_ref_apple_clang_O2":109.85}},{"id":"geomean","candidate":26.203,"reference":122.31,"speedup":4.668,"baselines":{"wpd_neon":14.532,"c_ref_apple_clang_O2":122.31}}]},"scores":{"gpt-6.1-terra-max":7.0,"gpt-6.1-sol-max":5.5,"opus-5.5-xhigh":5.0,"deepseek-v4.1-flash-max":5.5,"muse-wonder-full-xhigh":5.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the probes, but stack traffic and duplicated stores make the implementation costly and difficult to maintain.","gpt-6.1-sol-max":"Correct output, but excessive copying and hand-repeated schedules make maintenance costly.","opus-5.5-xhigh":"Correct, but 980 lines of copy-paste, stack spills, scalar gathers and weak tests","deepseek-v4.1-flash-max":"correct, but 980 lines of unrolled near-duplicate code and the slowest h path.","muse-wonder-full-xhigh":"verified correct, but hand-expanded to 3× peer size with masked directed tests"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Spill-heavy hot paths","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Duplicated row-store sequences","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Forbidden unchanged writes go undetected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Full input and output copies burden every call","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Memory permissions are not exercised","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Row packing is manually repeated sixteen times","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"verbosity","title":"All four functions are hand-expanded: spill and widen, two kernel passes, and 16 unrolled byte-gather row stores, repeated with only offsets changed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Rows are spilled to the stack and reloaded per half, results round-trip through the stack, and h rows are gathered byte by byte","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"redundancy","title":"48 copy `mov`s in the transposes, a 3-instruction blend macro (`mov`/`bsl`/`mov`), and `ldrb w6` followed by `mov w7, w6`","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page or ABI tests, and the directed cases repeat one line across all 16 lanes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"verbosity","title":"the 16-row output gather is written out in full sixteen times, twice","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"performance","title":"every tap and every result is round-tripped through a 256-byte stack frame","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"the long store blocks carry a single one-line comment","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"redundancy","title":"File is hand macro-expansion of one kernel","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Stack preserve/reload plus fully scalar stores","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Directed cases repeat one line into all 16 lines with positive-only steps","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No file header or register map","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Unindented macro bodies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.405,"lines":388,"significant":958,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-04","reviewed":false},"score":5.7,"slop":{"score":2.2,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":3.0,"gpt-6.1-sol-max":1.0,"muse-wonder-full-xhigh":3.0,"deepseek-v4.1-flash-max":3.0}},"defect_points":6.2,"score_cross_vendor":5.333,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":57.59,"tokens":{"uncached_input":0,"cache_read":0,"cache_write":0,"output":0,"reasoning":0},"cost_usd":0.0,"cost_breakdown":{"input":0.0,"cache_read":0.0,"cache_write":0.0,"output":0.0},"cost_usd_as_run":0.0,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":null},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":530,"code":418,"comment":97,"blank":15},{"path":"tests/selftest.c","role":"test","lines":468,"code":393,"comment":44,"blank":31}],"impl_lines":530,"impl_code":418,"impl_comment":97,"test_lines":468,"comment_ratio":0.188,"tests_declared":2,"impl_trailing_comments":128},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":16.035,"reference":136.435,"speedup":8.509,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":136.435}},{"id":"h_mb","candidate":45.909,"reference":119.582,"speedup":2.605,"baselines":{"wpd_neon":21.253,"c_ref_apple_clang_O2":119.582}},{"id":"v_inner","candidate":10.647,"reference":131.853,"speedup":12.384,"baselines":{"wpd_neon":12.845,"c_ref_apple_clang_O2":131.853}},{"id":"h_inner","candidate":35.448,"reference":111.416,"speedup":3.143,"baselines":{"wpd_neon":26.703,"c_ref_apple_clang_O2":111.416}},{"id":"geomean","candidate":22.959,"reference":124.425,"speedup":5.42,"baselines":{"wpd_neon":14.801,"c_ref_apple_clang_O2":124.425}}]},"scores":{"gpt-6.1-terra-max":7.0,"opus-5.5-xhigh":5.5,"gpt-6.1-sol-max":6.5,"muse-wonder-full-xhigh":7.0,"deepseek-v4.1-flash-max":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run, but costly horizontal stores and weaker footprint tests need work","opus-5.5-xhigh":"Correct, but has the most avoidable overhead (stack spills, scalar store loop, shuffle chains) and the weakest tests (no guard pages)","gpt-6.1-sol-max":"Correct in probes, but unnecessary memory traffic and weaker safety tests need work","muse-wonder-full-xhigh":"Correct and well commented, but the heaviest implementation with the weakest tests","deepseek-v4.1-flash-max":"correct stub-free implementation, but it is the slowest of the five and its own tests never exercise the memory or ABI rules"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"performance","title":"Hot paths spill and gather through the stack","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Directed adjacent-limit coverage misses outer comparisons","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No protected-page footprint test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Repeated masks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Edge orientations are reversed","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"performance","title":"Scalar, branchy h-store loop through a stack spill","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Macroblock kernel round-trips its low-half results through the stack","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"`LF_LOAD_H` uses two 8×8 transposes and 32 extra shuffles","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"No guard-page test and no ABI check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Interior-pair sweep tests p0|q0 instead of q3|q2","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"`bic` masks are computed twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"`LF_MASK` overstates its clobbers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"performance","title":"Horizontal stores scalarize vector results","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Avoidable macroblock result spills","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Allocated buffers cannot enforce access bounds","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI preservation is untested","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Interior-pair table omits q2/q3","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"performance","title":"Scalar stack-based h store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"No guard-page test at all","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Macroblock kernel spills half-results through the stack","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"LOAD_H shuttles taps through three register sets","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Sparse targeted thresholds and no ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"LF_MASK clobber list overstates what it touches","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicate bic recomputations","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No guard-page and no register-preservation test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"h stores use stack scratch plus a 16-iteration byte-gather loop","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"Two 8-lane half passes for the whole kernel","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"Wrong clobber list","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Duplicated `bic` derivations","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":44,"flags":1,"audited":"2026-11-04","reviewed":true},"score":6.6,"slop":{"score":0.8,"per_seat":{"gpt-6.1-terra-max":1.0,"opus-5.5-xhigh":1.0,"gpt-6.1-sol-max":0.0,"muse-wonder-full-xhigh":1.0,"deepseek-v4.1-flash-max":1.0}},"defect_points":6.2,"score_cross_vendor":6.6,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1-b2","task":"vp8lf-neon","lead":"opus-5.5-xhigh","replicate":1,"roster":null,"date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":6.46,"tokens":{"cache_read":3248896,"cache_write":0,"output":22611,"reasoning":13792,"uncached_input":112648},"cost_usd":0.3881,"cost_breakdown":{"cache_read":0.162445,"cache_write":0.0,"input":0.112648,"output":0.113055},"cost_usd_as_run":0.3881,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":37.47,"tokens":{"cache_read":1848020,"cache_write":0,"output":21295,"reasoning":11234,"uncached_input":171550},"cost_usd":1.4817,"cost_breakdown":{"cache_read":0.369604,"cache_write":0.0,"input":0.6862,"output":0.4259},"cost_usd_as_run":1.4817,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":1}},{"seat":"gpt-6-astra-xhigh","judge":"gpt-6-astra-xhigh","model":"gpt-6-astra","effort":"xhigh","label":"GPT‑6 Astra xhigh","usage":{"wall_minutes":9.85,"tokens":{"cache_read":2404736,"cache_write":0,"output":16123,"reasoning":7679,"uncached_input":127715},"cost_usd":4.488,"cost_breakdown":{"cache_read":2.404736,"cache_write":0.0,"input":1.27715,"output":0.80615},"cost_usd_as_run":4.488,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":10.81,"tokens":{"cache_read":2764544,"cache_write":0,"output":21186,"reasoning":13030,"uncached_input":133572},"cost_usd":0.7555,"cost_breakdown":{"cache_read":0.276454,"cache_write":0.0,"input":0.267144,"output":0.21186},"cost_usd_as_run":0.7555,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":1}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":12.41,"tokens":{"cache_read":8558720,"cache_write":0,"output":139801,"reasoning":107197,"uncached_input":111853},"cost_usd":0.1263,"cost_breakdown":{"cache_read":0.025676,"cache_write":0.0,"input":0.016778,"output":0.083881},"cost_usd_as_run":0.1263,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":1}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":14.11,"tokens":{"cache_read":3739371,"cache_write":201282,"output":60307,"reasoning":44239,"uncached_input":52},"cost_usd":3.5645,"cost_breakdown":{"cache_read":0.747874,"cache_write":1.610256,"input":0.000208,"output":1.20614},"cost_usd_as_run":3.5645,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316","sessions":1}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"W","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"W","score":6.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":7.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"W","score":7.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":8.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":10.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"W","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"W","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"W","score":6.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-b2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5}],"panel":["gpt-6.1-terra-max","muse-wonder-full-xhigh","gpt-6.1-sol-max","deepseek-v4.1-flash-max","opus-5.5-xhigh"]},"entries":[{"letter":"X","model":"gemini-4-argon","effort":"high","config":"gemini-4-argon@high","label":"Gemini 4 Argon high","harness":{"provider":"antigravity","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":20.78,"tokens":{"uncached_input":115570,"cache_read":703750,"cache_write":0,"output":73262,"reasoning":25121},"cost_usd":1.0341,"cost_breakdown":{"input":0.23114,"cache_read":0.070375,"cache_write":0.0,"output":0.73262},"cost_usd_as_run":1.0341,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Antigravity ACP 1.4.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":428,"code":297,"comment":83,"blank":48},{"path":"tests/selftest.c","role":"test","lines":802,"code":672,"comment":74,"blank":56}],"impl_lines":428,"impl_code":297,"impl_comment":83,"test_lines":802,"comment_ratio":0.218,"tests_declared":5,"impl_trailing_comments":138},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.833,"reference":134.02,"speedup":17.109,"baselines":{"wpd_neon":6.605,"c_ref_apple_clang_O2":134.02}},{"id":"h_mb","candidate":20.781,"reference":116.957,"speedup":5.628,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.957}},{"id":"v_inner","candidate":12.627,"reference":129.469,"speedup":10.253,"baselines":{"wpd_neon":12.536,"c_ref_apple_clang_O2":129.469}},{"id":"h_inner","candidate":25.357,"reference":109.796,"speedup":4.33,"baselines":{"wpd_neon":26.181,"c_ref_apple_clang_O2":109.796}},{"id":"geomean","candidate":15.11,"reference":122.176,"speedup":8.086,"baselines":{"wpd_neon":14.535,"c_ref_apple_clang_O2":122.176}}]},"scores":{"gpt-6.1-terra-max":8.0,"muse-wonder-full-xhigh":8.0,"gpt-6-astra-xhigh":8.5,"gpt-6.1-sol-max":8.5,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run; its ABI wrapper both misses violations and assumes extra preservation.","muse-wonder-full-xhigh":"Correct and well documented; ABI test is self-tailored, 6-tap is serial","gpt-6-astra-xhigh":"Correct, well-structured filter; the test harness depends on incidental register preservation.","gpt-6.1-sol-max":"Correct, readable core; the ABI wrapper and memory tests need correction.","deepseek-v4.1-flash-max":"bit-exact and the most readable source, with strong tests; the ABI harness holds caller-saved pointers across the call and the h store is 2× wide","opus-5.5-xhigh":"clean code with no wasted moves and good docs; the ABI wrapper in its tests is unsound"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI check misses x29","status":"valid","defect":"VP8-007","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI wrapper relies on caller-saved pointers after the call","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard test misses unchanged out-of-window stores","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Partial loads introduce avoidable dependencies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Saturation example conflates two operations","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"ABI wrapper does not check sp or x29 despite claiming sp","status":"valid","defect":"VP8-007","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"ABI wrapper keeps pointers in caller-saved registers across `blr`","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Lane-insert low-half loads create a false dependency","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"`smull`+`add` with serial register reuse for 27/18/9","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Guard claim overstates write-window verification","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"style","title":"Bitwise-OR merges signed adjustments","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"misleading-docs","title":"“2-tap on hev” misnames the filter","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"ABI wrapper retains pointers in caller-saved registers","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Guards miss forbidden unchanged writes","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Retains output pointers in caller-saved registers","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Misses forbidden unchanged stores","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Claims an absent stack-pointer check","status":"valid","defect":"VP8-007","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"ABI harness keeps output pointers in caller-saved x7 and x16 across `blr`","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"h macroblock store costs 2× the necessary store uops","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"Constant and result share v25","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"The ABI wrapper uses caller-saved registers after the call","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"The comment says sp is checked, but it isn't","status":"valid","defect":"VP8-007","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No read-only check around the write window","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"A check that can never fail","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"2-tap\" names the wrong filter","status":"valid","detail":"","location":""}],"isolation":{"tool_calls":15,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.1,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"muse-wonder-full-xhigh":0.5,"gpt-6.1-sol-max":1.0,"deepseek-v4.1-flash-max":0.5,"opus-5.5-xhigh":2.0}},"defect_points":5.4,"score_cross_vendor":8.1,"check":{"lint":{"diagnostics":0}}}]},{"schema":1,"id":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","task":"vp8lf-neon","lead":"opus-5.5-xhigh","replicate":1,"roster":"standard","date":"2026-11-02","status":"complete","hidden_suite_version":1,"council":{"seats":[{"seat":"gpt-6.1-terra-max","judge":"gpt-6.1-terra-max","model":"gpt-6.1-terra","effort":"max","label":"GPT‑6.1 Terra max","usage":{"wall_minutes":45.17,"tokens":{"cache_read":22479104,"cache_write":0,"output":114712,"reasoning":65206,"uncached_input":904488},"cost_usd":2.602,"cost_breakdown":{"cache_read":1.123956,"cache_write":0.0,"input":0.904488,"output":0.57356},"cost_usd_as_run":2.602,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":5}},{"seat":"deepseek-v4.1-flash-max","judge":"deepseek-v4.1-flash-max","model":"deepseek-v4.1-flash","effort":"max","label":"DeepSeek V4.1 Flash max","usage":{"wall_minutes":95.41,"tokens":{"cache_read":55757312,"cache_write":0,"output":620434,"reasoning":455132,"uncached_input":659234},"cost_usd":0.6384,"cost_breakdown":{"cache_read":0.167273,"cache_write":0.0,"input":0.098885,"output":0.37226},"cost_usd_as_run":0.6384,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5","sessions":5}},{"seat":"gpt-6.1-sol-max","judge":"gpt-6.1-sol-max","model":"gpt-6.1-sol","effort":"max","label":"GPT‑6.1 Sol max","usage":{"wall_minutes":99.0,"tokens":{"cache_read":19797248,"cache_write":0,"output":162029,"reasoning":109489,"uncached_input":1034747},"cost_usd":5.6695,"cost_breakdown":{"cache_read":1.979724,"cache_write":0.0,"input":2.069494,"output":1.62029},"cost_usd_as_run":5.6695,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":5}},{"seat":"opus-5.5-xhigh","judge":"opus-5.5-xhigh","model":"claude-opus-5.5","effort":"xhigh","label":"Opus 5.5 xhigh","usage":{"wall_minutes":116.35,"tokens":{"cache_read":32791754,"cache_write":1186463,"output":310986,"reasoning":200684,"uncached_input":374},"cost_usd":22.2713,"cost_breakdown":{"cache_read":6.55835,"cache_write":9.491704,"input":0.001496,"output":6.21972},"cost_usd_as_run":22.2713,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316","sessions":5}},{"seat":"gpt-6-astra-xhigh","judge":"gpt-6-astra-xhigh","model":"gpt-6-astra","effort":"xhigh","label":"GPT‑6 Astra xhigh","usage":{"wall_minutes":90.99,"tokens":{"cache_read":17638784,"cache_write":0,"output":102045,"reasoning":50926,"uncached_input":788101},"cost_usd":30.622,"cost_breakdown":{"cache_read":17.638784,"cache_write":0.0,"input":7.88101,"output":5.10225},"cost_usd_as_run":30.622,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0","sessions":5}},{"seat":"muse-wonder-full-xhigh","judge":"muse-wonder-full-xhigh","model":"muse-wonder-full","effort":"xhigh","label":"Muse Wonder Full xhigh","usage":{"wall_minutes":315.49,"tokens":{"cache_read":21103908,"cache_write":0,"output":172351,"reasoning":91279,"uncached_input":1190380},"cost_usd":12.4293,"cost_breakdown":{"cache_read":4.220781,"cache_write":0.0,"input":4.76152,"output":3.44702},"cost_usd_as_run":12.4293,"cost_basis":"list price incl. cache reads and writes, standard tier","fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1","sessions":4}}],"format":"v2","basis":"spec+rubric","rounds":1,"revisions":[],"anchors":[{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":6.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"E","score":7.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":8.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"E","score":7.0},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":6.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"E","score":8.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":7.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"E","score":8.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":10.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reL","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":7.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s2","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-s3","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"B","score":9.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":7.5},{"judge":"gpt-6.1-terra-max","session":"gpt-6.1-terra-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":7.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":8.5},{"judge":"gpt-6-astra-xhigh","session":"gpt-6-astra-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":10.0},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":7.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":8.5},{"judge":"gpt-6.1-sol-max","session":"gpt-6.1-sol-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":10.0},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":8.5},{"judge":"deepseek-v4.1-flash-max","session":"deepseek-v4.1-flash-max-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":7.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":8.5},{"judge":"opus-5.5-xhigh","session":"opus-5.5-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J","score":8.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L","score":9.0},{"judge":"muse-wonder-full-xhigh","session":"muse-wonder-full-xhigh-reG","source_run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"F","score":9.5}],"panel":["gpt-6.1-terra-max","deepseek-v4.1-flash-max","gpt-6.1-sol-max","opus-5.5-xhigh","muse-wonder-full-xhigh"]},"entries":[{"letter":"A","model":"gpt-6.1-terra","effort":"xhigh","config":"gpt-6.1-terra@xhigh","label":"GPT‑6.1 Terra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.34,"tokens":{"uncached_input":15613,"cache_read":420096,"cache_write":0,"output":10904,"reasoning":1610},"cost_usd":0.0911,"cost_breakdown":{"input":0.015613,"cache_read":0.021005,"cache_write":0.0,"output":0.05452},"cost_usd_as_run":0.0911,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":287,"code":263,"comment":13,"blank":11},{"path":"tests/selftest.c","role":"test","lines":373,"code":326,"comment":24,"blank":23}],"impl_lines":287,"impl_code":263,"impl_comment":13,"test_lines":373,"comment_ratio":0.047,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.434,"reference":133.863,"speedup":18.006,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.863}},{"id":"h_mb","candidate":20.434,"reference":117.061,"speedup":5.729,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":117.061}},{"id":"v_inner","candidate":13.868,"reference":129.73,"speedup":9.354,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.73}},{"id":"h_inner","candidate":25.316,"reference":109.778,"speedup":4.336,"baselines":{"wpd_neon":26.174,"c_ref_apple_clang_O2":109.778}},{"id":"geomean","candidate":15.197,"reference":122.224,"speedup":8.043,"baselines":{"wpd_neon":14.522,"c_ref_apple_clang_O2":122.224}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6-astra-xhigh":8.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct filter, but several directed mask cases cannot observe the comparison.","opus-5.5-xhigh":"Correct but nearly undocumented, with avoidable copies and no ABI tests","deepseek-v4.1-flash-max":"correct and readable, but documentation is minimal and ABI untested","gpt-6-astra-xhigh":"Correct implementation with ineffective outer-boundary tests and avoidable instructions.","gpt-6.1-sol-max":"Correct and compact; some directed comparison tests cannot expose regressions.","muse-wonder-full-xhigh":"correct and safe, but terse docs, no ABI test, and avoidable h-path moves"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Interior-boundary cases can pass with the wrong mask","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Safety checks miss ABI corruption and unchanged writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner filters convert unused p2/q2 twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header or contract notes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Eight moves after the transpose (src/vp8_lf_neon.S:60–67), plus `mov v20.16b, v19.16b` on the inner path (src/vp8_lf_neon.S:123) where v19 could be used directly.","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"src/vp8_lf_neon.S:1-3","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"tests/selftest.c","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:210-257","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Claimed decisive interior boundaries are unobservable","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Guard tests accept forbidden unchanged writes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Horizontal loading contains avoidable dependencies and copies","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"redundancy","title":"Inner functions unnecessarily convert outer taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Outer interior comparisons have no observable effect","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal loads retain overwritten high halves","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Lane-insert h loads create false dependencies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Eight avoidable moves after h transpose and sequential mbedge","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No file-level documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI preservation test and guard claim overstates coverage","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0456,"lines":12,"significant":263,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":12,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.4,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":3.1,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"B","model":"claude-opus-5.5","effort":"medium","config":"claude-opus-5.5@medium","label":"Opus 5.5 medium","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":8.76,"tokens":{"uncached_input":32,"cache_read":1238192,"cache_write":91453,"output":45001,"reasoning":15033},"cost_usd":1.8794,"cost_breakdown":{"input":0.000128,"cache_read":0.247638,"cache_write":0.731624,"output":0.90002},"cost_usd_as_run":1.8794,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":420,"code":290,"comment":108,"blank":22},{"path":"tests/selftest.c","role":"test","lines":767,"code":671,"comment":64,"blank":32}],"impl_lines":420,"impl_code":290,"impl_comment":108,"test_lines":767,"comment_ratio":0.271,"tests_declared":8,"impl_trailing_comments":60},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.393,"reference":135.165,"speedup":18.284,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":135.165}},{"id":"h_mb","candidate":18.649,"reference":119.036,"speedup":6.383,"baselines":{"wpd_neon":20.739,"c_ref_apple_clang_O2":119.036}},{"id":"v_inner","candidate":12.655,"reference":130.473,"speedup":10.31,"baselines":{"wpd_neon":12.621,"c_ref_apple_clang_O2":130.473}},{"id":"h_inner","candidate":25.099,"reference":110.589,"speedup":4.406,"baselines":{"wpd_neon":26.506,"c_ref_apple_clang_O2":110.589}},{"id":"geomean","candidate":14.466,"reference":123.436,"speedup":8.533,"baselines":{"wpd_neon":14.625,"c_ref_apple_clang_O2":123.436}}]},"scores":{"gpt-6.1-terra-max":9.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":8.0,"gpt-6-astra-xhigh":9.0,"gpt-6.1-sol-max":9.5,"muse-wonder-full-xhigh":9.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, well-documented filter with strong tests and two small omissions.","opus-5.5-xhigh":"Clear, well-factored, accurately documented, strong tests; two nits","deepseek-v4.1-flash-max":"correct, lean v paths, good docs and tests; one redundant instruction","gpt-6-astra-xhigh":"Strong implementation and tests, with a narrow memory-test omission and one redundant instruction.","gpt-6.1-sol-max":"Strong implementation and verification, with one redundant instruction.","muse-wonder-full-xhigh":"correct, best-documented register discipline, near-complete tests; only nits remain"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Bias constant is reloaded unnecessarily","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI wrapper does not check x29","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"A dead re-materialisation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"An overbroad claim","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"src/vp8_lf_neon.S:170","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:358-389","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Write-protection coverage excludes horizontal functions","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"redundancy","title":"Bias constant is unnecessarily reloaded","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Macroblock path reloads an unchanged bias","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Duplicate no-op constant reload in mbedge path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Write-window test silently passes for h functions","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0552,"lines":16,"significant":290,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":15,"flags":3,"audited":"2026-11-02","reviewed":true},"score":8.9,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5}},"defect_points":1.2,"score_cross_vendor":8.875,"check":{"lint":{"diagnostics":0}}},{"letter":"C","model":"gpt-6.1-terra","effort":"high","config":"gpt-6.1-terra@high","label":"GPT‑6.1 Terra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":1.55,"tokens":{"uncached_input":24324,"cache_read":401792,"cache_write":0,"output":5487,"reasoning":224},"cost_usd":0.0718,"cost_breakdown":{"input":0.024324,"cache_read":0.02009,"cache_write":0.0,"output":0.027435},"cost_usd_as_run":0.0718,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":278,"code":263,"comment":6,"blank":9},{"path":"tests/selftest.c","role":"test","lines":301,"code":261,"comment":20,"blank":20}],"impl_lines":278,"impl_code":263,"impl_comment":6,"test_lines":301,"comment_ratio":0.022,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.424,"reference":134.146,"speedup":18.069,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.146}},{"id":"h_mb","candidate":20.413,"reference":117.796,"speedup":5.771,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":117.796}},{"id":"v_inner","candidate":13.943,"reference":130.473,"speedup":9.358,"baselines":{"wpd_neon":12.57,"c_ref_apple_clang_O2":130.473}},{"id":"h_inner","candidate":25.611,"reference":110.992,"speedup":4.334,"baselines":{"wpd_neon":26.489,"c_ref_apple_clang_O2":110.992}},{"id":"geomean","candidate":15.252,"reference":122.993,"speedup":8.064,"baselines":{"wpd_neon":14.574,"c_ref_apple_clang_O2":122.993}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":6.5,"deepseek-v4.1-flash-max":6.5,"gpt-6-astra-xhigh":7.5,"gpt-6.1-sol-max":7.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct vector filter, with sparse invariant documentation and safety-test gaps.","opus-5.5-xhigh":"Correct code, but undocumented, oddly formatted, with thinner tests","deepseek-v4.1-flash-max":"correct, fastest-reading of the low-doc entries, but nearly undocumented and the weakest tests","gpt-6-astra-xhigh":"Correct implementation, but boundary tests miss defects and horizontal paths contain avoidable work.","gpt-6.1-sol-max":"Correct filtering, but the combined boundary sweep often tests only rejection.","muse-wonder-full-xhigh":"the code itself is correct, but tests and readability need real work before merge"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner filters convert unused p2/q2 twice","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Safety checks miss ABI corruption and unchanged writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"docs","title":"Filter register and arithmetic invariants are hard to follow","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The guard test never checks results","status":"valid","defect":"VP8-014","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"Every instruction is indented one space with unaligned operands","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Eight moves after the transpose (src/vp8_lf_neon.S:165–172) and `mov v20.16b, v19.16b` (src/vp8_lf_neon.S:55).","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"src/vp8_lf_neon.S:1-278","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"tests/selftest.c:213-228","status":"valid","defect":"VP8-014","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"tests/selftest.c","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:166-250","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"tests","title":"tests/selftest.c:149-182","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Boundary cases do not isolate comparisons","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Illegal unchanged stores escape detection","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Avoidable horizontal-load dependencies and copies","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"redundancy","title":"Unused inner-edge conversions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Coupled limits mask the intended boundary tests","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal loads retain overwritten vector data","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Guard test never checks the result","status":"valid","defect":"VP8-014","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Boundary sweep does not isolate comparisons despite its claim","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Lane-insert loads and transpose moves","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Minimal documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"style","title":"Unindented assembly throughout","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1141,"lines":30,"significant":263,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":13,"flags":0,"audited":"2026-11-02","reviewed":false},"score":6.9,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":4.1,"score_cross_vendor":6.5,"check":{"lint":{"diagnostics":0}}},{"letter":"D","model":"gpt-6-astra","effort":"high","config":"gpt-6-astra@high","label":"GPT‑6 Astra high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":3.6,"tokens":{"uncached_input":22856,"cache_read":347008,"cache_write":0,"output":9239,"reasoning":1452},"cost_usd":1.0375,"cost_breakdown":{"input":0.22856,"cache_read":0.347008,"cache_write":0.0,"output":0.46195},"cost_usd_as_run":1.0375,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":305,"code":277,"comment":15,"blank":13},{"path":"tests/selftest.c","role":"test","lines":417,"code":363,"comment":30,"blank":24}],"impl_lines":305,"impl_code":277,"impl_comment":15,"test_lines":417,"comment_ratio":0.051,"tests_declared":6,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.403,"reference":133.621,"speedup":18.05,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.621}},{"id":"h_mb","candidate":19.825,"reference":117.135,"speedup":5.908,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.135}},{"id":"v_inner","candidate":13.251,"reference":129.581,"speedup":9.779,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.581}},{"id":"h_inner","candidate":25.004,"reference":109.724,"speedup":4.388,"baselines":{"wpd_neon":26.171,"c_ref_apple_clang_O2":109.724}},{"id":"geomean","candidate":14.85,"reference":122.138,"speedup":8.225,"baselines":{"wpd_neon":14.529,"c_ref_apple_clang_O2":122.138}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.0,"gpt-6-astra-xhigh":8.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct, compact filter with useful directed tests and a safety-coverage gap.","opus-5.5-xhigh":"Correct and compact, but barely documented, verbose stores, and no ABI tests","deepseek-v4.1-flash-max":"correct and compact, but the documentation is one line and ABI is untested","gpt-6-astra-xhigh":"Compact, correct implementation with useful directed tests and minor remaining improvements.","gpt-6.1-sol-max":"Correct, maintainable implementation with useful directed tests.","muse-wonder-full-xhigh":"correct and compactly tested, but minimal documentation and no ABI test"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Safety checks miss ABI corruption and unchanged writes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost no documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Eight register moves after the transpose","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"verbosity","title":"store_h wraps each of 16 half stores in its own `.if !\\inner`","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"src/vp8_lf_neon.S:1","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"tests/selftest.c","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:19-34 and 112-175","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Read guards do not enforce the narrower write window","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Horizontal loading adds unnecessary dependencies and moves","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal loads preserve unused old halves","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Lane-insert h loads create false dependencies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Eight avoidable moves after h transpose","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Almost no file documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI preservation test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0505,"lines":14,"significant":277,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":10,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.6,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.4,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"E","model":"gpt-6.1-sol","effort":"max","config":"gpt-6.1-sol@max","label":"GPT‑6.1 Sol max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":16.36,"tokens":{"uncached_input":38636,"cache_read":1132672,"cache_write":0,"output":40853,"reasoning":18725},"cost_usd":0.5991,"cost_breakdown":{"input":0.077272,"cache_read":0.113267,"cache_write":0.0,"output":0.40853},"cost_usd_as_run":0.5991,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":291,"code":256,"comment":23,"blank":12},{"path":"tests/selftest.c","role":"test","lines":600,"code":538,"comment":37,"blank":25}],"impl_lines":291,"impl_code":256,"impl_comment":23,"test_lines":600,"comment_ratio":0.082,"tests_declared":6,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.33,"reference":133.8,"speedup":18.255,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":133.8}},{"id":"h_mb","candidate":18.87,"reference":117.166,"speedup":6.209,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":117.166}},{"id":"v_inner","candidate":12.641,"reference":129.723,"speedup":10.262,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.723}},{"id":"h_inner","candidate":24.519,"reference":109.816,"speedup":4.479,"baselines":{"wpd_neon":26.174,"c_ref_apple_clang_O2":109.816}},{"id":"geomean","candidate":14.389,"reference":122.246,"speedup":8.496,"baselines":{"wpd_neon":14.523,"c_ref_apple_clang_O2":122.246}}]},"scores":{"gpt-6.1-terra-max":8.5,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":7.5,"gpt-6-astra-xhigh":8.5,"gpt-6.1-sol-max":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct vector filter with useful differential tests; safety checks have gaps.","opus-5.5-xhigh":"Lean, correct core with st3/st4 stores; thin docs, no callee-saved check, and one test that checks a copy","deepseek-v4.1-flash-max":"leanest filter code in the set; documentation and ABI testing are thin","gpt-6-astra-xhigh":"Sound arithmetic and strong directed tests, with minor coverage, performance, and documentation issues.","gpt-6.1-sol-max":"Correct, clear SIMD with broad tests and one avoidable load dependency.","muse-wonder-full-xhigh":"lean, correct, well-tested core; only test gaps around register preservation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Unchanged out-of-window writes escape the guard test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Callee-saved registers are not checked","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No callee-saved register check","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The transpose test checks a copy","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Almost no documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False saturation claim","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"src/vp8_lf_neon.S:1-2","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"tests/selftest.c","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Guard tests miss unchanged forbidden writes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Initial horizontal loads retain unnecessary dependencies","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Benchmark inputs are not identical","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal loads merge unnecessary old high halves","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Lane-insert h loads create false dependencies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No callee-saved register preservation test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Transpose test duplicates the loader inline","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":256,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":18,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.2,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":2.2,"score_cross_vendor":7.833,"check":{"lint":{"diagnostics":0}}},{"letter":"F","model":"claude-opus-5.5","effort":"xhigh","config":"claude-opus-5.5@xhigh","label":"Opus 5.5 xhigh","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":26.87,"tokens":{"uncached_input":80,"cache_read":5695502,"cache_write":192888,"output":130673,"reasoning":77993},"cost_usd":5.296,"cost_breakdown":{"input":0.00032,"cache_read":1.1391,"cache_write":1.543104,"output":2.61346},"cost_usd_as_run":5.296,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":437,"code":268,"comment":146,"blank":23},{"path":"tests/selftest.c","role":"test","lines":1032,"code":854,"comment":129,"blank":49}],"impl_lines":437,"impl_code":268,"impl_comment":146,"test_lines":1032,"comment_ratio":0.353,"tests_declared":7,"impl_trailing_comments":122},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.204,"reference":133.968,"speedup":18.598,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.968}},{"id":"h_mb","candidate":14.281,"reference":117.082,"speedup":8.199,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":117.082}},{"id":"v_inner","candidate":12.6,"reference":129.486,"speedup":10.276,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.486}},{"id":"h_inner","candidate":24.38,"reference":109.809,"speedup":4.504,"baselines":{"wpd_neon":26.188,"c_ref_apple_clang_O2":109.809}},{"id":"geomean","candidate":13.333,"reference":122.205,"speedup":9.166,"baselines":{"wpd_neon":14.529,"c_ref_apple_clang_O2":122.205}}]},"scores":{"gpt-6.1-terra-max":9.5,"opus-5.5-xhigh":9.5,"deepseek-v4.1-flash-max":9.0,"gpt-6-astra-xhigh":10.0,"gpt-6.1-sol-max":10.0,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Strong implementation and coverage, with a minor test-runner false pass.","opus-5.5-xhigh":"Mergeable: lean, accurate docs and the strongest tests","deepseek-v4.1-flash-max":"best of the set: leanest hot paths, accurate docs, strongest tests","gpt-6-astra-xhigh":"Complete implementation with effective arithmetic, memory, and ABI tests.","gpt-6.1-sol-max":"Correct, carefully scheduled SIMD with the strongest ABI and memory-protection tests.","muse-wonder-full-xhigh":"correct, fastest structure on paper, and the most thorough test suite"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Unknown test names report success without running tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Boundary lines are best-effort, not guaranteed on-target","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":268,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":42,"flags":1,"audited":"2026-11-02","reviewed":true},"score":9.5,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":0.3,"score_cross_vendor":9.5,"check":{"lint":{"diagnostics":0}}},{"letter":"G","model":"qwen3.8-flash","effort":"xhigh","config":"qwen3.8-flash@xhigh","label":"Qwen3.8‑Flash xhigh","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":77.54,"tokens":{"uncached_input":738,"cache_read":28767683,"cache_write":359718,"output":289256,"reasoning":0},"cost_usd":0.6683,"cost_breakdown":{"input":0.000111,"cache_read":0.460283,"cache_write":0.071944,"output":0.13595},"cost_usd_as_run":0.6683,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":454,"code":314,"comment":125,"blank":15},{"path":"tests/selftest.c","role":"test","lines":776,"code":675,"comment":66,"blank":35}],"impl_lines":454,"impl_code":314,"impl_comment":125,"test_lines":776,"comment_ratio":0.285,"tests_declared":8,"impl_trailing_comments":82},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.945,"reference":134.104,"speedup":141.894,"baselines":{"wpd_neon":6.573,"c_ref_apple_clang_O2":134.104}},{"id":"h_mb","candidate":0.945,"reference":116.778,"speedup":123.562,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":116.778}},{"id":"v_inner","candidate":0.939,"reference":129.659,"speedup":138.038,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.659}},{"id":"h_inner","candidate":0.939,"reference":109.958,"speedup":117.064,"baselines":{"wpd_neon":26.167,"c_ref_apple_clang_O2":109.958}},{"id":"geomean","candidate":0.942,"reference":122.239,"speedup":129.738,"baselines":{"wpd_neon":14.518,"c_ref_apple_clang_O2":122.239}}]},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Interior-limit sweep can pass with a missing comparison","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Inline call has an incomplete clobber list","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI check rejects permitted SIMD changes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner filters bias unused outer taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"misleading-docs","title":"Transpose explanation is inaccurate","status":"valid","defect":"VP8-012","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Incomplete inline-assembly clobbers","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"ABI check rejects permitted SIMD changes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Guard tests miss forbidden unchanged stores","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Incorrect explanation of TRN","status":"valid","defect":"VP8-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Guard tests miss forbidden unchanged writes","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Interior-boundary fixtures often cannot expose mask errors","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Incorrect explanation of TRN","status":"valid","defect":"VP8-012","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"redundancy","title":"Inner filters perform unused conversions","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"The TRN claim","status":"valid","defect":"VP8-012","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"h macroblock store uses 32 single-element stores","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"dead XORs in the inner path","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"The guard test only proves the read window's ends","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"misleading-docs","title":"False reasoning for the transpose","status":"valid","defect":"VP8-012","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"Inline asm in the register test leaves out registers the call changes","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"misleading-docs","title":"Load-layout comment is wrong","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"Heaviest h path","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Same missing write-window check as A","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Wasted instructions in the filter","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Leftovers in the tests","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"misleading-docs","title":"false TRN/transpose rationale","status":"valid","defect":"VP8-012","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"layout and flim_i sweeps do not cover what they claim","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"adversarial builder does not achieve its claimed `p1-q1`/hev matrix","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"ABI test omits x29/sp and the inline-asm `blr` lacks an x30 clobber","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"h transpose finishes with twelve movs including eight lane inserts","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":123,"flags":18,"audited":"2026-11-02","reviewed":true},"score":7.6,"slop":{"score":2.2,"per_seat":{"gpt-6.1-terra-max":1.5,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":2.0,"opus-5.5-xhigh":4.0,"muse-wonder-full-xhigh":2.0}},"defect_points":6.0,"score_cross_vendor":7.6,"scores":{"gpt-6.1-terra-max":8.0,"gpt-6-astra-xhigh":8.0,"gpt-6.1-sol-max":8.0,"deepseek-v4.1-flash-max":8.0,"opus-5.5-xhigh":6.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct in the checks run, with test-harness weaknesses and minor redundant work.","gpt-6-astra-xhigh":"Correct filters, but ABI tests and transpose documentation need attention","gpt-6.1-sol-max":"Correct filtering in all exercised cases; test blind spots and avoidable inner-filter work.","deepseek-v4.1-flash-max":"correct and very well tested, but the largest h code and a false ISA claim in its justification","opus-5.5-xhigh":"correct, but the transpose comment is false, the h path is the heaviest, and the register test lies to the compiler","muse-wonder-full-xhigh":"correct core, but false transpose rationale and several overstated tests"},"check":{"lint":{"diagnostics":0}}},{"letter":"H","model":"glm-5.3-flash","effort":"max","config":"glm-5.3-flash@max","label":"GLM‑5.3‑Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":55.06,"tokens":{"uncached_input":263791,"cache_read":11276160,"cache_write":0,"output":168285,"reasoning":113274},"cost_usd":0.462,"cost_breakdown":{"input":0.039569,"cache_read":0.338285,"cache_write":0.0,"output":0.084142},"cost_usd_as_run":0.462,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":438,"code":300,"comment":117,"blank":21},{"path":"tests/selftest.c","role":"test","lines":764,"code":617,"comment":98,"blank":49}],"impl_lines":438,"impl_code":300,"impl_comment":117,"test_lines":764,"comment_ratio":0.281,"tests_declared":6,"impl_trailing_comments":145},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.582,"reference":133.212,"speedup":17.571,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.212}},{"id":"h_mb","candidate":20.791,"reference":116.841,"speedup":5.62,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.841}},{"id":"v_inner","candidate":14.038,"reference":129.578,"speedup":9.23,"baselines":{"wpd_neon":12.573,"c_ref_apple_clang_O2":129.578}},{"id":"h_inner","candidate":25.15,"reference":109.762,"speedup":4.364,"baselines":{"wpd_neon":26.171,"c_ref_apple_clang_O2":109.762}},{"id":"geomean","candidate":15.359,"reference":121.977,"speedup":7.942,"baselines":{"wpd_neon":14.533,"c_ref_apple_clang_O2":121.977}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":8.5,"deepseek-v4.1-flash-max":8.0,"gpt-6-astra-xhigh":7.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation, but its claimed HEV and ABI tests have material blind spots.","opus-5.5-xhigh":"Correct, clean and very well documented; small doc errors and an ABI test that doesn't test what it says","deepseek-v4.1-flash-max":"correct, thoroughly documented and tested; only the h store path and one wrong header line hold it back","gpt-6-astra-xhigh":"Correct assembly, but directed and ABI tests contain substantial mistakes.","gpt-6.1-sol-max":"Correct filtering, but several ABI and directed tests are unreliable.","muse-wonder-full-xhigh":"correct with extensive tests, but the ABI header is materially wrong and the h store is the heaviest"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"HEV boundary loop plants no test lanes","status":"valid","defect":"VP8-011","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI argument poisoning changes the C arguments","status":"valid","defect":"VP8-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Register-use description omits scratch registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner filters convert unused p2/q2 twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The \"garbage upper halves\" test is in the wrong bits","status":"valid","defect":"VP8-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"x18 is written without being declared clobbered","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"The register list is wrong","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"False reason for the 16-bit path","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Ternary with identical branches","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"src/vp8_lf_neon.S:26","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:182-243","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"HEV boundary tests contain no edges","status":"valid","defect":"VP8-011","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"ABI probes pass arguments outside the contract","status":"valid","defect":"VP8-009","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Forbidden unchanged writes are not detected","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Register-use description is stale","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"redundancy","title":"Inner filtering converts unused outer taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI wrapper omits caller-saved clobbers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"HEV sweep contains only flat lines","status":"valid","defect":"VP8-011","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Edge sweep constructs the wrong metric","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal loads preserve unnecessary old vector halves","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"Register-use summary is stale","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Header's register claim is false","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Lane-insert h loads create false dependencies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Over-engineered h mbedge store","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Case counts are inflated and lanes mostly idle","status":"valid","defect":"VP8-011","detail":"","location":""}],"quality":{"duplication":{"ratio":0.12,"lines":36,"significant":300,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":75,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.7,"slop":{"score":1.2,"per_seat":{"gpt-6.1-terra-max":2.0,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.5,"muse-wonder-full-xhigh":1.0}},"defect_points":4.3,"score_cross_vendor":7.7,"check":{"lint":{"diagnostics":0}}},{"letter":"I","model":"muse-wonder-full","effort":"xhigh","config":"muse-wonder-full@xhigh","label":"Muse Wonder Full xhigh","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":44.13,"tokens":{"uncached_input":194638,"cache_read":3912320,"cache_write":0,"output":53010,"reasoning":26031},"cost_usd":2.6212,"cost_breakdown":{"input":0.778552,"cache_read":0.782464,"cache_write":0.0,"output":1.0602},"cost_usd_as_run":2.6212,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":398,"code":267,"comment":104,"blank":27},{"path":"tests/selftest.c","role":"test","lines":652,"code":550,"comment":67,"blank":35}],"impl_lines":398,"impl_code":267,"impl_comment":104,"test_lines":652,"comment_ratio":0.28,"tests_declared":5,"impl_trailing_comments":67},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.413,"reference":133.863,"speedup":18.057,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.863}},{"id":"h_mb","candidate":19.447,"reference":116.967,"speedup":6.015,"baselines":{"wpd_neon":20.581,"c_ref_apple_clang_O2":116.967}},{"id":"v_inner","candidate":12.56,"reference":129.639,"speedup":10.322,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.639}},{"id":"h_inner","candidate":24.94,"reference":109.684,"speedup":4.398,"baselines":{"wpd_neon":26.167,"c_ref_apple_clang_O2":109.684}},{"id":"geomean","candidate":14.578,"reference":122.152,"speedup":8.379,"baselines":{"wpd_neon":14.525,"c_ref_apple_clang_O2":122.152}}]},"scores":{"gpt-6.1-terra-max":8.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":7.5,"gpt-6-astra-xhigh":8.0,"gpt-6.1-sol-max":8.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct filter with good directed tests and avoidable inner-edge work.","opus-5.5-xhigh":"Thorough, accurate docs and good directed tests; no ABI test, avoidable copies","deepseek-v4.1-flash-max":"best prose in the set, correct; a few needless instructions and no ABI test","gpt-6-astra-xhigh":"Correct filters with clear arithmetic explanations, but some boundary tests are ineffective.","gpt-6.1-sol-max":"Correct, well-documented SIMD with minor testing and load inefficiencies.","muse-wonder-full-xhigh":"correct with excellent exactness notes, but no ABI test and extra h-path moves"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner filters bias and unbias unused taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI and unchanged-write violations lack probes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Eight `orr` copies after the transpose","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"An overbroad claim","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:94,99,187,192","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:336-367","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"tests/selftest.c","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"style","title":"src/vp8_lf_neon.S:293-300","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Outer interior-limit tests cannot observe filtering","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Unchanged out-of-window stores remain invisible","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"performance","title":"Horizontal transpose performs avoidable copies","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"redundancy","title":"Inner functions perform unused bias conversions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Four interior-boundary probes are inert","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal loads merge unused old data","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Lane-insert h loads create false dependencies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Eight `orr`-as-move instructions after transpose","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI preservation test","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0449,"lines":12,"significant":267,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":35,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.9,"slop":{"score":0.4,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.5,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"J","model":"deepseek-v4.1-flash","effort":"max","config":"deepseek-v4.1-flash@max","label":"DeepSeek V4.1 Flash max","harness":{"provider":"opencode","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":15.21,"tokens":{"uncached_input":51353,"cache_read":9610880,"cache_write":0,"output":139566,"reasoning":103186},"cost_usd":0.1203,"cost_breakdown":{"input":0.007703,"cache_read":0.028833,"cache_write":0.0,"output":0.08374},"cost_usd_as_run":0.1203,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.1203,"fast_mode":false,"tier":"standard","harness_version":"OpenCode 2.2.5"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":419,"code":292,"comment":109,"blank":18},{"path":"tests/selftest.c","role":"test","lines":776,"code":677,"comment":61,"blank":38}],"impl_lines":419,"impl_code":292,"impl_comment":109,"test_lines":776,"comment_ratio":0.272,"tests_declared":7,"impl_trailing_comments":98},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.812,"reference":133.705,"speedup":17.114,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.705}},{"id":"h_mb","candidate":21.453,"reference":116.788,"speedup":5.444,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":116.788}},{"id":"v_inner","candidate":13.828,"reference":129.615,"speedup":9.373,"baselines":{"wpd_neon":12.573,"c_ref_apple_clang_O2":129.615}},{"id":"h_inner","candidate":25.374,"reference":109.643,"speedup":4.321,"baselines":{"wpd_neon":26.201,"c_ref_apple_clang_O2":109.643}},{"id":"geomean","candidate":15.572,"reference":122.052,"speedup":7.838,"baselines":{"wpd_neon":14.541,"c_ref_apple_clang_O2":122.052}}]},"scores":{"gpt-6.1-terra-max":7.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":7.5,"gpt-6-astra-xhigh":7.0,"gpt-6.1-sol-max":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct filter, but the ABI test can falsely pass corrupted registers.","opus-5.5-xhigh":"Correct and clearly documented, but two tests are weaker than they claim","deepseek-v4.1-flash-max":"solid and well tested; a wrong register claim and a store-heavy h path","gpt-6-astra-xhigh":"Correct filtering, undermined by false-negative ABI checks and defective test construction.","gpt-6.1-sol-max":"Correct kernel, but ABI checks and directed-case generation need repair.","muse-wonder-full-xhigh":"correct with strong docs/tests, marred by a masked ABI check and one false header claim"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"ABI comparison accepts register corruption","status":"valid","defect":"VP8-006","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Arithmetic sweep reads uninitialized lanes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Core output description is inaccurate for inner edges","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"redundancy","title":"Inner filters set up unused values","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The d8–d15 check passes if any byte still matches","status":"valid","defect":"VP8-006","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The \"hev on\" half of the q0−p0 sweep never filters anything","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"\"Only x0, x1, x9 and x10 are used\"","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Four moves in transpose stage 3","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"misleading-docs","title":"src/vp8_lf_neon.S:26","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"misleading-docs","title":"src/vp8_lf_neon.S:20-21","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"src/vp8_lf_neon.S:353-384","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"performance","title":"src/vp8_lf_neon.S:42-51","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"ABI checker accepts register corruption","status":"valid","defect":"VP8-006","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Interior-boundary tests can pass without checking the mask","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Final arithmetic batch contains uninitialized lanes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Write-window violations escape guard tests","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"GPR inventory omits an actual temporary","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"nit","category":"dead-code","title":"Inner expansions calculate unused values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"ABI checker accepts corrupted saved registers","status":"valid","defect":"VP8-006","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"medium","category":"tests","title":"Reversed edge cases miss the requested boundary","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Final arithmetic batch reads uninitialized lanes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal loads preserve overwritten vector halves","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"misleading-docs","title":"GPR-use documentation omits x11","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"Header omits a used GPR","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"d8–d15 ABI check cannot detect low-half clobbers","status":"valid","defect":"VP8-006","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Uninitialized lanes in arithmetic test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Lane-insert h loads create false dependencies","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0411,"lines":12,"significant":292,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":72,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.6,"slop":{"score":1.2,"per_seat":{"gpt-6.1-terra-max":2.0,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":1.0}},"defect_points":4.2,"score_cross_vendor":7.625,"check":{"lint":{"diagnostics":0}}},{"letter":"K","model":"gpt-6.1-sol","effort":"high","config":"gpt-6.1-sol@high","label":"GPT‑6.1 Sol high","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":7.19,"tokens":{"uncached_input":35593,"cache_read":646656,"cache_write":0,"output":18840,"reasoning":2402},"cost_usd":0.3243,"cost_breakdown":{"input":0.071186,"cache_read":0.064666,"cache_write":0.0,"output":0.1884},"cost_usd_as_run":0.3243,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":260,"code":236,"comment":12,"blank":12},{"path":"tests/selftest.c","role":"test","lines":451,"code":397,"comment":32,"blank":22}],"impl_lines":260,"impl_code":236,"impl_comment":12,"test_lines":451,"comment_ratio":0.048,"tests_declared":4,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.476,"reference":133.474,"speedup":17.852,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.474}},{"id":"h_mb","candidate":20.172,"reference":116.873,"speedup":5.794,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.873}},{"id":"v_inner","candidate":13.211,"reference":129.652,"speedup":9.814,"baselines":{"wpd_neon":12.55,"c_ref_apple_clang_O2":129.652}},{"id":"h_inner","candidate":25.038,"reference":109.619,"speedup":4.378,"baselines":{"wpd_neon":26.184,"c_ref_apple_clang_O2":109.619}},{"id":"geomean","candidate":14.945,"reference":122.024,"speedup":8.165,"baselines":{"wpd_neon":14.528,"c_ref_apple_clang_O2":122.024}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6-astra-xhigh":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":7.0},"council_summaries":{"gpt-6.1-terra-max":"Correct vector paths, with several test-coverage gaps","gpt-6-astra-xhigh":"Correct implementation with avoidable transpose copies and incomplete regression tests.","gpt-6.1-sol-max":"Correct shared core and useful directed tests, with remaining test gaps.","opus-5.5-xhigh":"Correct code; part of the test suite checks a C model rather than the asm, with some redundancy and minimal docs","deepseek-v4.1-flash-max":"correct and reasonably fast, but undocumented and 8 copies heavier than needed","muse-wonder-full-xhigh":"Correct, but its headline transpose test never runs the entry code"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Unchanged writes to forbidden taps pass the guard test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Outer interior-limit tests are ineffective","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Callee-saved registers are not verified","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The transpose unit test tests separate intrinsics","status":"valid","defect":"VP8-013","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Horizontal transpose has an avoidable copy stage","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Interior-boundary cases can produce identical outcomes","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"ABI failures are not asserted","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Unchanged out-of-window writes pass","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"redundancy","title":"Eight unnecessary transpose-result copies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI preservation lacks assertions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Forbidden writebacks remain invisible","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Nearest-tap maximum is calculated twice","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"The \"transpose unit test\" never calls the deliverable","status":"valid","defect":"VP8-013","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"One-line file documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"dead-code","title":"Dead stores in the edge-threshold test","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Duplicated work in the filter","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"One-line header","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h` loads merge into the old register contents","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Eight register copies after the transpose","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Self-confirming transpose unit test","status":"valid","defect":"VP8-013","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI or read-only write-window test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Eight avoidable register copies after the transpose","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"One-line file header","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Dead assignments in the edge-sweep builder","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Merging lane loads in `load_h`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0678,"lines":16,"significant":236,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":15,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.4,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.0,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":1.0,"muse-wonder-full-xhigh":0.5}},"defect_points":4.2,"score_cross_vendor":7.0,"check":{"lint":{"diagnostics":0}}},{"letter":"L","model":"grok-4.7","effort":"xhigh","config":"grok-4.7@xhigh","label":"Grok 4.7 xhigh","harness":{"provider":"grok","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":61.76,"tokens":{"uncached_input":278423,"cache_read":2780160,"cache_write":0,"output":255700,"reasoning":114840},"cost_usd":3.4811,"cost_breakdown":{"input":0.556846,"cache_read":1.39008,"cache_write":0.0,"output":1.5342},"cost_usd_as_run":3.4811,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Grok Build 1.0.70"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":429,"code":306,"comment":93,"blank":30},{"path":"tests/selftest.c","role":"test","lines":868,"code":799,"comment":31,"blank":38}],"impl_lines":429,"impl_code":306,"impl_comment":93,"test_lines":868,"comment_ratio":0.233,"tests_declared":6,"impl_trailing_comments":87},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.781,"reference":133.716,"speedup":17.185,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.716}},{"id":"h_mb","candidate":21.358,"reference":116.873,"speedup":5.472,"baselines":{"wpd_neon":20.592,"c_ref_apple_clang_O2":116.873}},{"id":"v_inner","candidate":13.835,"reference":129.713,"speedup":9.376,"baselines":{"wpd_neon":12.556,"c_ref_apple_clang_O2":129.713}},{"id":"h_inner","candidate":25.282,"reference":110.151,"speedup":4.357,"baselines":{"wpd_neon":26.181,"c_ref_apple_clang_O2":110.151}},{"id":"geomean","candidate":15.527,"reference":122.241,"speedup":7.873,"baselines":{"wpd_neon":14.53,"c_ref_apple_clang_O2":122.241}}]},"scores":{"gpt-6.1-terra-max":8.0,"gpt-6-astra-xhigh":8.0,"gpt-6.1-sol-max":8.5,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":8.5,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct filter with a brittle ABI test and limited write-boundary coverage","gpt-6-astra-xhigh":"Correct filter with avoidable transpose work and several test-harness defects.","gpt-6.1-sol-max":"Correct core with minor test-contract mistakes and avoidable horizontal-load work","opus-5.5-xhigh":"correct, well-commented assembly; the test suite has an unsound ABI harness and a silently skipped half of the arithmetic sweep","deepseek-v4.1-flash-max":"correct, accurately documented, fully ABI-tested; the h path carries avoidable instructions","muse-wonder-full-xhigh":"Correct and thoroughly commented, with the highest instruction count"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"performance","title":"Low-lane loads retain an unnecessary dependency","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"The ABI test rejects conforming implementations","status":"valid","defect":"VP8-008","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests miss unchanged writes outside individual rows","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"The h macroblock store and transpose are heavier than necessary","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Write-window strictness is untested","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"ABI harness omits caller-saved x9–x15 from its clobbers","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"the hev-off arithmetic sweep never runs","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"nothing checks writes outside the write window","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"performance","title":"8 avoidable `mov`s at the end of the transpose","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"selftest header is stale","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"the ABI check is stricter than AAPCS64","status":"valid","defect":"VP8-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Inline call omits caller-clobbered registers","status":"valid","defect":"VP8-005","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"ABI assertions exceed the contract","status":"valid","defect":"VP8-008","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Guards miss forbidden unchanged writes","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Interior-boundary cases can be ineffective","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"redundancy","title":"Transpose requires eight avoidable copies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI check requires preservation of entire vectors","status":"valid","defect":"VP8-008","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Threshold poisoning changes the actual int arguments","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guards miss unchanged writes outside the write window","status":"valid","defect":"VP8-004","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Horizontal loads preserve dead register contents","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Avoidable copies and address work","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"Thin ABI coverage and no read-only test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Merging lane loads in `load_h`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"verbosity","title":"Padding register remark","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Stale selftest header","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":306,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":30,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.0,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":4.2,"score_cross_vendor":8.0,"check":{"lint":{"diagnostics":0}}},{"letter":"M","model":"muse-wonder-full","effort":"medium","config":"muse-wonder-full@medium","label":"Muse Wonder Full medium","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":39.6,"tokens":{"uncached_input":88279,"cache_read":1848388,"cache_write":0,"output":36837,"reasoning":17544},"cost_usd":1.4595,"cost_breakdown":{"input":0.353116,"cache_read":0.369678,"cache_write":0.0,"output":0.73674},"cost_usd_as_run":1.4595,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":355,"code":271,"comment":73,"blank":11},{"path":"tests/selftest.c","role":"test","lines":547,"code":483,"comment":39,"blank":25}],"impl_lines":355,"impl_code":271,"impl_comment":73,"test_lines":547,"comment_ratio":0.212,"tests_declared":8,"impl_trailing_comments":80},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.466,"reference":133.621,"speedup":17.897,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.621}},{"id":"h_mb","candidate":20.014,"reference":117.019,"speedup":5.847,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":117.019}},{"id":"v_inner","candidate":13.825,"reference":129.673,"speedup":9.38,"baselines":{"wpd_neon":12.566,"c_ref_apple_clang_O2":129.673}},{"id":"h_inner","candidate":25.224,"reference":109.836,"speedup":4.354,"baselines":{"wpd_neon":26.232,"c_ref_apple_clang_O2":109.836}},{"id":"geomean","candidate":15.109,"reference":122.161,"speedup":8.085,"baselines":{"wpd_neon":14.543,"c_ref_apple_clang_O2":122.161}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6-astra-xhigh":9.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":7.5,"muse-wonder-full-xhigh":7.5},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation with useful directed tests and one misleading example","gpt-6-astra-xhigh":"Correct, well-explained implementation with meaningful numerical tests.","gpt-6.1-sol-max":"Correct, clearly documented core with efficient transpose allocation.","opus-5.5-xhigh":"Correct, well commented, good boundary tests, but no ABI test at all","deepseek-v4.1-flash-max":"well documented and fast for its class, but no ABI coverage","muse-wonder-full-xhigh":"Correct with decisive threshold tables, but no ABI test and a serial core"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Saturation example attributes the wrong result to narrowing the difference","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard checks cannot observe unchanged forbidden writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The driver does not check callee-saved registers","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"No explicit ABI regression coverage","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Identical forbidden writes remain invisible","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Preserved registers lack explicit checks","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Write-window enforcement checks values only","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"ABI never exercised","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Unreachable exactness example","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h` loads merge into the old register contents","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI or read-only write-window test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Serial dependence chains","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Merging lane loads in `load_h`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Stale selftest header","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":271,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":19,"flags":3,"audited":"2026-11-02","reviewed":true},"score":8.1,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.0}},"defect_points":2.3,"score_cross_vendor":8.25,"check":{"lint":{"diagnostics":0}}},{"letter":"N","model":"claude-haiku-5.5","effort":"max","config":"claude-haiku-5.5@max","label":"Haiku 5.5 max","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":20.68,"tokens":{"uncached_input":76,"cache_read":9528115,"cache_write":314707,"output":266769,"reasoning":222876},"cost_usd":1.1691,"cost_breakdown":{"input":3.4e-05,"cache_read":0.47062,"cache_write":0.278723,"output":0.419727},"cost_usd_as_run":1.1691,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":431,"code":288,"comment":123,"blank":20},{"path":"tests/selftest.c","role":"test","lines":930,"code":767,"comment":100,"blank":63}],"impl_lines":431,"impl_code":288,"impl_comment":123,"test_lines":930,"comment_ratio":0.299,"tests_declared":7,"impl_trailing_comments":129},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.245,"reference":133.621,"speedup":18.442,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.621}},{"id":"h_mb","candidate":18.891,"reference":116.894,"speedup":6.188,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":116.894}},{"id":"v_inner","candidate":12.607,"reference":129.608,"speedup":10.28,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.608}},{"id":"h_inner","candidate":24.909,"reference":109.626,"speedup":4.401,"baselines":{"wpd_neon":26.194,"c_ref_apple_clang_O2":109.626}},{"id":"geomean","candidate":14.399,"reference":122.054,"speedup":8.477,"baselines":{"wpd_neon":14.535,"c_ref_apple_clang_O2":122.054}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6-astra-xhigh":8.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Correct implementation with strong memory tests and a misleading arithmetic explanation","gpt-6-astra-xhigh":"Correct filtering, but some test assertions and explanatory comments are misleading.","gpt-6.1-sol-max":"Strong implementation and memory tests, with an ABI-test omission and inaccurate comment.","opus-5.5-xhigh":"Tightest code (467 instructions), excellent invariant documentation, thorough tests including a write trace and ABI","deepseek-v4.1-flash-max":"best documentation, fewest instructions, one (smaller) merge-load cost","muse-wonder-full-xhigh":"Correct with excellent tests, but one false proof statement and a serial core"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"misleading-docs","title":"Inner-edge explanation incorrectly says the filter value becomes zero","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Outer interior-limit probes have zero adjustments","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"ABI test omits x29 and does not compare dirty-argument output","status":"valid","defect":"VP8-010","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Dirty-argument calls never check pixel results","status":"valid","defect":"VP8-010","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Outer-boundary fixtures cannot expose mask errors","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Horizontal unchanged writes are unchecked","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"misleading-docs","title":"Incorrect inner-filter invariant","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI wrapper hides x29 corruption","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"misleading-docs","title":"Clearing c does not clear w","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Same unreachable exactness example","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h` loads merge into the old register contents","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"misleading-docs","title":"False zero-w claim for the inner edge","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Serial dependence chains in mask and mbedge passes","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Merging lane loads in `load_h`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Non-interleaved h mbedge stores","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1111,"lines":32,"significant":288,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":43,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.8,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":1.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":1.9,"score_cross_vendor":8.75,"check":{"lint":{"diagnostics":0}}},{"letter":"O","model":"gpt-6.1-sol","effort":"xhigh","config":"gpt-6.1-sol@xhigh","label":"GPT‑6.1 Sol xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":17.5,"tokens":{"uncached_input":40056,"cache_read":1003136,"cache_write":0,"output":33625,"reasoning":16613},"cost_usd":0.5167,"cost_breakdown":{"input":0.080112,"cache_read":0.100314,"cache_write":0.0,"output":0.33625},"cost_usd_as_run":0.5167,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":309,"code":271,"comment":25,"blank":13},{"path":"tests/selftest.c","role":"test","lines":689,"code":625,"comment":36,"blank":28}],"impl_lines":309,"impl_code":271,"impl_comment":25,"test_lines":689,"comment_ratio":0.084,"tests_declared":7,"impl_trailing_comments":8},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.644,"reference":137.937,"speedup":18.044,"baselines":{"wpd_neon":6.815,"c_ref_apple_clang_O2":137.937}},{"id":"h_mb","candidate":20.55,"reference":121.251,"speedup":5.9,"baselines":{"wpd_neon":21.337,"c_ref_apple_clang_O2":121.251}},{"id":"v_inner","candidate":12.98,"reference":134.637,"speedup":10.373,"baselines":{"wpd_neon":12.983,"c_ref_apple_clang_O2":134.637}},{"id":"h_inner","candidate":25.957,"reference":113.62,"speedup":4.377,"baselines":{"wpd_neon":27.11,"c_ref_apple_clang_O2":113.62}},{"id":"geomean","candidate":15.168,"reference":126.473,"speedup":8.338,"baselines":{"wpd_neon":15.041,"c_ref_apple_clang_O2":126.473}}]},"scores":{"gpt-6.1-terra-max":9.0,"gpt-6-astra-xhigh":9.5,"gpt-6.1-sol-max":9.5,"opus-5.5-xhigh":8.0,"deepseek-v4.1-flash-max":8.0,"muse-wonder-full-xhigh":8.5},"council_summaries":{"gpt-6.1-terra-max":"Clear implementation with broad differential and ABI coverage","gpt-6-astra-xhigh":"Clear implementation with effective boundary and ABI tests.","gpt-6.1-sol-max":"Clear correct implementation with strong arithmetic and ABI tests.","opus-5.5-xhigh":"Correct, solid tests including an ABI canary wrapper and guard pages; documentation is adequate but thin","deepseek-v4.1-flash-max":"best-in-class docs and tests, one measurable load-dependency cost","muse-wonder-full-xhigh":"Correct and economical, missing only the read-only write check"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard checks miss unchanged writes outside the write window","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Write-window checks only observe changed values","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Forbidden unchanged writes escape guard tests","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"docs","title":"Thin top-level docs","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h` loads merge into the old register contents","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No read-only write-window test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Merging lane loads in `load_h`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Avoidable copy on the inner path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Minimal file header","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0517,"lines":14,"significant":271,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":20,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.6,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":1.2,"score_cross_vendor":8.167,"check":{"lint":{"diagnostics":0}}},{"letter":"T","model":"gpt-6.1-sol","effort":"medium","config":"gpt-6.1-sol@medium","label":"GPT‑6.1 Sol medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.38,"tokens":{"uncached_input":25612,"cache_read":281600,"cache_write":0,"output":5769,"reasoning":322},"cost_usd":0.1371,"cost_breakdown":{"input":0.051224,"cache_read":0.02816,"cache_write":0.0,"output":0.05769},"cost_usd_as_run":0.1371,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":275,"code":260,"comment":6,"blank":9},{"path":"tests/selftest.c","role":"test","lines":327,"code":285,"comment":22,"blank":20}],"impl_lines":275,"impl_code":260,"impl_comment":6,"test_lines":327,"comment_ratio":0.023,"tests_declared":3,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.403,"reference":133.947,"speedup":18.094,"baselines":{"wpd_neon":6.594,"c_ref_apple_clang_O2":133.947}},{"id":"h_mb","candidate":20.287,"reference":117.376,"speedup":5.786,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":117.376}},{"id":"v_inner","candidate":13.173,"reference":129.53,"speedup":9.833,"baselines":{"wpd_neon":12.556,"c_ref_apple_clang_O2":129.53}},{"id":"h_inner","candidate":25.214,"reference":109.762,"speedup":4.353,"baselines":{"wpd_neon":26.198,"c_ref_apple_clang_O2":109.762}},{"id":"geomean","candidate":14.945,"reference":122.274,"speedup":8.182,"baselines":{"wpd_neon":14.54,"c_ref_apple_clang_O2":122.274}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6-astra-xhigh":7.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":6.5,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct core, but important boundary and ABI cases are weakly tested","gpt-6-astra-xhigh":"Correct filters, but several directed tests cannot detect their intended regressions.","gpt-6.1-sol-max":"Correct filtering, but weaker boundary and contract testing.","opus-5.5-xhigh":"Correct; undocumented, no ABI tests, minor redundancy","deepseek-v4.1-flash-max":"correct, but undocumented, 8 redundant copies and two dead XORs per inner call","muse-wonder-full-xhigh":"Correct core, but masked edge sweep and the sparsest documentation"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Unchanged writes outside the write window go unnoticed","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Directed threshold sweeps include ineffective cases","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Callee-saved registers are not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Inner filters convert unused outer taps","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Threshold fixtures mask the comparisons being tested","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"ABI regressions survive the suite","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Forbidden unchanged stores are invisible","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"redundancy","title":"Avoidable vector work","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"ABI corruption is not asserted","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guard pages miss forbidden unchanged stores","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Upper edge-limit probes are masked by the interior limit","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Inner functions transform unused taps","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file header or contract","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks, and the directed tests repeat one line in all 16 lanes","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"8 post-transpose `mov`s and 4 no-op inner-edge `eor`s","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"One-line header","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h` loads merge into the old register contents","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Eight register copies after the transpose","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No ABI test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"p2/q2 XORed into and out of the signed domain on the inner edge","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Edge-limit sweep is masked for large limits","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI or read-only write-window test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Eight avoidable register copies after the transpose","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"No file header in the `.S`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Wasted work on the inner path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Merging lane loads in `load_h`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1154,"lines":30,"significant":260,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":9,"flags":1,"audited":"2026-11-02","reviewed":true},"score":7.1,"slop":{"score":0.7,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":1.5,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":0.5}},"defect_points":4.0,"score_cross_vendor":6.667,"check":{"lint":{"diagnostics":0}}},{"letter":"U","model":"claude-opus-5.5","effort":"high","config":"claude-opus-5.5@high","label":"Opus 5.5 high","harness":{"provider":"claudeAgent","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":11.74,"tokens":{"uncached_input":44,"cache_read":2080210,"cache_write":112969,"output":66352,"reasoning":33151},"cost_usd":2.647,"cost_breakdown":{"input":0.000176,"cache_read":0.416042,"cache_write":0.903752,"output":1.32704},"cost_usd_as_run":2.647,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"standard","harness_version":"Claude Code 2.1.316"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":424,"code":300,"comment":108,"blank":16},{"path":"tests/selftest.c","role":"test","lines":910,"code":782,"comment":79,"blank":49}],"impl_lines":424,"impl_code":300,"impl_comment":108,"test_lines":910,"comment_ratio":0.265,"tests_declared":9,"impl_trailing_comments":109},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.204,"reference":133.369,"speedup":18.514,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.369}},{"id":"h_mb","candidate":18.565,"reference":116.946,"speedup":6.299,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":116.946}},{"id":"v_inner","candidate":12.587,"reference":129.659,"speedup":10.301,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.659}},{"id":"h_inner","candidate":24.499,"reference":109.897,"speedup":4.486,"baselines":{"wpd_neon":26.221,"c_ref_apple_clang_O2":109.897}},{"id":"geomean","candidate":14.25,"reference":122.098,"speedup":8.568,"baselines":{"wpd_neon":14.543,"c_ref_apple_clang_O2":122.098}}]},"scores":{"gpt-6.1-terra-max":9.0,"gpt-6-astra-xhigh":9.5,"gpt-6.1-sol-max":10.0,"opus-5.5-xhigh":9.0,"deepseek-v4.1-flash-max":9.5,"muse-wonder-full-xhigh":9.5},"council_summaries":{"gpt-6.1-terra-max":"Broad coverage and a well-documented, vectorized implementation","gpt-6-astra-xhigh":"Strong implementation with comprehensive numerical and ABI coverage.","gpt-6.1-sol-max":"Correct implementation with comprehensive contract-focused tests.","opus-5.5-xhigh":"Correct and clean; thorough docs and the most complete tests (ABI, guard pages, read-only rows); nits only","deepseek-v4.1-flash-max":"fastest measured, best documented, best-tested; nothing I would change","muse-wonder-full-xhigh":"Correct, well-scheduled core with the strongest test suite"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"abi","title":"Apple test wrapper uses the reserved x18 register","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"The standalone transpose test exercises a copy","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Horizontal forbidden-write coverage is incomplete","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"misleading-docs","title":"Exactness note uses an example that can never reach the output","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The transpose unit test copies the asm verbatim into inline asm","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"redundancy","title":"Transpose unit test copies the assembly verbatim","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Silent fallback in threshold probe","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":300,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":22,"flags":3,"audited":"2026-11-02","reviewed":true},"score":9.4,"slop":{"score":0.3,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":0.9,"score_cross_vendor":9.5,"check":{"lint":{"diagnostics":0}}},{"letter":"V","model":"gpt-6.1-terra","effort":"max","config":"gpt-6.1-terra@max","label":"GPT‑6.1 Terra max","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":6.3,"tokens":{"uncached_input":90212,"cache_read":953472,"cache_write":0,"output":29926,"reasoning":13577},"cost_usd":0.2875,"cost_breakdown":{"input":0.090212,"cache_read":0.047674,"cache_write":0.0,"output":0.14963},"cost_usd_as_run":0.2875,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":311,"code":279,"comment":21,"blank":11},{"path":"tests/selftest.c","role":"test","lines":501,"code":451,"comment":25,"blank":25}],"impl_lines":311,"impl_code":279,"impl_comment":21,"test_lines":501,"comment_ratio":0.07,"tests_declared":6,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.665,"reference":133.232,"speedup":17.381,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.232}},{"id":"h_mb","candidate":20.592,"reference":116.967,"speedup":5.68,"baselines":{"wpd_neon":20.602,"c_ref_apple_clang_O2":116.967}},{"id":"v_inner","candidate":12.658,"reference":129.968,"speedup":10.268,"baselines":{"wpd_neon":12.617,"c_ref_apple_clang_O2":129.968}},{"id":"h_inner","candidate":25.231,"reference":109.985,"speedup":4.359,"baselines":{"wpd_neon":26.326,"c_ref_apple_clang_O2":109.985}},{"id":"geomean","candidate":14.984,"reference":122.169,"speedup":8.153,"baselines":{"wpd_neon":14.569,"c_ref_apple_clang_O2":122.169}}]},"scores":{"gpt-6.1-terra-max":8.5,"gpt-6-astra-xhigh":9.0,"gpt-6.1-sol-max":9.0,"opus-5.5-xhigh":7.5,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":8.0},"council_summaries":{"gpt-6.1-terra-max":"Correct and readable implementation with useful boundary tests","gpt-6-astra-xhigh":"Correct, clear vector implementation with two safety-test gaps.","gpt-6.1-sol-max":"Correct SIMD core with broad numerical coverage.","opus-5.5-xhigh":"Correct and reasonably commented; tests check garbage upper argument bits but not callee-saved registers","deepseek-v4.1-flash-max":"correct and lean, but lightly documented and its \"dirty argument\" test checks nothing about registers","muse-wonder-full-xhigh":"Correct and clean, with gaps in ABI/write-window testing"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard tests miss unchanged writes to read-only taps","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Dirty-argument calls do not verify preserved registers","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Preserved registers are not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Unchanged forbidden writes escape detection","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Dirty-argument wrapper does not verify preserved registers","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Guard checks permit forbidden unchanged writebacks","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Partial ABI coverage","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"Sparse documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"One-line header","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h` loads merge into the old register contents","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"`call_dirty` exercises garbage upper halves but verifies nothing","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No callee-saved ABI test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"\"Full write windows\" claim overstates the guard test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Merging lane loads in `load_h`","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Avoidable copy on the inner path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"Sparse file documentation","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":279,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":19,"flags":0,"audited":"2026-11-02","reviewed":false},"score":8.0,"slop":{"score":0.1,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.5}},"defect_points":2.5,"score_cross_vendor":7.5,"check":{"lint":{"diagnostics":0}}},{"letter":"W","model":"gpt-6-astra","effort":"medium","config":"gpt-6-astra@medium","label":"GPT‑6 Astra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":2.97,"tokens":{"uncached_input":30536,"cache_read":237312,"cache_write":0,"output":5232,"reasoning":83},"cost_usd":0.8043,"cost_breakdown":{"input":0.30536,"cache_read":0.237312,"cache_write":0.0,"output":0.2616},"cost_usd_as_run":0.8043,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":267,"code":260,"comment":7,"blank":0},{"path":"tests/selftest.c","role":"test","lines":300,"code":262,"comment":19,"blank":19}],"impl_lines":267,"impl_code":260,"impl_comment":7,"test_lines":300,"comment_ratio":0.026,"tests_declared":2,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.413,"reference":136.792,"speedup":18.452,"baselines":{"wpd_neon":6.605,"c_ref_apple_clang_O2":136.792}},{"id":"h_mb","candidate":20.581,"reference":117.114,"speedup":5.69,"baselines":{"wpd_neon":20.644,"c_ref_apple_clang_O2":117.114}},{"id":"v_inner","candidate":13.235,"reference":131.453,"speedup":9.933,"baselines":{"wpd_neon":12.648,"c_ref_apple_clang_O2":131.453}},{"id":"h_inner","candidate":25.228,"reference":109.558,"speedup":4.343,"baselines":{"wpd_neon":26.194,"c_ref_apple_clang_O2":109.558}},{"id":"geomean","candidate":15.024,"reference":123.246,"speedup":8.204,"baselines":{"wpd_neon":14.579,"c_ref_apple_clang_O2":123.246}}]},"scores":{"gpt-6.1-terra-max":7.5,"gpt-6-astra-xhigh":7.5,"gpt-6.1-sol-max":8.0,"opus-5.5-xhigh":7.0,"deepseek-v4.1-flash-max":7.0,"muse-wonder-full-xhigh":6.5},"council_summaries":{"gpt-6.1-terra-max":"Correct core with ineffective boundary cases and limited ABI coverage","gpt-6-astra-xhigh":"Correct implementation, with substantial gaps in directed tests and redundant vector work.","gpt-6.1-sol-max":"Correct filters with weaker verification and some unused inner-mode work.","opus-5.5-xhigh":"Correct and compact, but nearly undocumented, no ABI tests, and some no-op instructions","deepseek-v4.1-flash-max":"correct and compact, but undocumented and carries three small avoidable costs","muse-wonder-full-xhigh":"Correct core, but thin and partly masked tests with sparse docs"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Guard checks accept forbidden unchanged writes","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Threshold probes can be masked or have zero adjustment","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Preserved registers are not checked","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Inner path performs unused work","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"medium","category":"tests","title":"Threshold sweeps contain ineffective cases","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"ABI violations survive testing","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Forbidden unchanged writes are undetected","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"redundancy","title":"Unnecessary vector instructions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Preserved-register corruption goes undetected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Read-window guards do not enforce write permissions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"High edge-limit boundaries are obscured by interior rejection","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"nit","category":"redundancy","title":"Inner mode performs unused work","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"No file or function documentation","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No ABI checks, and failures are silent","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"redundancy","title":"Wasted register traffic","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"docs","title":"No file-level documentation","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"performance","title":"`h` loads merge into the old register contents","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"redundancy","title":"Eight register copies after the transpose","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"No ABI test","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"p2/q2 are XORed into and out of the signed domain on the inner edge","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"redundancy","title":"`store_v` repeats the identical `sub x5, x0, x1, lsl #1` in both arms","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"medium","category":"tests","title":"Edge-limit sweep is masked for large limits","status":"valid","defect":"VP8-003","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No ABI or read-only write-window test","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Eight avoidable register copies after the transpose","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"docs","title":"Minimal documentation","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"dead-code","title":"Dead copy on the inner path","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"redundancy","title":"Unconditional signed-domain conversion of p2/q2","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"performance","title":"Merging lane loads in `load_h`","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.1154,"lines":30,"significant":260,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":8,"flags":0,"audited":"2026-11-02","reviewed":false},"score":7.2,"slop":{"score":0.9,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":2.0,"gpt-6.1-sol-max":0.5,"opus-5.5-xhigh":0.5,"muse-wonder-full-xhigh":1.0}},"defect_points":4.2,"score_cross_vendor":6.833,"check":{"lint":{"diagnostics":0}}},{"letter":"X","model":"gpt-6.1-terra","effort":"medium","config":"gpt-6.1-terra@medium","label":"GPT‑6.1 Terra medium","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":0.24,"tokens":{"uncached_input":23855,"cache_read":124672,"cache_write":0,"output":659,"reasoning":93},"cost_usd":0.0334,"cost_breakdown":{"input":0.023855,"cache_read":0.006234,"cache_write":0.0,"output":0.003295},"cost_usd_as_run":0.0334,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":23,"code":13,"comment":6,"blank":4},{"path":"tests/selftest.c","role":"test","lines":216,"code":181,"comment":18,"blank":17}],"impl_lines":23,"impl_code":13,"impl_comment":6,"test_lines":216,"comment_ratio":0.316,"tests_declared":1,"impl_trailing_comments":0},"hidden":{"passed":3,"total":13,"failed":[]},"perf":{"correct":false,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":0.945,"reference":133.674,"speedup":141.439,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.674}},{"id":"h_mb","candidate":0.945,"reference":117.04,"speedup":123.839,"baselines":{"wpd_neon":20.613,"c_ref_apple_clang_O2":117.04}},{"id":"v_inner","candidate":0.939,"reference":129.869,"speedup":138.262,"baselines":{"wpd_neon":12.563,"c_ref_apple_clang_O2":129.869}},{"id":"h_inner","candidate":0.97,"reference":114.034,"speedup":117.585,"baselines":{"wpd_neon":27.14,"c_ref_apple_clang_O2":114.034}},{"id":"geomean","candidate":0.95,"reference":123.376,"speedup":129.903,"baselines":{"wpd_neon":14.667,"c_ref_apple_clang_O2":123.376}}]},"scores":{"gpt-6.1-terra-max":1.0,"gpt-6-astra-xhigh":0.0,"gpt-6.1-sol-max":1.0,"opus-5.5-xhigh":0.5,"deepseek-v4.1-flash-max":0.5,"muse-wonder-full-xhigh":0.0},"council_summaries":{"gpt-6.1-terra-max":"All four exported functions remain no-op stubs","gpt-6-astra-xhigh":"All four filters remain unimplemented.","gpt-6.1-sol-max":"All four filters remain unimplemented.","opus-5.5-xhigh":"Unmodified stub: nothing implemented, nothing tested","deepseek-v4.1-flash-max":"the untouched scaffold stub; it does not implement the contract at all","muse-wonder-full-xhigh":"Unmodified scaffold stub; no filtering at all"},"findings":[{"seat":"gpt-6.1-terra-max","severity":"high","category":"correctness","title":"Valid edges are never filtered","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"critical","category":"correctness","title":"Return-only entry points","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"high","category":"correctness","title":"Return-only entry points","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"critical","category":"correctness","title":"All four entry points are a bare `ret`","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"medium","category":"tests","title":"selftest.c is the unextended scaffold","status":"valid","defect":"VP8-002","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"critical","category":"correctness","title":"All four entry points are empty","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"high","category":"tests","title":"No test work was done","status":"valid","defect":"VP8-002","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"critical","category":"correctness","title":"All four entry points return without filtering","status":"valid","defect":"VP8-001","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"high","category":"tests","title":"Selftest is the unextended scaffold","status":"valid","defect":"VP8-002","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":13,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":5,"flags":0,"audited":"2026-11-02","reviewed":false},"score":0.6,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":5.2,"score_cross_vendor":0.333,"check":{"lint":{"diagnostics":0}}},{"letter":"Y","model":"muse-wonder-full","effort":"high","config":"muse-wonder-full@high","label":"Muse Wonder Full high","harness":{"provider":"muse","fast_mode":false,"service_tier":"standard"},"usage":{"wall_minutes":50.52,"tokens":{"uncached_input":194516,"cache_read":4183864,"cache_write":0,"output":58528,"reasoning":28698},"cost_usd":2.7854,"cost_breakdown":{"input":0.778064,"cache_read":0.836773,"cache_write":0.0,"output":1.17056},"cost_usd_as_run":2.7854,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":0.0,"fast_mode":false,"tier":"standard","harness_version":"Muse Code 1.4.14-R7309.1"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":400,"code":296,"comment":95,"blank":9},{"path":"tests/selftest.c","role":"test","lines":734,"code":647,"comment":60,"blank":27}],"impl_lines":400,"impl_code":296,"impl_comment":95,"test_lines":734,"comment_ratio":0.243,"tests_declared":7,"impl_trailing_comments":155},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.739,"reference":138.577,"speedup":17.906,"baselines":{"wpd_neon":6.825,"c_ref_apple_clang_O2":138.577}},{"id":"h_mb","candidate":20.749,"reference":121.734,"speedup":5.867,"baselines":{"wpd_neon":21.453,"c_ref_apple_clang_O2":121.734}},{"id":"v_inner","candidate":13.855,"reference":129.462,"speedup":9.344,"baselines":{"wpd_neon":12.56,"c_ref_apple_clang_O2":129.462}},{"id":"h_inner","candidate":25.15,"reference":109.724,"speedup":4.363,"baselines":{"wpd_neon":26.204,"c_ref_apple_clang_O2":109.724}},{"id":"geomean","candidate":15.38,"reference":124.419,"speedup":8.09,"baselines":{"wpd_neon":14.816,"c_ref_apple_clang_O2":124.419}}]},"scores":{"deepseek-v4.1-flash-max":8.5,"gpt-6.1-terra-max":8.0,"gpt-6.1-sol-max":9.0,"gpt-6-astra-xhigh":8.5,"opus-5.5-xhigh":8.5,"muse-wonder-full-xhigh":8.5},"council_summaries":{"deepseek-v4.1-flash-max":"Correct, ABI-clean and the best-motivated arithmetic comments; only a small test gap and some chatty store comments.","gpt-6.1-terra-max":"Good differential coverage, but its ABI test rejects permitted implementations.","gpt-6.1-sol-max":"Correct, well documented, and thoroughly tested; arithmetic can be leaner","gpt-6-astra-xhigh":"Correct core with useful differential tests, but incomplete write-boundary checks and avoidable copies.","opus-5.5-xhigh":"Correct and exactly in bounds, the best-documented filter core and strong tests; its tests miss out-of-window stores of unchanged values, plus a few small redundancies","muse-wonder-full-xhigh":"Correct and well documented, with moderate instruction waste and a masked ABI test"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"low","category":"tests","title":"Write-window rule is not negatively tested","status":"valid","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"verbosity","title":"Per-store row comments restate the store order","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"medium","category":"tests","title":"Dirty-argument test passes invalid thresholds","status":"valid","defect":"VP8-009","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Forbidden unchanged writes escape the guard test","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"nit","category":"redundancy","title":"Horizontal transpose uses avoidable copies","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Widened sum costs six unnecessary instructions","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"Guard tests miss unchanged forbidden stores","status":"valid","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"redundancy","title":"Four avoidable copies per horizontal call","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"Stores of unchanged values outside the write window go undetected","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The \"dirty args\" test sets bits 8–31 of the `int` arguments","status":"valid","defect":"VP8-009","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"tests","title":"The `call_dirty_abi` inline asm breaks GCC's asm operand rules","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"performance","title":"The third transpose stage goes through a temporary","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"Six avoidable instructions per h_inner call plus serial dependencies","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"ABI test is a counted pass on non-Linux platforms","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Guard-page coverage is four cases per function at minimum stride only","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":296,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":40,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.5,"slop":{"score":0.2,"per_seat":{"gpt-6.1-terra-max":0.5,"deepseek-v4.1-flash-max":0.5,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.2,"score_cross_vendor":8.5,"check":{"lint":{"diagnostics":0}}},{"letter":"Z","model":"gpt-6-astra","effort":"xhigh","config":"gpt-6-astra@xhigh","label":"GPT‑6 Astra xhigh","harness":{"provider":"codex","fast_mode":false,"service_tier":"default"},"usage":{"wall_minutes":16.38,"tokens":{"uncached_input":39705,"cache_read":1022464,"cache_write":0,"output":27589,"reasoning":12893},"cost_usd":2.799,"cost_breakdown":{"input":0.39705,"cache_read":1.022464,"cache_write":0.0,"output":1.37945},"cost_usd_as_run":2.799,"cost_basis":"list price incl. cache reads and writes, standard tier","cost_recorded_usd":null,"fast_mode":false,"tier":"default","harness_version":"Codex CLI 0.166.0"},"status":"scored","code":{"files":[{"path":"src/vp8_lf_neon.S","role":"impl","lines":282,"code":238,"comment":28,"blank":16},{"path":"tests/selftest.c","role":"test","lines":561,"code":504,"comment":30,"blank":27}],"impl_lines":282,"impl_code":238,"impl_comment":28,"test_lines":561,"comment_ratio":0.105,"tests_declared":6,"impl_trailing_comments":0},"hidden":{"passed":13,"total":13,"failed":[]},"perf":{"correct":true,"host":"mac-dev (Apple M1, macOS 27.2, Apple clang version 21.0.0 (clang-2100.4.1.1))","proxy":false,"metrics":[{"id":"v_mb","candidate":7.361,"reference":133.936,"speedup":18.195,"baselines":{"wpd_neon":6.584,"c_ref_apple_clang_O2":133.936}},{"id":"h_mb","candidate":19.983,"reference":117.061,"speedup":5.858,"baselines":{"wpd_neon":20.623,"c_ref_apple_clang_O2":117.061}},{"id":"v_inner","candidate":12.648,"reference":129.656,"speedup":10.251,"baselines":{"wpd_neon":12.553,"c_ref_apple_clang_O2":129.656}},{"id":"h_inner","candidate":25.099,"reference":109.697,"speedup":4.371,"baselines":{"wpd_neon":26.215,"c_ref_apple_clang_O2":109.697}},{"id":"geomean","candidate":14.7,"reference":122.201,"speedup":8.313,"baselines":{"wpd_neon":14.539,"c_ref_apple_clang_O2":122.201}}]},"scores":{"deepseek-v4.1-flash-max":8.0,"gpt-6.1-terra-max":8.5,"gpt-6.1-sol-max":8.5,"gpt-6-astra-xhigh":9.0,"opus-5.5-xhigh":8.0,"muse-wonder-full-xhigh":9.0},"council_summaries":{"deepseek-v4.1-flash-max":"Leanest and clearest code of the four, but its own suite never exercises the ABI gate.","gpt-6.1-terra-max":"Correct in the checks run, with gaps in memory-boundary and ABI tests.","gpt-6.1-sol-max":"Correct and clearly structured, with missing ABI regression coverage","gpt-6-astra-xhigh":"Correct, compact core with strong memory tests; ABI regression coverage is missing.","opus-5.5-xhigh":"Correct, the leanest code and strong boundary and write-window tests; the file is thinly documented and its tests never check the ABI rules","muse-wonder-full-xhigh":"Leanest code and strong directed tests, but no ABI test and a thin header"},"findings":[{"seat":"deepseek-v4.1-flash-max","severity":"medium","category":"tests","title":"No ABI test","status":"valid","defect":"VP8-015","detail":"","location":""},{"seat":"deepseek-v4.1-flash-max","severity":"nit","category":"docs","title":"Header omits the memory contract","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"Horizontal write bounds are not protected","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-terra-max","severity":"low","category":"tests","title":"No register-preservation test","status":"valid","defect":"VP8-015","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"performance","title":"Widened sum costs six unnecessary instructions","status":"valid","detail":"","location":""},{"seat":"gpt-6.1-sol-max","severity":"low","category":"tests","title":"Register preservation is untested","status":"valid","defect":"VP8-015","detail":"","location":""},{"seat":"gpt-6-astra-xhigh","severity":"low","category":"tests","title":"No explicit ABI checks","status":"valid","defect":"VP8-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"tests","title":"No test for callee-saved registers, x18, d8–d15 or garbage upper argument bits","status":"valid","defect":"VP8-015","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"low","category":"docs","title":"The filter's register use is undocumented","status":"valid","detail":"","location":""},{"seat":"opus-5.5-xhigh","severity":"nit","category":"style","title":"`test_readonly_rows` hard-codes `f == 2` to mean \"inner v\"","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"tests","title":"No test of callee-saved, x18, d8-d15, or sp preservation","status":"valid","defect":"VP8-015","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"low","category":"performance","title":"h lane loads depend on garbage destination contents","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"tests","title":"Read-only-row test is one fixed case and vacuous for h","status":"valid","detail":"","location":""},{"seat":"muse-wonder-full-xhigh","severity":"nit","category":"docs","title":"File header omits the memory/domain contract","status":"valid","detail":"","location":""}],"quality":{"duplication":{"ratio":0.0,"lines":0,"significant":238,"window":6},"complexity":null,"files":["src/vp8_lf_neon.S"]},"isolation":{"tool_calls":20,"flags":1,"audited":"2026-11-02","reviewed":true},"score":8.4,"slop":{"score":0.0,"per_seat":{"gpt-6.1-terra-max":0.0,"deepseek-v4.1-flash-max":0.0,"gpt-6.1-sol-max":0.0,"opus-5.5-xhigh":0.0,"muse-wonder-full-xhigh":0.0}},"defect_points":2.4,"score_cross_vendor":8.333,"check":{"lint":{"diagnostics":0}}}]}],"briefs":{},"defects":{"bounded-tool":[{"id":"BT-001","title":"Result is dropped when the tool finishes before the caller's :await arrives","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"G"}],"method":"hidden-suite"},{"id":"BT-002","title":"An improper args list returns :unavailable instead of :invalid_argument","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"M"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A"}],"method":"hidden-suite"},{"id":"BT-003","title":"A fast tool is reported :unavailable because init requires its /proc identity","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Y"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A"}],"method":"hidden-suite"},{"id":"BT-004","title":"Group kill is skipped when the BEAM's own process group reads as unknown","severity":"high","category":"robustness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B"}],"method":"hidden-suite"},{"id":"BT-005","title":"Trapping caller receives the port's {:EXIT, port, :normal}","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"E"}],"method":"hidden-suite"},{"id":"BT-006","title":"Slash paths are rewritten before use (~ expanded, .. collapsed)","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"J"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"N"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"V"}],"method":"hidden-suite"},{"id":"BT-007","title":"Runner init crashes when a fast tool's port has closed before its OS pid or identity is read","severity":"high","category":"robustness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"D"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"K"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"M"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"T"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"V"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Z"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"K"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"L"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"N"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"V"}],"method":"probe"},{"id":"BT-008","title":"A result completed before the deadline is lost or turned into :timeout when the caller is delayed","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"H"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"M"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Z"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"W"}],"method":"probe"},{"id":"BT-009","title":"A trailing slash is stripped, so a regular file named with one runs","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"D"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"G"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"H"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"I"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"J"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"K"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"L"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"M"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"N"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"O"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"T"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"U"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"V"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Y"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Z"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1-b2","letter":"Z"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"Z"}],"method":"probe"},{"id":"BT-010","title":"`..` in a slash path is collapsed lexically before resolution","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"Z"}],"method":"probe"},{"id":"BT-011","title":"Executables with non-ASCII paths are reported :unavailable","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F"}],"method":"probe"},{"id":"BT-012","title":"An invalid later duplicate of :timeout or :max_bytes is ignored and the tool runs","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"E"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"G"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"H"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"J"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"K"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"M"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"N"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Y"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Z"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"C"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"K"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"O"}],"method":"probe"},{"id":"BT-013","title":"Valid duplicate options are rejected","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"L"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"D"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"Y"}],"method":"probe"},{"id":"BT-014","title":"The last of duplicated options wins instead of the first","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1-b2","letter":"Z"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1-b2","letter":"Y"}],"method":"probe"},{"id":"BT-015","title":"Valid atom pool names (true, false, nil) are rejected or get the wrong error","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"C"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"T"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"Y"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"U"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1-b2","letter":"Z"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1-b2","letter":"Y"}],"method":"probe"},{"id":"BT-016","title":"An overflow that happened before the deadline is reported as :timeout when processed late","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"K"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"L"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"M"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"N"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"V"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"W"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"X"}],"method":"probe"},{"id":"BT-017","title":"An exit status processed after the deadline is reported as a normal exit","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B"}],"method":"probe"},{"id":"BT-018","title":"The deadline is measured from after validation, not from the call","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"O"}],"method":"probe"},{"id":"BT-019","title":"An empty PATH entry resolves to a bare name, so the relay shell runs its builtin","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"C"}],"method":"probe"},{"id":"BT-020","title":"Stopping the pool leaves running tools alive","severity":"high","category":"robustness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1-b2","letter":"Y"}],"method":"probe"},{"id":"BT-021","title":"A runner killed with :kill permanently consumes its concurrency slot","severity":"high","category":"robustness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A"}],"method":"probe"},{"id":"BT-022","title":"The shared slot table dies with whichever process started the first pool","severity":"high","category":"robustness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A"}],"method":"probe"},{"id":"BT-023","title":"A failed duplicate start_link changes the running pool's limit","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A"}],"method":"probe"},{"id":"BT-024","title":"Every run forks an extra /bin/sh for an access check","severity":"low","category":"performance","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"E"}],"method":"probe"},{"id":"BT-025","title":"Test helper crashes when /proc/<pid>/stat returns :esrch, so the own suite is flaky","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"U"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"V"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"X"}],"method":"probe"},{"id":"BT-026","title":"Own tests are a handful of cases that never check that OS processes die","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"M"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"W"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"J"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"N"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"V"}],"method":"inspection"},{"id":"BT-027","title":"Own tests never exercise the 64 MiB output budget","severity":"low","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"H"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1-b2","letter":"Z"}],"method":"inspection"},{"id":"BT-028","title":"Tests loosen the 1 s bounds and avoid the :kill runner crash","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"A"}],"method":"inspection"},{"id":"BT-029","title":"Descendant-cleanup tests record the parent shell's PID","severity":"low","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"B"}],"method":"inspection"},{"id":"BT-030","title":"A resolution test overwrites and deletes a fixed file in the user's home directory","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"F"}],"method":"inspection"},{"id":"BT-031","title":"Test setup deletes the project's existing tmp directory","severity":"high","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"J"}],"method":"inspection"},{"id":"BT-032","title":"Test teardown can SIGKILL a recycled PID","severity":"high","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"J"}],"method":"inspection"},{"id":"BT-033","title":"Moduledoc still says the file is a stub","severity":"low","category":"misleading-docs","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"B"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"U"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"V"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"X"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1-b2","letter":"Z"}],"method":"inspection"},{"id":"BT-034","title":"Public API has no @doc, @spec or @typedoc","severity":"low","category":"docs","entries":[{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"J"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"L"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"M"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"N"},{"run":"2026-11-02-bounded-tool-gpt-6.1-sol-max-r1","letter":"V"}],"method":"inspection"},{"id":"BT-035","title":"Own tests have no runner-crash or VM-SIGKILL case","severity":"low","category":"tests","entries":[{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-bounded-tool-opus-5.5-xhigh-r1","letter":"Z"}],"method":"inspection"}],"cdef-avx512":[{"id":"CDEF-001","title":"Filter entry point is a bare ret","severity":"critical","category":"correctness","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"E"},{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"L"}],"method":"hidden-suite"},{"id":"CDEF-002","title":"find_dir is a stub that returns 0 and never writes *var","severity":"critical","category":"correctness","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"E"}],"method":"hidden-suite"},{"id":"CDEF-003","title":"Selftest is the unmodified scaffold","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"E"},{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"L"}],"method":"inspection"},{"id":"CDEF-004","title":"find_dir is slower than the scalar C reference","severity":"high","category":"performance","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"J"},{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"L"}],"method":"probe"},{"id":"CDEF-005","title":"find_dir is a hand-unrolled straight-line block of about 1,060 lines","severity":"medium","category":"verbosity","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"G"}],"method":"probe"},{"id":"CDEF-006","title":"Assembly and its tables are effectively undocumented","severity":"low","category":"docs","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"G"},{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"J"},{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"K"},{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"L"}],"method":"inspection"},{"id":"CDEF-007","title":"find_dir guard fixture writes past its 16-page mapping","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"B"}],"method":"inspection"},{"id":"CDEF-008","title":"ABI probe overwrites its own failure mask before returning","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"H"}],"method":"inspection"},{"id":"CDEF-009","title":"Guard tests leave readable slack past the buffers","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"H"}],"method":"probe"},{"id":"CDEF-010","title":"The only added test is one guard case with edges = 0 and one strength set","severity":"low","category":"tests","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"J"}],"method":"inspection"},{"id":"CDEF-011","title":"ABI test passes top and bottom pointers with no leading padding","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"N"}],"method":"probe"},{"id":"CDEF-012","title":"Callee-saved register check is compiled away","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-cdef-avx512-gpt-6.1-sol-max-r1","letter":"V"}],"method":"probe"}],"exr-piz":[{"id":"EXR-001","title":"Decompress is not implemented","severity":"critical","category":"correctness","entries":[{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"N"}],"method":"hidden-suite"},{"id":"EXR-002","title":"No test file","severity":"high","category":"tests","entries":[{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"N"}],"method":"inspection"},{"id":"EXR-003","title":"A 58-bit code that starts at bit offset 7 mod 8 is misread","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"L"}],"method":"hidden-suite"},{"id":"EXR-004","title":"A table with no code for the run-length symbol is accepted","severity":"high","category":"correctness","entries":[{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B"}],"method":"hidden-suite"},{"id":"EXR-005","title":"Per-channel bookkeeping breaks the 4n + 4 MiB allocation bound on many-channel layouts","severity":"high","category":"robustness","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"Z"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1-b2","letter":"W"}],"method":"probe"},{"id":"EXR-006","title":"ErrTooLarge is returned before later channel types are validated","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"L"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"H"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"U"}],"method":"probe"},{"id":"EXR-007","title":"Header fields narrowed to int panic on 32-bit targets","severity":"high","category":"robustness","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B"}],"method":"probe"},{"id":"EXR-008","title":"Cancellation is not checked (or not acted on) while traversing many small channel planes","severity":"medium","category":"robustness","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"D"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"F"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"G"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"J"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"K"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"O"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"U"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"V"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"A"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"F"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"I"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"L"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"O"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"W"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"Z"}],"method":"probe"},{"id":"EXR-009","title":"Wide rows are processed with no cancellation check inside the row","severity":"medium","category":"robustness","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"G"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"J"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"L"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"O"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"Z"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1-b2","letter":"W"}],"method":"probe"},{"id":"EXR-010","title":"Huffman decoding polls by output position, so zero-length runs never reach a check","severity":"medium","category":"robustness","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"B"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"V"}],"method":"probe"},{"id":"EXR-011","title":"Slower than the oracle on realistic chunks","severity":"low","category":"performance","entries":[{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"Z"}],"method":"probe"},{"id":"EXR-012","title":"Package comment is still the scaffold's stub text","severity":"low","category":"misleading-docs","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"T"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"Y"}],"method":"inspection"},{"id":"EXR-013","title":"Exported types, constants and errors are undocumented","severity":"low","category":"docs","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"V"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"C"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"K"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"T"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"U"}],"method":"inspection"},{"id":"EXR-014","title":"Own tests cover only the golden vectors","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"V"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"U"}],"method":"inspection"},{"id":"EXR-015","title":"Tests leave the Huffman validation rules unchecked or discard their results","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"I"},{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"Y"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"C"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"J"}],"method":"inspection"},{"id":"EXR-016","title":"Malformed-chunk tests fail before reaching the checks they name","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1","letter":"L"},{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"Z"}],"method":"inspection"},{"id":"EXR-017","title":"Test table encoder writes a single zero code length as packet 58","severity":"low","category":"tests","entries":[{"run":"2026-11-02-exr-piz-gpt-6.1-sol-max-r1","letter":"B"}],"method":"inspection"},{"id":"EXR-018","title":"Cancellation timing test starts its clock only when the decoder observes the cancel","severity":"low","category":"tests","entries":[{"run":"2026-11-02-exr-piz-opus-5.5-xhigh-r1-b2","letter":"W"}],"method":"inspection"}],"range-coder":[{"id":"RC-001","title":"encode_icdf mis-encodes a symbol after zero-frequency leading symbols","severity":"high","category":"correctness","entries":[],"method":"hidden-suite"},{"id":"RC-002","title":"decode_icdf panics on malformed tables (unchecked multiply)","severity":"high","category":"robustness","entries":[],"method":"hidden-suite"},{"id":"RC-003","title":"Pending carry counter panics on 32-bit targets","severity":"medium","category":"robustness","entries":[],"method":"probe"},{"id":"RC-004","title":"Bit-total and carry counters panic on overflow on every target","severity":"medium","category":"robustness","entries":[],"method":"probe"},{"id":"RC-005","title":"Encoder back counter overflows on 32-bit targets after 2^32 dropped raw bytes","severity":"medium","category":"robustness","entries":[],"method":"probe"},{"id":"RC-006","title":"Test suite covers a fraction of the required categories","severity":"medium","category":"tests","entries":[],"method":"inspection"},{"id":"RC-007","title":"Public documentation far below the required level","severity":"medium","category":"docs","entries":[],"method":"inspection"},{"id":"RC-008","title":"decode_icdf documentation claims a scaling check the code does not make","severity":"low","category":"misleading-docs","entries":[],"method":"probe"},{"id":"RC-009","title":"Tests let contract violations through (sticky-error order, ICDF bound check)","severity":"low","category":"tests","entries":[],"method":"probe"},{"id":"RC-010","title":"Encoder writes the pending 0xFF run on every emitted byte, even when there is none","severity":"low","category":"performance","entries":[],"method":"probe"},{"id":"RC-011","title":"No-op defensive clamp in decode_icdf, with a comment implying it is needed","severity":"low","category":"redundancy","entries":[],"method":"inspection"},{"id":"RC-012","title":"Encoder and decoder are not implemented","severity":"critical","category":"correctness","entries":[{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"I"}],"method":"hidden-suite"},{"id":"RC-013","title":"No tests","severity":"high","category":"tests","entries":[{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"I"}],"method":"inspection"},{"id":"RC-014","title":"Deferred-carry counter panics on overflow","severity":"medium","category":"robustness","entries":[{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"X"},{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"V"}],"method":"probe"},{"id":"RC-015","title":"Deferred-carry counter saturates or is wider than libopus's wrapping u32, so finish() differs after 2^32 carry bytes","severity":"medium","category":"correctness","entries":[{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"V"},{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"W"},{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1-b2","letter":"W"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"K"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"V"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"W"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"X"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1-b2","letter":"X"}],"method":"probe"},{"id":"RC-016","title":"Public methods do not document their preconditions or what a rejected call does","severity":"low","category":"docs","entries":[{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"W"},{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1-b2","letter":"W"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"F"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"I"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"K"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"U"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"W"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"X"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1-b2","letter":"X"}],"method":"inspection"},{"id":"RC-017","title":"decode_bit_logp documented with the probability of the wrong bit value","severity":"low","category":"misleading-docs","entries":[{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"H"}],"method":"inspection"},{"id":"RC-018","title":"BITRES documented as the unit of logp","severity":"low","category":"misleading-docs","entries":[{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"H"}],"method":"inspection"},{"id":"RC-019","title":"Tests let pending-decode rule violations through","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"W"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"E"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"H"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"I"},{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"X"}],"method":"probe"},{"id":"RC-020","title":"Test suite is a single golden test","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"K"}],"method":"inspection"},{"id":"RC-021","title":"Determinism check in the garbage-decode test asserts nothing","severity":"low","category":"tests","entries":[{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"H"}],"method":"inspection"},{"id":"RC-022","title":"Module doc calls the delayed-carry coder carry-less","severity":"low","category":"misleading-docs","entries":[{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"T"}],"method":"inspection"},{"id":"RC-023","title":"Comments attribute nonexistent od_ec_*/OD_EC_* names to libopus","severity":"low","category":"misleading-docs","entries":[{"run":"2026-11-02-range-coder-opus-5.5-xhigh-r1","letter":"V"}],"method":"inspection"},{"id":"RC-024","title":"finish() walks the whole buffer to zero the gap","severity":"low","category":"performance","entries":[{"run":"2026-11-02-range-coder-gpt-6.1-sol-max-r1","letter":"Z"}],"method":"probe"}],"vp8lf-neon":[{"id":"VP8-001","title":"All four entry points are a bare ret","severity":"critical","category":"correctness","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"X"},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"N"},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"Z"}],"method":"hidden-suite"},{"id":"VP8-002","title":"Selftest is the unextended scaffold","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"X"},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"N"}],"method":"inspection"},{"id":"VP8-003","title":"Threshold boundary tests cannot observe the comparison they target","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"A"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"C"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"G"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"I"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"T"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"W"}],"method":"inspection"},{"id":"VP8-004","title":"Guard tests cannot detect a forbidden store of an unchanged value","severity":"low","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"G"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1-b2","letter":"X"}],"method":"probe"},{"id":"VP8-005","title":"ABI test harness relies on registers a conforming callee may change","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"G"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1-b2","letter":"X"}],"method":"probe"},{"id":"VP8-006","title":"ABI checker accepts a corrupted d8","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"J"}],"method":"probe"},{"id":"VP8-007","title":"ABI check does not verify x29 or sp","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1-b2","letter":"X"}],"method":"probe"},{"id":"VP8-008","title":"ABI test rejects code that changes the upper halves of v8-v15","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"L"}],"method":"probe"},{"id":"VP8-009","title":"Dirty-argument test corrupts the low 32 bits of the int thresholds","severity":"low","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"H"},{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Y"}],"method":"inspection"},{"id":"VP8-010","title":"Dirty-argument calls never compare the filtered pixels","severity":"low","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"N"}],"method":"probe"},{"id":"VP8-011","title":"HEV boundary sub-test plants no test lanes","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"H"}],"method":"inspection"},{"id":"VP8-012","title":"Transpose comment says TRN1/TRN2 cannot build a transpose","severity":"low","category":"misleading-docs","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"G"}],"method":"inspection"},{"id":"VP8-013","title":"Transpose unit test checks a C re-implementation, not the deliverable","severity":"low","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"K"}],"method":"inspection"},{"id":"VP8-014","title":"Guard test never compares the output","severity":"low","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"C"}],"method":"inspection"},{"id":"VP8-015","title":"No callee-saved register (ABI) test","severity":"low","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-opus-5.5-xhigh-r1","letter":"Z"}],"method":"inspection"},{"id":"VP8-016","title":"Guard test fails on page sizes above 4096, so the Apple test target cannot pass","severity":"high","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"A"}],"method":"inspection"},{"id":"VP8-017","title":"Guard tests call the filter with strides outside the contract","severity":"low","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"C"}],"method":"inspection"},{"id":"VP8-018","title":"Own tests cannot detect a one-byte over-read past the read window","severity":"low","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"D"},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"I"}],"method":"probe"},{"id":"VP8-019","title":"Read-only guard rejects allowed write-backs of unchanged values","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"D"}],"method":"probe"},{"id":"VP8-020","title":"test_boundaries writes past both static buffers at stride 4096","severity":"high","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"I"}],"method":"inspection"},{"id":"VP8-021","title":"Guard test protects and clears memory beyond its 12-page mapping","severity":"high","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1-b2","letter":"W"}],"method":"probe"},{"id":"VP8-022","title":"Directed arithmetic corpus never passes the mask","severity":"medium","category":"tests","entries":[{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"G"}],"method":"inspection"},{"id":"VP8-023","title":"Assembly is effectively undocumented","severity":"medium","category":"docs","entries":[{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"H"},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"I"},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"J"},{"run":"2026-11-02-vp8lf-neon-gpt-6.1-sol-max-r1","letter":"O"}],"method":"inspection"}]}}