← task board · runs · Qwen 3.6 27B · pi + subagents

query-optimize — correct and 145× faster than the original, but 1.73× slower than the invisible golden bar

FAIL in fast__qwen3.6-27b__20260708-153432 (bkHKdLi, ~28 min wall incl. a 7m28s verifier). 5 of 6 verifier tests passed — output byte-identical to golden, no DB modifications, single small query — only test_compare_golden_vs_solution_runtime failed: solution median 1.155s vs golden 0.666s, bar is <= 1.05 x golden (0.699s). The agent's query is ~145× faster than the original it was asked to optimize (2m45s → ~1.2s in-container) and still loses.

Why the query is 1.73× slower than golden

Both queries are the same complexity class (no quadratic blowup like the 35b trial qTiRRpV's 3-way-join disaster); the gap is a filter-ordering constant factor:

Why the agent stopped there

The instruction says "make the query as efficient as possible" — the 1.05× bar and /tests/golden.sql are invisible to the agent. It measured its solution at 0.573s vs the original's 2m45s (~290×), verified the output IDENTICAL with a real diff, and reasonably declared victory. Same failure class as both 35b analyses ("optimized against the original, never against golden") but the nearest miss yet: 1.73× vs plain-pi 35b's 1.36× and the 35b subagents trial's ~140×. NB the 07-06 27b SubagentsPi run PASSED this task at 0.99× golden — with the bar invisible, whether the model lands on the filter-first shape is effectively a coin flip.

Harness mechanisms in this trial (all healthy — first live outing of the 2026-07-08 evening fixes)

Verdict attribution

Model optimization judgment against an unobservable bar — not a harness failure and not fabrication (verification was real and passed; the scored axis has a hidden reference). For scoring intuition, treat this task's near-miss FAILs as the norm: the pass depends on landing the filter-first query shape. A general lever (also in the 35b analyses): on tasks whose single scored axis is runtime, "fast enough" has no observable definition — trying 2–3 structurally different query shapes and keeping the fastest would turn the coin flip into a search.