← task board · runs · Qwen 3.6 27B · pi + subagents
FAIL in fast__qwen3.6-27b__20260708-153432 (bkHKdLi, ~28 min wall incl. a
7m28s verifier). 5 of 6 verifier tests passed — output byte-identical to
golden, no DB modifications, single small query — only
test_compare_golden_vs_solution_runtime failed: solution median 1.155s
vs golden 0.666s, bar is <= 1.05 x golden (0.699s). The agent's query
is ~145× faster than the original it was asked to optimize (2m45s → ~1.2s
in-container) and still loses.
Both queries are the same complexity class (no quadratic blowup like the 35b trial qTiRRpV's 3-way-join disaster); the gap is a filter-ordering constant factor:
word_stats does ONE
aggregate pass over the 3-way join with HAVING (≥2 synsets, ≥2 domains,
≥2 senses), cutting to the ~17.8k qualifying words; the window/ranking CTE
then joins senses only for those words (a 2-way join, no synsets
needed for ranking).base materializes the FULL
words×senses×synsets join once; ranked runs DENSE_RANK() OVER
(PARTITION BY wordid ...) over EVERY word in the dictionary (ranking work
for rows that are then discarded); aggregated scans the full base a
second time for the aggregates; the HAVING filter only applies at the
end.The instruction says "make the query as efficient as possible" — the 1.05×
bar and /tests/golden.sql are invisible to the agent. It measured its
solution at 0.573s vs the original's 2m45s (~290×), verified the output
IDENTICAL with a real diff, and reasonably declared victory. Same failure
class as both 35b analyses ("optimized against the original, never against
golden") but the nearest miss yet: 1.73× vs plain-pi 35b's 1.36× and the
35b subagents trial's ~140×. NB the 07-06 27b SubagentsPi run PASSED this
task at 0.99× golden — with the bar invisible, whether the model lands on
the filter-first shape is effectively a coin flip.
SUBAGENT_TIMEOUT cap at exactly 600000ms — first live firing; the root
read the partial artifact and finished cleanly. Pre-cap this was the
17–62-min looping-child shape.Model optimization judgment against an unobservable bar — not a harness failure and not fabrication (verification was real and passed; the scored axis has a hidden reference). For scoring intuition, treat this task's near-miss FAILs as the norm: the pass depends on landing the filter-first query shape. A general lever (also in the 35b analyses): on tasks whose single scored axis is runtime, "fast enough" has no observable definition — trying 2–3 structurally different query shapes and keeping the fastest would turn the coin flip into a search.