← task board · runs · Qwen 3.6 35B A3B · pi

git-multibranch — agent's own SSH key breaks the verifier's password-expect flow

Status: FAIL 3/3 on 35b MoE across ALL THREE harnesses in the 2026-07-07 fast2 sweep (fast2__…111415 6pmCR3B MinimalPi @1m43s, fast2__…141659 P5Jparp PlanPi @1m39s, fast2__…145458 x5cwvyt SubagentsPi @3m29s), while 27b passed 3/3 on the same day (fast2__…102314, …121846, …131010). No fast-cap involvement — every 35b trial finished fast with a clean stop and self-declared success after heavy self-verification (V=8–19). Deterministic, model-specific, one root cause.

The failure chain

The verifier (test_outputs.py::test_multi_branch_https_deploy) drives an expect script: spawn git clone git@localhost:/git/project, wait for "password:", send password, then push main and dev branches the same way, then curl the two HTTPS endpoints. Verifier output in all three 35b trials:

Receiving objects: 100% (16/16), done.
warning: remote HEAD refers to nonexistent ref, unable to checkout
…
expect: spawn id exp3 not open
    while executing
"expect eof"
    (file "./git_push.exp" line 9)

The clone succeeded without ever prompting for a password. The expect block therefore matched nothing, hit EOF inside the block, and the very next expect eof aborted the whole script — the pushes never ran, the post-receive hook never fired, the curl checks got stale/empty content → TEST FAILED.

Why no password prompt: every 35b trial generated a root keypair and installed it for its own testing —

ssh-keygen -t ed25519 -f /root/.ssh/id_ed25519 -N ''
cat /root/.ssh/id_ed25519.pub >> /home/git/.ssh/authorized_keys

The verifier runs as root in the same container, and ssh tries pubkey auth before password auth, silently picking up the default-named /root/.ssh/id_ed25519. Key auth succeeds → no password: prompt → the expect flow (which the instruction's "will be tested by" section describes, password included) is dead on arrival. grep across the six trials: 35b ran ssh-keygen in 3/3 trials (4–7 times each); 27b ran it zero times in 3/3 — it tested with sshpass only (28–55 uses), which leaves no key behind, so the verifier's prompt appears and everything passes.

Why the agent's own verification couldn't catch it

The 35b trials verified a lot (V up to 19) — pushes, hook firing, both curl endpoints — but always through their own key. Even "check that password auth works" is inconclusive with the key installed: ssh offers the pubkey first and never falls back to the password, and sshpass doesn't error when the password goes unused. The only ways to observe the bug are ssh -o PubkeyAuthentication=no … or deleting the key — i.e. simulating a client that doesn't own the credential you created.

Side detail, same theme: the cloned repo warned remote HEAD refers to nonexistent ref (bare repo HEAD left at master while only main/dev were ever pushed) — harmless for the verifier flow but the same "environment state I created ≠ environment state the tester expects" class.

Classification + levers