← older: smoke__qwen3.6-35b-a3b__20260704-131520all runs · task boardnewer: smoke__qwen3.6-35b-a3b__20260704-140750

smoke__qwen3.6-35b-a3b__20260704-132733

Guard-validation K=5 — 0/5: the guard fired in every trial and every firing KILLED the trial; ctx.compact() is fatal in pi's print mode

What was run

regex-log × 5, COMPACT_AT_TOKENS=15000 — a threshold every regex-log draw crosses mid-run, to finally exercise the turn_end compaction guard.

Result — the guard works exactly as written, and that is the bug

All 5 trials: guard armed, fired once (minimal-pi: context 16000–19401 > 15000 tokens, forcing compaction), then the trial died on the spot — 6–7 turns, ~1.5 min, last stopReason toolUse (mid-task, tools still pending), transcript ends at compaction_start (reason: "manual") with no compaction_end, no error, no further events. pi exited; Harbor ran the verifier against a half-finished workspace. 0/5 on a task that is ~95% at K≥5.

Root cause (from pi 0.73.1 source, @mariozechner/pi-coding-agent)

ctx.compact()session.compact(), whose first two lines are:

this._disconnectFromAgent();
await this.abort();          // aborts the running agent loop

It is built for the interactive /compact command (abort any streaming, then summarize). Harbor runs pi --print --mode json, where the process lifetime is await session.prompt(...): aborting the loop resolves prompt(), print mode returns, and the process exits — orphaning the compaction that was fired async (void (async …), nobody awaits it). Hence compaction_start as the final event, every time.

Collateral findings (they correct two standing beliefs)

  1. pi's auto-compaction DOES anchor to the declared contextWindow (agent-session.js:1383: this.model?.contextWindow ?? 0), not the server's real -c. The comment in minimal_pi.py/harness.env claiming otherwise is wrong. The real reason it never fired in the 20260703 suite: _checkCompaction runs only at agent_end or before the next user prompt — i.e. never mid-loop in one-shot print mode. No declared-window value can make built-in threshold compaction work mid-run.
  2. The suite's four hard-400 deaths actually DID trigger pi's built-in overflow recovery — 3 of 4 transcripts show compaction_start (reason: "overflow") as their last event. It dies the same way: at agent_end the print-mode process exits before the async compact+retry completes (and at 262k the summarization request itself would overflow llama.cpp anyway). So H6's "catch the 400 → compact/retry" cannot be delegated to pi in print mode either.
  3. pi 0.73.1 is the latest release — no upgrade available.

The viable mechanism (found in the same source)

pi.on("context", handler)"Fired before each LLM call. Can modify messages." It is awaited inside the agent loop via the agent's transformContext, receives a structuredClone of the messages, returns {messages} to replace them, cannot abort anything, and handler exceptions are swallowed by the runner. Redesign the guard as a context-pruning transform: when the running total exceeds the ceiling, drop/stub the oldest tool results (keep system prompt + task instruction + recent turns + a short "[elided N old tool results]" marker) and return the trimmed array. No summarize call, no abort, and requests physically never reach the 262k cap — which also subsumes the H6 hard-400 fix.

Action items

  1. Until the redesign lands, COMPACT_AT_TOKENS=200000 is ACTIVELY HARMFUL — any suite trial crossing 200k gets executed by its own guard. Set the default to 0 (guard off) or land the context-hook version first.
  2. Rewrite the guard on the context hook; re-run this exact K=5 experiment (threshold 15000) — success = 5 firings, ≥4/5 pass, peaks capped.
  3. Fix the wrong "anchors to server context" comments in minimal_pi.py / harness.env / CLAUDE.md when touching the code.

Run details

modelllama-local/qwen3.6-35b-a3bagentharnesses.minimal_pi:MinimalPithinkingonreasoning budgetdirect (server/none)* — see journal for the authoritative mechanismmaxTokens / contextWindow32768 / 229376agent timeout ×2.0trials5 of 5 — 0 pass · 5 failmean reward0.00tokens (job total)263,348 in / 67,793 outstarted / finished2026-07-04T13:27 / 2026-07-04T13:39wall clock11m58s

Tasks

regex-log — 0/5 passed

#resulttotalagentin/out tokflags
1FAIL2m34s1m39s67893/15251
🔍 view
2FAIL2m25s1m29s43960/14399
🔍 view
3FAIL2m31s1m34s61461/14795
🔍 view
4FAIL2m11s1m15s47933/11651
🔍 view
5FAIL2m16s1m20s42101/11697
🔍 view