MinimalPi harness tweaks — inventory + usefulness (2026-07-21)
A catalogue of every default-on modification the MinimalPi harness applies on
top of stock pi, each rated by how much the logs actually justify it.
Evidence is firing frequency in the just-finished full suite
runs/suite__qwen3.6-35b-a3b__20260720-025830 (89 trials, MinimalPi, 35b, K=1,
AGENT_TIMEOUT_MULT=2.0) plus the documented A/B results in CLAUDE.md /
AGENT_TODOS.md. Ratings: Load-bearing (trials die without it) · High ·
Medium · Low · Neutral/off.
Headline for that run: 40/89 = 44.9%, up from 35/89 (20260718), +5 net with
14 gained / 9 lost (large K=1 churn — trust firing frequencies over per-task
flips).
Tier 1 — Foundational infrastructure (load-bearing; prevent whole death classes)
- pi 0.80.2 pin + maxTokens 65536 + removal of the hidden 32000 output clamp.
Without it every large write truncated at exactly 32000 output tokens. Base
everything else sits on.
- Print-mode stream-tick patch (OOM fix). Before it, big streaming writes
killed the process with exit 134 (V8 heap). This run: 0 exit-134/OOM events
across 89 trials.
- Post-session exit patch (main.js process.exit). Before it, a leaked handle
kept finished sessions alive until Harbor's cap fired AgentTimeoutError on
already-solved trials. This run: no solved-but-ERR teardown hangs.
- GitHub-free install (node from nodejs.org). Removed overnight rate-limit
hard-fails. Reliability, not score.
- Context window 196608 / compact-at 180000. Roll-up shows 0 dead trials at
the 262k hard-400 limit even though 22/89 trials exceeded the old declared
131k window — so this directly saved ~22 trials from context-death.
- Reasoning control + server-side reasoning-budget 8000. Documented: regex-log
6/6 at budget 8000 vs 3/5 at 4000. Keeps thinking from eating the whole budget.
Tier 2 — Active guards that fire constantly and clearly earn their place
- Bash default timeout (90s) — High. Fired 20 times this run (20 trials got a
"Command timed out" recovery instead of silently hanging to the cap). Highest-
frequency behavioral guard; clearest day-to-day win.
- Context overflow pruner (prune, never compact) — Load-bearing/High. Fired in
27 trials, 209 prune events. Replaced the
ctx.compact() that once killed
5/5 trials; the reason long-context trials survive.
- Loop guard + escalation — High. Fired in 16 trials (116 blocks); the
escalation hard-stopped 8 trials (with 8 halfway nudges) that would
otherwise have spun to the 2-hour cap. Big wall-clock + interpretability win.
Tier 3 — Rare but high-value when they fire
- Write guard (truncated-write block) — Medium/High. Only 3 trials hit a
length-stop this run and the guard fired in 1 — but when it fires it prevents
silently corrupting a file. Low frequency, high stakes.
- Write-guard storm hard-stop (added 2026-07-20, WRITE_GUARD_STOP_AFTER=8) —
Medium (insurance). Didn't fire this run (the
wg:2547 storm didn't recur),
but last run that exact shape burned a whole 30-min cap. Correct backstop,
unmeasured upside.
- Dash-prompt prefix (added 2026-07-20) — High, deterministic. Turned
pytorch-model-recovery from never ran (instant "Unknown option: -") into a
PASS. Guaranteed permanent +1; most cost-effective line changed.
Tier 4 — Moderate / hard to isolate
- Runaway + empty-final recovery — Medium. Fired in 7 trials (2 oversized-
text, 3 empty-after-runaway-thinking, 3 empty-final). Nudges dead-ended turns
back to productive action; several are near-misses, so real but noisy value.
- Runaway-text trim (added 2026-07-20, TRIM_RUNAWAY_TEXT=on) — Medium,
promising but unmeasured. Its target
torch-tensor-parallelism flipped
FAIL→PASS, but there's no per-trim marker yet (logged follow-up), so trim
engagement isn't counted. Needs the marker + K≥3 to confirm.
- Preamble RULE 2 "verify before declaring done" — Medium. Documented FAIL→PASS
on openssl-selfsigned-cert (3/3). This run: 25 of 30 self-declared-done fails
had actually run a check, so models do verify — but 25 near-misses remain, so
it lifts the floor without closing the gap. Best lever on the largest failure
class.
Tier 5 — Low / neutral (kept, but logs don't show a score gain)
- Qwen-native tool surface (read_file/write_file/edit/run_shell_command +
--no-builtin-tools) — Neutral on score. Documented A/B: score unchanged
(35b 5/9 = plain baseline) with 0 schema errors. Clean, research-backed default
that costs nothing, but on these tasks it removed friction that wasn't there.
Kept as the standard surface, not a score lever.
- Preamble RULE 1 "chunk writes ≤300 lines" — Low/ignored. Documented zero
effect — the model writes 2–8× the limit anyway. The mechanical write guard,
not this rule, is what protects. Effectively dead text.
Tier 6 — Correctly OFF by default (the logs are why)
- Context strip (
strip_thinking / keep_tool_results) — off. Measured
harmful on the MoE (35b fast 7/9→3/9, amnesia + fabricated completions). The
token win wasn't worth the score loss.
subagent_nudges / force_review — on but inert in MinimalPi. Both gate on
subagents=on; in the MinimalPi suite they fired 0 times. Only matter for
SubagentsPi.
- Fast-fail per-task timeouts — off for suite. Not used in the full-suite
config (they'd distort wall-clock comparisons); a
make fast iteration tool.
Net read
Two buckets do the work. Infrastructure fixes (pi version/maxTokens, OOM
patch, exit patch, context window, pruner) make trials finishable at all — they
never show as flips but without them large swaths of the suite die. The active
guards that pay off most visibly are the bash timeout (20 firings), the
loop-guard hard-stop (8 trials rescued from the cap), and the two new
deterministic wins (dash-prompt = +1 guaranteed, runaway-text trim = 1 target
flip). The weakest-justified defaults are qwen-tools (measured no score
change) and preamble RULE 1 (ignored by the model) — harmless but not carrying
weight. Everything measured harmful is already off.
Caveat: K=1, with 14 gained / 9 lost churn, so per-task flips are noisy — the
firing frequencies here are the more reliable signal than any single verdict.