RULE-RELAXED recoverability axis — NOT a baseline/leaderboard number. The medic is deliberately told the verifier failed and shown the exact failing-check output; this run exists to measure how recoverable a failure is in one debugging round, never to be folded into the standard score.
Phase 1 (baseline): runs/smoke__qwen3.6-35b-a3b__20260710-190154
(MinimalPi, MEDIC_CAPTURE=on). cancel-async-tasks FAIL (agent 17s — a fast
"confident false success": the asyncio.gather + semaphore draft passes 5/6 but
mishandles cancellation above max_concurrent).
Phase 2 (this run, MedicPi, MEDIC_SIGNAL=full MEDIC_LOG_ACCESS=dump
MEDIC_WORKSPACE=restore): the base agent's broken /app/run.py was restored in
place and the failing pytest output injected into the system prompt. The medic
ran 6m19s (2.71M input / 40k output tokens; 147 run_shell_command, 15 read_file,
12 write_file, 12 edit — it edited /app/run.py repeatedly).
test_tasks_cancel_above_max_concurrent now PASSES; the medic
over-corrected and regressed a different test
(test_tasks_obey_max_concurrent), so the trial is still 5/6 → FAIL. So this
is "fixed the reported bug, broke an invariant," not "made no progress."Every P0 mechanism worked on a real failing 35b task, verified from the logs:
1. phase-1 workspace capture (workspace.tar.gz = ./run.py);
2. failed-set selection (medic_lib.py select → cancel-async-tasks);
3. input staging (workspace + failing-check output + run log, keyed by task);
4. workspace restore — the base draft's distinctive
except (KeyboardInterrupt, asyncio.CancelledError) block appears 24× in the
medic's context → it opened onto the real broken attempt, not a pristine file;
5. failure-signal injection — minimal-pi: MEDIC recovery prompt active
(signal=full) fired; the medic targeted the exact named failing test;
6. real second verdict + medic_recovery.json diff.
query_log A/B is the intended lever.💬 1 analyst comment inline below (from runs/medic__qwen3.6-35b-a3b__20260710-190552/COMMENTS.md).
llama-local/qwen3.6-35b-a3bagentharnesses.medic_pi:MedicPithinkingonreasoning budgetdirect (server/none)* — see journal for the authoritative mechanismmaxTokens / contextWindow65536 / 196608agent timeout ×2.0trials1 of 1 — 0 pass · 1 failmean reward0.00tokens (job total)2,713,973 in / 40,221 outstarted / finished2026-07-10T19:05 / 2026-07-10T19:12wall clock7m05sMedic recovery (P0) still-fails, but with genuine targeted progress: it opened
onto the restored broken /app/run.py (the base draft's except
(KeyboardInterrupt, asyncio.CancelledError) block appears 24× in its context)
with the failing pytest output in the system prompt (MEDIC recovery prompt
active, signal=full), fixed the originally-failing
test_tasks_cancel_above_max_concurrent, but over-corrected and broke
test_tasks_obey_max_concurrent → still 5/6. Not a harness miss — the restore
and signal both landed; the model just over-edited (and churned ~21
loop-guard/recovery events over 6m19s). Single K=1 datapoint.
| # | result | total | agent | in/out tok | flags | |
|---|---|---|---|---|---|---|
| 1 | FAIL | 7m05s | 6m19s | 2713973/40221 | 🔍 view |