Audit the scaffold, not the checkpoint for coding agents
The point
A paper argues that checking whether a model was retrained or its weights stayed frozen is an incomplete rule for AI self improvement. For coding agents, the scaffold of tools, tests, and task breakdown around a frozen model can be rewritten, growing the set of changes an agent can reach.
Why it matters
For builders, audits should track scaffold changes, not only model changes.
Key details
- 30 agents from one model family failed together, and a majority failed 23/55 (42%) of tasks
- Evidence across 401 sessions: code changed fell off commit to commit
Original ~10 min → 25 sec here Source: arXiv cs.AI, Nov 13, 2026