Terminal-Bench audit finds issues in roughly 30 percent of tasks

The point

Researchers audited Terminal-Bench 2.0 and found issues in roughly 30 percent of the benchmark. They released repairs as Terminal-Bench 2.1 through a workflow combining community reports, agent traces, oracle checks, and human gated repairs.

Why it matters

For builders, benchmarks can change scores and ranks, so treat leaderboards as moving measurements.

Key details

Original ~12 min → 25 sec here Source: arXiv cs.AI, Nov 6, 2026

← All points