AuthGateBench: LLMs still block actions users explicitly authorized

The point

AuthGateBench is the first policy guided benchmark for authorization gating under benign requests. Its 1,400 instances span five failure modes from Claude Code incidents, using contrast pairs that flip between block and allow. Models block most actions even when explicitly authorized.

Why it matters

For builders, auto approval gates in Claude Code and Codex need policy context, not action alone.

Key details

Original ~3 min → 25 sec here Source: arXiv cs.AI, Nov 9, 2026

← All points