ServerBench: coding agents design their own LLM serving systems
The point
ServerBench benchmarks coding agents designing complete, specialized LLM serving systems. Agents work toward dollars per dollar, latency qualified token value per dollar of GPU cost, with stateful replay and independent verification.
Why it matters
For builders, it tests whether agents can improve the serving stack they run on.
Key details
- Claude Sonnet 5.5 reached 2.97 dollars per dollar on Gemma E4B, 4.87 times the vLLM reference, across 11 models and three deployments
- 22 of 33 runs yielded verified systems, and 13 exceeded the strongest baseline
- Top designs coordinated capacity improvements with load selection
Original ~12 min → 25 sec here Source: arXiv cs.AI, Nov 4, 2026