Tests as data: ensuring AI-generated code does what we want
The point
Matteo Vaccari argues that when AI writes both production and test code, developers stop trusting tests because agents refactor tests as code changes. He proposes customer tests written as data: input, expected output, and setup in readable text files, run by a smaller test runner.
Why it matters
For builders, tests become a contract agents cannot quietly rewrite.
Key details
- Interviews with 26 Thoughtworks developers in early 2026 found they did not trust AI generated tests
- Customer tests use customer language and test the system from the outside, without implementation details
Original ~20 min → 25 sec here Source: Martin Fowler, Nov 11, 2026