Dosu uses evaluation driven development to keep its agent reliable at scale
The point
Dosu helps develop, maintain, and support software, taking triage and questions off maintainers. A small prompt tweak can help in one area and regress another, so Dosu benchmarks every change with evals and adds production failures to those evals.
Why it matters
For builders, that loop makes agent changes measurable at scale.
Key details
- Installed on thousands of repositories, with dozens of submodules
- LangSmith tracing took minutes with the traceable decorator and shows prompts and output in one trace
- Failure search uses feedback, sentiment, internal errors, and response time
Original ~9 min → 25 sec here Source: LangChain Blog, Nov 9, 2026