AI & Tech··1 sources verified
BuggyMLE Injects Bugs to Build a Harder Benchmark for AI Coding Agents
Summarised by Relevant News AI · Read time: 3 min
Researchers introduced BuggyMLE, a framework that injects controlled bugs into real machine learning repositories to create difficulty-graded tasks for testing AI agents. Their BuggyMLE-Bench-Lite drops the top solve rate from 95% to 9% as difficulty rises, and a model fine-tuned on the curriculum (Qwen3.5-9B) also improved on SWE-bench Lite and LiveCodeBench. The paper also proposes a Code Complexity Index that predicts solve rates across eight coding benchmarks.
Why it matters: It offers a more realistic, repository-level way to measure whether agents can actually perform ML engineering, and shows that targeted training on such tasks can transfer to general coding performance.
All sources