Google Uses 29 AI Agents to Build a 50-Task Causal Reasoning Benchmark

Google Research reports that 29 agents spent 11.5 hours converting 13 public data tables into a 50-task benchmark. The tasks ask models to decide whether the data can identify a causal effect and, where it can, to estimate that effect against a known answer. Separate reviewers and auditors then found wording cues and shortcuts that let models answer without doing the intended reasoning, and the team turned those problems into tests for future benchmark drafts.

Why it matters: It shows a multi-agent workflow for building evaluations, along with a review step that catches shortcut-driven flaws in benchmarks that could otherwise overstate model reasoning.