AI & Tech··1 sources verified
AI Agents Run Multiverse Analysis Showing Most Finance Natural-Experiment Claims Hinge on Specification Choices
Summarised by Relevant News AI · Read time: 3 min
Researchers used AI agents to rebuild data and reproduce 324 finance papers that reuse 13 natural experiments, then re-estimated each main effect across 62,409 defensible specifications. They could reproduce only 42% of signed main estimates under the papers' own specifications, and on average just 25% of specifications kept the published sign with p < 0.10. Regression discontinuity claims proved more robust than difference-in-differences claims, and the authors argue that agentic coding makes large-scale multiverse analysis feasible.
Why it matters: The study demonstrates a practical use of agentic coding for large-scale reproducibility and robustness audits, a method that could be applied to other empirical research fields.
All sources