AI & Tech··1 sources verified
Calibrated Decision Rewards Let Language Models Train Without Answer Keys
Summarised by Relevant News AI · Read time: 3 min
Researchers propose RLCDR, which turns an LLM into a decision model that outputs calibrated probabilities for questions about a response, such as whether the final answer is correct or which step is the first error. Training Qwen3-1.7B with GRPO on MATH, the reference-free reward lifted MATH500 accuracy from 72.2% to 76.8%, roughly matching training on the answer key (76.9%), while a random reward gained only 0.65 points. The paper also reports that the reward is better calibrated than a softmax judge on the same backbone and finds errors better than a dedicated 7B process reward model on ProcessBench, though it becomes overconfident when trained only on problems the base model mostly fails.
Why it matters: It offers a practical route to reinforcement learning on tasks without reference answers, where the reward signal's probabilities are trustworthy enough to avoid reward hacking, and it documents where that approach breaks down.
All sources