AI & Tech··1 sources verified
Researchers Run 6-Bit MXFP6 Ad Ranking Models on AMD MI350X Without Losing Accuracy
Summarised by Relevant News AI · Read time: 3 min
A new arXiv paper evaluates MXFP8 and 6-bit MXFP6 quantization for production recommendation and ranking models on AMD's MI350X, holding to a strict 0.01% Normalized Entropy quality margin. On two production ads ranking models, MXFP6 GEMM kernels ran 1.73 to 1.99x faster than FP16, and selective MXFP6 raised end-to-end inference throughput 1.5 to 14.3% over FP8 configurations. The authors also report that MXFP8 matches row-wise FP8 accuracy, so the faster kernel can be chosen per layer.
Why it matters: Teams serving latency- and cost-sensitive inference can see that 6-bit microscaling formats may cut serving costs for ranking models without breaching strict quality bars, a finding that has been hard to achieve in this domain.
All sources