Diffusion large language models are an alternative to autoregressive models, but existing RL methods for them have two blind spots, this work argues: they treat every denoising step as equally important, so there is no temporal credit assignment across the denoising trajectory, and policy optimization relies on mean-field likelihood estimates that are biased and high-variance.

The authors propose DACA-GRPO (Denoising-Aware Credit Assignment for GRPO), a lightweight, plug-and-play enhancement for any GRPO-style trainer. It adds two complementary mechanisms. Denoising Progress Scores extract per-token importance weights from intermediate predictions without an additional forward pass. Stratified Masking Likelihood partitions token positions into strata so that each token is predicted with most of the sequence available as context, reducing the mean-field bias.

DACA-GRPO was applied on top of three GRPO base methods across seven benchmarks covering mathematical reasoning, code generation, constraint satisfaction, and constrained generation. The reported gains are up to 5.6 percentage points on math reasoning, 7.4 on code generation, 36.3 on constraint satisfaction, and 5.9 on JSON schema adherence. The method is designed to slot into existing GRPO training pipelines rather than replace them.