DiScoFormer: One transformer for density and score, across distributions

Many problems in machine learning and the sciences reduce to recovering a distribution from a finite set of samples, which requires estimating two quantities: density (how common values are) and score (the gradient of log-density, pointing toward more probable regions).

Score estimation is central to diffusion-based generative models, Bayesian sampling, and particle simulations, but existing tools force a painful trade-off.

Kernel density estimation (KDE) is generalizable and needs no training, but its accuracy collapses in high dimensions, while neural score-matching models stay accurate in high dimensions but must be retrained from scratch for every new distribution.

This tension leaves practitioners choosing between generality and precision.