Activation steering modifies generative model behavior by shifting internal representations, but standard methods apply the same intervention to every input, which hurts quality when steering is not needed. Dynamically Scaled Activation Steering (DSAS) decouples when to steer from how to steer: it computes context-dependent scaling factors that modulate the strength of any existing steering transformation per layer and per token. At generation time, DSAS detects undesired behavior and intervenes strongly only where necessary, leaving the rest of the computation untouched.

The method is method-agnostic and can be jointly optimized end-to-end with the steering function. In experiments on LLM toxicity mitigation, combining DSAS with existing steering methods consistently improves the Pareto front compared to steering alone, meaning a better trade-off between reducing toxic output and preserving utility. The authors also demonstrate DSAS on a text-to-image diffusion model, showing adaptive steering can modulate specific concepts without uniform edits.

DSAS adds minimal computational overhead and improves interpretability by pinpointing which tokens require steering and the magnitude needed. The paper was accepted at the UniReps workshop at NeurIPS 2025. Code will be released on GitHub.