Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP

The article exposes the tension between what developers expect from torch.compile and what it actually does. A common reflex is to apply compile whenever a model is slow, but for a single nn.

Linear with bias, compile has almost nothing to fuse—the bias is already folded into the GEMM epilogue via aten::addmm, and the only overhead removed is the CPU dispatch of the transpose view, not any GPU kernel.

The real payoff only appears when multiple operations (GeLU + mul) are fused, as in a compiled MLP, but that fusion comes with shape specialization and recompilation risk.