Achilli/Pham et al. Diffusion Memorization-Transition DDPMs

Research (Achilli, Ventura, Pham, Raya, Krotov, Lucibello, Ambrogioni et al.; PixelCNN++-backbone DDPMs, arXiv:2410.08727 / arXiv:2505.21777)

Structures found in this family (2)

Curvature profile of the representation manifold, Dimensional collapse

By model (5)

DDPM (CelebA-HQ, custom-trained, memorization-transition study)

Curvature profile of the representation manifold, Dimensional collapse

DDPM (CIFAR-10, custom-trained, memorization-transition study)

Curvature profile of the representation manifold, Dimensional collapse

DDPM (Fashion-MNIST, custom-trained, memorization-transition study)

Curvature profile of the representation manifold, Dimensional collapse

DDPM (LSUN-Church, custom-trained, memorization-transition study)

Curvature profile of the representation manifold, Dimensional collapse

DDPM (MNIST, custom-trained, memorization-transition study)

Curvature profile of the representation manifold, Dimensional collapse

Observations (2)

Real trained DDPMs lose spectral-gap dimension smoothly as training-set size shrinks into a memorization regime

Diffusion-model score-Jacobian curvature maps onto a dense associative-memory energy landscape whose memorized/spurious/generalized transition scales with model capacity

Papers

Losing Dimensions: Geometric Memorization in Generative Diffusion (2024), Memorization to Generalization: Emergence of Diffusion Models from Associative Memory (2025)