MATH · IN · MODELS

Diffusion-model score-Jacobian curvature maps onto a dense associative-memory energy landscape whose memorized/spurious/generalized transition scales with model capacity

measured in 1 paper

Pham, Raya, Negri, Zaki, Ambrogioni & Krotov map a diffusion model's own energy function E_DM(x_t,t) directly onto a Dense Associative Memory (Hopfield-network) energy E_AM(x), with noise variance playing the role of inverse temperature, and reuse the score-Jacobian SVD/spectral-gap curvature diagnostic from Ventura et al. (2025) and Achilli et al. (2024) to quantify local energy curvature around real DDPM training points [pham-etal-2025-memorization-generalization-diffusion-associative-memory] Across real DDPMs (PixelCNN++-backbone U-Nets) trained from scratch on MNIST, Fashion-MNIST, CIFAR-10 and LSUN-Church at 38 training-set sizes per dataset, nearest-neighbor-distance classification against real and synthetic sample sets identifies memorized, spurious, and generalized regimes whose transition point matches the dimensional-collapse critical points found on the same checkpoints by Achilli et al. (2024) (e.g. CIFAR-10 A=2000/B=16000 of 50000 total) [pham-etal-2025-memorization-generalization-diffusion-associative-memory] A U-Net capacity ablation (widths 64/96/128, 8.9M-109.7M parameters) on CIFAR-10 and LSUN-Church shows the critical training-set size at which the spurious-state fraction peaks scales up with model capacity (CIFAR-10 U-Net64 peak at K=2568 vs. U-Net96 peak at K=5862), directly tying the memorization-geometry transition to model capacity rather than dataset size alone [pham-etal-2025-memorization-generalization-diffusion-associative-memory]

Papers

Memorization to Generalization: Emergence of Diffusion Models from Associative Memory — Pham, Bao, Raya, Gabriel, Negri, Matteo, Zaki, Mohammed J., Ambrogioni, Luca, Krotov, Dmitry2025 · arXiv:2505.21777