Latent-manifold principal curvature (MAPC) tracks CNN generalization

Kaufman & Azencot estimate principal curvatures of the latent image manifold at hidden layers of real trained ResNet (18/50/101) and VGG (13/16/19) classifiers across CIFAR-10/100, TinyImageNet, and ImageNet [kaufman-azencot-2023-latent-image-manifolds] Curvature is computed via CAML (Li 2018) over SVD-generated synthetic neighbors, taking the eigenvalues of the manifold's Hessian, and summarized per layer as the mean absolute principal curvature (MAPC) [kaufman-azencot-2023-latent-image-manifolds] The curvature gap between the last two layers correlates with generalization: larger gaps are associated with more accurate models and smaller gaps with inferior performance [kaufman-azencot-2023-latent-image-manifolds] Regularizers such as mixup flatten intermediate-layer representations, but high curvature in the last layer is itself fundamental to performance; the study is purely observational [kaufman-azencot-2023-latent-image-manifolds]