Object-manifold radius and dimension track memorization and double descent

Stephenson et al. apply mean-field manifold capacity theory to AlexNet, VGG-16, and ResNet-18 classifiers trained on CIFAR-100 and Tiny-ImageNet, tracked jointly across depth and training epochs [stephenson-etal-2021-geometry-of-generalization-and-memorization] Object-manifold radius and dimension track memorization: manifolds for memorized examples differ systematically from generalized ones [stephenson-etal-2021-geometry-of-generalization-and-memorization] Manifold dimension undergoes double descent (shown for a ResNet-18 width-sweep on CIFAR-100 with 10% label noise), while radius and center correlation stay monotonic with model size [stephenson-etal-2021-geometry-of-generalization-and-memorization] Individual-example manifold radius and dimension shrink as examples move from a memorized to a generalized regime; purely observational [stephenson-etal-2021-geometry-of-generalization-and-memorization]