CNN intrinsic dimension is hunchback-shaped; last-layer ID predicts accuracy

Ansuini et al. estimate TwoNN intrinsic dimension of layer-wise activations in real trained ImageNet CNNs (AlexNet, VGG, ResNet), finding ID rises sharply in early layers then contracts to a low plateau, a hunchback profile [ansuini-etal-2019-intrinsic-dimension-of-data-representations] Intrinsic dimension is orders of magnitude below the nominal unit count at every layer [ansuini-etal-2019-intrinsic-dimension-of-data-representations] The last hidden layer's intrinsic dimension predicts the network's own test accuracy across architectures (r approximately 0.94) [ansuini-etal-2019-intrinsic-dimension-of-data-representations] This foundational result predates and matches the expansion-then-compression pattern later confirmed in Transformers [ansuini-etal-2019-intrinsic-dimension-of-data-representations] No causal intervention is performed [ansuini-etal-2019-intrinsic-dimension-of-data-representations]