Replica mean-field theory from statistical physics that predicts a class's linear separability (manifold capacity) directly from three measured geometric properties of its activation cloud — radius, dimension, and inter-class center correlation.
Used in (4 observations)
structure: Linear Separability · models: ResNet-50 (SimCLR contrastive pretraining, ImageNet) · paper: Linear Classification of Neural Manifolds with Correlated Variability
structure: Linear Separability · models: BERT-base-cased, RoBERTa-base, ALBERT-base-v1, DistilBERT-base-uncased, GPT-1 (OpenAI GPT) · paper: Emergence of Separable Manifolds in Deep Language Representations
structure: Linear Separability · models: ResNet-50 (Maximum Manifold Capacity Representations, self-supervised, ImageNet), ResNet-50 (MoCo v2, unsupervised contrastive pretraining, ImageNet), ResNet-50 (SimCLR contrastive pretraining, ImageNet) · paper: Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations
structure: Linear Separability · models: AlexNet (ImageNet image classifier, supervised), VGG (image classifier, various depths), ResNet (image classifier, various depths) · paper: Separability and Geometry of Object Manifolds in Deep Neural Networks