MATH · IN · MODELS

ResNet

Microsoft Research

Structures found in this family (12)

Linear Subspace, Linear Separability, Intrinsic-dimension profile across depth, Sphere, Decision boundary (as a codimension-1 hypersurface), Attribute-Induced Embedding Folding, Polytope (Simplex), Dimensional collapse, Curvature profile of the representation manifold, Representational-similarity trajectory across depth, Persistent-homology / Betti profile across depth, Linear Centroids Hypothesis

By model (13)

ResNet (image classifier, various depths)
Intrinsic-dimension profile across depth, Curvature profile of the representation manifold, Representational-similarity trajectory across depth, Polytope (Simplex), Linear Separability
ResNet-50 (supervised, ImageNet)
Linear Separability, Decision boundary (as a codimension-1 hypersurface), Linear Centroids Hypothesis
ResNet-50 (MoCo v2, unsupervised contrastive pretraining, ImageNet)
Sphere, Linear Subspace, Linear Separability
ArcFace ResNet100 (trained on MS1MV3/IBUG-500K)
Sphere, Attribute-Induced Embedding Folding
ResNet-50 (hard-negative supervised/unsupervised contrastive, CIFAR-100)
Polytope (Simplex), Dimensional collapse
ResNet-18 (supervised, ImageNet)
Linear Subspace, Persistent-homology / Betti profile across depth
ArcFace ResNet50 (trained on CASIA/VGG2)
Sphere
FaceNet
Attribute-Induced Embedding Folding
ResNet-18 (multi-label, MLab-CIFAR10)
Polytope (Simplex)
ResNet-101 (CIFAR-10)
Decision boundary (as a codimension-1 hypersurface)
ResNet-34 (supervised, ImageNet)
Intrinsic-dimension profile across depth
ResNet-50 (Maximum Manifold Capacity Representations, self-supervised, ImageNet)
Linear Separability
WideResNet-28-10 (CIFAR-10)
Decision boundary (as a codimension-1 hypersurface)

Observations (25)

A fitted linear projection (PISCO) disentangles style from content
Linear-probe separability increases monotonically with CNN depth
CNN intrinsic dimension is hunchback-shaped; last-layer ID predicts accuracy
Additive angular margin gives a constant geodesic margin on the hypersphere
Contrastive loss decomposes into alignment plus hypersphere uniformity
Standard classifiers have higher decision-boundary curvature than robust ones
A multiscale (macroscale identity-cloud, microscale intra-identity) geometric structure in face-recognition embedding spaces is measurably shaped by interpretable attributes, with FaceNet showing stronger attribute-dependence than ArcFace
A real ResNet-50 trained with hard-negative contrastive losses and unit-ball normalization converges to Neural-Collapse ETF geometry on CIFAR-100, while omitting either yields Dimensional-Collapse instead
Latent-manifold principal curvature (MAPC) tracks CNN generalization
CKA recovers layer and network correspondences where CCA metrics fail
The hump-shaped ID profile is likely a TwoNN estimator artifact
Penultimate-layer manifold geometry predicts OOD generalization failure
Lower local intrinsic dimension predicts vision-model alignment and generalization
A real ResNet-18 trained on multi-label CIFAR-10 obeys a class-frequency-weighted prototype synthesis rule rather than uniform averaging, with collapse metrics tracking label imbalance
Topological descriptors (persistent homology) and intrinsic-dimension estimates of the last-layer data manifold in real trained CNNs change measurably across layers and training, and predict generalization performance
Mid-level vision decodability weakly correlates with recognition accuracy
MMCR trains competitive SSL via a distinct low-dim, large-radius geometry
An affine map between two vision models matches their PCA components
Last-layer class means converge to a Simplex Equiangular Tight Frame
Object manifolds untangle across CNN depth, causally driven by geometry
Decision-boundary margin is small along low-frequency discriminative directions
SVCCA reveals low effective dimensionality and bottom-up convergence
Object-manifold radius and dimension track memorization and double descent
Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probes
ICA after PCA-whitening recovers a universal sparse axis-aligned decomposition

Papers

Simple Disentanglement of Style and Content in Visual Representations (2023), Understanding Intermediate Layers Using Linear Classifier Probes (2016), Intrinsic Dimension of Data Representations in Deep Neural Networks (2019), ArcFace: Additive Angular Margin Loss for Deep Face Recognition (2019), Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere (2020), Curvature Dynamic Black-box Attack: Revisiting Adversarial Robustness via Dynamic Curvature Estimation (2025), Attributes Shape the Embedding Space of Face Recognition Models (2025), Hard-Negative Sampling for Contrastive Learning: Optimal Representation Geometry and Neural- vs Dimensional-Collapse (2024), Data Representations' Study of Latent Image Manifolds (2023), Similarity of Neural Network Representations Revisited (2019), Rethinking Intrinsic Dimension Estimation in Neural Representations (2026), Diagnosing Generalization Failures from Representational Geometry Markers (2026), Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and the Human Brain (2026), How Label Imbalance Shapes Geometry: A General Spectral Analysis of Multi-Label Neural Collapse (2026), Topology and Geometry of Data Manifold in Deep Learning (2022), Probing the Mid-level Vision Capabilities of Self-Supervised Learning (2024), Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations (2023), Text-to-Concept (and Back) via Cross-Model Alignment (2023), Prevalence of Neural Collapse during the terminal phase of deep learning training (2020), Are Neurons Actually Collapsed? On the Fine-Grained Structure in Neural Representations (2023), Separability and Geometry of Object Manifolds in Deep Neural Networks (2020), Hold Me Tight! Influence of Discriminative Features on Deep Network Boundaries (2020), SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability (2017), On the Geometry of Generalization and Memorization in Deep Neural Networks (2021), The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts (2026), Discovering Universal Geometry in Embeddings with ICA (2023)