AlexNet

University of Toronto

Structures found in this family (5)

Intrinsic-dimension profile across depth, Sphere, Linear Subspace, Linear Separability, Polytope (Simplex)

By model (2)

AlexNet (ImageNet image classifier, supervised)

Intrinsic-dimension profile across depth, Linear Subspace, Linear Separability, Polytope (Simplex)

AlexNet-based encoder (contrastive/alignment-uniformity, STL-10)

Sphere

Observations (5)

CNN intrinsic dimension is hunchback-shaped; last-layer ID predicts accuracy

Contrastive loss decomposes into alignment plus hypersphere uniformity

Net2Vec shows most CNN concepts need several jointly-fit filters

Object manifolds untangle across CNN depth, causally driven by geometry

Object-manifold radius and dimension track memorization and double descent

Papers

Intrinsic Dimension of Data Representations in Deep Neural Networks (2019), Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere (2020), Net2Vec: Quantifying and Explaining How Concepts Are Encoded by Filters in Deep Neural Networks (2018), Separability and Geometry of Object Manifolds in Deep Neural Networks (2020), On the Geometry of Generalization and Memorization in Deep Neural Networks (2021)