MATH · IN · MODELS

SimCLR

Google Research

Structures found in this family (4)

Linear Subspace, Dimensional collapse, Linear Separability, Polytope (Simplex)

By model (3)

ResNet-50 (SimCLR contrastive pretraining, ImageNet) · 25M
Dimensional collapse, Linear Separability, Linear Subspace
ResNet-18 (SimCLR contrastive pretraining, CIFAR-10)
Linear Subspace
SimCLR ResNet-18 (mini-ImageNet)
Polytope (Simplex)

Observations (7)

A fitted linear projection (PISCO) disentangles style from content
Contrastive SSL embeddings undergo provable covariance-spectrum rank collapse
A correlated manifold-capacity formula beats the low-rank approximation
SSL encoders collapse variance along each task's decision axis, aiding few-shot transfer
Mid-level vision decodability weakly correlates with recognition accuracy
MMCR trains competitive SSL via a distinct low-dim, large-radius geometry
An affine map between two vision models matches their PCA components

Papers

Simple Disentanglement of Style and Content in Visual Representations (2023), Understanding Dimensional Collapse in Contrastive Self-supervised Learning (2022), Linear Classification of Neural Manifolds with Correlated Variability (2023), Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learning (2026), Probing the Mid-level Vision Capabilities of Self-Supervised Learning (2024), Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations (2023), Text-to-Concept (and Back) via Cross-Model Alignment (2023)