MATH · IN · MODELS

Contrastive SSL embeddings undergo provable covariance-spectrum rank collapse

measured in 1 paper

- Contrastive self-supervised embeddings (SimCLR, ResNet-50, ImageNet) undergo dimensional collapse: many singular values of the embedding covariance drop to numerical zero, in both the projection head and the backbone. [jing-etal-2022-dimensional-collapse] - Two causes are proven for a linear model under InfoNCE: strong augmentation drives weights toward zero wherever augmentation variance exceeds data variance (Lemma 1), and multilayer implicit regularization under SGD drives low rank even under weak augmentation (Corollary 2). [jing-etal-2022-dimensional-collapse] - The collapse is diagnosed via the log singular-value spectrum of the covariance; DirectCLR (a fixed low-rank projector) recovers accuracy (ImageNet linear-probe Top-1 66.5% full projector, 51.5% none, 62.7% DirectCLR). [jing-etal-2022-dimensional-collapse]

Context

contrastive self-supervised learning (SimCLR), embedding covariance singular-value spectrum, strong-augmentation-driven weight shrinkage (Lemma 1), multilayer implicit regularization under SGD (Corollary 2), DirectCLR (fixed low-rank projector)

Method

Papers

Understanding Dimensional Collapse in Contrastive Self-supervised Learning — Jing, Li, Vincent, Pascal, LeCun, Yann, Tian, Yuandong2022 · arXiv:2110.09348