MATH · IN · MODELS
methods / Theoretical / Analytical / Alignment/Uniformity Loss Decomposition

Alignment/Uniformity Loss Decomposition

Techniqueadvanced

Decomposes the asymptotic contrastive loss into an alignment term (closeness of positive pairs) and a uniformity term (a Gaussian-potential measure of how close the induced feature distribution is to the uniform measure on the hypersphere), then causally tests the decomposition by training directly on the two decomposed losses in place of the original contrastive objective.

Used in (1 observation)

structure: Sphere · models: AlexNet-based encoder (contrastive/alignment-uniformity, STL-10), ResNet-50 (MoCo v2, unsupervised contrastive pretraining, ImageNet) · paper: Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere