SSL training drives emergent semantic-class clustering beyond augmentation invariance

Ben-Shaul et al. train self-supervised models with VICReg and SimCLR objectives on a ResNet backbone, evaluated on CIFAR-100 and FOOD101 using nearest-class-mean clustering accuracy [ben-shaul-etal-2023-reverse-engineering-self-supervised-learning] The SSL objective's regularization term drives emergent clustering of representations by semantic class, not just by the sample/augmentation-invariance it explicitly optimizes [ben-shaul-etal-2023-reverse-engineering-self-supervised-learning] Semantic-class alignment increases with both training time and network depth, aligning more strongly with true classes than with equal-size random partitions [ben-shaul-etal-2023-reverse-engineering-self-supervised-learning]