SSL encoders collapse variance along each task's decision axis, aiding few-shot transfer

Luthra et al. extend Neural Collapse to self-supervised encoders via directional class-distance-normalized variance (directional CDNV), which measures within-class variance only along a task's own class-separating axis [luthra-etal-2026-directional-neural-collapse] Across SimCLR, VICReg, MAE, and DINOv2 encoders (trained from scratch on mini-ImageNet plus off-the-shelf checkpoints), directional CDNV falls sharply while isotropic CDNV stays large, so SSL collapses variance selectively along task-relevant axes [luthra-etal-2026-directional-neural-collapse] When this holds across many independently-sampled tasks at once, their decision axes become near-orthogonal (pairwise cosine), geometrically explaining multitask few-shot transfer [luthra-etal-2026-directional-neural-collapse] No causal intervention is performed [luthra-etal-2026-directional-neural-collapse]