Null-calibrated representational similarity

methods / Representation Alignment / Null-calibrated representational similarity

Null-calibrated representational similarity

Techniqueadvanced

Corrects any representational-similarity metric (CKA, mutual k-NN, RSA, CCA, Procrustes) for a scale confound by subtracting off its expected value under an explicit null model of independent representations, using permutation-based estimation; global spectral metrics have a non-vanishing O(d/n) null baseline that grows with width/depth, while local-neighborhood metrics have a much smaller O(k/n) baseline.

Used in (1 observation)

After correcting CKA and mutual k-NN alignment for a proven width/depth-dependent null-baseline confound (O(d/n) for global spectral metrics vs O(k/n) for local-neighborhood metrics), the Platonic Representation Hypothesis's global cross-modal convergence largely vanishes while its local-neighborhood convergence survives, across real image-text and video-language pretrained model pairsmeasured

structure: Aristotelian Representation Hypothesis · models: BLOOMZ-560M, BLOOMZ-1.1B, BLOOMZ-1.7B, BLOOMZ-3B, BLOOMZ-7.1B, OpenLLaMA-3B, OpenLLaMA-7B, OpenLLaMA-13B, LLaMA-7B, LLaMA-13B, LLaMA-30B, LLaMA-65B, Gemma-2-9B-it, ViT-Base, ViT-Small, MAE ViT-Base (Masked Autoencoder), DINOv2 ViT-B/14, DINOv2 ViT-L/14, CLIP ViT-B/16, CLIP ViT-L/14, VideoMAE-base, VideoMAE-v2 · paper: Revisiting the Platonic Representation Hypothesis: An Aristotelian View