MATH · IN · MODELS
methods / Theoretical / Analytical / Geometric analysis / SVCCA (SVD + canonical correlation) representation similarity

SVCCA (SVD + canonical correlation) representation similarity

Techniqueintermediate

Compares two neural network representations by first projecting each onto its own top singular directions (SVD, discarding low-variance noise dimensions) and then applying canonical correlation analysis (CCA) between the two reduced representations — an affine-invariant similarity index usable to compare layers within one network, across training time, or across independently trained networks.

Used in (1 observation)

SVCCA reveals low effective dimensionality and bottom-up convergencemeasured
structure: Representational-similarity trajectory across depth · models: ResNet (image classifier, various depths), Convolutional network image classifier (generic) · paper: SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability