Centered Kernel Alignment (CKA) representation similarity

methods / Theoretical / Analytical / Geometric analysis / Centered Kernel Alignment (CKA) representation similarity

Centered Kernel Alignment (CKA) representation similarity

Techniqueintermediate

Compares two neural network representations via Centered Kernel Alignment, a similarity index invariant to orthogonal transformation and isotropic scaling (but not arbitrary invertible linear transformation), shown to reliably recover known layer/network correspondences where CCA-family metrics fail when representation dimension is large relative to sample count.

Used in (1 observation)

CKA recovers layer and network correspondences where CCA metrics failmeasured

structure: Representational-similarity trajectory across depth · models: Convolutional network image classifier (generic), ResNet (image classifier, various depths), Transformer image classifier (generic) · paper: Similarity of Neural Network Representations Revisited