Projection-Weighted CCA (PWCCA) representation similarity

methods / Theoretical / Analytical / Geometric analysis / Projection-Weighted CCA (PWCCA) representation similarity

Projection-Weighted CCA (PWCCA) representation similarity

Techniqueintermediate

Refines SVCCA by weighting each canonical-correlation direction by how much it actually contributes to the layer's own output (rather than treating every canonical direction equally), giving a similarity index more robust to low-variance noise directions than plain SVCCA.

Used in (1 observation)

Wider networks converge to more similar representations, measured by PWCCAmeasured

structure: Representational-similarity trajectory across depth · models: 11-layer CIFAR-10 convnet (Morcos et al. 2018), LSTM language model (AWD-LSTM setup, PTB/WikiText-2) · paper: Insights on Representational Similarity in Neural Networks with Canonical Correlation Analysis