MATH · IN · MODELS
methods / Causal Validation / Orthogonal subspace collapse (aggregated-PCA)

Orthogonal subspace collapse (aggregated-PCA)

Techniqueintermediate

Finds a multi-dimensional 'attribute subspace' by running PCA on a matrix of per-class mean representations (one row per class, e.g. one per speaker), then causally ablates that attribute by projecting every representation onto the orthogonal complement of the top-k resulting principal directions — a whole-subspace generalization of a single erasure direction.

Used in (1 observation)

structure: Linear Subspace · models: CPC-big (LibriLight 6k hrs), CPC-small (LibriSpeech-100h), APC (LibriSpeech-360h) · paper: Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspaces