MATH · IN · MODELS
methods / Dimensionality Reduction / Demixed PCA (dPCA)

Demixed PCA (dPCA)

Techniqueadvanced

A supervised variant of PCA that finds a subspace maximally explaining one labeled variable's variance (e.g. an answer identity) while separating it from other variables (e.g. a query's string length), imported from neuroscience population-coding analysis; its null space can then be used to selectively remove one variable's information via projection.

Used in (1 observation)

structure: Linear Direction · models: LLaMA-7B · paper: Label Words as Local Task Vectors in In-Context Learning