Whitening transformation

methods / Theoretical / Analytical / Geometric analysis / Whitening transformation

Techniqueintermediate

Classical closed-form linear transformation — subtract the mean, then decorrelate and rescale every axis to unit variance via the eigendecomposition of the covariance matrix — applied as a post-hoc fix for anisotropic sentence embeddings. Unlike the learned normalizing-flow fix, this requires no training: a direct, exact linear map computed once from the embedding covariance.

Used in (2 observations)

Whitening removes anisotropy and improves sentence similaritymeasured

structure: Anisotropy · models: BERT-base-uncased, RoBERTa-base, DistilBERT-base-uncased, LaBSE · paper: WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach

Whitening matches BERT-flow at far lower dimensionmeasured

structure: Anisotropy · models: BERT-base-uncased, BERT-large-uncased · paper: Whitening Sentence Representations for Better Semantics and Faster Retrieval