Direct gender bias is rank-1; associative WEAT bias is distributed

measured in 1 paper

Kresin et al. build the Bolukbasi gender-difference matrix from definitional word pairs, form its covariance and analyze the PCA spectrum on GloVe, word2vec and FastText embeddings (the paper does not specify the exact training corpora) [kresin-etal-2026-what-does-debiasing-really-remove] The spectrum is strongly low-rank: PC1 alone captures ~48% of variance, the first 5 ~80%, and the first 10 exceed 93% [kresin-etal-2026-what-does-debiasing-really-remove] Direct projection-based bias collapses to near-zero after removing only PC1 and stays flat, so it is genuinely rank-1-dominated across all three embeddings [kresin-etal-2026-what-does-debiasing-really-remove] WEAT associative bias is not reduced by removing any single dominant component and stays distributed across many higher-dimensional directions [kresin-etal-2026-what-does-debiasing-really-remove] Mean vector displacement rises monotonically and neighbor stability declines as more components are removed, a distortion trade-off with no universal optimal k [kresin-etal-2026-what-does-debiasing-really-remove]

Structure

Linear Subspace

Context

explained-variance spectrum of a PCA-derived bias subspace (48%/80%/93% at k=1/5/10), dissociation between direct (rank-1) and associative/WEAT (distributed) bias, cumulative multi-PC removal vs. single-PC ablation as complementary component-removal regimes, mean vector displacement and neighbor stability as geometric-distortion metrics, cross-embedding consistency (GloVe, word2vec, FastText) of the rank-1 direct-bias finding

Confirmed in models

GloVe (Wikipedia + Gigaword, uncased) · GloVe, word2vec (Google News, 300d) · word2vec, FastText (bag-of-word-vectors) · FastText

Method

PCA

Papers

What Does Debiasing Really Remove? A Geometric Study of PCA-Based Gender Debiasing in Word Embeddings — Kresin, Alexey, Dieffi, Tchifou M., Caspi, Tomer2026 · arXiv:2606.07964