Hard-debiasing a real word2vec/GloVe gender subspace leaves the residual geometry recoverably gendered

Gonen & Goldberg apply Bolukbasi et al. (2016)'s hard-debiasing projection (removing the linear gender direction) to real word2vec-GoogleNews and GloVe-Wikipedia embeddings [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] K-means clustering of the top gender-biased words on the "debiased" embeddings still recovers the original male/female clustering with 92-98.4% accuracy [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] An RBF-kernel SVM trained on original (biased) word vectors to classify male- vs female-stereotyped words transfers to the debiased vectors with comparable accuracy, and nearest-neighbor profession word lists remain measurably gender-skewed after debiasing [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] These findings show gender bias in these embeddings is not confined to the single linear direction removed by hard-debiasing, so linear-subspace projection alone does not eliminate the bias structure, only its visibility along that one axis [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias]