FastText

Facebook AI Research

Structures found in this family (2)

Linear Subspace, Pinched Manifold (Singular Quotient)

By model (3)

fastText (Common Crawl, 300d)

Linear Subspace

FastText (bag-of-word-vectors)

Linear Subspace

fastText (self-trained on 65M-token SemEval-2010 corpus)

Pinched Manifold (Singular Quotient)

Observations (6)

MUSE aligns monolingual embeddings with an unsupervised orthogonal map

Fourier periodicity in number embeddings is universal; linear separability is not

Topological Polysemy (TPS), a persistent-homology-based singularity measure, correlates with real word-sense counts in a trained fastText embedding, and not with frequency

Direct gender bias is rank-1; associative WEAT bias is distributed

Analogy preservation is necessary and sufficient for a linear cross-lingual map

ICA after PCA-whitening recovers a universal sparse axis-aligned decomposition

Papers

Word Translation Without Parallel Data (2018), Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026), Topology of Word Embeddings: Singularities Reflect Polysemy (2020), What Does Debiasing Really Remove? A Geometric Study of PCA-Based Gender Debiasing in Word Embeddings (2026), Understanding Linearity of Cross-Lingual Word Embedding Mappings (2020), Discovering Universal Geometry in Embeddings with ICA (2023)