MATH · IN · MODELS
methods / Theoretical / Analytical / Paraphrase-error decomposition

Paraphrase-error decomposition

Techniqueadvanced

Defines 'word transformation' via an information-theoretic paraphrase condition (small KL divergence between the context distributions two word sets induce), then algebraically decomposes any embedding-sum residual into named, exactly-quantified error terms (paraphrase error, plus two conditional/unconditional dependence error terms) rather than assuming or spectrally deriving a generative structure for co-occurrence.

Used in (1 observation)

structure: Concept Crystals (Parallelogram/Trapezoid Structure) · models: word2vec SGNS (text8, 300d) · paper: Analogies Explained: Towards Understanding Word Embeddings