MATH · IN · MODELS
methods / Dictionary Learning / Sparse coding for discourse atoms

Sparse coding for discourse atoms

Techniqueadvanced

Classical (pre-neural-SAE) k-SVD-style sparse dictionary learning applied directly to a trained word-embedding matrix, recovering a small set of overcomplete 'atoms of discourse' such that each word vector is approximately a sparse linear combination of a few atoms.

Used in (1 observation)

structure: Linear Subspace · models: Arora et al. SN embeddings (Wikipedia 3B tokens, 300d) · paper: Linear Algebraic Structure of Word Senses, with Applications to Polysemy