MATH · IN · MODELS
methods / Theoretical / Analytical / Manifold capture theory (subspace capture / tiling / dilution)

Manifold capture theory (subspace capture / tiling / dilution)

Techniqueadvanced

A formal criterion for when a sparse dictionary (e.g. an SAE) 'captures' a known manifold — a small, fixed group of atoms whose span contains it and whose encoder consistently selects that group — plus a taxonomy of three regimes (compact capture, shattering/tiling, dilution) describing how real dictionaries fall short of that ideal.

Used in (1 observation)

structure: Circle, Affine Subspace, Paraboloid (circular × pinched continuum) · models: Llama-3.1-8B · paper: Do Sparse Autoencoders Capture Concept Manifolds?