Mamba

CMU / Princeton (Gu & Dao)

Structures found in this family (4)

Linear Subspace, Linear Direction, Circle, Anisotropy

By model (3)

Mamba-2.8B · 2.8B

Linear Subspace, Circle

Mamba-130m (state-spaces/mamba-130m-hf) · 130M

Anisotropy

Mamba-8B · 8B

Linear Direction

Observations (4)

Fourier periodicity in number embeddings is universal; linear separability is not

In-context task representations are k-NN-decodable and causally patchable across real LLMs and an SSM

Knowledge editing shatters transformers' natural cyclic manifold geometry

Frozen Mamba collapses into an extreme anisotropy cone

Papers

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026), Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder-Decoder Perspective (2025), Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing (2024), Lost in State Space: Probing Frozen Mamba Representations (2026)