SmolLM2

Hugging Face

Structures found in this family (3)

Linear Direction, Tangent-Aligned Anisotropy Hypothesis, Linear Subspace

By model (2)

SmolLM2-1.7B · 1.7B

Linear Direction, Tangent-Aligned Anisotropy Hypothesis, Linear Subspace

SmolLM2-360M · 360M

Tangent-Aligned Anisotropy Hypothesis, Linear Subspace

Observations (2)

Concept directions anti-concentrate in the unembedding's low-variance spectral tail

Anisotropy from frequency-blind sampling and self-reinforcing tangent gradients

Papers

Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations (2026), Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics (2026)