MATH · IN · MODELS

OLMo 2

Allen Institute for AI

Structures found in this family (5)

Linear Direction, Lissajous Curves, Linear Separability, Dimensional collapse, Linear Subspace

By model (5)

OLMo 2 7B · 7B
Lissajous Curves, Linear Direction, Linear Separability, Dimensional collapse, Linear Subspace
OLMo 2 1B · 1B
Lissajous Curves, Dimensional collapse, Linear Direction
OLMo 2 13B · 13B
Lissajous Curves, Linear Direction
OLMo 2 32B · 32B
Linear Direction, Lissajous Curves
OLMo 2 7B Instruct · 7B
Linear Direction, Linear Subspace

Observations (7)

ICL answer tokens are affine functions in a PCA subspace
Sinusoidal number geometry recurs across eight LLMs and is causal
Cue-induced bias directions are installed by alignment tuning
Morphology stays linearly separable while lexical identity turns nonlinear with depth
Pretraining passes through three effective-rank phases
A 40-dim SVD emotional subspace with four interpretable axes steers emotion
A sinusoidal probe decodes integers near-perfectly across nine LLMs

Papers

Functional Subspace, where language models can use vector algebra to solve problems (2026), Unravelling the Mechanisms of Manipulating Numbers in Language Models (2025), How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? (2026), Finding Lexical Identity and Inflectional Morphology in Modern Language Models (2025), Tracing the Representation Geometry of Language Models from Pretraining to Post-training (2025), Emotions Where Art Thou: Characterizing the Emotional Latent Space of LLMs (2025), Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers (2025)