MATH · IN · MODELS

DeepSeek

DeepSeek AI

Structures found in this family (5)

Linear Direction, Linear Subspace, Linear Separability, Platonic Representation Hypothesis, Intrinsic-dimension profile across depth

Hypotheses argued for by this family (1)

Platonic Representation Hypothesis

By model (9)

DeepSeek-R1-Distill-Qwen-1.5B · 1.5B
Linear Direction, Linear Subspace, Linear Separability, Intrinsic-dimension profile across depth
DeepSeek-R1-Distill-Qwen-32B · 32B
Linear Direction, Linear Separability, Intrinsic-dimension profile across depth, Linear Subspace
DeepSeek-R1-Distill-Llama-8B · 8B
Linear Direction, Platonic Representation Hypothesis, Linear Separability
DeepSeek-R1-Distill-Qwen-14B · 14B
Linear Direction, Linear Subspace, Intrinsic-dimension profile across depth
DeepSeek-R1-Distill-Qwen-7B · 7B
Linear Direction, Linear Subspace, Linear Separability
DeepSeek-LLM-7B-Chat · 7B
Linear Separability, Linear Direction
DeepSeek-LLM-7B-Base · 7B
Linear Direction
DeepSeek-R1-Distill-Llama-70B · 70B
Linear Separability
DeepSeek-V3 · 671B (37B active)
Linear Subspace

Observations (15)

A mean-difference vector between execution and reflection/transition thoughts, added to hidden states, causally calibrates reasoning length and accuracy
A difference-in-means direction at the <think> position, shared across MATH difficulty levels, causally sets the number of reasoning tokens
Contextual displacement forms a vector field tracking concreteness
Fourier periodicity in number embeddings is universal; linear separability is not
Linear H-probes recover tree depth and pairwise tree distance from low-dimensional, causally important subspaces of language-model representations in synthetic tree-traversal tasks, with analogous but weaker structure in real mathematical reasoning traces
Temporal-probe decodability predicts reasoning in high-resource languages
Orthogonal Procrustes alignment maps a Gemma Scope SAE 'Concept Atlas' onto three independently-trained Llama-3.1-8B models' activation spaces, with quantified retrieval/translation quality (AUROC 0.82-0.86, AP 0.39-0.49 vs. a 0.046 random baseline), and the transferred concept directions can be added back into the subject model's residual stream to steer generation
Reasoning behaviors are distinct but correlated steerable linear directions
A probe reads answer correctness before reasoning models state it
Chain-of-thought trajectories collapse to under ten intrinsic dimensions
The gains of reinforcement learning with verifiable rewards (RLVR) can be reproduced by a very low-capacity steering-vector intervention whose effective control directions lie mainly in the low-variance complement of the activation principal subspace, with geometry that is stable within a task and base model
Small Tower-of-Hanoi transformers, Qwen3.6-27B and DeepSeek-R1-Distill-Qwen-32B linearly encode a geometrically faithful Sierpinski-triangle state-space representation, and in the large models it decays over long planning
Chain-of-thought traverses step-specific subspaces that sharpen with depth
Crosscoder pre-'wait' feature directions set which reasoning pattern follows
A mid-layer head group decodes another agent's beliefs and steers theory-of-mind

Papers

SEAL: Steerable Reasoning Calibration of Large Language Models for Free (2025), On Reasoning Strength Planning in Large Reasoning Models (2025), Language Models Represent and Transform Concepts with Shared Geometry (2026), Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026), H-Probes: Extracting Hierarchical Structures From Latent Representations of Language Models (2026), What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026), Atlas-Alignment: Making Interpretability Transferable Across Language Models (2025), Understanding Reasoning in Thinking Language Models via Steering Vectors (2025), Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification (2025), Reasoning emerges from constrained inference manifolds in large language models (2026), Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectors (2026), Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking (2026), LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals (2026), Internal states before "wait" modulate reasoning patterns (2025), Language Models Represent Beliefs of Self and Others (2024)