MATH · IN · MODELS

DeepSeek

DeepSeek AI

Structures found in this family (5)

Linear Direction, Linear Subspace, Linear Separability, Platonic Representation Hypothesis, Intrinsic-dimension profile across depth

Hypotheses argued for by this family (1)

Platonic Representation Hypothesis

By model (9)

DeepSeek-R1-Distill-Llama-8B · 8B
Platonic Representation Hypothesis, Linear Direction, Linear Separability
DeepSeek-R1-Distill-Qwen-1.5B · 1.5B
Linear Direction, Linear Separability, Intrinsic-dimension profile across depth
DeepSeek-LLM-7B-Chat · 7B
Linear Separability, Linear Direction
DeepSeek-R1-Distill-Qwen-14B · 14B
Linear Direction, Intrinsic-dimension profile across depth
DeepSeek-R1-Distill-Qwen-32B · 32B
Linear Separability, Intrinsic-dimension profile across depth
DeepSeek-R1-Distill-Qwen-7B · 7B
Linear Direction, Linear Separability
DeepSeek-LLM-7B-Base · 7B
Linear Direction
DeepSeek-R1-Distill-Llama-70B · 70B
Linear Separability
DeepSeek-V3 · 671B (37B active)
Linear Subspace

Observations (10)

Contextual displacement forms a vector field tracking concreteness
Fourier periodicity in number embeddings is universal; linear separability is not
Temporal-probe decodability predicts reasoning in high-resource languages
Orthogonal Procrustes alignment maps a Gemma Scope SAE 'Concept Atlas' onto three independently-trained Llama-3.1-8B models' activation spaces, with quantified retrieval/translation quality (AUROC 0.82-0.86, AP 0.39-0.49 vs. a 0.046 random baseline), and the transferred concept directions can be added back into the subject model's residual stream to steer generation
Reasoning behaviors are distinct but correlated steerable linear directions
A probe reads answer correctness before reasoning models state it
Chain-of-thought trajectories collapse to under ten intrinsic dimensions
Chain-of-thought traverses step-specific subspaces that sharpen with depth
Crosscoder pre-'wait' feature directions set which reasoning pattern follows
A mid-layer head group decodes another agent's beliefs and steers theory-of-mind

Papers

Language Models Represent and Transform Concepts with Shared Geometry (2026), Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026), What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026), Atlas-Alignment: Making Interpretability Transferable Across Language Models (2025), Understanding Reasoning in Thinking Language Models via Steering Vectors (2025), Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification (2025), Reasoning emerges from constrained inference manifolds in large language models (2026), LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals (2026), Internal states before "wait" modulate reasoning patterns (2025), Language Models Represent Beliefs of Self and Others (2024)