MATH · IN · MODELS

GPT-2

OpenAI

Structures found in this family (9)

Dimensional collapse, Linear Direction, Linear Subspace, Intrinsic-dimension profile across depth, Platonic Representation Hypothesis, Representational-similarity trajectory across depth, Linear Separability, Anisotropy, Continuous causal attention operator (CCT)

Hypotheses argued for by this family (1)

Platonic Representation Hypothesis

By model (8)

GPT-2 Small · 124M
Linear Direction, Linear Subspace, Platonic Representation Hypothesis, Representational-similarity trajectory across depth, Linear Separability, Anisotropy
GPT-2 Medium · 345M
Linear Subspace, Linear Direction, Intrinsic-dimension profile across depth, Linear Separability
GPT-2 Large · 774M
Linear Subspace, Linear Direction, Linear Separability
GPT-2 XL · 1.5B
Linear Direction, Intrinsic-dimension profile across depth, Linear Separability
GPT-2
Dimensional collapse, Continuous causal attention operator (CCT)
Custom GPT-2-Small (trained from scratch on synthetic grid-navigation token sequences) · 12-layer (GPT-2-Small architecture), 124M
Linear Subspace
nanoGPT-style GPT-2 124M (FineWeb-10B) · 124M
Intrinsic-dimension profile across depth
Custom 12-layer GPT-2-Small (trained from scratch, 100M tokens) · 12-layer (GPT-2-Small architecture)
Intrinsic-dimension profile across depth

Observations (28)

Attention outputs collapse to a ~60% low-rank subspace
BatchTopK relaxes per-sample sparsity to a per-batch budget
Causal-tracing localization does not predict the best edit layer
Answer-correctness sits in a 3-8D linear subspace, causally steerable
FFN value-vector directions causally suppress toxicity
A ~14-D temporal subspace is fragmented by SAEs but captured whole by SASA
Trajectory-subspace curvature perturbations causally modulate next-token entropy
Representational similarity analysis shows 15 language and vision-language models align more strongly with brain regions whose activity is consistent across sentence, word-cloud, and image presentations of the same concept than with less modality-consistent regions
Linear relational embeddings are faithful and causal for half of relations
An invariant reasoning subspace stabilizes latent chain-of-thought refinement
LLMs implement word2vec-style relational vector offsets, causally patchable
Number-task subspaces are distinct yet linearly inter-transformable
Real GPT-2, TinyLlama, and Qwen2.5 representation trajectories show higher curvature for reasoning than lexical tasks, and a universal three-phase layerwise structure
A participation-ratio spectral signal finds attention circuits label-free across scale
Behavioral grounding survives rotation but collapses under random reassignment
A GPT-2 trained on grid navigation forms an orthogonal xy coordinate map
Outlier dimensions persist and can solve GLUE tasks
ROME localizes facts to a two-peak causal site and edits via a rank-one direction
Sentiment is a single convergent linear direction, causally validated
A continuous-time attention operator makes fixed LLMs behave continuously
A summarization motif routes sentiment through non-valenced tokens
Linear safety separability emerges above a hidden-dimension threshold
Token trajectories trace a curved low-dimensional Langevin manifold
Topic subspaces sharpen with depth; a centroid direction induces CoT
TopK SAEs give clean scaling laws and mitigate dead latents
Transcoders linearize MLP circuit attribution and match SAE quality
Othello-move representations from seven architecturally distinct real models (GPT-2, BART, T5, Flan-T5-XL, LLaMA-2-7B, Mistral-7B, Qwen2.5-7B) Procrustes-align into a shared geometric space at cosine similarity up to 97.2% (unsupervised), with PCA trajectories converging across models and nearest-neighbor tile embeddings recovering the board's actual physical adjacency structure
NTP implicitly SVD-factorizes co-occurrence; orthants recover semantic categories

Papers

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning (2025), BatchTopK Sparse Autoencoders (2024), Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models (2023), The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026), Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (2022), Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability (2026), Representational Curvature Modulates Behavioral Uncertainty in Large Language Models (2026), Language models align with brain regions that represent concepts across modalities (2025), Linearity of Relation Decoding in Transformer Language Models (2023), Invariant Reasoning Directions in Latent Trajectories of Language Models (2026), Language Models Implement Simple Word2Vec-style Vector Arithmetic (2024), The Representational Geometry of Number (2026), Trajectory Geometry of Transformer Representations Across Layers (2026), Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers (2026), Mapping Language Models to Grounded Conceptual Spaces (2022), Cognitive Maps in Language Models: A Mechanistic Analysis of Spatial Planning (2025), Outlier Dimensions Encode Task-Specific Knowledge (2023), Locating and Editing Factual Associations in GPT (2022), Linear Representations of Sentiment in Large Language Models (2023), Language Models Are Implicitly Continuous (2025), The Blessing and Curse of Dimensionality in Safety Alignment (2025), Lines of Thought in Large Language Models (2024), Large Language Models Encode Semantics and Alignment in Linearly Separable Representations (2025), Scaling and Evaluating Sparse Autoencoders (2024), Transcoders Find Interpretable LLM Feature Circuits (2024), Revisiting the Othello World Model Hypothesis (2025), Geometry of Semantics in Next-Token Prediction: How Optimization Implicitly Organizes Linguistic Representations (2025)