structure: 1D continuum manifold · models: Gemma-2-2B, Gemma-2-9B, Qwen3-4B · paper: Geometry of Ordinal Representations in Language Models, When Models Manipulate Manifolds: The Geometry of a Counting Task

structure: Linear Subspace · models: Llama-3.2-3B, Llama 3.1 70B, Qwen3-1.7B, Qwen3-32B · paper: Semantic Structure of Feature Space in Large Language Models

structure: Linear Subspace, Linear Direction · models: Gemma 2 27B Instruct, Gemma-2-27B, Qwen3-32B, Llama 3.3 70B Instruct · paper: The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models

structure: Linear Subspace, Linear Direction · models: Qwen3-8B, Llama-3.1-8B-Instruct · paper: The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models

structure: Linear Subspace · models: Llama-3-8B-Instruct, Qwen3-8B · paper: Cell-Based Representation of Relational Binding in Language Models

structure: Linear Subspace, Linear Direction · models: Llama-2-7B, Llama-3-8B, Qwen1.5-7B, Pythia-6.9B, Float-7B (code fine-tuned LM) · paper: Representational Analysis of Binding in Language Models

structure: Linear Direction · models: Qwen3-VL-4B-Instruct, Qwen3-VL-8B-Instruct, LLaVA-OneVision-1.5-4B-Instruct · paper: Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow

structure: Linear Direction, Linear Subspace · models: Gemma 3 4B Instruct, Gemma 3 4B, Gemma 3 27B Instruct, Qwen3-4B-Instruct, Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct · paper: Tool Calling Is Linearly Readable and Steerable in Language Models

structure: Circle, 1D continuum manifold · models: Llama-3.1-8B · paper: Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior

structure: Intrinsic-dimension profile across depth, Curvature profile of the representation manifold · models: AlphaEarth (satellite/Earth-observation foundation model) · paper: Characterizing AlphaEarth Embedding Geometry for Agentic Environmental Reasoning

structure: Intrinsic-dimension profile across depth · models: Llama-3-8B-Instruct, Llama-3.1-70B-Instruct · paper: The Geometry of Thought: How Scale Restructures Reasoning in Large Language Models

structure: Concept Cluster Heterogeneity · models: Llama-3-8B-Instruct, Llama-3.1-70B-Instruct · paper: The Geometry of Thought: How Scale Restructures Reasoning in Large Language Models

structure: Dimensional collapse · models: Llama-3-8B-Instruct, Llama-3.1-70B-Instruct · paper: The Geometry of Thought: How Scale Restructures Reasoning in Large Language Models

structure: Anisotropy · models: ELMo (AllenNLP biLM, 1B Word Benchmark), BERT-base-cased, GPT-2-small · paper: How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings

structure: Linear Direction · models: GPT-J-6B, Llama-3.1-8B, OLMo 2 32B, Pythia-12B, Gemma 3 27B Instruct, GPT-NeoX-20B · paper: Functional Subspace, where language models can use vector algebra to solve problems

structure: Linear Subspace · models: Aurora (weather foundation model) · paper: Does Aurora Encode Atmospheric Structure? Latent Regime Analysis and Attribution

structure: Circle · models: Autoencoder (trained on J.S. Bach's Well-Tempered Clavier) · paper: The Circle of Fifths as Latent Geometry in Bach's Well-Tempered Clavier

structure: Linear Subspace, Linear Direction · models: Bayesian Wind Tunnel Transformer (bijection task, 6 layers, 6 heads, d_model=192), Bayesian Wind Tunnel Transformer (HMM filtering task, 9 layers, 8 heads, d_model=256) · paper: The Bayesian Geometry of Transformer Attention

structure: Conceptual Belief Space Hypothesis, Linear Subspace · models: Llama-3.1-8B-Instruct · paper: Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space

structure: Linear Subspace · models: BERT-base-cased · paper: Visualizing and Measuring the Geometry of BERT

structure: Linear Direction, Linear Separability · models: ESM-2 (650M), ESM-2 (15B), ESM3 (1.4B, OPEN), ESM3 (98B, LARGE), ESMC (600M), ESMC (6B), ProGen2-base, EvoDiff OA-DM · paper: Viral Proteins Reveal Geometry of Protein Language Models

structure: Linear Direction · models: word2vec (Google News, 300d) · paper: Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

structure: Circle, Constructive Interference Hypothesis · models: BOWS Autoencoder (tied-weight, ReLU), BOWS Autoencoder (tied-weight, linear — no ReLU, baseline), BOWS Toy Transformer (1 block, 8 heads, d_model=768) · paper: From Data Statistics to Feature Geometry: How Correlations Shape Superposition

structure: Linear Direction · models: 128-dim dot-product matrix-factorization recommender, trained on Alibaba/Tianchi mobile-recommendation logs (Cheng) · paper: A Rank-One Popularity Component in Dot-Product Recommender Scores: Population Theory and Prior-Separation Evidence

structure: Linear Direction · models: Claude Sonnet 4.5 · paper: Emotion Concepts and their Function in a Large Language Model

structure: Dimensional collapse · models: CLIP ViT-B/16, CLIP ViT-B/32, CLIP ViT-L/14, SigLIP, SigLIP 2 · paper: Your CLIP has 164 dimensions of noise: Exploring the embeddings covariance eigenspectrum of contrastively pretrained vision-language transformers

structure: Paraboloid (circular × pinched continuum) · models: Llama-3.1-8B · paper: Do Sparse Autoencoders Capture Concept Manifolds?

structure: Linear Direction · models: Llama-2-7B-Chat, Llama-2-13B-Chat, Llama-2-70B-Chat, Vicuna-13B, Vicuna-33B-Uncensored, DeBERTa-xxlarge-v2-MNLI · paper: Representation Engineering: A Top-Down Approach to AI Transparency

structure: Intrinsic-dimension profile across depth · models: Gemma-2-27B · paper: Context Structure Reshapes the Representational Geometry of Language Models

structure: Linear Direction · models: Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-3B, Qwen-2.5-7B, Qwen2.5-14B, Qwen2.5-32B, Llama-2-7B, Llama-2-13B, Llama-3-8B, Llama-3.1-8B, Mistral Small 3 (2501), Ministral-3-3B-Base-2512, Ministral-3-8B-Base-2512, Ministral-3-14B-Base-2512, Gemma-2-2B, Gemma-2-9B, DeepSeek-LLM-7B-Base, Pythia-410M, Pythia-1B, Pythia-1.4B, Pythia-2.8B, Pythia-6.9B, Pythia-12B · paper: Language Models Represent and Transform Concepts with Shared Geometry

structure: Affine Subspace · models: Llama-3.1-8B · paper: Do Sparse Autoencoders Capture Concept Manifolds?

structure: Local hyperboloid-type quadric patch · models: all-mpnet-base-v2, all-MiniLM-L6-v2 · paper: Controlled Paraphrase Geometry in Sentence Embedding Space: Local Manifold Modeling and Latent Probing

structure: Linear Direction · models: LLaMA-7B · paper: Label Words as Local Task Vectors in In-Context Learning

structure: Linear Direction · models: StyleGAN2 (trained on FFHQ, 1024x1024), BigGAN-deep (512px) · paper: GANSpace: Discovering Interpretable GAN Controls

structure: Curvature profile of the representation manifold, Linear Subspace · models: Llama-3.2-1B · paper: The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors

structure: Affine Subspace · models: Llama-2-7B, Llama-2-13B, Llama-2-70B, Pythia-6.9B, Gemma-2-2B, EmbeddingGemma, word2vec (trained on Wikipedia) · paper: Language Models Represent Space and Time, Symmetry in Language Statistics Shapes the Geometry of Model Representations

structure: Intrinsic-dimension profile across depth · models: Stable Diffusion XL, SDXL-DMD (4-step distilled) · paper: ELROND: Exploring and Decomposing Intrinsic Capabilities of Diffusion Models

structure: Linear Direction · models: DDPM (CelebA-HQ 256x256, HuggingFace Diffusers), DDPM (LSUN-Church 256x256, HuggingFace Diffusers), DDPM (LSUN-Bedroom 256x256, HuggingFace Diffusers) · paper: Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models

structure: Circle, 2D Grid (Square Lattice) · models: Llama-3.1-8B, Llama-3.2-1B, Llama-3.1-8B-Instruct, Gemma-2-2B, Gemma-2-9B · paper: In-Context Learning of Representations

structure: Intrinsic-dimension profile across depth · models: ESM-2 (35M), ESM-2 (650M), ESM-2 (3B), iGPT-S, iGPT-M, iGPT-L, Llama-2-70B, GPT-2-XL, Gemma-2-2B · paper: The Geometry of Hidden Representations of Large Transformer Models, The Geometry of Concepts: Sparse Autoencoder Feature Structure

structure: Anisotropy · models: GPT-1 (OpenAI GPT), GPT-2-small, BERT-base-uncased, DistilBERT-base-uncased · paper: IsoScore: Measuring the Uniformity of Embedding Space Utilization

structure: Belief State Geometry Hypothesis (Mixed-State Presentation) · models: Custom GPT-2-style transformer (87M params, trained on synthetic Poker Hand History trajectories) · paper: Emergent World Beliefs: Exploring Transformers in Stochastic Games

structure: Linear Subspace · models: GloVe (Wikipedia + Gigaword, uncased), word2vec (Google News, 300d), FastText (bag-of-word-vectors) · paper: What Does Debiasing Really Remove? A Geometric Study of PCA-Based Gender Debiasing in Word Embeddings

structure: Linear Direction · models: T5-11B, UnifiedQA-11B (T5-based), T0++, GPT-J-6B, RoBERTa-large-MNLI, DeBERTa-xxlarge-v2-MNLI · paper: Discovering Latent Knowledge in Language Models Without Supervision

structure: Intrinsic-dimension profile across depth · models: Pythia-410M, Pythia-1.4B, Pythia-6.9B, Llama-3-8B, Mistral-7B · paper: Geometric Signatures of Compositionality Across a Language Model's Lifetime

structure: Intrinsic-dimension profile across depth, Linear Direction · models: GPT-2-Medium, Llama-3.1-8B-Instruct, Gemma-2-9B-it · paper: Shared Global and Local Geometry of Language Model Embeddings

structure: Linear Direction · models: Gemma 3 4B, Qwen3-8B · paper: Just-in-Time and Distributed Task Representations in Language Models

structure: Linear Separability · models: MAE ViT-Base (Masked Autoencoder), DINOv2 ViT-B/14, CLIP ViT-H/14, ConvNeXt (image classifier, various sizes, ImageNet) · paper: From Local Cues to Global Percepts: Emergent Gestalt Organization in Self-Supervised Vision Models

structure: Linear Subspace, Linear Direction · models: CodeLlama-13B, Gemma-2-2B, Llama 3.1 70B · paper: Do Language Models Track Entities Across State Changes?

structure: 1D continuum manifold · models: Pythia-2.8B, Llama-2-7B, Llama-3.1-8B, Llama-3.2-1B, GPT-2-Large, Mistral-7B, Llama-3.1-8B-Instruct, Llama-3.2-1B-Instruct, Qwen2.5-3B-Instruct, Qwen2.5-3B, Llama-3.2-3B-Instruct, Llama-3.2-3B, Gemma-2-2B-it, Gemma-2-2B, Llama-3.1-70B-Instruct, Llama-3-8B-Instruct, Llama-3-8B, Mistral-7B-Instruct-v0.3, Qwen2.5-7B-Instruct · paper: Number Representations in LLMs: A Computational Parallel to Human Perception, Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling, Weber's Law in Transformer Magnitude Representations: Efficient Coding, Representational Geometry, and Psychophysical Laws in Language Models, LLMs Know More About Numbers than They Can Say

structure: Intrinsic-dimension profile across depth · models: ConvNeXt (image classifier, various sizes, ImageNet), ResNet (image classifier, various depths), Vision Transformer (ViT, image classifier, various sizes, ImageNet), ResMLP (image classifier, various sizes, ImageNet) · paper: Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and the Human Brain

structure: Linear Subspace · models: Llama-3.2-3B, Llama-3-8B · paper: Do Linear Probes Generalize Better in Persona Coordinates?

structure: Polytope (Simplex), Belief State Geometry Hypothesis (Mixed-State Presentation) · models: Bayesian Wind Tunnel Mamba (HMM filtering task, 9 layers, d_model=256, state dim 16) · paper: The Bayesian Geometry of Transformer Attention

structure: Linear Direction · models: Molmo-7B-O, NVILA-Lite-2B, Qwen2.5-VL-3B-Instruct, RoboRefer-2B-SFT, Qwen3-VL-235B-A22B-Instruct · paper: Why Far Looks Up: Probing Spatial Representation in Vision-Language Models

structure: Linear Subspace · models: LSTM (sentiment classification, Yelp/IMDB/SST), GRU (sentiment classification, Yelp/IMDB/SST), Update Gate RNN (sentiment classification, Yelp/IMDB/SST), Vanilla RNN (sentiment classification, Yelp/IMDB/SST) · paper: How Recurrent Networks Implement Contextual Processing in Sentiment Analysis

structure: Torus, Circle · models: Grokking Modular-Arithmetic Transformer (1 layer, 4 heads, d_model=128, mod 113), Clock/Pizza Transformer, Model A (1 layer, constant attention alpha=0, width 128, mod 59), Clock/Pizza Transformer, Model B (1 layer, normal attention alpha=1, width 128, mod 59) · paper: Progress Measures for Grokking via Mechanistic Interpretability, The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks, On the Geometry and Topology of Representations: The Manifolds of Modular Addition

structure: Anisotropy · models: word2vec (Google News, 300d), GloVe (840B token Common Crawl) · paper: All-but-the-Top: Simple and Effective Postprocessing for Word Representations

structure: Linear Direction · models: Llama-3.1-8B, Mistral-7B-v0.1 · paper: How Language Models Process Negation

structure: Circle · models: NLLB-200 (distilled, 600M) · paper: Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry

structure: Concept Crystals (Parallelogram/Trapezoid Structure) · models: NLLB-200 (distilled, 600M) · paper: Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry

structure: Linear Subspace · models: Custom GPT-2-Small (trained from scratch on synthetic grid-navigation token sequences) · paper: Cognitive Maps in Language Models: A Mechanistic Analysis of Spatial Planning

structure: Linear Direction, Polytope (Simplex) · models: Pythia-160M · paper: (How) Do Language Models Track State?

structure: Circle · models: LSTM VAE (371 J.S. Bach chorales, 6 compared input encodings: piano roll, MIDI-like, ABC, Tonnetz, Pitch DFT, Pitch-Class DFT) · paper: Exploring Latent Spaces of Tonal Music using Variational Autoencoders

structure: Tree Metric Embedding, Concept Crystals (Parallelogram/Trapezoid Structure) · models: Llama-2-7B, Mistral-7B-v0.1, BERT-large · paper: A polar coordinate system represents syntax in large language models

structure: Intrinsic-dimension profile across depth · models: Qwen1.5-0.5B, Qwen2-0.5B, Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Llama-3-8B · paper: The Geometry of Reasoning: Flowing Logics in Representation Space

structure: Linear Direction · models: Gemma-2-9B-it, Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct · paper: Refusal Direction is Universal Across Safety-Aligned Languages

structure: Linear Direction · models: Llama-3-70B-Instruct, Gemma-7B-it, Qwen-1.8B-Chat, Vicuna-13B, Qwen1.5-1.8B-Chat, Qwen1.5-32B-Chat, Llama-2-13B-Chat, Llama-3.1-8B-Instruct, NeuralDaredevil-8B-abliterated, Hermes-2-Pro-Llama-3-8B, OLMo-7B-SFT, Zephyr-7B-Beta, H2O-Danube3-4B-Chat, Gemma-2-9B-it, Qwen2.5-7B-Instruct · paper: Refusal in Language Models Is Mediated by a Single Direction, Representation Engineering: A Top-Down Approach to AI Transparency, Programming Refusal with Conditional Activation Steering, Refusal Direction is Universal Across Safety-Aligned Languages

structure: Linear Subspace · models: Llama-3.1-8B, Llama-3.1-8B-Instruct, OLMo 2 7B, OLMo 2 7B Instruct, Ministral-8B-Instruct-2410 · paper: Emotions Where Art Thou: Characterizing the Emotional Latent Space of LLMs

structure: 1D continuum manifold · models: Gemma-2-2B, EmbeddingGemma, word2vec (trained on Wikipedia), Claude 3.5 Haiku · paper: Symmetry in Language Statistics Shapes the Geometry of Model Representations, When Models Manipulate Manifolds: The Geometry of a Counting Task

structure: Circle, Affine Subspace, Paraboloid (circular × pinched continuum) · models: Llama-3.1-8B · paper: Do Sparse Autoencoders Capture Concept Manifolds?

structure: Linear Subspace · models: Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, Qwen2.5-7B-Instruct · paper: Understanding and Preserving Safety in Fine-Tuned LLMs

structure: Linear Direction · models: GPT-2 Small, Pythia-1.4B, Pythia-2.8B · paper: Linear Representations of Sentiment in Large Language Models

structure: Line Attractor · models: LSTM (sentiment classification, Yelp/IMDB/SST), GRU (sentiment classification, Yelp/IMDB/SST), Update Gate RNN (sentiment classification, Yelp/IMDB/SST), Vanilla RNN (sentiment classification, Yelp/IMDB/SST) · paper: Reverse Engineering Recurrent Networks for Sentiment Classification Reveals Line Attractor Dynamics

structure: Linear Direction, Linear Separability · models: Llama-2-7B, Llama-2-13B, Llama-2-70B, Llama-3-8B, Llama-3-70B, Gemma-2B, Gemma-7B · paper: Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning

structure: 1D continuum manifold · models: 1-Layer Ordinal Local-Comparison Transformer, Qwen2.5-1.5B · paper: Emergent Ordinal Geometry in Transformers Trained on Local Comparisons

structure: Linear Direction · models: SDXL-DMD (4-step distilled), Stable Diffusion XL, SDXL Turbo, FLUX.1 [schnell] · paper: SliderSpace: Decomposing the Visual Capabilities of Diffusion Models

structure: Linear Direction, Concept Crystals (Parallelogram/Trapezoid Structure) · models: Llama-3.2-3B-Instruct, Llama-3.2-1B-Instruct, Qwen3-1.7B · paper: Linear Spatial World Models Emerge in Large Language Models

structure: Linear Subspace, Linear Direction · models: Gemma-2-2B-it, Llama-2-7B-Chat · paper: The Cylindrical Representation Hypothesis for Language Model Steering

structure: Tangent-Aligned Anisotropy Hypothesis, Linear Subspace · models: EuroBERT-210m, EuroBERT-610m, Pythia-160M, Pythia-410M, Pythia-1B, Pythia-1.4B, SmolLM2-360M, SmolLM2-1.7B · paper: Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics

structure: Linear Direction · models: GraphCast (weather foundation model) · paper: Mechanistic Interpretability Tool for AI Weather Models

structure: Linear Direction · models: Qwen3-4B-Instruct-2507 · paper: Temporal Preference Concepts and Their Functions in a Large Language Model

structure: Linear Separability, Linear Direction · models: Mistral Small 3 (2501), Mistral-7B, Llama-3.1-8B, Llama-3.1-8B-Instruct, Llama-3.2-3B, Gemma-2-9B, Gemma-2-2B, GPT-J-6B, GPT-2 XL, GPT-2 Large, GPT-2 Medium, GPT-2 Small · paper: Large Language Models Encode Semantics and Alignment in Linearly Separable Representations

structure: Circle, Cone, Platonic Representation Hypothesis · models: GPT-2-small, Mistral-7B, Llama-3-8B, Gemma-2-2B, EmbeddingGemma, word2vec (trained on Wikipedia), Qwen2.5-3B-Instruct, Qwen2.5-3B, Llama-3.2-3B-Instruct, Llama-3.2-3B, Gemma-2-2B-it, Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, Llama-3.1-8B · paper: Symmetry in Language Statistics Shapes the Geometry of Model Representations, Not All Language Model Features Are One-Dimensionally Linear, Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling, Do Sparse Autoencoders Capture Concept Manifolds?

structure: Cone · models: Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, Gemma-2-2B-it, Gemma-2-9B-it · paper: From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs

structure: Linear Direction · models: Llama-2-7B, Llama-2-13B, Llama-2-70B, Llama-2-7B-Chat, Llama-2-13B-Chat, Mistral-7B-v0.1 · paper: The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets, On the Universal Truthfulness Hyperplane Inside LLMs

structure: Circle, Linear Direction · models: Llama-3.1-8B-Instruct, Qwen3-8B, Qwen3-14B, Apertus-8B-Instruct-2509, Gemma 4 E4B-it · paper: Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control, Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

structure: Linear Centroids Hypothesis · models: ResNet-50 (supervised, ImageNet), DINOv2 ViT-L/14, DINOv3 ViT-B/16, GPT-2-Large, Llama-3.1-8B · paper: The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts

structure: Linear Separability · models: Qwen2.5-7B-Instruct, Llama-3.1-8B, Falcon3-7B · paper: Monitoring Emergent Reward Hacking During Generation via Internal Activations

structure: Linear Subspace · models: 12-layer, 8-head causal Transformer (d_model=512, RoPE, synthetic variable-assignment-program task) · paper: How Do Transformers Learn Variable Binding in Symbolic Programs?

structure: Linear Direction · models: Qwen3-32B, Llama 3.3 70B Instruct · paper: Rhetorical Questions in LLM Representations: A Linear Probing Study

structure: Linear Separability · models: Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Llama-3.2-1B, Llama-3.2-3B, Llama-3.1-8B · paper: Decoding Emotion in the Deep: A Systematic Study of How LLMs Represent, Retain, and Express Emotion