Pythia

Representational Analysis of Binding in Language Models (2024), The Communication Map of a Transformer (2026), Superposition as Lossy Compression: Measure with Sparse Autoencoders and Connect to Adversarial Vulnerability (2025), Functional Subspace, where language models can use vector algebra to solve problems (2026), Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning (2025), Investigating Representation Universality: Case Study on Genealogical Representations (2024), Transferring Linear Features Across Language Models With Model Stitching (2025), Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs (2024), Language Models Represent and Transform Concepts with Shared Geometry (2026), Polar probe linearly decodes semantic structures from LLMs (2026), On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders (2026), Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models (2025), How Do Language Models Bind Entities in Context? (2023), Function Vectors in Large Language Models (2024), In-Context Learning Creates Task Vectors (2023), Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (2025), Do Different Prompting Methods Yield a Common Task Representation in Language Models? (2025), Persistent Topological Features in Large Language Models (2024), Improving Dictionary Learning with Gated Sparse Autoencoders (2024), Language Models Represent Space and Time (2024), Symmetry in Language Statistics Shapes the Geometry of Model Representations (2026), Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence (2025), Inference-Time Causal Probing in LLMs (2026), Language Models Use Trigonometry to Do Addition (2025), Representational Curvature Modulates Behavioral Uncertainty in Large Language Models (2026), Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoders (2024), Semantic Convergence: Investigating Shared Representations Across Scaled LLMs (2025), Abstraction Induces the Brain Alignment of Language and Speech Models (2026), Geometric Signatures of Compositionality Across a Language Model's Lifetime (2024), Finding Lexical Identity and Inflectional Morphology in Modern Language Models (2025), Rethinking Intrinsic Dimension Estimation in Neural Representations (2026), Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models (2026), Number Representations in LLMs: A Computational Parallel to Human Perception (2025), Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling (2025), Weber's Law in Transformer Magnitude Representations: Efficient Coding, Representational Geometry, and Psychophysical Laws in Language Models (2026), LLMs Know More About Numbers than They Can Say (2026), Laguerre Geometry for Interpreting Large Language Models (2026), Latent Semantic Manifolds in Large Language Models (2026), Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformers (2026), Attention Sinks and Compression Valleys in LLMs Are Two Sides of the Same Coin (2025), Talking Heads: Understanding Inter-layer Communication in Transformer Language Models (2024), How Reliable are Causal Probing Interventions? (2025), LEACE: Perfect Linear Concept Erasure in Closed Form (2023), Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers (2026), (How) Do Language Models Track State? (2025), Outlier Dimensions Encode Task-Specific Knowledge (2023), Tracing the Representation Geometry of Language Models from Pretraining to Post-training (2025), The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models (2024), Token Embeddings Violate the Manifold Hypothesis (2025), Linear Representations of Sentiment in Large Language Models (2023), Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks (2026), Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics (2026), Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings (2026), Measuring Intrinsic Dimension of Token Embeddings (2025), Transcoders Find Interpretable LLM Feature Circuits (2024), What are you sinking? A geometric approach on attention sink (2025), Eliciting Latent Predictions from Transformers with the Tuned Lens (2023), The Intrinsic Dimension of Prompts in Internal Representations of Large Language Models (2025), Probing Geometry of Next-Token Prediction Using Cumulant Expansion of the Softmax Entropy (2025), Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning (2026), Scale Determines Whether Language Models Organize Representation Geometry for Prediction (2026), Which Attention Heads Matter for In-Context Learning? (2025), When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception (2026)