Custom Research Transformer (small, purpose-built for interpretability studies)

various (academic)

Structures found in this family (7)

Linear Subspace, Circle, Intrinsic-dimension profile across depth, Dimensional collapse, Curvature profile of the representation manifold, Polytope (Simplex), Representation-Weight-Gradient Alignment (Canonical Representation Hypothesis)

By model (12)

Custom 3-layer decision-pretraining Transformer (Fang & Rajan 2026) · 3 layers, 512-dim embeddings, trained via decision-pretraining (DPT) meta-RL on gridworld and tree-maze tasks

Linear Subspace

Self-supervised masked-reconstruction transformer on Hierarchical Object Dataset

Linear Subspace

Small GPT-2-style transformer (2-layer, 1-head, from scratch, numeric/continuous ICL parameterization tasks)

Linear Subspace

LLaMA-style decoder-only language models trained from scratch (BLiMP grokking-analogue study) · 35M, 60M, 130M

Linear Subspace

Custom Transformer (trained from scratch on a structured synthetic knowledge graph) · small (research-scale)

Circle

Pereira & Zuidema 2026 GPT-2-style Tower of Hanoi transformer (6 layers, d=128) · 6 layers, hidden 128

Linear Subspace

Custom controlled 7B Transformer (trained from scratch for architecture/normalization ablations) · 7B

Dimensional collapse

Graph Transformer (GT/GraphiT/SAN), trained on LRGB peptides-func/peptides-struct (Tori, Bini, Sorbi, Marchand-Maillet & Ginis)

Curvature profile of the representation manifold

TS-51M (custom 8-layer x 512d x 16-head transformer, TinyStories, 6 pretraining seeds) · 51M

Intrinsic-dimension profile across depth

Custom GPT-2-style causal transformer (205M, TinyStories, largest of a 3.4M-205M width/depth/epoch grid) · 205M

Polytope (Simplex)

Ziyin et al. 2024 6-Layer Transformer (OpenWebText, from scratch)

Representation-Weight-Gradient Alignment (Canonical Representation Hypothesis)

Custom ViT (10-layer, 8-head, 384-dim, trained from scratch with VICReg/SimCLR objectives on CIFAR-100/FOOD101) · 10-layer, 384-dim

no structures recorded for this checkpoint specifically

Observations (11)

A bespoke small GPT-2-style transformer (2-layer, 1-head), trained from scratch on in-context-learning tasks parameterized by a latent numerical/continuous concept, has add-k task vectors on a 1D linear manifold (PC1 > 99.9% of variance, offsets ordered, interpolation steers k); its Circular-Trajectory task vectors lie on a smooth low-dimensional manifold that the paper does not call linear

An in-context-RL transformer linearly decodes XY position by layer 2

Self-supervised transformer on synthetic hierarchical objects encodes object-membership abstractions in low-dimensional, linearly readable directions

In LLaMA-style language models trained from scratch, grammatical concept vectors become more predictive of acceptability and occupy a higher-dimensional subspace after delayed (grokking-like) generalization on BLiMP phenomena

Knowledge editing shatters transformers' natural cyclic manifold geometry

A participation-ratio spectral signal finds attention circuits label-free across scale

Small Tower-of-Hanoi transformers, Qwen3.6-27B and DeepSeek-R1-Distill-Qwen-32B linearly encode a geometrically faithful Sierpinski-triangle state-space representation, and in the large models it decays over long planning

SwiGLU spikes collapse keys into a sink-forming subspace

A real trained graph transformer's attention-weighted effective graph undergoes curvature collapse, concentrating on and worsening bottleneck edges rather than smoothing them

Neural-Collapse signatures emerge in causal language models, but not self-duality

Latent representations, weights, and neuron gradients become mutually aligned during training, unifying Neural Collapse and the Neural Feature Ansatz

Papers

Latent Concept Disentanglement in Transformer-based Language Models (2025), From Memories to Maps: Mechanisms of In-Context Reinforcement Learning in Transformers (2026), Emergence and Function of Abstract Representations in Self-Supervised Transformers (2023), A Pre-Training Analogue of Grokking in Language Models: Tracing Delayed Grammatical Generalization (2026), Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing (2024), Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers (2026), Transformers Struggle to Use Their Emergent World Models: Revisiting the Tower of Hanoi, and the Illusion of Thinking (2026), The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks (2026), Probing Graph Neural Network Activation Patterns Through Graph Topology (2026), Linguistic Collapse: Neural Collapse in (Large) Language Models (2024), Formation of Representations in Neural Networks (2024)