Dimensional collapse

structures / Linear Structures / Dimensional collapse

PROPERTYdistribution-propertyintermediatehow it's classified →

A representation space exhibits dimensional collapse if the embedding covariance matrix's singular-value spectrum decays toward zero well before the ambient dimension, so the population of embedding vectors spans a genuine lower-dimensional linear subspace rather than the full available space — a rank-of-the-cloud property, distinct from anisotropy's directional-concentration property of the same cloud.

Replicationcomputed from the corpus — never hand-assigned

16 papers · no shared authors6 architecture classes · across papers4 domains · across papers24 model families · across papers

Filled = two or more values reported by papers that share no author — replication. Outlined = two or more values, but all from a single study — breadth, not replication. Grey = a single value. Derived from paper authorship and each model's architecture class, domain and family; it updates itself when a paper is added.

Definition

Given a set of embedding vectors $Z=\{z_1,\dots,z_n\}\subset\mathbb{R}^d$ with covariance $C=\frac1n\sum_i (z_i-\bar z)(z_i-\bar z)^\top$ , decompose $C=U\Sigma U^\top$ . Dimensional collapse is the empirical finding that a substantial number of the sorted singular values $\sigma_1 \ge \sigma_2 \ge \dots \ge \sigma_d$ decay toward (numerical) zero — the embedding population occupies an effective linear subspace of dimension $\ll d$ , not merely a directionally-skewed cloud that still uses every dimension.

Relative to anisotropy

Anisotropy and dimensional collapse are both distribution-property statistics of the same activation cloud, but they measure different things: anisotropy is about directional concentration (do vectors point similarly, measured via mean pairwise cosine or the mean-direction norm) and survives even at full rank (a full-rank cloud can still be directionally skewed); dimensional collapse is about rank (how many singular values of the covariance are non-negligible) and can occur even in a directionally-isotropic cloud restricted to a genuine subspace. Anisotropy is dominated by outlier directions; dimensional collapse is dominated by outlier (near-zero) singular values — a cone vs. a flat disk, in the ambient space.

Key evidence

Jing, Vincent, LeCun & Tian (2022) find dimensional collapse in contrastive self-supervised learning (SimCLR, ResNet-50, ImageNet): sorting singular values of the embedding covariance and plotting $\log \sigma_k$ against rank index $k$ shows a number of singular values collapsing to numerical zero, both in the 128-dim contrastive embedding head and in the 2048-dim backbone representation itself when the projector head is removed. They prove two independent causes for a linear model $z=Wx$ trained with InfoNCE under gradient flow: (1) strong augmentation — when augmentation variance along a direction exceeds the data’s own variance along it, gradient flow drives $W$ ‘s weight toward zero along that direction (Lemma 1); (2) implicit regularization — for an over-parameterized multi-layer linear network ( $W_2 W_1$ ), SGD dynamics push the product’s singular values toward low rank even under weak augmentation, an effect specific to networks with more than one layer (Corollary 2), verified in a toy 16x16 linear MLP. See contrastive-ssl-embeddings-undergo-provable-rank-collapse-of-the-covariance-spectrum-under-strong-augmentation-and-multilayer-implicit-regularization.

Wang, Ge, Shu, He & Qiu (2025) find dimensional collapse within a single transformer, comparing activation types rather than training regimes: computing SVD-based intrinsic dimension (smallest number of directions needed for 99% of variance) for attention-block outputs, MLP outputs, and residual-stream activations, attention outputs consistently occupy only about 60% of the ambient dimension across layers, model families and datasets, while MLP outputs and the residual stream remain near full rank at about 90% — a gap traced to the attention output projection matrix $W^O$ compressing the concatenated per-head outputs. Evaluating the open-source LlamaScope SAE suite on Llama-3.1-8B shows the number of dead SAE features tracks this per-layer intrinsic dimension: lower-intrinsic- dimension (more collapsed) activations have systematically more dead features, and constraining SAE feature initialization to the activations’ own active subspace (rather than random init) cuts dead features from 87% to below 1% in a 1M-feature attention-output SAE. See attention-outputs-occupy-a-60-percent-intrinsic-dimension-subspace-versus-90-percent-for-mlp-and-residual-stream-and-subspace-constrained-sae-initialization-cuts-dead-features-from-87-percent-to-below-1-percent.

Li, Agrawal, Ghosh, Teru, Santoro, Lajoie & Richards (2025) track dimensional collapse along a third axis — training time itself, across pretraining checkpoints of OLMo (1B/7B, OLMo-2 1B/7B) and Pythia (160M-12B) — using RankMe (Von Neumann entropy of the covariance eigenvalue distribution) and alpha-ReQ (the eigenspectrum’s power-law decay exponent). Both statistics trace the same non-monotonic three-phase trajectory in every model and every layer tested: a “warmup” collapse, an “entropy-seeking” expansion that tracks rising n-gram memorization, and a “compression-seeking” phase of anisotropic consolidation (variance preserved along dominant eigendirections, contracted — not zeroed — along others) that tracks a sharp downstream-accuracy improvement (SciQ correlations up to r=0.89 with RankMe, p<3e-4, across model scales). A formal linear-encoder derivation (proven theorems, not just curve-fitting) shows this three-phase shape requires both a skewed token-frequency distribution and a representational bottleneck $d\ll|V|$ ; removing either condition eliminates compression- seeking. Post-training pushes the same spectral statistics in opposite directions: SFT/DPO drive further entropy-seeking (better in-distribution, worse out-of-distribution performance) while RLVR drives compression- seeking (better reward-aligned accuracy, less generation diversity). See pretraining-passes-through-three-effective-rank-phases-tracking-n-gram-memorization-and-downstream-performance-explained-by-cross-entropy-plus-bottleneck-theory.

Queipo-de-Llano, Arroyo, Barbero, Dong, Bronstein, LeCun & Shwartz-Ziv (2025) prove that massive residual-stream activations necessarily induce representational compression (tight entropy-reduction and spectral-dominance bounds), and track normalized representation entropy, sink rate, and BOS-token norm across Pythia 410M/6.9B training checkpoints (steps 1k-143k): compression valleys, attention sinks, and massive activations emerge synchronized around step 1k and persist, with the layer index of emergence fixed per model. Staged causal ablation of the massive-activation mechanism progressively — and, when fully ablated, completely — eliminates the compression valley, confirming the theoretical link; broader validation extends to GPT-OSS-20B and Gemma-7B. See massive-residual-stream-activations-provably-cause-representational-compression-valleys-that-emerge-synchronized-with-attention-sinks-and-are-eliminated-by-staged-causal-ablation.

Vemula (2026) tests whether massive activations can be architecturally designed away: training real decoder-only transformers from scratch on FineWeb-Edu (160M/290M parameters) under four matched configurations — vanilla, QK-normalized “suppress”, a novel Ledger-Residuals architecture that splits the residual stream into a mutable Deliberation stream and a protected, decode-only Commitment stream, and a Commitment variant with a 6x stronger commit-sparsity penalty. QK-normalization essentially removes the effect (fixed-dimension ratio 28.3 -> 6.8, start-token concentration 1.01 -> 0.13 at 290M), but the architecturally-isolated Commitment stream rebuilds its own massive activation at a different dimension (persistence 0.58, ratio 7.0, concentration 3.36), and a stronger sparsity penalty makes the re-emerged activation worse, not better (persistence rising to 0.96). The same qualitative re-emergence replicates at 160M parameters. This is evidence that massive activations are not merely an artifact of one particular residual-stream design choice removable by architectural isolation alone, complementing Sun et al.’s mechanistic account below of why they form in standard architectures. See vemula-2026-massive-activations-re-emerge-in-real-from-scratch-trained-transformers-under-a-protected-commitment-residual-stream-and-are-only-removed-by-qk-normalization.

Sun, Canziani, LeCun & Zhu (2026) trace the mechanism one level deeper: SwiGLU feed-forward blocks act as directional quadratic amplifiers producing massive activations along a shared “spike direction,” and pre-norm RMSNorm’s bounded-range property collapses these tokens’ normalized key vectors into a low-dimensional, near-constant subspace that “sink heads’” query subspaces geometrically align with — a mechanistic account of attention-sink formation as geometric alignment to a collapsed key subspace, rather than semantic relevance. Systematic causal ablations on a controlled 7B model trained from scratch (SwiGLU->GeLU/linear, Sandwich Norm/QKNorm/DynamicTanh, explicit gating, training-context-length manipulation) show spikes and sinks can be independently suppressed — e.g. DynamicTanh eliminates spikes without hurting sinks or perplexity, while long-context-only training sharply reduces sink ratio. See swiglu-driven-massive-activations-collapse-into-a-low-dimensional-spike-key-subspace-that-geometrically-explains-attention-sink-formation-and-is-independently-suppressible-via-normalization-swaps.

A latent-reasoning-model study applies a four-metric shape suite (Latent-trajectory shape metrics (entropy, effective rank, anisotropy, intrinsic dimension): entropy, effective rank, anisotropy, intrinsic dimension) to Huginn-3.5B’s iterated latent-thought states, finding incorrect reasoning trajectories collapse into flatter, lower-entropy, lower-anisotropy regions of latent space relative to correct trajectories, which instead trace a richer, higher-dimensional path. The discovered correctness-predictive geometry is then used as a training-time reward signal (Latent Thinking Optimization), rather than via a direct steering/ablation intervention, so the finding rests on criterion (a) alone. See huginn-3-5bs-correct-latent-thought-trajectories-are-geometrically-distinguishable-from-incorrect-ones-via-entropy-effective-rank-anisotropy-and-intrinsic-dimension.

Li, Jia, Zhang, Li, Shi, Zhou, Zhu, Shen & Qu (2026) apply self-supervised evaluation principles to diffusion models, decomposing internal features at each noise level into an invariant component and a residual component and computing the Invariant Contamination Ratio (a Fisher-based measure of how much residual variation contaminates the invariant signal). In a real pretrained diffusion model (Stable Diffusion), invariance peaks at intermediate noise levels — the same levels that yield the best downstream classification performance — and ICR tracks generative quality without sampling and anticipates memorization onset during training. See invariant-contamination-ratio-in-diffusion-model-features-identifies-optimal-noise-level-generative-quality-and-memorization-onset.

Grzywaczewski, Płudowski & Biecek (2026) precisely quantify the collapsed (“noise”) subspace dimensionality in six real pretrained CLIP/SigLIP vision-language models via a sharp covariance-eigenspectrum decay elbow — ranging from 3% of ambient dimensions (CLIP ViT-B/32) up to 40% (SigLIP2 ViT-L/16) — and show this noise subspace is shared across unrelated ImageNet classes (over 90% cross-class overlap in the two largest models). Causally projecting the identified noise subspace away leaves zero-shot accuracy essentially unchanged (e.g. CLIP ViT-L/14: 91.2%→91.1%), while removing an equal number of random directions measurably degrades it — a clean geometry-tied ablation distinguishing the collapsed subspace from an arbitrary same-sized one. See clip-siglip-embedding-covariance-eigenspectrum-splits-into-a-signal-subspace-and-a-shared-noise-subspace-that-safely-ablates-away.

Anderson (2026) tracks PCA effective dimension ( $d_{95}$ ) along chain-of- thought reasoning trajectories in real Llama-3-8B-Instruct and Llama-3.1-70B-Instruct, finding domain-specific collapse: legal-reasoning trajectories’ $d_{95}$ falls 501->274 (45%) from the 8B to the 70B model, a pattern the paper calls “Crystallization,” distinct from code reasoning’s discrete-cluster-formation pattern at the same two scales (see Concept Cluster Heterogeneity). Single-author preprint; observational only. See anderson-2026-legal-reasoning-trajectories-in-real-llama-3-8b-instruct-and-llama-3-1-70b-instruct-undergo-a-45-percent-collapse-in-effective-dimensionality-with-scale.

Jiang, Nguyen, Aeron & Ishwar (2024) show Dimensional Collapse and Neural Collapse (see Polytope (Simplex)) are two distinct, empirically distinguishable failure/success modes of the same contrastive-learning training setup: a real ResNet-50 trained via Adam on real CIFAR-100 reaches the Neural-Collapse Simplex-ETF geometry only when hard-negative sampling and unit-ball/unit-sphere feature normalization are both present; omitting either instead produces Dimensional Collapse, measured via the same representation’s covariance spectrum failing to reach full effective rank. See jiang-etal-2023-a-real-resnet-50-trained-with-hard-negative-contrastive-losses-and-unit-ball-normalization-converges-to-neural-collapse-etf-geometry-on-cifar100-while-omitting-either-yields-dimensional-collapse-instead.

Sadok & Alameda-Pineda (2026, InsideSSL) track von Neumann entropy of the per-layer Gram matrix — a compression/collapse-adjacent statistic distinct from the covariance-rank measurements elsewhere in this node — across real Wav2Vec2, WavLM, HuBERT, and Data2Vec-audio speech-SSL models: most models sustain high entropy throughout the network, but Wav2Vec2-base alone undergoes a sharp entropy collapse toward near-baseline levels at layer 11, coinciding with a spike in an InfoNCE-based invariance-loss measure — a training-objective-specific collapse event distinguishing one architecture’s optimization regime from the others tested. See sadok-alameda-pineda-2026-real-speech-ssl-models-wav2vec2-wavlm-hubert-and-data2vec-show-distinct-layerwise-entropy-and-trajectory-curvature-regimes-with-wav2vec2-undergoing-a-late-layer-entropy-collapse.

Zolfaghari (2026) probes five real LoRA-fine-tuned honest/deceptive model pairs (Pythia-1.4B, Gemma-2-2B/9B, Qwen2.5-7B, Llama-3.1-8B) with a mechanistic geometry suite (effective rank, participation ratio, Fisher Discriminant Ratio, centroid distance, adjacent-layer direction cosine) applied to the linearly-decodable honest-vs-deceptive activation cloud, finding an architecture-dependent bifurcation: Pythia, Llama, and Qwen collapse the deception-direction cloud to near-rank-1 effective rank (1.06-1.07), while Gemma-2 retains a much higher effective rank (60-234) for the same labeled contrast — the same collapse-vs-distributed split documented elsewhere on this node for other labeled concepts, here shown to hold even though linear-probe decodability itself is uniformly near-ceiling (AUC>=0.99) across all five architectures. See zolfaghari-2026-a-real-multi-architecture-study-finds-the-honest-vs-deceptive-activation-cloud-collapses-to-near-rank-1-effective-rank-in-pythia-llama-and-qwen-but-stays-high-rank-in-gemma-2-despite-uniformly-near-ceiling-linear-probe-decodability.

How to detect it

Compute the covariance matrix of a batch of embeddings, take its SVD, and plot sorted log-singular-values against rank index; a sharp knee/plateau well before the ambient dimension indicates dimensional collapse. Compare against a randomly-initialized network’s spectrum as a control.

Key papers

Jing, Vincent, LeCun & Tian (2022). Understanding Dimensional Collapse in Contrastive Self-supervised Learning. ICLR 2022, arXiv:2110.09348 — origin of the term and its dual (augmentation / implicit-regularization) theoretical account for contrastive SSL.
Wang, Ge, Shu, He & Qiu (2025). Attention Layers Add Into Low- Dimensional Residual Subspaces. arXiv:2508.16929 — attention outputs collapse to ~60% intrinsic dimension vs. ~90% for MLP/residual, traced to $W^O$ , with a practical SAE-initialization fix for the resulting dead-feature problem.
Li, Agrawal, Ghosh, Teru, Santoro, Lajoie & Richards (2025). Tracing the Representation Geometry of Language Models from Pretraining to Post-training. arXiv:2509.23024 — RankMe/alpha-ReQ trace a proven, three-phase collapse/expansion/anisotropic-compression trajectory across pretraining and post-training.
Vemula, M. (2026). Massive Activations Are Architecturally Robust: A Controlled Scratch/Commitment Residual Stream Test. arXiv:2606.20743 — massive activations re-emerge under an architecturally-isolated Commitment residual stream in real from-scratch-trained transformers.

Found in (16 observations · 24 families)

Achilli/Pham et al. Diffusion Memorization-Transition DDPMs

Losing Dimensions: Geometric Memorization in Generative Diffusion (2024)measured

Real trained DDPMs lose spectral-gap dimension smoothly as training-set size shrinks into a memorization regime

Details

Achilli, Ventura, Silvestri, Pham, Raya, Krotov, Lucibello & Ambrogioni retarget the score-Jacobian spectral-gap estimator from Ventura et al. (2025) -- built for the true (population) score -- to the empirical score of real trained DDPMs, finding new spectral gaps appear that only a theory of the empirical score, not the true score, predicts [achilli-etal-2024-losing-dimensions-geometric-memorization-diffusion] Across real DDPMs (PixelCNN++-backbone U-Nets, 24.5M-61.7M parameters) trained from scratch on MNIST, Fashion-MNIST, CIFAR-10, CelebA-HQ and LSUN-Church at 38 dataset-size splits per dataset, the spectral-gap-estimated local intrinsic dimension declines smoothly across a training-set-size window of roughly 1000-10000 samples (dataset-specific critical points A/B, e.g. CIFAR-10 A=2000/B=16000 of 50000 total) before collapsing toward zero below it [achilli-etal-2024-losing-dimensions-geometric-memorization-diffusion] A secondary participation-ratio diagnostic (effective active-sample count) tracks the same three empirical regimes -- generalization, geometric memorization, and full memorization -- as training-set size shrinks [achilli-etal-2024-losing-dimensions-geometric-memorization-diffusion]

models: DDPM (MNIST, custom-trained, memorization-transition study), DDPM (Fashion-MNIST, custom-trained, memorization-transition study), DDPM (CIFAR-10, custom-trained, memorization-transition study), DDPM (CelebA-HQ, custom-trained, memorization-transition study), DDPM (LSUN-Church, custom-trained, memorization-transition study) · method: Score-Jacobian spectral-gap analysis

Memorization to Generalization: Emergence of Diffusion Models from Associative Memory (2025)measured

Diffusion-model score-Jacobian curvature maps onto a dense associative-memory energy landscape whose memorized/spurious/generalized transition scales with model capacity

Details

Pham, Raya, Negri, Zaki, Ambrogioni & Krotov map a diffusion model's own energy function E_DM(x_t,t) directly onto a Dense Associative Memory (Hopfield-network) energy E_AM(x), with noise variance playing the role of inverse temperature, and reuse the score-Jacobian SVD/spectral-gap curvature diagnostic from Ventura et al. (2025) and Achilli et al. (2024) to quantify local energy curvature around real DDPM training points [pham-etal-2025-memorization-generalization-diffusion-associative-memory] Across real DDPMs (PixelCNN++-backbone U-Nets) trained from scratch on MNIST, Fashion-MNIST, CIFAR-10 and LSUN-Church at 38 training-set sizes per dataset, nearest-neighbor-distance classification against real and synthetic sample sets identifies memorized, spurious, and generalized regimes whose transition point matches the dimensional-collapse critical points found on the same checkpoints by Achilli et al. (2024) (e.g. CIFAR-10 A=2000/B=16000 of 50000 total) [pham-etal-2025-memorization-generalization-diffusion-associative-memory] A U-Net capacity ablation (widths 64/96/128, 8.9M-109.7M parameters) on CIFAR-10 and LSUN-Church shows the critical training-set size at which the spurious-state fraction peaks scales up with model capacity (CIFAR-10 U-Net64 peak at K=2568 vs. U-Net96 peak at K=5862), directly tying the memorization-geometry transition to model capacity rather than dataset size alone [pham-etal-2025-memorization-generalization-diffusion-associative-memory]

models: DDPM (MNIST, custom-trained, memorization-transition study), DDPM (Fashion-MNIST, custom-trained, memorization-transition study), DDPM (CIFAR-10, custom-trained, memorization-transition study), DDPM (LSUN-Church, custom-trained, memorization-transition study) · method: Score-Jacobian spectral-gap analysis

Llama

The Geometry of Thought: How Scale Restructures Reasoning in Large Language Models (2026)measured

Legal-reasoning trajectories collapse ~45% in effective dimension with scale

Details

- Recording chain-of-thought hidden-state trajectories during legal reasoning, PCA effective dimensionality d95 (components for 95% variance) falls from 501 to 274 (a 45% collapse) from Llama-3-8B-Instruct to Llama-3.1-70B-Instruct. [anderson-2026-geometry-of-thought] - The paper frames legal reasoning as "Crystallization" - a global-dimensionality collapse, distinct from the discrete-cluster ("Lattice") pattern found in code reasoning. [anderson-2026-geometry-of-thought] - Observational only (a covariance-spectrum statistic on two real checkpoints); no causal intervention. [anderson-2026-geometry-of-thought]

models: Llama-3-8B-Instruct, Llama-3.1-70B-Instruct · method: PCA

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning (2025)measured

Attention outputs collapse to a ~60% low-rank subspace

Details

- Across model families, attention-block outputs occupy an intrinsic dimension of only ~60% of the ambient space (directions for 99% variance), versus ~90% for MLP outputs and the residual stream, traced to the attention output projection W^O. [wang-etal-2025-attention-low-rank-subspaces] - The dead-feature rate of SAEs tracks activation intrinsic dimension layer-by-layer: lower-dimensional activations yield more dead features. [wang-etal-2025-attention-low-rank-subspaces] - Active Subspace Initialization (constraining SAE weights to the activations' top singular vectors) cuts dead features from 87% to below 1% in 1M-feature attention-output SAEs while lowering reconstruction loss. [wang-etal-2025-attention-low-rank-subspaces] - Measured on GPT-2, Llama-3.1-8B, Gemma-2-9B, Qwen3-8B/4B and Pythia-160M/2.8B. [wang-etal-2025-attention-low-rank-subspaces]

models: Llama-3.1-8B · method: SVD, Sparse Autoencoders (SAE)

Tracing the Representation Geometry of Language Models from Pretraining to Post-training (2025)measured

Pretraining passes through three effective-rank phases

Details

- Two spectral statistics of last-token representations - RankMe (von Neumann effective rank) and alpha-ReQ (eigenspectrum power-law exponent) - trace a consistent three-phase trajectory across OLMo and Pythia: warmup collapse, entropy-seeking expansion, then compression-seeking consolidation. [li-etal-2025-tracing-representation-geometry] - The entropy-seeking phase tracks peaking short-context n-gram memorization; the compression-seeking phase coincides with a grokking-like rise in downstream (TriviaQA/SciQ) accuracy. Across Pythia 1B-12B and OLMo2-1B, SciQ accuracy correlates with alpha-ReQ at r=0.54-0.84 (p<2.6e-3) and RankMe at r=-0.62 to -0.89. [li-etal-2025-tracing-representation-geometry] - Ablating all but the top-k eigendirections destroys downstream accuracy (Pythia-1B SciQ 0.838 to 0.225 at top-10), so the full spectrum is load-bearing; a proven cross-entropy + bottleneck (d<<|V|, Zipfian) derivation reproduces the phases, and removing either condition or using MSE eliminates compression-seeking. [li-etal-2025-tracing-representation-geometry] - Post-training diverges: SFT/DPO (Tulu-3 pipeline on Llama-3.1-8B) push entropy-seeking while RLVR pushes compression-seeking, with matching diversity-vs-accuracy tradeoffs; the pretraining phases are reported as correlational. [li-etal-2025-tracing-representation-geometry]

models: Llama-3.1-8B, Llama-3.1-Tulu-3-8B-SFT, Llama-3.1-Tulu-3-8B-DPO · method: SVD

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks (2026)measured

SwiGLU spikes collapse keys into a sink-forming subspace

Details

- SwiGLU feed-forward blocks act as directional quadratic amplifiers, creating massive activations for tokens aligned with a shared "spike direction". [sun-etal-2026-the-spike-the-sparse-and-the-sink] - Pre-norm RMSNorm's bounded-range property (Theorem B.3) collapses these spike tokens' normalized key vectors into a low-dimensional near-constant subspace, and sink heads' query subspaces align with it - explaining attention sinks as geometric alignment, not semantic relevance. [sun-etal-2026-the-spike-the-sparse-and-the-sink] - Causal ablations on a controlled 7B model trained from scratch show spikes and sinks are separable: DynamicTanh eliminates spikes without hurting sinks or perplexity, and long-context-only training sharply reduces the sink ratio. [sun-etal-2026-the-spike-the-sparse-and-the-sink] - Broader analysis applied to pre-norm Llama-family models (approximated here by Llama-2-7B). [sun-etal-2026-the-spike-the-sparse-and-the-sink]

models: Llama-2-7B · method: Causal interventions (steering)

GPT-2

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning (2025)measured

Attention outputs collapse to a ~60% low-rank subspace

Details

models: GPT-2 · method: SVD, Sparse Autoencoders (SAE)

Gemma

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning (2025)measured

Attention outputs collapse to a ~60% low-rank subspace

Details

models: Gemma-2-9B · method: SVD, Sparse Autoencoders (SAE)

Attention Sinks and Compression Valleys in LLMs Are Two Sides of the Same Coin (2025)measured

Massive activations provably cause compression valleys synced with attention sinks

Details

- Queipo-de-Llano et al. prove (Theorem 1) that massive residual-stream activations necessarily induce representational compression, with tight bounds on entropy reduction and singular-value dominance. [queipodellano-etal-2025-attention-sinks-and-compression-valleys] - Across Pythia 410M and 6.9B training checkpoints (steps 1k-143k), compression valleys, attention sinks and massive activations emerge synchronized around step 1k and persist, at a fixed layer per model. [queipodellano-etal-2025-attention-sinks-and-compression-valleys] - Staged causal ablation of the massive-activation mechanism progressively - and fully, when all stages are ablated - eliminates the compression valley, confirming the causal link; validated further on GPT-OSS-20B and Gemma-7B. [queipodellano-etal-2025-attention-sinks-and-compression-valleys]

models: Gemma-7B · method: Causal interventions (steering)

When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception (2026)measured

The honest-vs-deceptive activation cloud collapses to near-rank-1 in some real architectures but not others

Details

Zolfaghari LoRA-fine-tunes five real model families (Pythia-1.4B, Gemma-2-2B/9B, Qwen2.5-7B, Llama-3.1-8B) into honest and deceptive variants and applies a mechanistic geometry suite (effective rank, participation ratio, Fisher Discriminant Ratio, centroid distance, adjacent-layer direction cosine) to the labeled honest-vs-deceptive activation cloud [zolfaghari-2026-multi-model-study-linear-representations-synthetic-deception] Pythia, Llama, and Qwen collapse the deception-direction cloud to near-rank-1 effective rank (1.06-1.07), while Gemma-2 retains a much higher effective rank (60-234) for the same labeled contrast, an architecture-dependent bifurcation [zolfaghari-2026-multi-model-study-linear-representations-synthetic-deception] Linear-probe decodability of the same honest-vs-deceptive contrast is uniformly near-ceiling (AUC >= 0.99) across all five architectures regardless of the rank-collapse split, showing decodability and rank structure are separate properties [zolfaghari-2026-multi-model-study-linear-representations-synthetic-deception]

models: Gemma-2-9B · method:

Qwen

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning (2025)measured

Attention outputs collapse to a ~60% low-rank subspace

Details

models: Qwen3-8B, Qwen3-4B · method: SVD, Sparse Autoencoders (SAE)

Pythia

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning (2025)measured

Attention outputs collapse to a ~60% low-rank subspace

Details

models: Pythia-160M, Pythia-2.8B · method: SVD, Sparse Autoencoders (SAE)

Attention Sinks and Compression Valleys in LLMs Are Two Sides of the Same Coin (2025)measured

Massive activations provably cause compression valleys synced with attention sinks

Details

models: Pythia-410M, Pythia-6.9B · method: Causal interventions (steering)

Tracing the Representation Geometry of Language Models from Pretraining to Post-training (2025)measured

Pretraining passes through three effective-rank phases

Details

models: Pythia-160M, Pythia-410M, Pythia-1B, Pythia-1.4B, Pythia-2.8B, Pythia-6.9B, Pythia-12B · method: SVD

When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception (2026)measured

The honest-vs-deceptive activation cloud collapses to near-rank-1 in some real architectures but not others

Details

models: Pythia-1.4B · method:

CLIP (Contrastive Language-Image Pretraining)

Your CLIP has 164 dimensions of noise: Exploring the embeddings covariance eigenspectrum of contrastively pretrained vision-language transformers (2026)measured

CLIP/SigLIP covariance splits into signal and shared-noise subspaces

Details

- The embedding covariance eigenspectrum of six CLIP/SigLIP models shows a sharp elbow (threshold ~10^-3.6) separating a semantic signal subspace from a shared noise subspace. [grzywaczewski-etal-2026-clip-covariance-eigenspectrum-noise] - The noise subspace's dimensionality is quantified per model (CLIP ViT-L/14 164/768 = 21%; SigLIP2 ViT-L/16 405/1024 = 40%) and is shared across unrelated classes (>50% overlap, >90% for the largest models). [grzywaczewski-etal-2026-clip-covariance-eigenspectrum-noise] - Projecting the noise subspace away leaves ImageNet zero-shot accuracy essentially unchanged (CLIP ViT-L/14 91.2% to 91.1%) while removing the same number of random directions degrades it (to 89.3%); text-image cosine similarity even improves. [grzywaczewski-etal-2026-clip-covariance-eigenspectrum-noise] - Tested on CLIP ViT-B/16, ViT-B/32, ViT-L/14, SigLIP ViT-B/16, SigLIP2 ViT-B/16 and ViT-L/16. [grzywaczewski-etal-2026-clip-covariance-eigenspectrum-noise]

models: CLIP ViT-B/16, CLIP ViT-B/32, CLIP ViT-L/14 · method: PCA, Causal interventions (steering)

SigLIP

Your CLIP has 164 dimensions of noise: Exploring the embeddings covariance eigenspectrum of contrastively pretrained vision-language transformers (2026)measured

CLIP/SigLIP covariance splits into signal and shared-noise subspaces

Details

models: SigLIP, SigLIP 2 · method: PCA, Causal interventions (steering)

SimCLR

Understanding Dimensional Collapse in Contrastive Self-supervised Learning (2022)measured

Contrastive SSL embeddings undergo provable covariance-spectrum rank collapse

Details

- Contrastive self-supervised embeddings (SimCLR, ResNet-50, ImageNet) undergo dimensional collapse: many singular values of the embedding covariance drop to numerical zero, in both the projection head and the backbone. [jing-etal-2022-dimensional-collapse] - Two causes are proven for a linear model under InfoNCE: strong augmentation drives weights toward zero wherever augmentation variance exceeds data variance (Lemma 1), and multilayer implicit regularization under SGD drives low rank even under weak augmentation (Corollary 2). [jing-etal-2022-dimensional-collapse] - The collapse is diagnosed via the log singular-value spectrum of the covariance; DirectCLR (a fixed low-rank projector) recovers accuracy (ImageNet linear-probe Top-1 66.5% full projector, 51.5% none, 62.7% DirectCLR). [jing-etal-2022-dimensional-collapse]

models: ResNet-50 (SimCLR contrastive pretraining, ImageNet) · method: SVD

Huginn

Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in its Latent Thoughts (2025)measured

Correct latent-reasoning trajectories are geometrically distinguishable in Huginn-3.5B

Details

- In the recurrent-depth model Huginn-3.5B, a four-metric shape suite (entropy, effective rank, anisotropy, intrinsic dimension) applied to latent-thought trajectories separates correct from incorrect reasoning. [latent-thinking-optimization-2025-latent-reasoning-encodes-reward-signals] - Correct trajectories show higher entropy, LOWER effective rank, and higher anisotropy/intrinsic dimension than incorrect ones - a richer but more consolidated latent region. [latent-thinking-optimization-2025-latent-reasoning-encodes-reward-signals] - The signal is reliable enough that a learned Latent Reward Model predicts answer correctness from latent geometry alone (no text decoding), and is used as a training-time reward (Latent Thinking Optimization). [latent-thinking-optimization-2025-latent-reasoning-encodes-reward-signals]

models: Huginn-3.5B (recurrent-depth latent-reasoning model) · method: Latent-trajectory shape metrics (entropy, effective rank, anisotropy, intrinsic dimension)

EDM

Evaluating the Representation Space of Diffusion Models via Self-Supervised Principles (2026)measured

Fisher invariant-contamination ratio marks a diffusion model's optimal noise level

Details

- The Invariant Contamination Ratio (ICR) is a Fisher-based signal-to-noise measure that splits diffusion features into an invariant component (shared across noisy views) and a residual, scoring how much residual contaminates the invariant signal at each noise level. [li-etal-2026-evaluating-representation-space-diffusion-models] - ICR is minimized at intermediate noise levels, which coincide with the best linear-probe classification features (e.g. sigma ~0.29 for EDM, t ~0.2 for SiT). [li-etal-2026-evaluating-representation-space-diffusion-models] - ICR co-moves with FID in data-rich training and its rise anticipates the onset of training-data memorization, enabling label-free early stopping. [li-etal-2026-evaluating-representation-space-diffusion-models] - Tested on EDM, SiT-XL/2 and SiT-B/2 over CIFAR-10/100 and ImageNet-64/256 (Stable Diffusion is only cited as context, not used); observational-diagnostic, no causal intervention. [li-etal-2026-evaluating-representation-space-diffusion-models]

models: EDM (Elucidating Diffusion Models) · method: Invariant Contamination Ratio (ICR)

SiT (Scalable Interpolant Transformers)

Evaluating the Representation Space of Diffusion Models via Self-Supervised Principles (2026)measured

Fisher invariant-contamination ratio marks a diffusion model's optimal noise level

Details

models: SiT-XL/2, SiT-B/2 · method: Invariant Contamination Ratio (ICR)

ResNet

Hard-Negative Sampling for Contrastive Learning: Optimal Representation Geometry and Neural- vs Dimensional-Collapse (2024)measured

A real ResNet-50 trained with hard-negative contrastive losses and unit-ball normalization converges to Neural-Collapse ETF geometry on CIFAR-100, while omitting either yields Dimensional-Collapse instead

Details

A real ResNet-50 is trained from scratch for 400 epochs (Adam, batch size 512) on real CIFAR-100 (and CIFAR-10, TinyImageNet) with representation dimension set to d = C - 1 and unit-ball/unit-sphere feature normalization, under supervised and unsupervised hard-negative contrastive losses (HSCL, HUCL) [jiang-etal-2023-hard-negative-sampling-neural-dimensional-collapse] Under these conditions the trained representations converge to Neural-Collapse (NC) geometry -- class means forming an Equiangular Tight Frame, measured via the zero-sum, unit-norm, and equal-inner-product metrics matching their theoretical minima -- for a range of hardness levels and batch sizes from 64 to 512 [jiang-etal-2023-hard-negative-sampling-neural-dimensional-collapse] Without incorporating hard negatives or feature normalization, Adam optimization of the same losses on the same real ResNet-50/CIFAR-100 setup instead produces Dimensional-Collapse (DC), failing to attain the NC-ETF geometry [jiang-etal-2023-hard-negative-sampling-neural-dimensional-collapse]

models: ResNet-50 (hard-negative supervised/unsupervised contrastive, CIFAR-100) · method: Geometric analysis

GPT-OSS

Attention Sinks and Compression Valleys in LLMs Are Two Sides of the Same Coin (2025)measured

Massive activations provably cause compression valleys synced with attention sinks

Details

models: GPT-OSS-20B · method: Causal interventions (steering)

Graph Convolutional Network (Kipf & Welling)

Are We Measuring Oversmoothing in Graph Neural Networks Correctly? (2025)measured

Numerical and effective rank of node representations collapse toward 1 with depth in real trained GCN/GAT networks, and correlate with test accuracy far better than Dirichlet energy

Details

Numerical Rank and Effective Rank of the last-hidden-layer node-representation matrix decline sharply as depth increases from 2 to 24 layers in real trained GCN and GAT networks, e.g. dropping from an effective rank of about 1084 to 13.6 on Cora as depth grows [zhang-etal-2025-are-we-measuring-oversmoothing-in-gnns-correctly] Across homophilic (Cora, Citeseer, Pubmed), heterophilic (Squirrel, Chameleon, Amazon Ratings), and large-scale (OGB-Arxiv) real benchmark graphs, rank-based metrics correlate with test accuracy far better than Dirichlet energy (e.g. Cora/GCN: NumRank correlation 0.59, Erank 0.97, vs. Dirichlet energy -0.79) [zhang-etal-2025-are-we-measuring-oversmoothing-in-gnns-correctly] Theorem-level result proves numerical rank converges to 1 for a broad family of GNN architectures as depth grows, paired with the empirical measurement (10 independently trained networks per depth/dataset configuration) rather than left purely theoretical [zhang-etal-2025-are-we-measuring-oversmoothing-in-gnns-correctly]

models: GCN depth ensemble, 2-24 layers (Zhang, Higham, Deidda & Tudisco) · method: SVD

Graph Attention Network (Veličković et al.)

Are We Measuring Oversmoothing in Graph Neural Networks Correctly? (2025)measured

Numerical and effective rank of node representations collapse toward 1 with depth in real trained GCN/GAT networks, and correlate with test accuracy far better than Dirichlet energy

Details

models: GAT depth ensemble, 2-24 layers (Zhang, Higham, Deidda & Tudisco) · method: SVD

OLMo

Tracing the Representation Geometry of Language Models from Pretraining to Post-training (2025)measured

Pretraining passes through three effective-rank phases

Details

models: OLMo-1B, OLMo-7B · method: SVD

OLMo 2

Tracing the Representation Geometry of Language Models from Pretraining to Post-training (2025)measured

Pretraining passes through three effective-rank phases

Details

models: OLMo 2 1B, OLMo 2 7B · method: SVD

wav2vec 2.0

InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective (2026)measured

Real speech-SSL models Wav2Vec2, WavLM, HuBERT, and Data2Vec show distinct layer-wise entropy and trajectory-curvature regimes, with Wav2Vec2 undergoing a late-layer entropy collapse

Details

Von Neumann entropy (from the per-layer Gram matrix) and average curvature of the trajectory traced by adjacent-token transition vectors are computed layer-by-layer for real pretrained Wav2Vec2-base, WavLM-base, HuBERT-base, and Data2Vec-audio-base on the real LibriSpeech test-clean corpus [sadok-alameda-pineda-2026-insidessl] WavLM, HuBERT, and Data2Vec sustain high normalized entropy throughout the network (about 0.82 declining only to about 0.75 in final layers), while Wav2Vec2-base instead undergoes an entropy collapse toward near-baseline levels at layer 11, coinciding with a sharp rise in InfoNCE-based invariance loss (average 1.0 to 3.0) across its final two layers [sadok-alameda-pineda-2026-insidessl] Trajectory curvature starts high (about 1.4) in early layers and decreases to stabilize around 1.2 in deeper layers for most models, indicating the embedding manifold "unfolds" toward linear separability with depth; WavLM, HuBERT and UniSpeech show highly correlated curvature trajectories (Pearson correlation above 0.96), while linear probing shows phoneme-classification accuracy peaks at the depth where curvature transitions from the high- to low-curvature regime [sadok-alameda-pineda-2026-insidessl]

models: wav2vec2-base-960h · method:

WavLM

InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective (2026)measured

Real speech-SSL models Wav2Vec2, WavLM, HuBERT, and Data2Vec show distinct layer-wise entropy and trajectory-curvature regimes, with Wav2Vec2 undergoing a late-layer entropy collapse

Details

models: WavLM Base · method:

HuBERT

InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective (2026)measured

Real speech-SSL models Wav2Vec2, WavLM, HuBERT, and Data2Vec show distinct layer-wise entropy and trajectory-curvature regimes, with Wav2Vec2 undergoing a late-layer entropy collapse

Details

models: HuBERT-base · method:

Data2Vec

InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective (2026)measured

Real speech-SSL models Wav2Vec2, WavLM, HuBERT, and Data2Vec show distinct layer-wise entropy and trajectory-curvature regimes, with Wav2Vec2 undergoing a late-layer entropy collapse

Details

models: Data2Vec Audio Base (LibriSpeech-960h) · method:

Custom Research Transformer (small, purpose-built for interpretability studies)

The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks (2026)measured

SwiGLU spikes collapse keys into a sink-forming subspace

Details

models: Custom controlled 7B Transformer (trained from scratch for architecture/normalization ablations) · method: Causal interventions (steering)

Vemula (2026) Custom Decoder-Only Transformers (FineWeb-Edu, from scratch)

Massive Activations Are Architecturally Robust: A Controlled Scratch/Commitment Residual Stream Test (2026)measured

Massive activations re-emerge under a protected residual stream and resist a stronger sparsity penalty

Details

Vemula trains real decoder-only transformers from scratch on FineWeb-Edu at 160M and 290M parameters under four matched configurations (vanilla, QK-normalized "suppress", a novel Ledger-Residuals architecture splitting the residual stream into a mutable Deliberation stream and a protected Commitment stream, and a Commitment variant with 6x stronger commit-sparsity penalty), measuring massive activations via excess kurtosis, fixed-dimension ratio, dominant-dimension persistence, and start-token concentration [vemula-2026-massive-activations-architecturally-robust] At 290M parameters and matched validation loss, the vanilla transformer shows a massive-activation channel at dimension 828 (persistence 0.92, fixed-dimension ratio 28.3, start-token concentration 1.01), which QK-normalization essentially removes (ratio 6.8, concentration 0.13), but the Ledger-Residuals commitment channel rebuilds a massive activation at a different dimension (dimension 22, persistence 0.58, ratio 7.0, concentration 3.36) despite architecturally isolating it from the mutable stream [vemula-2026-massive-activations-architecturally-robust] A 6x stronger commit-sparsity penalty on the same Ledger-Residuals architecture makes the re-emerged massive activation worse rather than better (persistence rises to 0.96, concentration to 4.69), and the same qualitative re-emergence pattern replicates at 160M parameters (vanilla kurtosis 81, ratio 34, persistence 54%; ledger commitment channel rebuilds at ratio 13, concentration 3.04) [vemula-2026-massive-activations-architecturally-robust]

models: Custom decoder-only transformer (160M, FineWeb-Edu, from scratch), Custom decoder-only transformer (290M, FineWeb-Edu, from scratch) · method: