Attention outputs collapse to a ~60% low-rank subspace

measured in 1 paper

- Across model families, attention-block outputs occupy an intrinsic dimension of only ~60% of the ambient space (directions for 99% variance), versus ~90% for MLP outputs and the residual stream, traced to the attention output projection W^O. [wang-etal-2025-attention-low-rank-subspaces] - The dead-feature rate of SAEs tracks activation intrinsic dimension layer-by-layer: lower-dimensional activations yield more dead features. [wang-etal-2025-attention-low-rank-subspaces] - Active Subspace Initialization (constraining SAE weights to the activations' top singular vectors) cuts dead features from 87% to below 1% in 1M-feature attention-output SAEs while lowering reconstruction loss. [wang-etal-2025-attention-low-rank-subspaces] - Measured on GPT-2, Llama-3.1-8B, Gemma-2-9B, Qwen3-8B/4B and Pythia-160M/2.8B. [wang-etal-2025-attention-low-rank-subspaces]

Structure

Dimensional collapse

Context

SVD-based intrinsic dimension (99%-variance threshold), attention output projection matrix W^O as the compressive mechanism, dead-feature rate tracks activation intrinsic dimension, Active Subspace Initialization for sparse autoencoders, universality across layers, model families and datasets

Confirmed in models

Llama-3.1-8B · Llama, GPT-2 · GPT-2, Gemma-2-9B · Gemma, Qwen3-8B · Qwen, Qwen3-4B · Qwen, Pythia-160M · Pythia, Pythia-2.8B · Pythia

Method

SVD, Sparse Autoencoders (SAE)

Papers

Dimensional Collapse in Transformer Attention Outputs: A Challenge for Sparse Dictionary Learning — Wang, Junxuan, Ge, Xuyang, Shu, Wentao, He, Zhengfu, Qiu, Xipeng2025 · arXiv:2508.16929