Layer intrinsic-dimension peak aligns with the best brain-predicting layer

measured in 1 paper

Cheng et al. estimate per-layer intrinsic dimension via the GRIDE estimator in OPT (125M/1.3B/13B), Pythia (160M/410M/6.9B), WavLM (base-plus, large), and Whisper (large encoder) [cheng-etal-2026-abstraction-brain-alignment] Layerwise intrinsic dimension correlates with how well each layer predicts real human brain responses (fMRI rho=0.76, ECoG rho=0.43, both p<0.05) [cheng-etal-2026-abstraction-brain-alignment] The ID-peak layer and the best-brain-predicting layer are usually within 0-1 layers of each other [cheng-etal-2026-abstraction-brain-alignment] Brain-tuning WavLM's best layer (layer 9) to predict fMRI causally raises both its intrinsic dimension and its semantic content, while a random-Fourier-features control shows raised ID alone is insufficient [cheng-etal-2026-abstraction-brain-alignment]

Structure

Intrinsic-dimension profile across depth

Context

GRIDE intrinsic dimension estimation, brain-alignment encoding performance, brain-tuning causal fine-tuning, cross-modal (text and speech) replication

Confirmed in models

OPT-125M · OPT, OPT-1.3B · OPT, OPT-13B · OPT, Pythia-160M · Pythia, Pythia-410M · Pythia, Pythia-6.9B · Pythia, WavLM-base-plus · WavLM, WavLM Large · WavLM, Whisper large · Whisper

Method

Intrinsic dimension estimation (TwoNN), Causal interventions (steering)

Papers

Abstraction Induces the Brain Alignment of Language and Speech Models — Cheng, Emily, Vaidya, Aditya R., Antonello, Richard2026 · arXiv:2602.04081