A grid-organized cell subspace supports relational binding in LLMs

Dai, Heinzerling & Inui (2026) use PLS-regression probing (vs PCA/ICA baselines) on frozen middle-layer activations (strongest layers 10-20) of Llama3-8B-Instruct and Qwen3-8B [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] They find a low-dimensional linear "Cell-Based Representation" subspace organized as a grid along entity-index x relation-index axes, with near-perfect PLS fits using only 2-5 components [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] Cross-context transfer R^2~0.8, ablation stability R^2~0.8, index-prediction R^2~0.95 across 13 discourse patterns, attribute accuracy 0.94-0.95 (beating a Hessian propositional-probe baseline) [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] Subspace perturbation degrades attribute accuracy vs a random-subspace control, and explicit steering vectors reliably shift entity/relation indices across 5 domains [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models]