A low-dimensional subspace of decode-time hidden states is shared across tasks in LLMs, and removing it only during decoding degrades decisions far more than removing a prefill-derived or random subspace

- Shao et al. propose DecodeShare, which identifies a low-dimensional subspace consistently shared across tasks in decode-time hidden states and tests its causal role by removing it only during decoding; removal degrades decision performance far more than removing a prefill-derived or random subspace at the same intervention budget, and common steering directions can overlap this task-general decode channel. The shared core is small (a few percent of d; 32D for Llama-2-7B-Chat layer 10), found by pooled PCA over task-centred decode-time states at layers such as 4, 10 and 24 in Llama-2-7B-Chat, Qwen2.5-7B-Instruct and Falcon-7B-Instruct; in the H3 grid decode-estimated removal cuts mean accuracy by 19.5 points at layer 10 (prefill-estimated removal 0.2) and patching back only the removed shared component over the full window rescues 100% of ablation-induced flips in all nine model-layer cells of Table 1 (a first-step patch rescues 65-100%), while patching outside the shared subspace rescues 0%. Appendix C.4 reports non-7B checks on Llama-3.2-3B (layer 27), Llama-2-13B (layers 10, 28, 39) and Llama-2-70B (layers 32, 48, 56): the shared set is a small fraction of the cross-task PCA dimension (7.2% at 3B, 3.3-4.8% at 13B, 0.05-0.60% at 70B), shared-subspace removal lowers accuracy on a four-task multiple-choice slice (mean -21.9 at 3B layer 27, -9.6 to -11.9 at 13B, -15.2 to -20.3 at 70B), and the decode-estimated basis hurts far more than the prefill-estimated basis at 3B layer 27, 13B layer 10 and 70B layer 56 (-22.9, -32.0, -34.8 vs -5.0, +0.4, -1.2); the paper states it leaves systematic scaling trends to future work and does not specify chat/base checkpoints for these models. [shao-etal-2026-decodeshare-tracing-the-shared-subspace-of-llm-decode-time-decisions]