A weights-only coupling coefficient between transformer components pools into a distinct two-dimensional residual-stream subspace whose deletion abolishes induction in six models up to Pythia-6.9B, different from activation-PCA and outlier-dimension subspaces

- Wang defines a coupling coefficient from the geometry of a transformer's weights (a generalization of the Elhage et al. composition score covering 18 connection classes) with an exact chance level, finds 70-89% of head pairs oriented far from chance, and shows that pooling every head's couplings identifies a distinct two-dimensional residual-stream subspace whose deletion abolishes induction in six models up to Pythia-6.9B and which differs from activation-PCA and outlier-dimension subspaces. Table 3 (full text) reports the deletion on GPT-2 small/medium/large, Pythia-160m, Pythia-2.8B and Pythia-6.9B (82-96% of induction gain destroyed versus 0.1-3.0% for a random 2-D subspace); GPT-Neo-125M appears in the coupling census but is excluded from the deletion test and not tagged. The subspace is computed from weights alone (RW-PCA, a two-direction selection from ten pooled bands) and compared with activation-PCA and outlier-dimension planes by principal cosines. [wang-2026-the-communication-map-of-a-transformer]