GPT-2 writes features into low-rank communication subspaces of the residual stream for later layers to read, and a 3D subspace causally implements positional list-indexing

- Merullo, Eickhoff & Pavlick (2024) analyze a mechanism used by GPT-2 Small (main text) and Pythia-160m (Appendix G) to selectively inhibit context items (inhibition heads feeding a mover head), finding that the models "write into low-rank subspaces of the residual stream to represent features which are then read out by later layers" (Abstract); the inhibition channel is a 1D component per inhibition head and the duplicate-token channel is 2D (Sec. 3, Fig. 4 caption), and the paper reports that key composition (previous-token to induction head) did not show such a channel. [merullo-etal-2024-talking-heads-inter-layer-communication] - In GPT-2 Small, the 3D space spanned by the top three inhibition-head components (7.9.6, 8.6.2, 8.6.10) is traversed on a grid (Sec. 5.2, Fig. 5): different regions of it make the mover head attend to different list items, and the structure fragments as the list grows past ~9-10 items. The authors propose this as an explanation of the model's order sensitivity on the Laundry List task ("we believe" the space fragments; this is argued from the traversal visualization and accuracy interventions, not independently tested). [merullo-etal-2024-talking-heads-inter-layer-communication] - Decomposing attention-head QK/OV matrices with SVD and scoring composition between single rank-1 components lets the authors identify the channels from weights (single components compose far more strongly, e.g. head 8.6 component 2 is 5 standard deviations above its other 63 components), and zeroing one component reduces the inhibition score by 7-14% (Sec. 3.3). Setting the three inhibition components to a region chosen by the correct item's index raises Laundry List accuracy from 64% to 86% at 3 objects and to about 51% at 8 objects (Sec. 5.2, Fig. 5); this improvement is an activation intervention, not a weight edit. [merullo-etal-2024-talking-heads-inter-layer-communication]