BERT causally uses a multi-dimensional number subspace for agreement

Lasri et al. apply INLP (erasure only, no AlterRep) within amnesic probing to BERT-base, removing the grammatical-number subspace at noun and verb positions (44-70 directions by layer, not a single direction) [lasri-etal-2022] Removing the noun-position subspace drops subject-verb agreement accuracy 0.22-0.32 in early layers; removing the verb-position subspace drops it 0.23-0.33 in layers 9-11 [lasri-etal-2022] A random-direction placebo removing the same number of directions drops accuracy only 0.00-0.06, confirming specificity to the targeted subspace [lasri-etal-2022] Noun and verb number encodings are separate: a projector trained at one position erases nothing useful at the other, and the probe weight vectors are near-opposite then near-orthogonal across layers [lasri-etal-2022] Correlating harmful-removal layers with an attention-ablation experiment locates number transfer from noun to verb in layers 2-8 [lasri-etal-2022]