A few rogue dimensions drive transformer anisotropy
measured in 1 paper- A handful of "rogue" dimensions, often a single one, with huge off-center means dominate cosine similarity and the anisotropy statistic; the top dimension contributes over 99% of expected cosine in XLNet (0.996 at layer 11; BERT 0.817, GPT-2 0.763, RoBERTa 0.726). [timkey-vanschijndel-2021] - Standardizing or removing ~5 such dimensions restores near-isotropy (A-hat <= 0.25) and improves correlation with human similarity judgments on RG65, WS353, SimLex-999 and SimVerb-3500. [timkey-vanschijndel-2021] - Ablating the rogue dimensions (zeroing their layer-norm gamma and beta) barely changes the output distribution (KL), so they dominate cosine similarity but not model behavior. [timkey-vanschijndel-2021] - Tested on BERT, RoBERTa, GPT-2 and XLNet (base size) plus word2vec (GoogleNews). [timkey-vanschijndel-2021]