Outlier dimensions persist and can solve GLUE tasks

- Outlier dimensions (variance >=5x the average) that arise in pretraining persist as outliers after fine-tuning regardless of task or seed (GPT-2's exact top-3 persist). [rudman-etal-2023-outlier-dimensions-encode-task-specific-knowledge] - A single principal outlier dimension with a learned threshold can complete GLUE binary-classification tasks, with model-dependent accuracy loss: low for GPT-2 (2.85% avg) and ALBERT (6.12%) but high for BERT (16.33%), RoBERTa (19.23%) and DistilBERT (19.12%); Pythia 70M/160M/410M lose 12.58/7.48/11.44%. [rudman-etal-2023-outlier-dimensions-encode-task-specific-knowledge] - The often-quoted "0-3.4%" figures are QNLI-only (ALBERT 3.42%, Pythia-410M 0.0%), not the cross-task averages; Pythia-410M (11.44% avg) is not low-loss. [rudman-etal-2023-outlier-dimensions-encode-task-specific-knowledge] - Tested on eight encoder and decoder models (BERT, ALBERT, DistilBERT, RoBERTa, GPT-2, Pythia-70M/160M/410M); the paper names families/sizes, not exact checkpoints. [rudman-etal-2023-outlier-dimensions-encode-task-specific-knowledge]