The minimal-distortion linear eraser is an oblique projection

Belrose et al. prove LEACE's closed-form eraser guards a concept against every linear classifier and convex loss simultaneously (Theorem 4.1) [belrose-etal-2023-leace] It is the least-squares-optimal minimal-distortion such eraser (Theorems 4.2-4.3) and is necessarily an oblique, not orthogonal, projection, unlike INLP/RLACE [belrose-etal-2023-leace] On BERT Bias-in-Bios gender removal it reaches chance accuracy at rank 1 with the smallest embedding MSE, cutting the TPR gap 0.198->0.084, about two orders of magnitude faster than RLACE [belrose-etal-2023-leace] Concept-scrubbing POS from every layer of Pythia (160M-12B) and LLaMA (7B-30B) raises LM loss far above a random-subspace control (e.g. LLaMA-7B 0.69->1.73 bpb) while removing fewer dimensions than a full-rank baseline [belrose-etal-2023-leace]