MATH · IN · MODELS

A representation intervention inverts back into gender-flipped text

measured in 1 paper

Avitan et al. invert BiasBios representations that have passed through a gender intervention (LEACE erasure or MiMiC/MiMiC+ steering) back into fluent text via a vec2text procedure [avitan-etal-2024-intervention-lens] The recovered text adds only modest fluency cost (Mistral-7B perplexity 18.3-19.1 vs 18.2 for inversion alone, far below the 22.6 of the originals) [avitan-etal-2024-intervention-lens] Human annotators judge the intended gender flip present with substantial reliability (MiMiC female-to-male labeled male 82% vs 3% for originals) [avitan-etal-2024-intervention-lens] A PMI analysis shows the interventions also shift subtler class-correlated vocabulary (female-to-male adds developer/managers; male-to-female adds domestic/maternal) beyond pronouns [avitan-etal-2024-intervention-lens]

Context

vec2text embedding inversion, PMI word-association analysis, human evaluation of counterfactual text

Papers

Intervention Lens: from Representation Surgery to String Counterfactuals — Avitan, Matan, Cotterell, Ryan, Goldberg, Yoav, Ravfogel, Shauli2024 · arXiv:2402.11355