R-LACE (Relaxed Linear Adversarial Concept Erasure)

methods / Causal Validation / R-LACE (Relaxed Linear Adversarial Concept Erasure)

R-LACE (Relaxed Linear Adversarial Concept Erasure)

Techniqueadvanced

Reformulates linear concept erasure as a minimax game between a rank-k erasure projection and a linear predictor trying to recover the concept despite it, solved exactly in closed form for some losses and via a convex (Fantope) relaxation for others — a theoretically-grounded alternative to INLP's iterative classifier-then-nullspace-projection heuristic, provably finding a smaller subspace in some cases.

Used in (3 observations)

Guardedness against a binary probe leaks to a multiclass softmaxmeasured

structure: Linear Subspace · models: BERT-base-uncased · paper: Log-linear Guardedness and its Implications

Nonlinear interventions are more reliable than linear subspace erasuremeasured

structure: Linear Subspace · models: BERT-base-uncased, GPT-2-small, Pythia-160M, Pythia-1.4B, Pythia-6.9B, Llama-3.2-3B-Instruct · paper: How Reliable are Causal Probing Interventions?

One adversarial direction erases gender where INLP needs dozensmeasured

structure: Linear Subspace · models: GloVe (Wikipedia + Gigaword, uncased), BERT-base-uncased · paper: Linear Adversarial Concept Erasure