Gradient-Based Interventions (nonlinear counterfactual push via adversarial attack)

methods / Causal Validation / Gradient-Based Interventions (nonlinear counterfactual push via adversarial attack)

Gradient-Based Interventions (nonlinear counterfactual push via adversarial attack)

Techniqueadvanced

Trains a nonlinear (MLP) probe on a target property, then uses a white-box adversarial attack (FGSM, PGD, or AutoAttack) to find a perturbation within an L∞-ball around a hidden vector that minimizes the probe's loss with respect to a counterfactual target value — the same counterfactual-substitution pipeline as AlterRep, but with the push direction found by attacking a nonlinear probe instead of projecting relative to a linear hyperplane.

Used in (1 observation)

Nonlinear interventions are more reliable than linear subspace erasuremeasured

structure: Linear Subspace · models: BERT-base-uncased, GPT-2-small, Pythia-160M, Pythia-1.4B, Pythia-6.9B, Llama-3.2-3B-Instruct · paper: How Reliable are Causal Probing Interventions?