Naturalistic Causal Probing (input-level dependency-tree reinflection)

methods / Causal Validation / Naturalistic Causal Probing (input-level dependency-tree reinflection)

Naturalistic Causal Probing (input-level dependency-tree reinflection)

Techniqueadvanced

Intervenes on a morphosyntactic property at the INPUT (sentence) level rather than in representation space: swaps a focus noun's lemma to change its gender/number, recursively re-inflects every grammatically-dependent word via the sentence's dependency tree, and estimates the average treatment effect on a model's internal representations using a paired (matched-counterfactual) estimator justified by the backdoor criterion — guaranteeing purity of the intervention at the cost of losing any explicit representation-space subspace or direction.

Used in (2 observations)

A naturalistic causal ATE direction nearly coincides with the top PCA directionmeasured

structure: Linear Direction · models: mBERT (BERT-base, Multilingual Cased), XLM-RoBERTa base, GPT2-small (Spanish) · paper: Naturalistic Causal Probing for Morpho-Syntax

Probing accuracy collapses under a naturalistic counterfactual testmeasured

structure: Linear Direction · models: mBERT (BERT-base, Multilingual Cased), XLM-RoBERTa base, GPT2-small (Spanish) · paper: Naturalistic Causal Probing for Morpho-Syntax