Inference-time causal probing separates causal from correlational directions

Khorasani et al. introduce HDMI, an inference-time causal probing method that fits linear probes while testing whether interventions along the candidate direction propagate to downstream behavior [khorasani-etal-2026-inference-time-causal-probing-in-llms] On Llama-3-8B-Instruct and Pythia-70M, directions selected by the causal criterion produce reliable behavioral shifts under activation-space intervention [khorasani-etal-2026-inference-time-causal-probing-in-llms] Several high-accuracy linear-probe directions that pass standard probing tests fail to causally affect behavior when intervened upon [khorasani-etal-2026-inference-time-causal-probing-in-llms] Linear separability alone therefore overstates how many directions in a model are behaviorally load-bearing [khorasani-etal-2026-inference-time-causal-probing-in-llms]