Refusal directions are approximately parallel across 14 languages and transfer

measured in 1 paper

Wang et al. extract diff-in-means refusal directions across 14 languages in Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it using the PolyRefuse dataset [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] PCA shows the per-language refusal directions are approximately parallel, with high cross-lingual cosine similarity peaking around layer 12 in Llama [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] Ablating a refusal vector extracted from English (or German/Thai) causes near-total loss of refusal across all 14 languages and all three models, while adding it induces refusal [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] Yoruba, a language the models were never safety-aligned in, transfers weakly, so the parallelism tracks safety-alignment coverage rather than language identity [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages]

Structure

Linear Direction

Context

refusal, safety, cross-lingual transfer, jailbreak, refusal geometry debate, multilingual alignment

Confirmed in models

Gemma-2-9B-it · Gemma, Llama-3.1-8B-Instruct · Llama, Qwen2.5-7B-Instruct · Qwen

Method

Difference-in-means direction, Causal interventions (steering), PCA

Papers

Refusal Direction is Universal Across Safety-Aligned Languages — Wang, Xinpeng, Wang, Mingyang, Liu, Yihong, Schütze, Hinrich, Plank, Barbara2025 · arXiv:2505.17306