Per-language SAE directions in Gemma-2 steer target-language generation

Chou et al. use Gemma Scope SAEs on Gemma-2-2B and Gemma-2-9B to identify per-language feature directions maximally differing between English and Chinese/Japanese/Spanish/French [chou-etal-2025-causal-language-control-in-multilingual-transformers-via-sparse-feature-steering] Additive steering of these SAE directions, decoded into the residual stream, forces generation into the target language [chou-etal-2025-causal-language-control-in-multilingual-transformers-via-sparse-feature-steering] On Gemma-2-9B, FastText language accuracy reaches 0.978 (Chinese) down to 0.852 (French), versus 0.356-0.786 for prompting [chou-etal-2025-causal-language-control-in-multilingual-transformers-via-sparse-feature-steering] LaBSE semantic similarity to the unsteered generation stays comparable to baseline, with effects concentrated in mid-to-late layers (29-36) [chou-etal-2025-causal-language-control-in-multilingual-transformers-via-sparse-feature-steering]