SAE decoder cosine similarity quantifies interference and transfers steering

Gong et al. train SAEs on Pythia-70M and GPT-2-Small and quantify feature-pair interference as the cosine similarity of their SAE decoder directions in activation space [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Using these directions as steering vectors (scaled +/-20), small-model interference geometry transfers to larger black-box instruction-tuned models [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] One high-interference direction steered into Llama-3.1-8B-Instruct transfers at over 95% success [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Pythia-derived success is 49.74% for high-interference pairs versus 21.76-22.87% for low-interference or random pairs [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Transfer is further validated on Llama-3.1-70B-Instruct and Gemma-2-9B-Instruct, with an additional Gemma-2-2B experiment [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence]