An OLS-fitted linear map transfers steering vectors across LLMs

measured in 1 paper

Huang et al. fit an ordinary-least-squares linear map T between paired sentence-representations of a source and target real LLM [huang-etal-2025-cross-model-platonic-transferability] Transferring a concept steering vector through T succeeds behaviorally in 212/216 cases, while no-map and random-map controls fail to beat baseline in 23/35 cases [huang-etal-2025-cross-model-platonic-transferability] Maps fit for different concepts are themselves highly similar (SSIM 0.87-0.95) versus a random-matrix baseline (0.05-0.13) [huang-etal-2025-cross-model-platonic-transferability] This cross-concept map consistency is a quantified geometric statistic about the transformation itself, beyond downstream task accuracy [huang-etal-2025-cross-model-platonic-transferability]

Structure

Linear Direction, Platonic Representation Hypothesis

Context

cross-model linear map, OLS regression, steering vector transfer, specificity control, cross-concept matrix similarity

Confirmed in models

Llama-2-7B-Chat · Llama, Qwen2-7B-Instruct · Qwen, Llama-3.1-8B-Instruct · Llama, Qwen2-0.5B-Instruct · Qwen

Method

Cross-model direction transfer via ridge regression

Papers

Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts — Huang, Chen, Huang, Youcheng, Feng, Duanyu, Lei, Wenqiang, Lv, Jiancheng2025 · arXiv:2501.02009