MATH · IN · MODELS
methods / Direction Extraction / Cross-model direction transfer via ridge regression

Cross-model direction transfer via ridge regression

Techniqueadvanced

Extracts a diff-in-means direction independently in each of several differently-sized/architected models, then fits a ridge-regression map between one model's activation space and another's (on paired activations from the same prompts) and transfers the source model's direction into the target model's space by applying that same linear map — turning 'do two unrelated models share a behavioral direction' into a directly testable geometric transfer, rather than a same-model-only diff-in-means claim.

Used in (4 observations)

structure: Linear Direction · models: GPT-2-small, GPT-2-Medium, Gemma-2-2B, Gemma-2-9B, Pythia-70M-deduped, Pythia-160M-deduped · paper: Transferring Linear Features Across Language Models With Model Stitching
structure: Linear Direction, Platonic Representation Hypothesis · models: Llama-2-7B-Chat, Qwen2-7B-Instruct, Llama-3.1-8B-Instruct, Qwen2-0.5B-Instruct · paper: Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts
structure: Intrinsic-dimension profile across depth, Linear Direction · models: GPT-2-Medium, Llama-3.1-8B-Instruct, Gemma-2-9B-it · paper: Shared Global and Local Geometry of Language Model Embeddings
structure: Linear Direction · models: Qwen2.5-1.5B-Instruct, Gemma-2-2B-it, Llama-3.2-1B-Instruct, Ministral-3-3B-Instruct · paper: Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families