MATH · IN · MODELS

Steering-direction orthogonalization battery

Techniqueadvanced

Applies several distinct exact-orthogonalization schemes (symmetric Löwdin whitening, greedy Gram-Schmidt, selective/partial orthogonalization, soft projection, hard symmetric orthonormalization) to a fixed set of concept steering directions, then measures whether forcing near-zero pairwise cosine similarity between the directions is sufficient to eliminate behavioral cross-concept interference when steering with them one at a time — distinguishing geometric independence of a direction set from behavioral independence of what steering along each direction actually does.

Used in (1 observation)

structure: Linear Direction · models: Llama-3-8B-Instruct, Ministral-8B-Instruct-2410 · paper: Do Personality Traits Interfere? Geometric Limitations of Steering in Large Language Models