MATH · IN · MODELS

Angle-Norm Decomposition Steering

Techniqueintermediate

Splits a hidden state into a radial component (its norm) and an angular component (a concept score given by its cosine alignment with a unit steering direction, plus an orthogonal residual), then compares steering variants that differ in whether they preserve the original norm, target a specific angular concept score, and/or explicitly rescale the radius — isolating how much of a concept's causal effect rides on angle versus magnitude.

Used in (1 observation)

structure: Linear Direction · models: Llama-3.1-8B-Instruct, Llama-3.1-8B, Llama-3.1-70B-Instruct, Llama-3.2-1B-Instruct, Qwen2.5-7B-Instruct, Qwen2.5-3B-Instruct, Gemma-2-9B-it · paper: A Geometric Account of Activation Steering through Angle-Norm Decomposition