MATH · IN · MODELS

Dual (Information-Geometric) Steering

Techniqueadvanced

Adds a concept probe direction in the dual (mean-parameter) coordinates of a softmax distribution's Bregman/information geometry, φ(λ_t) := φ(λ_0) + tβ_W, rather than in the natural/logit coordinates as ordinary additive steering does — provably minimizing the reverse KL divergence of the off-target (concept-factored) distribution among all interventions reaching a target concept score, rather than minimizing Euclidean distance in logit space.

Used in (1 observation)

structure: Linear Direction · models: Gemma 3 4B, MetaCLIP 2 · paper: The Information Geometry of Softmax: Probing and Steering