Dual steering in information-geometric coordinates is provably KL-optimal

measured in 1 paper

Park et al. argue softmax distributions have a Bregman information geometry where KL divergence equals a Bregman divergence of the log-normalizer [park-etal-2026-information-geometry-softmax-probing-steering] Dual steering adds the probe in the dual (mean) parameter space rather than the logit space of ordinary CAA-style steering [park-etal-2026-information-geometry-softmax-probing-steering] Theorem 3.1 proves dual steering is the exact KL-minimizing intervention reaching a target concept logit-score, with a minimal-collateral-change guarantee under concept-factorizability [park-etal-2026-information-geometry-softmax-probing-steering] On Gemma-3-4B and MetaCLIP-2, dual steering outperforms Euclidean steering across all three robustness metrics [park-etal-2026-information-geometry-softmax-probing-steering]

Structure

Linear Direction

Context

Bregman/information geometry of softmax distributions as the natural coordinate system for steering, dual (mean-parameter) steering as the provably KL-optimal intervention, contrasted with causal-inner-product's logit-space framework, cross-modality validation (LLM + vision-language model) of the same geometric steering principle

Confirmed in models

Gemma 3 4B · Gemma, MetaCLIP 2 · MetaCLIP

Method

Linear probing, Dual (Information-Geometric) Steering

Papers

The Information Geometry of Softmax: Probing and Steering — Park, Kiho, Nief, Todd, Choe, Yo Joong, Veitch, Victor2026 · arXiv:2602.15293