MATH · IN · MODELS
methods / Causal Validation / Causal interventions (steering) / Activation Steering (Addition) / Modality Linear Representation-Steering (MoReS)

Modality Linear Representation-Steering (MoReS)

Techniqueintermediate

A trained, reversible linear map that steers a modality's tokens (e.g. visual tokens in an MLLM) into and back out of a low-rank subspace at every layer, rather than adding a single fixed direction — the intervention itself, not just its effect, is constrained to a learned k-dimensional subspace.

Used in (1 observation)

structure: Linear Subspace · models: Phi-2, Vicuna-7B, Vicuna-13B · paper: LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering