Representation-alignment loss regularization

methods / Causal Validation / Causal interventions (steering) / Representation-alignment loss regularization

Representation-alignment loss regularization

Techniqueintermediate

A training-time (not inference-time) causal intervention: an auxiliary cosine-similarity loss term, added alongside the standard training objective, explicitly pulls a model's own intermediate representations toward a target geometric reference (e.g. a frozen foundation model's features) during training, and the resulting change in downstream behavior is measured after training completes.

Used in (1 observation)

MLLM visual tokens diverge from frozen VFM features; alignment loss recovers accuracymeasured

structure: Representational-similarity trajectory across depth · models: LLaVA-1.5-7B, LLaVA-1.5-13B · paper: Visual Representation Alignment for Multimodal Large Language Models