A PCA 'Ordering ID' direction localizes and steers binding

measured in 1 paper

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

Structure

Linear Subspace, Linear Direction

Context

relational-binding, order-id-representation

Confirmed in models

Llama-2-7B · Llama, Llama-3-8B · Llama, Qwen1.5-7B · Qwen, Pythia-6.9B · Pythia, Float-7B (code fine-tuned LM) · Float

Method

PCA, Direction Extraction, Causal interventions (steering)

Papers

Representational Analysis of Binding in Language Models — Dai, Qin, Heinzerling, Benjamin, Inui, Kentaro2024 · arXiv:2409.05448