Measures a single component's (e.g. one attention head's) direct contribution to the correct next-token prediction by dot-producting its (LayerNorm-scaled) output against the difference between the correct token's unembedding and a reference (mean) unembedding — a forward-pass-only attribution, not a causal test, used to locate candidate components for follow-up causal work.
Used in (7 observations)
structure: Linear Subspace · models: Pythia-2.8B, GPT-2-XL, GPT-J-6B, Pythia-6.9B · paper: Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs
structure: Linear Direction · models: hallway (maze-solving transformer, forkless mazes), jirpy (maze-solving transformer, forking mazes) · paper: Structured World Representations in Maze-Solving Transformers
structure: Linear Separability · models: Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, Qwen3-4B-Instruct · paper: Tracing Relational Knowledge Recall in Large Language Models
structure: Linear Direction · models: GPT-2 Small, Pythia-2.8B · paper: Linear Representations of Sentiment in Large Language Models
structure: Linear Subspace, Linear Direction · models: Llama-3-8B, Llama-3.1-8B, Llama-3.2-3B, Qwen2-7B (base), Qwen2.5-32B, Yi-34B (base) · paper: Task Recognition and Task Learning Heads Align In-Context Hidden States with a Label-Unembedding Task Subspace
structure: Linear Direction · models: Pythia-410M, Pythia-1B, Pythia-1.4B, Qwen-2.5-7B, GPT-2-Medium · paper: Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning
structure: Linear Direction, Constraint-Algebra Basis Hypothesis · models: Sudoku Transformer (8 layers, 8 heads, d_model=576) · paper: Transformers Linearly Represent Highly Structured World Models