Causal inner product
Techniqueadvanced
A covariance-whitened inner product on the unembedding space, ⟨γ,γ'⟩_C = γᵀΣγ⁻¹γ', constructed so that causally separable concepts become approximately orthogonal under it — turning 'is concept A independent of concept B' into an angle you can measure directly, rather than a raw cosine similarity that correlated training data can distort.
Used in (2 observations)
structure: Polytope (Simplex) · models: Gemma-2B, Llama-3-8B, Qwen3-4B, Mistral-7B-v0.3 · paper: The Geometry of Categorical and Hierarchical Concepts in Large Language Models, When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs
structure: Linear Direction, Polytope (Simplex) · models: Gemma-2B, Llama-3-8B, Qwen3-4B, Mistral-7B-v0.3 · paper: The Geometry of Categorical and Hierarchical Concepts in Large Language Models, When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs