Cone probe (order-embedding for descendant relations)

methods / Causal Validation / Cone probe (order-embedding for descendant relations)

Cone probe (order-embedding for descendant relations)

Techniqueadvanced

Fits a differentiable relaxation of a Heaviside order-embedding score, p(E_i,E_j)=sigma(E_i,1-E_j,1)*sigma(E_i,0-E_j,0), so that 'j is a descendant of i' corresponds to E_j lying inside a fixed coordinate-wise cone anchored at E_i -- provably transitive and antisymmetric by construction -- then validates the fitted subspace causally via projector-based activation patching, comparing its effect against a same-dimensionality top-PCA-component subspace and a full-layer patch.

Used in (1 observation)

A cone probe isolates a causal tree-like subspace for genealogymeasured

structure: Cone · models: Llama-3.1-8B-Instruct, Llama-3-8B-Instruct, Llama-3.2-3B-Instruct, Gemma-2-2B-it, Gemma-2-9B-it, OPT-1.3B, OPT-2.7B, OPT-6.7B, Pythia-410M, Pythia-1.4B, Pythia-2.8B, Mistral-7B-Instruct-v0.1 · paper: Investigating Representation Universality: Case Study on Genealogical Representations