A cone probe isolates a causal tree-like subspace for genealogy

measured in 1 paper

Baek et al. define a differentiable order-embedding cone probe (provably transitive and antisymmetric for a tree's descendant-of relation) and fit it on a 15-node family tree, then on 10-dimensional PCA-reduced activations of five instruction-tuned LLMs [baek-etal-2024-cone-probe-genealogical-representation-universality] Activation patching restricted to the fitted cone subspace produces a causal effect on genealogy Q&A comparable to or larger than a same-rank top-PCA subspace, but smaller than an unrestricted full-layer patch [baek-etal-2024-cone-probe-genealogical-representation-universality] The cone subspace thus captures real but incomplete causal structure, and a context-shuffle control degrades performance, consistent with reliance on depth-ordered structure [baek-etal-2024-cone-probe-genealogical-representation-universality] A complementary model-stitching experiment (OPT/Pythia/Mistral/Llama, 410M-8B) finds low next-token-loss degradation when splicing early/mid layers between independently-trained models [baek-etal-2024-cone-probe-genealogical-representation-universality] The authors explicitly hedge the universality conclusion given the lack of ground-truth graph representations [baek-etal-2024-cone-probe-genealogical-representation-universality]

Structure

Cone

Context

cone probe -- differentiable order-embedding score isolating a tree-like (descendant-of) subspace, provably transitive/antisymmetric by construction, projected activation patching restricted to a fitted subspace, compared against a same-rank PCA-subspace control and a full-layer control, dissociation between a specific fitted subspace's causal contribution and the residual stream's total causal capacity (cone patch < full-layer patch), cross-architecture model stitching (linear adapter splicing layers of independently-trained models) as a second, complementary universality test, explicitly hedged conclusion on universality, motivated by the general difficulty of establishing ground-truth representations for graph-structured knowledge

Confirmed in models

Llama-3.1-8B-Instruct · Llama, Llama-3-8B-Instruct · Llama, Llama-3.2-3B-Instruct · Llama, Gemma-2-2B-it · Gemma, Gemma-2-9B-it · Gemma, OPT-1.3B · OPT, OPT-2.7B · OPT, OPT-6.7B · OPT, Pythia-410M · Pythia, Pythia-1.4B · Pythia, Pythia-2.8B · Pythia, Mistral-7B-Instruct-v0.1 · Mistral

Method

Cone probe (order-embedding for descendant relations), Activation patching

Papers

Investigating Representation Universality: Case Study on Genealogical Representations — Baek, David D., Li, Yuxiao, Tegmark, Max2024 · arXiv:2410.08255