A polar probe splits in-context relational graphs into existence and type readouts

Diego-Simon et al. extend the structural probe's distance/depth decomposition from parse trees to arbitrary in-context relational graphs across five domains (arithmetic ordering, spatial layouts, thematic roles, family trees, metro maps) [diego-simon-etal-2026-polar-probe-semantic-structures] Within a learned probe subspace, a pair's difference vector is decomposed into an existence readout (Euclidean distance) and a type readout (cosine similarity to learned per-relation-type prototype vectors) [diego-simon-etal-2026-polar-probe-semantic-structures] Tested on Llama3-8B, OLMo-7B (9 pretraining checkpoints), and the Pythia sweep (10M-6.8B), existence decoding peaks around 0.80 and type around 0.50-0.70 in middle layers, near chance in a random-init control and scaling monotonically with model size [diego-simon-etal-2026-polar-probe-semantic-structures] Performance saturates by probe rank ~32, and the five domains' probe subspaces are largely disjoint except a pronounced spatial-layout/variable-ordinality overlap [diego-simon-etal-2026-polar-probe-semantic-structures] Mapping each prototype back into the residual stream via the probe's pseudoinverse and adding or subtracting it causally raises or lowers the correct-token probability, though the method underperforms on non-commutative or many-to-many graphs [diego-simon-etal-2026-polar-probe-semantic-structures]