MATH · IN · MODELS

Relations bind tokens into an oriented frame whose shape is causally steerable

measured in 1 paper

Kobrosly studies the relation-frame, the ordered hidden-state configuration of the k token positions jointly bound by a k-argument relation [kobrosly-2026] Using Plucker sign entropy over sampled token tuples across Llama-3.1 8B/70B/405B-Instruct, true relation tuples show more orientation-consistent (lower-entropy) signs at the arity-matched rank k=r than scrambled or random-tuple controls [kobrosly-2026] The pattern is most uniform in the 405B model across templates and constructors, while 8B and 70B retain broadly positive but more template-specific support [kobrosly-2026] Patching the corrupt relation-marker cloud's centered shape toward the clean shape (leaving the centroid fixed) nearly matches full replacement (endpoint recovery ~1.0), while centroid-only translation (~0.002) and equal-norm noise (~0.02) recover almost nothing [kobrosly-2026] An ordering audit shows the ordered clean shape itself is required (permuting, reflecting, or transplanting a corrupt-state shape all fail); single-author preprint, no external replication [kobrosly-2026]

Context

relational binding, orientation, Plücker coordinates, Grassmannian geometry, activation patching

Papers

Relational Rank Geometry in Transformers: Detecting and Steering Hidden-State Relation Frames — Kobrosly, Mazen2026 · arXiv:2605.29634