Definition

Where Linear Direction and Linear Subspace each describe a single feature site, a relation frame is the hidden-state configuration of the $k$ token positions that jointly instantiate a $k$-argument relation (e.g. the three role-tokens of a ternary relation). Rigorously it is an ordered configuration $F=(h_1,\ldots,h_k)\in(\mathbb{R}^d)^k$, from which we read three independently-manipulable, well-defined pieces: the centroid $\bar h=\frac1k\sum_i h_i$; the centered shape $Z=[(h_1-\bar h)^\top;\ldots;(h_k-\bar h)^\top]$; and the spanned subspace $V_F=\operatorname{rowspan}(Z)$, a point of the Grassmannian $\mathrm{Gr}(r,d)$ with $r=\dim V_F$. The orthogonally-invariant summary of the shape is the Gram matrix $G_F=ZZ^\top$ (invariant under any orthogonal change of the ambient basis).

Caveats on the “orientation sign” (corrected). A tempting scalar handedness label $\operatorname{sign}\det(U_k[t,:])$ built from left singular vectors of the projected tuple is not a data invariant as usually stated:

• SVD singular vectors are defined only up to a per-column sign; flipping $u_j\mapsto -u_j$ flips the determinant. With repeated singular values the corresponding singular subspace can be orthogonally rotated, making the sign still less determined. So a bare $\pm1$ requires an explicit, fixed sign/orientation convention to mean anything.
• Any determinant sign is meaningful only after fixing an orientation of the ambient space; an orthogonal change of coordinates with determinant $-1$ flips it.
• The centered configuration has rows summing to zero, so $\mathrm{rank}(Z)\le k-1$: there is generically no honest “$k$-th” direction for the centered shape — the natural cap is $k-1$.

If oriented information is genuinely needed, use the exterior product $v_1\wedge\cdots\wedge v_r$ / Plücker coordinates after fixing orientation conventions, rather than a single unqualified sign.

Relative to linear-subspace and linear-direction

A Linear Direction or Linear Subspace is a single fixed geometric object shared across many activations (the same “truth direction” for every true/false statement). A relation frame is instead a per-instance, multi-token configuration — a different ordered tuple of vectors for every relation instance in every prompt — and the object of interest is how that tuple’s own internal shape and orientation vary with whether the tokens are genuinely relation-bound (versus scrambled or random). It generalizes the single-site view to an explicitly higher-arity one: a $k$-argument relation should show its clearest signature at Plücker rank $k$, not at rank 1.

Key evidence

Kobrosly (2026) uses Plücker sign entropy — the entropy of orientation signs over many sampled tuples — to test this on synthetic controlled-arity prompts ($r=3,\ldots,6$ arguments) across Llama-3.1 8B/70B/405B-Instruct: true relation tuples have consistently lower sign entropy (i.e. more orientation-consistent) at the arity-matched rank $k=r$ than scrambled or random-tuple controls, with positive diagonal margins in every headline row and in a held-out layer-selection audit, most uniform in the 405B model. In a separate edge-grid clean/corrupt steering assay (8×8 grid, YES/NO relation markers), patching the shape of a corrupt relation frame toward the clean frame’s shape — while leaving the centroid fixed — recovers clean-answer behavior and clean residual relation geometry in Llama-3.1-70B/405B-Instruct nearly as well as patching the whole frame, whereas moving only the centroid, or adding equal-norm noise, recovers almost nothing. See Plücker sign entropy and Relation-frame shape steering for the two techniques.

How to detect it

Fix a relation of known arity $r$, sample matched true/scrambled/random token tuples, project into a shared low-rank analysis subspace, and compare orientation-sign entropy across candidate ranks $k$: enrichment specifically at $k=r$ (not at other tested ranks) indicates the relation is represented as an $r$-way oriented configuration rather than reducible to $r$ independent single-token directions.

Key papers

• Kobrosly (2026). Relational Rank Geometry in Transformers: Detecting and Steering Hidden-State Relation Frames. arXiv:2605.29634 — origin of Plücker sign entropy and relation-frame steering; independent-researcher preprint, single-paper evidence so far.