The family of linear relation-decoding operators is highly compressible and clusters by shared coarse-grained property, not by individual relation identity

- Christ et al. extend Hernandez et al.'s single-relation Linear Relational Embedding (LRE) decoders to collections of relations (47 in the Hernandez dataset, 79 in an extended dataset) and show that order-3 tensor networks (SimpleOrder3Network, TriangleTensorNetwork) trained end-to-end compress the collection: with under one million parameters they outperform baselines (stacked per-relation matrices, about 788 million parameters, mean faithfulness 0.41) in faithfulness; main-text results are for GPT-J, with Llama 3.1 8B and GPT-NeoX-20B in Appendix C. [christ-etal-2025-relation-decoding-operator-structure] - A cross-evaluation protocol, applying each relation's decoder to the subjects of every other relation, yields off-diagonal faithfulness above 0 (several above 0.7) and visible block structure, e.g. characteristic gender, university degree gender and occupation gender sharing the concept of gender; some overlaps are syntactic rather than semantic (Sec. 4.1-4.2, Fig. 2-3). [christ-etal-2025-relation-decoding-operator-structure] - The authors conclude the decoders extract recurring coarse-grained properties rather than distinct relations; this is their interpretation of the cross-evaluation results, and they note that "tensor networks capture structural regularities beyond the simple low-rank property of individual relations" (Sec. 4.3). The paper does not measure a shared linear subspace. [christ-etal-2025-relation-decoding-operator-structure]