A tensor-product probe matches linear-probe accuracy on real OthelloGPT with fewer parameters, revealing a shared low-rank role-filler factorization and a saddle-shaped role manifold

A tensor-product representation (TPR) probe -- 64 role (square) embeddings and 3 filler (color) embeddings bound by a shared bilinear matrix -- is fit to real OthelloGPT's board-state representation, trained on 20 million real Othello game transcripts [lee-etal-2026-tensor-product-representation-probes] The TPR probe matches ordinary 192-independent-direction linear-probe accuracy (about 99%) while using only 57.5% of the parameters (56,582 vs. 98,304), demonstrating the 192 previously-independent linear directions share a common low-rank factorized basis [lee-etal-2026-tensor-product-representation-probes] Isomap applied to the learned role embeddings reveals a saddle-shaped manifold separating board rows from columns, while the first two principal components of the filler embeddings separate empty-vs-occupied and current-vs-opponent color [lee-etal-2026-tensor-product-representation-probes]