OthelloGPT

Research (Li et al. 2023, custom-trained on synthetic Othello move sequences)

Structures found in this family (3)

Linear Subspace, Nonlinear World-Model Decodability, Linear Direction

By model (1)

OthelloGPT

Linear Subspace, Nonlinear World-Model Decodability, Linear Direction

Observations (4)

A tensor-product probe matches linear-probe accuracy on real OthelloGPT with fewer parameters, revealing a shared low-rank role-filler factorization and a saddle-shaped role manifold

Nonlinear (small-MLP) probes reliably decode the full Othello board state from an 8-layer, 512-dim GPT-style transformer's internal activations at every move, and activation-patching interventions that flip the decoded board-state representation causally change the model's subsequent move predictions

Othello-GPT's board direction is decodable early but causally used only deep

Othello-GPT's board state is linear under the model's own relative frame

Papers

Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions (2026), Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task (2023), Linear Latent World Models in Simple Transformers: A Case Study on Othello-GPT (2023), Emergent Linear Representations in World Models of Self-Supervised Sequence Models (2023)