A poker-trained transformer's own activations show a triangular PCA belief-state geometry

Kamel, Rastogi, Ma, Ranganathan & Zhu (2025) train a real GPT-2-style transformer (87M params, 12 heads) from scratch on 2M+ synthetically-generated poker hand histories (PHH format) [kamel-etal-2025-emergent-world-beliefs-transformers-stochastic-games] A linear probe decodes deterministic hand-rank at ~80% held-out accuracy, and PCA/t-SNE/UMAP of the model's own activations reveals genuine clustering by hand-rank and conceptual similarity, including a recurring triangular geometry resembling a belief-state simplex [kamel-etal-2025-emergent-world-beliefs-transformers-stochastic-games] The equity/belief-state sub-claim rests only on a nonlinear (2-layer MLP) probe (r=0.59); no causal intervention (steering along probe directions) is performed, flagged as suggestive rather than confirmatory evidence [kamel-etal-2025-emergent-world-beliefs-transformers-stochastic-games]