Chess-GPT linearly encodes editable board state and player skill
measured in 1 paperKarvonen trains two character-level chess GPTs from scratch on 16 million Lichess games with no rules given [karvonen-2024-chess-gpt] A per-square linear probe recovers board state at up to 99.6% (16L) and 99.1% (8L), versus about 75% on a randomly-initialized model [karvonen-2024-chess-gpt] Subtracting a piece's probe direction from the residual stream raises the edited-board legal-move rate from about 40% to 90-92% [karvonen-2024-chess-gpt] A diff-in-means skill direction (high- minus low-skill games) predicts latent Elo, with a binary skill probe at 90.5% / 88.6% [karvonen-2024-chess-gpt] Adding the skill direction raises the 16-layer model's win rate versus Stockfish level 0 from 16.7% to 43.2% on random openings, and lowers it to 5.9% with the negative direction [karvonen-2024-chess-gpt]