MATH · IN · MODELS

Code-reasoning trajectories form discrete clusters that sharpen with scale

measured in 1 paper

- Chain-of-thought hidden-state trajectories on code-reasoning tasks organize into a small number of discrete, silhouette-validated clusters ("strategic modes"), found via PCA plus a k-means sweep maximizing silhouette. [anderson-2026-geometry-of-thought] - The clustering sharpens with scale: the optimal cluster count rises k=2 (Llama-3-8B-Instruct) to k=5 (Llama-3.1-70B-Instruct) and the maximum silhouette from 0.133 to 0.417 (a 213% gain, the study's highest). [anderson-2026-geometry-of-thought] - This "Lattice" regime is one of three in the paper's scale-dependent phase diagram; other domains behave differently (Law collapses/aligns, Math and Science are scale-invariant), so the discrete-mode result is specific to code. [anderson-2026-geometry-of-thought] - Observational only, on two Llama-3 instruct checkpoints (8B, 70B); the author flags correlation-not-causation and runs no activation-level intervention. [anderson-2026-geometry-of-thought]

Context

k-means clustering, silhouette score, chain-of-thought, code reasoning, discrete modes, scale

Papers

The Geometry of Thought: How Scale Restructures Reasoning in Large Language Models — Anderson, Samuel Cyrenius2026 · arXiv:2601.13358