Token trajectories trace a curved low-dimensional Langevin manifold

Sarfati et al. track each token's hidden-state trajectory through GPT-2-Medium's 24 layers (replicated on Llama-2-7B, Mistral-7B-v0.1, Llama-3.2-1B/3B) and find ensembles cluster on a low-dimensional curved manifold [sarfati-etal-2024-lines-of-thought] Truncating to the top ~256 of 1024 per-layer SVD dimensions preserves ~90% of the next-token distribution's information [sarfati-etal-2024-lines-of-thought] Layer-to-layer evolution follows a rotation-and-stretch law plus exponentially-growing Gaussian noise, a linear Langevin/Ornstein-Uhlenbeck SDE extracted from ensemble statistics [sarfati-etal-2024-lines-of-thought] Simulated SDE trajectories reproduce real statistics (a linear SVM separates them at near-chance 46-61%), while untrained-model trajectories move in straight parallel lines, so the structure is training-dependent [sarfati-etal-2024-lines-of-thought]