The effective dimension of a chain-of-thought hidden-state trajectory's covariance spectrum predicts reasoning task hardness

- Masoomi, Bazzaz, Javanmard & Mirrokni (2026) formalize each chain-of-thought reasoning trace as a discrete curve in the hidden-state space of a real LLM (Qwen2.5-0.5B-Instruct), characterizing it via spectral, positional, and kinematic geometric functionals, and define an effective dimension d_rho as the minimum number of principal components of the trajectory covariance needed to capture a rho fraction of variance (Def. 3; Sec. 5.1: 3 MATH500 categories, 9 questions each, 10 trajectories per question at T=0.7). [masoomi-etal-2026-geometric-signatures-of-reasoning] - Theoretically they prove only a linear-algebraic fact (a flat spectrum maximizes d_rho, Prop. 2); the paper's Limitations state the analysis "does not fully explain how task difficulty induces this flatness in a trained transformer". Empirically, hard problems (MATH500 level 5) have higher d_rho than easy ones (level 1): at layer 21, mean d_rho about 170 vs 121.6, a 40% gap (Sec. 5.3, Fig. 3). [masoomi-etal-2026-geometric-signatures-of-reasoning] - Under leave-one-question-out cross-validation the d_rho features reach AUC 0.93 (MLP, best layer 21; AUC 0.81 at layer 0) for easy-vs-hard classification (Sec. 5.3). Separately, seven kinematic/positional features from the first 20% of generated tokens give correctness AUC 0.806 (logistic regression), 0.723 (MLP) and 0.641 (GRU) under a question-level split (Sec. 5.2). All results are from one small model (Qwen2.5-0.5B-Instruct) and a small question set; the authors list this as a limitation. [masoomi-etal-2026-geometric-signatures-of-reasoning]