Concept difference matrix has a layer-stable leading singular direction (Spectral Principal Path) in four LLMs

Tian et al. take the leading right singular vector of the last-token concept-difference matrix D^(l) (80 COCO-derived paired sequences per concept) at every layer as the Spectral Principal Path (SPP) [tian-etal-2025-spectral-principal-paths] Across Qwen 2.5-7B, Llama 3.1-8B-Instruct, Mistral 7B-Instruct-v0.3 and Wizard-Vicuna-30B the SPP's spectral gap sigma1/sigma2 exceeds a shuffled baseline, most clearly in the first layers [tian-etal-2025-spectral-principal-paths] Adjacent-layer |cos(v1^(l), v1^(l+1))| exceeds 0.85 in most layers; mean 0.853 / 0.847 / 0.881 / 0.785 for the four models (Table 1, diverse contexts) [tian-etal-2025-spectral-principal-paths] Prepending a shared long prefix (homogeneous contexts) lowers mean angular stability by 0.06-0.09 and the mean spectral gap by 0.23-0.50, which the authors read as context incoherence being necessary for SPP stability [tian-etal-2025-spectral-principal-paths] The destructive-interference ratio of per-sample perturbations is consistently 2.7-2.9x; the input-space linearity hypothesis and Wedin-bound stability guarantee are the authors' own theory [tian-etal-2025-spectral-principal-paths]