Word-cooccurrence eigenvectors form a spectral tree recurring in Gemma

The successive leading eigenvectors of a normalized word co-occurrence (PMI-related) Gram matrix progressively separate taxonomic branches from broad to fine, a coarse-to-fine spectral splitting tree [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] Top-k eigenspace alignment far exceeds a shuffled baseline and fits an exponential decay f(d)=1.967*e^(-1.235*d) in WordNet distance d [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] The same spectral-tree geometry, first validated in word2vec, recurs in Gemma-2B unembeddings and in the mid-layer residual stream of Gemma-2B [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] Parent-to-child innovation vectors are near-orthogonal to the parent (a linear-representation signature); the geometry is derived as a prediction of a co-occurrence model, not tested causally [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence]