Fourier periodicity in number embeddings is universal; linear separability is not

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]