LLMs encode each decimal digit as its own circle

Levy & Geva show Llama-3-8B and GPT-4o make numerical errors scattered by digit-edit-distance rather than distributed around the true value, suggesting digit-wise rather than value-space representation [levy-geva-2025-digit-representations-base-10] Circular probes fitting [cos(2*pi*x/b), sin(2*pi*x/b)] per digit reconstruct digits best in base 10 (91% Llama, 92% Mistral), far above every other base (all under ~20%) and above a matched linear probe [levy-geva-2025-digit-representations-base-10] Each digit position occupies its own circle, orthogonal to the others, in both models despite different tokenization (Llama whole-number, Mistral per-digit) [levy-geva-2025-digit-representations-base-10] Rotating a hidden state by +5 mod 10 along one digit's circular-probe directions shifts the generated digit (exact hit up to 15%, off-by-one up to 50%), versus under 1% for a matched linear intervention and 0.1% random [levy-geva-2025-digit-representations-base-10] Each digit's code is a single base-10 circle (one dominant frequency), distinct from the multi-frequency Fourier findings for addition computation [levy-geva-2025-digit-representations-base-10]