Digit basins stratified by carry-potential fibers predict off-by-one errors
measured in 1 paperWen et al. apply UMAP to Qwen3-4B's final-layer activations during multi-digit addition and find ten discrete digit basins (trustworthiness 0.9953), one per output digit [wen-etal-2026-shape-of-addition]
Each basin is stratified by continuous Iso-Raw-Sum-Trajectory fibers indexed by a Carry Potential whose integer part is the true carry [wen-etal-2026-shape-of-addition]
A noisy-quantization bathtub model fits the per-position error rate (R^2=0.80, replicated at R^2>=0.70 in Qwen3-8B and Gemma-3-4B-IT), explaining why 93.19% of errors are off-by-one and the basins do not wrap circularly [wen-etal-2026-shape-of-addition]
Contrastive-centroid steering flips the predicted digit at a magnitude that depends continuously on fiber position, and an inference-time Dual-Stream correction improves accuracy across all three models [wen-etal-2026-shape-of-addition]
Context
discrete digit-basin clustering (UMAP, trustworthiness 0.9953) coexisting with continuous carry-potential fiber stratification within and across basins, Carry Potential Phi_p as a continuous fiber coordinate whose fractional part indexes position relative to a carry-flip boundary, noisy-quantization bathtub-curve model (R^2=0.80-0.70 across 3 models) linking fiber position to arithmetic error probability, explanation of the 93.19% off-by-one error rate via local fiber-boundary proximity rather than global rotational/circular dynamics, contrastive-centroid steering with boundary-dependent flip thresholds, and an inference-time Dual-Stream correction method validated across 3 models, explicit positioning as a refinement of Nanda et al.'s circular modular-arithmetic geometry and Kantamneni & Tegmark's generalized-helix hypothesis
Papers
The Shape of Addition: Geometric Structures of Arithmetic in Large Language Models — Wen, Liuyuan, Zhu, Xun, Huang, Lihao, Li, Wenbin, Gao, Yang2026 · arXiv:2606.03645