MATH · IN · MODELS

Modular-addition networks implement an approximate Chinese Remainder Theorem

measured in 1 paper

McCracken et al. propose that networks solving modular addition universally implement an approximate Chinese Remainder Theorem algorithm built from approximate cosets [mccracken-etal-2025-universal-abstract-algorithm-modular-addition] Across neurons, neuron clusters, and whole trained MLPs and transformers, they measure that neurons activate on approximate cosets in early layers or on linear combinations of them in deeper layers, rather than exclusively on single cosets [mccracken-etal-2025-universal-abstract-algorithm-modular-addition] The measured feature count matches the theory's predicted O(log n) scaling [mccracken-etal-2025-universal-abstract-algorithm-modular-addition] The work extends the Fourier/group-representation-theoretic universality cluster of Nanda et al. (2023) and Chughtai et al. (2023) with a coset-structure measurement, and performs no causal intervention [mccracken-etal-2025-universal-abstract-algorithm-modular-addition]

Structure

Context

approximate coset structure, Chinese Remainder Theorem algorithm, feature-count scaling

Papers

Uncovering a Universal Abstract Algorithm for Modular Addition in Neural Networks — McCracken, Gavin, Moisescu-Pareja, Gabriela, Letourneau, Vincent, Precup, Doina, Love, Jonathan2025 · arXiv:2505.18266