A 1-layer RNN solves modular addition via six Fourier frequencies

Rangamani trains a single-layer tanh RNN from scratch on modular addition mod 113 to grokking, then SVDs each weight matrix and finds a few singular directions capture nearly all energy (rank 12 for W_E/W_fc/W_ih, rank 32 for W_hh) [rangamani-2025-fourier-rnn-modular-addition] Just 6 dominant Fourier frequencies (k in {6,15,20,29,47,54}) account for nearly all signal energy, and the 12 significant singular vectors of the unembedding group into exactly 6 frequency-dominated pairs [rangamani-2025-fourier-rnn-modular-addition] This reproduces the multi-frequency Fourier circuit previously found only in transformers inside a recurrent architecture instead [rangamani-2025-fourier-rnn-modular-addition] Removing any single frequency barely hurts, but sequential ablation degrades accuracy monotonically (28.3% after 4, 6.2% after 5, chance 0.885% after all 6), and restricting weights to the orthogonal complement of the low-rank subspace also drops accuracy to chance [rangamani-2025-fourier-rnn-modular-addition]