Fine-tuned GPT-2-XL solves addition with sparse Fourier features

measured in 1 paper

Zhou et al. fine-tune GPT-2-XL on addition (99.74% test accuracy) and use the logit lens to show it progressively refines its answer layer by layer rather than retrieving a memorized fact [zhou-etal-2024-fourier-features-for-addition] Fourier-transforming MLP and attention logits reveals sparse outlier components with periods ~2, 2.5, 5, 10, the same periods present in the pretrained token embeddings [zhou-etal-2024-fourier-features-for-addition] MLP layers concentrate low-frequency magnitude-approximating components while attention layers concentrate high-frequency modular-classification components [zhou-etal-2024-fourier-features-for-addition] Fourier-domain ablation confirms causality: removing high-frequency attention/MLP components drops accuracy to 27.08% with small-magnitude errors, and removing low-frequency MLP components drops it to 35.89% with off-by-10/50/100 errors [zhou-etal-2024-fourier-features-for-addition] A from-scratch model shows no Fourier structure and lower accuracy (94.44%), while injecting pretrained embeddings rescues both the structure and the accuracy [zhou-etal-2024-fourier-features-for-addition]

Structure

Lissajous Curves

Context

sparse Fourier decomposition of addition logits, MLP low-frequency magnitude approximation vs. attention high-frequency modular classification, frequency-domain ablation confirms causal role of each component, pre-training is necessary for Fourier features to emerge

Confirmed in models

GPT-2-XL · GPT

Method

Fourier analysis of weights and activations, Causal interventions (steering), Logit Lens

Papers

Pre-trained Large Language Models Use Fourier Features to Compute Addition — Zhou, Tianyi, Fu, Deqing, Sharan, Vatsal, Jia, Robin2024 · arXiv:2406.03445