Input numbers and arithmetic calculation results are linearly encoded in LLM hidden states, and are causally controllable via simple vector addition

- Zhu, Dai & Sui construct a synthetic addition-problem dataset and use linear probes to read out input numbers from real LLaMA-2-7B/13B and Mistral-7B hidden states across layers, finding number values (probed as log2 of the value, Ridge regression) extractable from early layers onward with high Pearson correlation and R^2, though not precisely: approximate accuracy (error under 1%) stays below 50% on all layers (Sec. 3.1); all models are base models without fine-tuning (Sec. 2.5). [zhu-etal-2024-lms-encode-value-of-numbers-linearly] - The same probe-based pattern holds for the model's own predicted output (probed at the last input token, not just the input operands), and two-layer MLP probes show no clear advantage over linear probes, which the authors read as the values being represented linearly 'or at least near-linearly' (Sec. 3.4). [zhu-etal-2024-lms-encode-value-of-numbers-linearly] - Causal vector-addition interventions on real Mistral-7B (adding alpha=2 times the normalized probe-coefficient direction to the residual stream on 4-digit addition prompts, Sec. 4.2) make the output larger than the original prediction in up to 73% of 1,000 test items (layers 14-19), well above null and random-direction baselines, supporting a causal role for the probed direction (success = a larger output, intervening on 6 consecutive layers at every generated token; the effect is layer-dependent and early-layer intervention destroys the output); the authors report that trying to shift the output to a specific new value o+o' did not work (App. I.3), and that probes cannot reconstruct the number precisely, so the control is of direction (larger/smaller), not exact value; activation patching (Sec. 4.1) is also used. [zhu-etal-2024-lms-encode-value-of-numbers-linearly]