A single function vector injected into the residual stream of Gemma-2-2B rescues zero-shot abstract-rule accuracy, but only when the downstream retrieval heads are intact

Wessel (2026) builds one function vector per rule (ABA / ABB identity rules over arbitrary tokens) in Gemma 2-2B by summing the mean outputs z W_O of the 29 causal-mediation-significant attention heads from 10-shot prompts [wessel-2026-in-context-learning-amplifies-a-latent-symbolic-circuit] Adding s=25 times this vector to the residual stream at layer 12 (last query position) of held-out 0-shot prompts raises accuracy from 1.0% to 86.0% on ABA and from 11.0% to 88.5% on ABB, n=200 per cell, while a norm-matched random vector gives 0.0% / 0.5% [wessel-2026-in-context-learning-amplifies-a-latent-symbolic-circuit] Rescue plateaus around s=25 across layers 6-16 and collapses at s=100 (ABA at layer 12: 21.5%) [wessel-2026-in-context-learning-amplifies-a-latent-symbolic-circuit] Ablating the 9 retrieval heads drops the rescue to 13.0% (ABA) and 28.5% (ABB), whereas ablating the 7 symbolic-induction heads leaves ABB unchanged (89.0%) and lowers ABA to 65.0%, so the vector substitutes for the induction stage but still needs the retrieval stage to be read out [wessel-2026-in-context-learning-amplifies-a-latent-symbolic-circuit] The vector is tested only in Gemma 2-2B; Llama 3.1-8B and Qwen 3-4B are used for the circuit-patching results, not the vector injection [wessel-2026-in-context-learning-amplifies-a-latent-symbolic-circuit]