Transcoder-decoded planning directions causally control future tokens

Hanna & Ameisen decompose real Qwen3 (0.6B-14B) MLP activations via transcoders and attribution graphs to find planning-feature directions representing a not-yet-generated token [hanna-ameisen-2026-latent-planning-emerges-with-scale] Ablating these features where the model succeeds harms performance, while upweighting them 5x where it fails drastically improves performance [hanna-ameisen-2026-latent-planning-emerges-with-scale] A parallel rhyme-planning circuit is verified the same way [hanna-ameisen-2026-latent-planning-emerges-with-scale] Directional up/down-weighting (x7 / x-3) redirects couplet completions toward or away from a target rhyme [hanna-ameisen-2026-latent-planning-emerges-with-scale]