Persona vectors from early pretraining checkpoints already steer traits and still steer the post-trained Instruct models

Moskvoretskii et al. extract difference-of-means persona vectors v in R^d (evil, sycophantic, impolite, humorous) at 17 OLMo-3-7B pretraining checkpoints and add them to the residual stream at layer 16 or 20; they steer the same checkpoint from about 0.22% of pretraining tokens, and humor barely emerges by the end [moskvoretskii-etal-2026-tracing-persona-vectors-through-llm-pretraining] Vectors extracted from base checkpoints still steer the post-trained OLMo-3-7B-Instruct, and Apertus-8B (15 checkpoints, first available at 1.4% of pretraining) shows the same qualitative pattern [moskvoretskii-etal-2026-tracing-persona-vectors-through-llm-pretraining] Matched-magnitude random-direction and label-shuffled controls stay near zero while the real vector's trait delta is 3-10x larger (five OLMo-3 checkpoints, three seeds) [moskvoretskii-etal-2026-tracing-persona-vectors-through-llm-pretraining] Cosine similarity of each checkpoint's vector with the final-pretraining vector rises from about 0.3 at the earliest extractable checkpoint, so the direction keeps refining while remaining steerable [moskvoretskii-etal-2026-tracing-persona-vectors-through-llm-pretraining]