Linear concept directions (e.g. factuality) in real Gemma-3-27B/12B/4B representations change dramatically over the course of a (simulated) conversation -- information represented as factual at the start can be represented as non-factual by the end and vice versa -- with the effect strongest in the largest model and content-dependent rather than uniform

- Lampinen, Li, Hosseini, Bhardwaj & Shanahan study the dynamics of linear concept directions in real Gemma-3-27B, Gemma-3-12B, and Gemma-3-4B representations across the course of simulated conversations, finding that a linear direction's readout along the same fixed direction can change dramatically over a conversation -- e.g. information represented as factual at the start of a conversation can be represented as non-factual by the end, and vice versa. [lampinen-etal-2026-linear-representations-change-over-conversation] - These representational changes are content-dependent (conversation-relevant information changes while generic information is preserved), robust even for directions constructed to disentangle factuality from more superficial response patterns, occur across different layers of the model, and do not require on-policy conversations -- even replaying a conversation script written by an entirely different model produces similar changes. Gemma-3-27B (the largest model tested) shows the strongest effect, Gemma-3-12B shows a qualitatively similar but weaker effect, and Gemma-3-4B shows little reliable change. [lampinen-etal-2026-linear-representations-change-over-conversation] - They further show (appendix B.8, using a separate pre-answer 'behavioral factuality' direction) that steering along a direction can have opposite behavioral effects in an empty context and after the chakras conversation, but more consistent effects for the consciousness conversation, so a direction's causal effect is not guaranteed to be conversation-invariant. The paper notes it evaluated a relatively small set of conversations and cannot verify the causal role of the post-answer representations themselves; the opposite-day flip is also reproduced (noisier) in Qwen3 14B. [lampinen-etal-2026-linear-representations-change-over-conversation]