Raw activations look unchanged under real SFT, but their SAE decomposition reveals sharp directional drift

Chopra fine-tunes real Gemma-3-1B-IT on four tasks (MultiNLI, GSM8K, WildJailbreak safety, tool-calling), finding raw hidden-activation cosine similarity between base and SFT checkpoints stays at 0.96-0.999 across layers [chopra-2026-mechanistic-investigation-of-supervised-fine-tuning] The same activations, projected through a frozen, independently pretrained GemmaScope-2 sparse autoencoder, show cosine similarity collapsing much further -- e.g. layer 22 falling to 0.557 on MultiNLI and 0.708 on GSM8K [chopra-2026-mechanistic-investigation-of-supervised-fine-tuning] SVD of the activation-delta matrix finds early-layer drift is low-rank (one principal component explains 78.7% of variance at layer 7 for MultiNLI) while late-layer drift is distributed (32.6% at layer 22), and the safety-tuning task shows an inverted early/late SAE-feature flip ratio (3.00) relative to the other three tasks (0.2-1.0) [chopra-2026-mechanistic-investigation-of-supervised-fine-tuning]