Sequential post-training (SFT, preference optimization, safety/refusal tuning, math/code specialization, long-CoT tuning, chained across stages) progressively compresses real LLMs' hidden-state representations into a low-rank, anisotropic, and homogeneous feature space

- Liu, Chen, Wang, Xu, Lin, Zhang, Zhou, Yang, Wu & Sun define a measurement suite for hidden states, logits, token trajectories, and LoRA updates, and apply it across chains of post-training stages (supervised fine-tuning, preference optimization, safety/refusal tuning, math and code specialization, long chain-of-thought tuning, under controlled stage orderings) on real Qwen2.5-1.5B, TinyLlama-1.1B, Llama-3.2-1B, OLMo-1B, Qwen2.5-7B, and Mistral-7B. [liu-etal-2026-representation-collapse-sequential-post-training] - For one reported model family (Table 4; the identical values are labelled Qwen2.5-1.5B in App. D Table 8), normalized effective rank falls from 0.74 (base) to 0.58 and anisotropy rises from 0.08 to 0.19 across the chain General SFT, Math SFT, Code SFT, Safety SFT, while the target score rises 54.2 to 70.1 and the future-task score falls 71.0 to 60.7. The paper frames this as concentration accumulating over the stage chain, with collapse strongest in middle and late blocks (Fig. 2) and dependent on the objective (Fig. 3); it says its claim is narrower than "post-training always collapses LLMs". [liu-etal-2026-representation-collapse-sequential-post-training] - The paper links this concentration to lower later plasticity (framed in the abstract as a hypothesis, tested with a mixed-effects regression; collapse index vs future score in Table 5 across TinyLlama-1.1B, Llama-3.2-1B, Qwen2.5-1.5B, Qwen2.5-7B), and states that the correlations are not by themselves causal evidence. Weaker out-of-domain generalization and poorer calibration are part of the stated hypothesis; App. B Fig. 9 has calibration/retention panels but the text gives no numerical values for them. The mitigations (mixed-domain replay, feature refresh, representation-diversity regularization, LoRA update decorrelation) are evaluated as a Pareto tradeoff (Fig. 5). [liu-etal-2026-representation-collapse-sequential-post-training]