A fitted linear map between pre- and post-continual-learning representation spaces predicts forgetting
measured in 1 paperKim, Kim & Sohn define a representation discrepancy D as the minimum, worst-case misalignment between a layer's real representation space before and after continual-learning steps, minimized over a purely linear map T with no bias term, and fit this map on real fixed-dimension ResNets, a 9-layer fully-connected ReLU network, and a ViT trained sequentially on Split-CIFAR100 (50 tasks) and ImageNet32 (50 tasks) [kim-etal-2025-measuring-representational-shifts-continual-learning-linear-transformation] The linear-transformation discrepancy correlates strongly with measured representational forgetting from frozen-feature linear probing, R-squared 0.88 on Split-CIFAR100 and 0.74 on ImageNet32 [kim-etal-2025-measuring-representational-shifts-continual-learning-linear-transformation] Forgetting saturates at a training-step horizon of 10 tasks on both datasets, with the saturation point decreasing for deeper layers (deeper layers forget faster) and increasing with network width (wider networks forget slower) [kim-etal-2025-measuring-representational-shifts-continual-learning-linear-transformation]