Neural regression collapse coincides with last-layer feature intrinsic dimension below that of the targets (ID_H < ID_Y), which over-compresses and hurts generalization, while non-collapsed regression models keep ID_H > ID_Y

- Andriopoulos, Dong, Adhikari and Ross estimate the intrinsic dimension of last-layer features (ID_H) and of regression targets (ID_Y) across control tasks and synthetic datasets and report that collapsed regression models show ID_H < ID_Y (over-compression, poor generalization) whereas non-collapsed models typically keep ID_H >= ID_Y, identifying over- and under-compressed regimes that depend on data quantity and noise. Sec. 5 further reports that collapsed last-layer features lie on increasingly lower-dimensional nonlinear (curved) manifolds within the n-dimensional linear subspace. Models are small bespoke regression networks trained by the authors: 3-layer MLPs of varying width on MuJoCo and synthetic image-regression tasks, plus a 4-layer CNN encoder with a 3-layer MLP head on Cheetah-run; a generic custom MLP entry is tagged. [andriopoulos-etal-2025-geometric-analysis-of-neural-regression-collapse-via-intrinsic-dimensi]