A ~14-D temporal subspace is fragmented by SAEs but captured whole by SASA

Dalili & Mahdavi prove standard single-direction sparse autoencoders are geometrically and dynamically forced to fragment multi-dimensional features into many near-collinear atoms (via a covering-number and basis-instability argument) [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] In GPT-2-Small's residual stream, with no SAE, a day/month/year temporal subspace has intrinsic dimension 14 (the 90%-variance dimension of 768) with a cyclic topology [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] Standard SAEs fragment this concept into 35 atoms (9 day + 16 month + 10 year, a count from Engels et al. 2025), while their Subspace-Aware SAE captures it in one rank-6 group preserving the cyclic topology [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] Feature absorption drops 37.2%->6.6% on GPT-2-Small and 24.0%->18.3% on Mistral-7B-v0.1 at half the training-token budget of a standard SAE [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability]