Frozen Mamba collapses into an extreme anisotropy cone
measured in 1 paper- Frozen Mamba-130m activations collapse into an extremely narrow anisotropy cone: mean pairwise cosine similarity 0.9999 (std 4.4e-5), against RoBERTa-base's ~0.9839. [wagh-singh-2026-lost-in-state-space-probing-frozen-mamba] - The collapse cripples frozen probes (CoLA MCC 0.000 for the raw final state; patched-Mamba 0.246 vs RoBERTa 0.475; STS-B 0.316 vs 0.650). [wagh-singh-2026-lost-in-state-space-probing-frozen-mamba] - A geometry-targeted orthogonal write-injection (eta=0.5) fails to widen the cone and worsens STS-B (0.316 to 0.259), a candid negative result implying geometry correction needs training-time adaptation. [wagh-singh-2026-lost-in-state-space-probing-frozen-mamba] - The comparison model is RoBERTa-base, not BERT, and the "~10x-tighter" figure is the angular ratio between cosine 0.9999 and 0.99, not a measured Mamba-vs-baseline cone ratio. [wagh-singh-2026-lost-in-state-space-probing-frozen-mamba]