A Mamba model discovers the corner geometry of the belief simplex

measured in 1 paper

Agarwal, Dalal & Misra train a from-scratch Mamba/SSM on HMM filtering and find its final-layer representations organize into five discrete clusters, one per hidden state [agarwal-etal-2026] The paper describes this as discovering the corner geometry of the belief simplex, concentrating at the vertices rather than the transformer's continuous interior-populated Mixed-State Presentation [agarwal-etal-2026] A permutation/relabeling test confirms the clustering tracks genuine posterior structure, not arbitrary state IDs [agarwal-etal-2026] Mamba realizes belief accumulation and belief transport but not random-access binding, distinguishing it from the transformer's richer continuous geometry [agarwal-etal-2026]

Structure

Polytope (Simplex), Belief State Geometry Hypothesis (Mixed-State Presentation)

Context

Bayesian inference, hidden Markov models, state-space models, belief state, computational mechanics, architecture comparison, toy/synthetic sequence model

Confirmed in models

Bayesian Wind Tunnel Mamba (HMM filtering task, 9 layers, d_model=256, state dim 16) · Bayesian Wind Tunnel Mamba

Method

PCA, Geometric analysis

Papers

The Bayesian Geometry of Transformer Attention — Agarwal, Naman, Dalal, Siddhartha R., Misra, Vishal2026 · arXiv:2512.22471