Relative to Attention–MLP Sufficiency Staging Hypothesis: both hypotheses descend from the same computational-mechanics tradition (Shai et al. 2024) and both concern a network’s internal representation of a process’s state, but this hypothesis claims a curved Mixed-State-Presentation simplex geometry for belief over a stationary, ergodic HMM’s hidden states, while Attention–MLP Sufficiency Staging Hypothesis claims a two-stage linear-subspace separation (a full world state vs. a compressed task-specific state, tied to a specific attention→MLP sublayer boundary) for a non-stationary, non-ergodic process — a genuinely different shape claim about a different architectural location, not a restatement.

Statement

Let a data-generating process have a finite set of hidden causal states with known transition and emission dynamics (a hidden Markov model or epsilon-machine). At each position $t$, an ideal Bayesian observer maintains a belief $b_t = p(\text{state}\mid x_{1:t})$ — a point in the probability simplex $\Delta^{k-1}$ over the $k$ hidden states. The hypothesis claims a sequence-predicting transformer trained on this process’s outputs linearly embeds $b_t$ in its residual stream, via a specific map called the Mixed-State Presentation (MSP): the image of $\Delta^{k-1}$ under repeated Bayesian updating, which is generically a curved, self-similar (sometimes fractal) point set rather than a flat simplex. The model’s next-token distribution should then decompose as a barycentric combination: $p(x_{t+1}\mid x_{1:t}) = \sum_{v} b_t^{(v)}\, p(x_{t+1}\mid v)$, a convex combination of each vertex’s own predictive distribution, weighted by belief coordinates.

Intuition

Most representation-geometry hypotheses on this map (Linear Representation Hypothesis, Conceptual Belief Space Hypothesis) posit structure for informally-defined, human-labeled concepts, where there is no independently derivable “correct” geometry to check against — only self-consistency and causal tests. This hypothesis is unusual in being checkable against a known ground truth: because the generating process’s causal states and their transition dynamics are specified in advance, the exact target geometry (the MSP, however curved or fractal) can be computed analytically and compared directly to what a trained network’s activations reveal, rather than being invented after the fact to fit the data.

Properties

• Grounded in a known generative process, unlike Conceptual Belief Space Hypothesis. Both hypotheses describe “belief that updates as a sequence is read,” but this one concerns a probability distribution over the states of a specified hidden Markov process (with a provably-derivable target geometry), while Conceptual Belief Space Hypothesis concerns belief over informally-defined semantic concepts (emotions, genres) whose target geometry has no independent derivation — a genuinely different grounding for a superficially similar idea, not a restatement of it.
• The target geometry is generically curved, not a flat simplex. Repeated Bayesian updating on processes like Mess3 (a 3-state HMM) or Tom Quantum/Bloch-Walk processes produces an MSP that is a curved, often fractal, self-similar subset of the simplex’s affine hull — the flat $(k-1)$-simplex of Polytope (Simplex) is the convex hull containing the MSP, not the MSP itself.
• Barycentric decomposability is the operational signature. The distinguishing, falsifiable prediction is that a convex combination of per-vertex predictive distributions, weighted by recovered belief coordinates, predicts the model’s actual next-token distribution better than the single best individual feature or latent direction can — a genuine mixture claim, not just “some feature correlates with state.”
• Must be distinguished from a tiling artifact. A set of latents that individually specialize to disjoint sub-regions of state space (see Manifold capture theory (subspace capture / tiling / dilution)‘s tiling/shattering regime) can mimic an apparent simplex-mixture signal near the vertices without any latent actually encoding a graded mixture; the discriminating test is whether the barycentric combination’s advantage holds specifically in the interior of the simplex, where no single specialized latent can substitute for a true mixture coordinate.
• Predicts vertex-directed causal steering. Adding an intervention vector that shifts the recovered belief coordinate toward a target vertex should shift the model’s generated continuations toward that vertex’s characteristic predictive distribution, in proportion to the shift — a causal, not just correlational, consequence of the geometry actually being used.

Key evidence

Shai, Marzen, Teixeira, Gietelink Oldenziel & Riechers (2024, NeurIPS spotlight) originate this hypothesis: they derive the Mixed-State Presentation analytically for known epsilon-machines (Mess3, RRXOR, and related processes) and then train real GPT-2-style transformers from scratch, via ordinary gradient descent, on sequences generated by these processes. Linear regression from the theoretically-derived belief coordinates to the trained network’s own residual-stream activations recovers the predicted (often fractal, Sierpinski-triangle-like for Mess3) MSP geometry with high fit quality at specific layers/token positions, and next-token predictions decompose as the predicted barycentric combination of per-vertex distributions — the founding empirical confirmation for this node, on a real trained network even though the training data is synthetically generated by a known HMM. See shai-etal-2024-real-gpt-2-style-transformers-trained-from-scratch-on-hmm-generated-sequences-linearly-embed-the-theoretically-predicted-fractal-mixed-state-presentation-belief-geometry-in-their-residual-stream.

Kamel, Rastogi, Ma, Ranganathan & Zhu (2025) extend this hypothesis beyond a hand-derivable epsilon-machine to a genuinely learned stochastic game with no closed-form target geometry: a GPT-2-style transformer (87M parameters, 12 heads) trained from scratch on 2M+ synthetically-generated poker hand histories (Poker Hand History / PHH format). PCA of the model’s own activations reveals a recurring triangular geometry — resembling a belief-state simplex — among hand-strength clusters, alongside a strong linear probe for deterministic hand-rank (~80% accuracy) and t-SNE/UMAP clustering by conceptual similarity. Unlike Shai et al.’s epsilon-machine setting, the ground-truth belief geometry here (equity under uncertain opponent hands) cannot be derived in advance and checked against, and the equity/belief-state sub-claim itself rests only on a nonlinear (2-layer MLP, r=0.59) probe with no causal steering performed — flagged accordingly as suggestive, not confirmatory, evidence for the hypothesis on a real learned game rather than a fully specified generative process. See kamel-etal-2025-a-trained-poker-transformer-shows-a-triangular-pca-belief-state-geometry-and-linear-hand-rank-decoding.

Exercises

Base

1. Why can the exact target geometry (the MSP) be computed in advance for this hypothesis, when it cannot for Conceptual Belief Space Hypothesis?

2. A flat 2-simplex (triangle) and a curved, fractal MSP inscribed within it are both described in the Statement. Which one is the object Polytope (Simplex) already covers, and which is new here?

Middle

3. Suppose an SAE cluster’s latents perfectly tile three disjoint regions near the vertices of a belief simplex, with no latent responding anywhere in the interior. Explain concretely why a barycentric-combination prediction could still numerically outperform the single best latent near the vertices, yet this would not count as confirming evidence for the hypothesis.

4. A study finds a positive causal-steering effect toward a target vertex, but no significant barycentric predictive advantage for that same cluster. What does this combination suggest about which part of the hypothesis is and is not supported?

Pro

5. Formalize the “tiling artifact” test using held-out $R^2$: let $R^2_{\text{bary}}$ be the barycentric combination’s held-out coefficient of determination and $R^2_{\text{best}}$ the best single latent’s, both computed separately on near-vertex and simplex-interior sample subsets. State a precise pair of inequalities across both subsets that would constitute strong, artifact-resistant evidence for the hypothesis, and explain why requiring the inequality on only the near-vertex subset is insufficient.