Conceptual Belief Space Hypothesis

structures / Hypotheses / Conceptual Belief Space Hypothesis

CLAIMhypothesisadvancedhow it's classified →

In-context updating is modeled as Bayesian belief updating over a Gärdenfors-style conceptual space: a belief state is a probability distribution over a low-dimensional metric space whose concepts are convex regions, tracing a trajectory as more context is read.

Replicationcomputed from the corpus — never hand-assigned

1 paper1 architecture class1 domain1 model family

Filled = two or more values reported by papers that share no author — replication. Outlined = two or more values, but all from a single study — breadth, not replication. Grey = a single value. Derived from paper authorship and each model's architecture class, domain and family; it updates itself when a paper is added.

Relative to Belief State Geometry Hypothesis (Mixed-State Presentation): both hypotheses describe belief that updates as a sequence is read, but ground this in different targets — this hypothesis concerns informally-defined semantic concepts (emotions, genres) with no independently derivable target geometry, while Belief State Geometry Hypothesis (Mixed-State Presentation) concerns a probability distribution over the known hidden states of a specified generative process (an HMM/epsilon-machine), whose target geometry (the Mixed-State Presentation) can be computed analytically in advance.

Statement

A belief state is modeled as a probability distribution $b_t = p(C \mid x_{1:t})$ over a metric conceptual space $\mathcal{C} = A_1\times A_2\times\cdots\times A_n$ , where each concept occupies a convex sub-region of $\mathcal{C}$ (a conceptual space, in the sense of Gärdenfors). As more of an input sequence $x_{1:t}$ is processed, the belief updates, and the sequence of belief states $b_{1:T}$ traces a trajectory through $\mathcal{C}$ — the geometric analogue of Bayesian posterior updating.

Intuition

Rather than treating “what does the model currently believe about topic X” as a single static point recovered once, this hypothesis treats belief as something that moves smoothly through a structured space as evidence accumulates — closer to watching a point drift across a map as new information arrives than to reading a single fixed coordinate off a snapshot.

Properties

Concepts as convex regions, not points. A concept in a Gärdenfors-style conceptual space occupies a convex sub-region of $\mathcal{C}$ , not a single coordinate — reflecting that a concept typically corresponds to a range of instances (e.g. many shades count as “red”), and that betweenness/interpolation between two clear instances of a concept should itself tend to lie within (or near) the same concept.
In the simplest confirmed case, the space is just a flat 2D subspace. For emotion concepts, the recovered $\mathcal{C}$ reduces to a 2-dimensional Valence–Arousal plane — a flat Linear Subspace of dimension 2, not an exotic curved manifold; the hypothesis’s distinguishing content is the trajectory dynamics on top of this space, not an unusual curvature claim.
A trajectory, not a fixed point, is the object of interest. The sequence $b_1,\ldots,b_T$ is what the hypothesis claims has structure — smoothness, consistency with Bayesian updating, and (claimed) correspondence between the trajectory recovered from external behavior and the trajectory recovered from internal representations.
Steering entanglement should be predictable from distance in $\mathcal{C}$ . If two concepts $c_1,c_2$ occupy nearby regions of $\mathcal{C}$ , intervening to shift belief toward $c_1$ is predicted to also shift belief (unintentionally) toward $c_2$ in proportion to their geometric proximity — a specific, falsifiable prediction linking the recovered geometry to the causal effect of interventions, not just to passive decoding.
Requires two independently-recovered geometries to agree. The hypothesis’s strongest form requires that the same low-dimensional structure emerges whether $\mathcal{C}$ is estimated from external behavior (e.g. judgments/outputs) alone or from internal representations alone — agreement between two independently-derived estimates is a much stronger claim than either one being individually well-structured.

Exercises

Base

If a “concept” occupied a single point rather than a convex region, what specific modeling capability would be lost — i.e. what could no longer be represented?

Solution

Graded or ambiguous membership: with only a single point per concept, there would be no way to represent “this instance is a clear example of joy” versus “this instance is a weak/ambiguous example of joy” as different degrees within the same concept — every instance would either be exactly that one point or not, with no notion of degree, typicality, or a continuum of instances belonging to varying extents.

A belief trajectory $b_1,\ldots,b_T$ is claimed to be “smooth.” Give a precise, checkable meaning for “smooth” in terms of the sequence of points, e.g. using consecutive distances.

Solution

One natural operational definition: the trajectory is smooth if consecutive belief states are close relative to the overall scale of the space — e.g. $\|b_{t+1}-b_t\|$ is small (bounded by some threshold, or decaying/controlled as a function of $t$ ) compared to typical distances between well-separated concepts, so that the path does not “teleport” between distant regions of $\mathcal{C}$ in a single update step.

Middle

Suppose $\mathcal{C}$ is exactly the 2D Valence–Arousal plane, and two concepts occupy disjoint convex regions $R_1,R_2$ . Prove that if a trajectory moves continuously (as a function of a continuous time parameter, an idealization of the discrete-step case) from a point in $R_1$ to a point in $R_2$ , it must pass through points belonging to neither region (assuming $R_1,R_2$ are open, or through the boundary if closed).

Solution

Suppose $\gamma:[0,1]\to\mathcal{C}$ is continuous with $\gamma(0)\in R_1$ , $\gamma(1)\in R_2$ , and $R_1,R_2$ are disjoint open sets. Suppose for contradiction $\gamma(t) \in R_1\cup R_2$ for every $t\in[0,1]$ . Then $[0,1] = \gamma^{-1}(R_1) \cup \gamma^{-1}(R_2)$ , two disjoint sets (since $R_1\cap R_2=\emptyset$ ), both open in $[0,1]$ (preimages of open sets under a continuous map), both non-empty ( $0$ is in the first, $1$ in the second) — but this contradicts the connectedness of $[0,1]$ (a connected space cannot be partitioned into two disjoint non-empty open sets). So $\gamma(t)$ must exit $R_1\cup R_2$ for some $t$ , i.e. pass through a point in neither region.

The hypothesis predicts steering-entanglement magnitude should be a function of geometric distance in $\mathcal{C}$ . Propose a specific, minimal quantitative form for this function (e.g. linking entanglement to distance) and identify what a null result — entanglement showing no relationship to distance — would imply about the hypothesis.

Solution

A minimal candidate: entanglement (e.g. the induced shift in belief toward concept $c_2$ per unit intended shift toward $c_1$ ) decays monotonically with the distance $d(c_1,c_2)$ in $\mathcal{C}$ — for instance, entanglement $\propto \exp(-d(c_1,c_2)/\ell)$ for some length scale $\ell$ , or more weakly, just “monotonically non-increasing in $d$ ” without committing to a specific functional form. A null result (entanglement uncorrelated with $d$ ) would directly undercut the hypothesis’s claim that the recovered geometry is causally meaningful — it would suggest that whatever low-dimensional structure was recovered from passive decoding does not correspond to the structure actually governing how interventions propagate, i.e. the geometry might be descriptively convenient (fits the data) without being the structure the underlying system’s interventions actually respect.

Pro

The hypothesis requires agreement between a geometry recovered from external behavior and one recovered from internal representations. Formalize “agreement” using a tool from this map (e.g. CKA, or principal angles between subspaces) and explain why simply checking that both recovered spaces are low-dimensional is insufficient evidence for agreement.

Solution

Formalize via CKA (see Platonic Representation Hypothesis): construct kernel matrices $K_{\text{behavior}}$ and $K_{\text{activation}}$ from pairwise similarities of the same set of belief states, estimated independently from behavioral judgments and from internal activations respectively, and require $\mathrm{CKA}(K_{\text{behavior}}, K_{\text{activation}})$ to be high. Merely checking that both recovered spaces are low-dimensional (e.g. both well-approximated by 2 or 3 principal components) is insufficient because low-dimensionality alone says nothing about whether the specific relational structure (which points are near which) agrees between the two estimates — two entirely different, unrelated 2D arrangements of the same underlying concepts would both be “low-dimensional” without agreeing at all; CKA (or an analogous relational-similarity measure) is needed specifically because it tests the pairwise-relationship structure, not just the ambient dimensionality, exactly as distinguished in Platonic Representation Hypothesis‘s own properties.

Suppose the recovered conceptual space for a “genuinely unstructured” control domain (concepts with no real semantic relationships to each other) still shows a statistically significant, low-dimensional trajectory structure. What does this imply about the measurement pipeline, and what specific control does this scenario show is necessary before accepting evidence for the hypothesis in a structured domain?

Solution

If an unstructured control domain (chosen specifically because it should have no real conceptual relationships to recover) still yields apparent low-dimensional structure, this implies the measurement/estimation pipeline itself (e.g. the dimensionality-reduction method, or systematic biases in how belief is elicited and represented) can manufacture apparent structure regardless of whether real conceptual structure is present — an artifact of the method, not evidence of genuine belief-space geometry. This shows that any claim of recovered structure in a domain of actual interest (e.g. emotions) needs a matched unstructured control run through the identical pipeline, with the structured domain’s result required to be distinguishably stronger than the control’s, before the recovered geometry can be attributed to real conceptual structure rather than to a pipeline artifact that would appear regardless of the input’s actual content.

Found in (1 observation · 1 family)

Llama

Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space (2026)measured

In-context belief updating traces a low-dimensional manifold

Details

Bigelow et al. measure Llama-3.1-8B-Instruct's belief about story concepts (emotions, genres, an arbitrary control) as an expected value over next-token digit probabilities, tracing a smooth trajectory through a UMAP/PCA-recovered low-dimensional manifold [bigelow-etal-2026] The same low-dimensional structure is recoverable from residual-stream activations, correlating with the behavioral geometry (r=.92) for structured domains but not the arbitrary control [bigelow-etal-2026] For emotions the manifold reduces to a 2D valence-arousal plane matching Russell's circumplex, a flat linearly-probeable subspace [bigelow-etal-2026] Activation-addition steering along a concept direction predictably shifts belief, and unintended steering entanglement is predictable from geometric distance between concepts [bigelow-etal-2026] The model-id and diff-in-means extraction were not confirmable from the available render, so those details are deferred, though the geometric findings hold [bigelow-etal-2026]

models: Llama-3.1-8B-Instruct · method: UMAP, PCA, Linear probing, Difference-in-means direction, Activation Steering (Addition), Geometric analysis