Toy transformers build an orthogonal hypothesis frame for Bayesian updating

measured in 1 paper

Agarwal et al. train small from-scratch transformers (a 6-layer/6-head/d192 bijection model, a 9-layer/8-head/d256 HMM model) on synthetic tasks with analytically known Bayesian posteriors [agarwal-etal-2026] The transformers match the Bayes-optimal predictive entropy to 10^-3-10^-4 bits, while capacity-matched MLPs fail by orders of magnitude [agarwal-etal-2026] Layer-0 attention keys form a near-orthogonal "hypothesis frame" (mean off-diagonal key cosine 0.052+/-0.004 vs 0.082+/-0.003 random), built by a single identifiable head [agarwal-etal-2026] Query-key similarity sharpens with depth onto still-feasible hypotheses, mirroring Bayesian conditioning as evidence accumulates [agarwal-etal-2026] PCA of attention-output values reveals a smooth low-dimensional entropy-tracking structure (a curved 1-D entropy manifold) that keeps refining after the key frame stabilizes [agarwal-etal-2026] Layer/head ablations and length-generalization support this as a reused computation; across architectures Transformers realize all three inference primitives, Mamba two, LSTM one, MLP none [agarwal-etal-2026]

Structure

Linear Subspace, Linear Direction

Context

Bayesian inference, in-context learning, attention mechanism, toy/synthetic sequence model, architecture comparison, cross-entropy optimum

Confirmed in models

Bayesian Wind Tunnel Transformer (bijection task, 6 layers, 6 heads, d_model=192) · Bayesian Wind Tunnel Transformer, Bayesian Wind Tunnel Transformer (HMM filtering task, 9 layers, 8 heads, d_model=256) · Bayesian Wind Tunnel Transformer

Method

Analytical derivation, PCA, Linear probing, Geometric analysis, Causal interventions (steering)

Papers

The Bayesian Geometry of Transformer Attention — Agarwal, Naman, Dalal, Siddhartha R., Misra, Vishal2026 · arXiv:2512.22471