Definition

Given an overcomplete archetype dictionary $A=(a_1,\ldots,a_c)$ partitioned into disjoint tiles $\{T_i\}$, with each tile’s polytope $P_i=\mathrm{conv}(A_{T_i})$ (the polytope structure over just that tile’s archetypes), the Minkowski Representation Hypothesis (MRH) claims a layer’s activation space $X$ satisfies:

1. Minkowski sum: $X = \bigoplus_i P_i = \{\sum_i p_i : p_i \in P_i\}$
2. Block-convex coding: any activation $x=\sum_{i\in S} z_i A_{T_i}$, $z_i \in \Delta^{|T_i|}$ (each active tile contributes barycentric coordinates over its own archetypes), with $|S|\ll m$ (only a handful of tiles active at once).

Unlike a single categorical-concept polytope (one convex hull over $k$ mutually exclusive values), this is a claim about the entire representation space’s global shape: many independent, tile-local polytopes summed together, with sparsity operating at the level of which tiles are active, not just which vertices within one polytope.

Where this comes from mechanistically

Not merely observed — provably follows from multi-head attention’s own algebra: a single attention head produces outputs in the convex hull of its projected value vectors (softmax weights are barycentric coordinates); affine transformations preserve convexity; and summing multiple heads’ outputs (as in standard multi-head aggregation) sums their individual convex hulls, which is exactly the Minkowski-sum operation — so MRH-consistent geometry is a structural consequence of the architecture, not an accident of training, under stated conditions.

Relative to a single polytope / feature lobes

Polytope (Simplex) describes one categorical concept’s convex hull in isolation; Feature Lobes (Spatial-Functional Modularity) describes coarse spatial-functional clustering of many features with no claim about how activations compose those regions. MRH is a distinct, more specific claim: the entire space is built by literally summing many tile polytopes, and any single activation’s position is explained by which small subset of tiles contributed and with what barycentric weights — a compositional, block-sparse generalization of the single-polytope idea to a whole layer.

Key evidence

Fel, Wang, Lepori, Kowal, Lee, Balestriero, Joseph, Lubana, Konkle, Ba & Wattenberg (ICLR 2026, arXiv:2510.08638) test MRH on DINOv2-B (ViT-Base, 4 register tokens, $d{=}768$) using a 32,000-atom SAE dictionary ($k{=}8$ active codes, $R^2{>}88\%$ reconstruction) trained on 1.4M ImageNet-1K images. Three converging tests: (1) curved geodesics — straight-line interpolation between tokens rapidly leaves the data manifold, while shortest paths along a token k-NN graph (consistent with walking across polytope faces) stay close to it; (2) convex coding — classical Archetypal Analysis (Archetypal Analysis / Simplex Fitting (AANet)), the single-tile ($|S|{=}1$) special case of MRH, matches SAE reconstruction quality with only 10 archetypes per image despite being far more constrained (points forced into the convex hull of observed tokens); (3) block structure — the archetypal coefficient matrix shows spontaneous block-diagonal co-activation clusters matching the tile partition. Also finds antipodal concept-direction pairs (e.g. vertical/horizontal orientation), a 2D positional subspace emerging by the final layer, and sharper singular-value decay / higher pairwise-inner-product coherence in the SAE dictionary than random or Grassmannian-frame baselines. The authors are explicit that this is compatible, not conclusive, evidence (“multiple mechanisms can mimic the same surface phenomena”) and run no causal steering experiment of their own. See dino-activation-space-is-consistent-with-a-minkowski-sum-of-tile-polytopes.