2D Grid (Square Lattice)

structures / Manifolds / 2D Grid (Square Lattice)

OBJECTgeometric-objectembedded-setK:zerosquare lattice (bounded 2D grid graph, open boundary)intermediatehow it's classified →

A finite 2-dimensional square-lattice arrangement of points — items placed at integer grid coordinates (i,j) with horizontal/vertical nearest-neighbour adjacency. Distinguished from a torus by being bounded and non-periodic (contractible, open boundary; no global wrap-around cycle vs. H1=Z^2), and from a 1D continuum by having two independent lattice coordinates rather than one.

Replicationcomputed from the corpus — never hand-assigned

1 paper1 architecture class1 domain2 model families · one study

Filled = two or more values reported by papers that share no author — replication. Outlined = two or more values, but all from a single study — breadth, not replication. Grey = a single value. Derived from paper authorship and each model's architecture class, domain and family; it updates itself when a paper is added.

Definition

A 2D grid (square lattice) is a finite set of points placed at integer coordinates $(i,j)$ with $1 \le i \le m$ , $1 \le j \le n$ , together with edges joining each point to its horizontal and vertical nearest neighbours $(i\pm1, j)$ and $(i, j\pm1)$ . Embedded in a $d$ -dimensional activation space, it appears as a set occupying two independent lattice coordinates over a bounded region.

Crucially, the grid is not periodic: there is no identification of the top edge with the bottom or the left with the right. Points on the boundary have fewer neighbours than interior points (corners have 2, edges 3, interior 4). This open boundary is the property that separates it from a torus, whose two directions wrap around ( $T^2 = \mathbb{R}^2/\mathbb{Z}^2$ , see Torus).

Intuition

A checkerboard, or a mesh. From an interior point you can walk in two independent directions — left/right and up/down — but eventually you hit an edge and stop, rather than reappearing on the far side. Two independent coordinates like a torus, but with a rim: a bounded patch, not a donut.

Properties

Two independent coordinates → intrinsic dimension 2, but the object is a discrete bounded lattice, not a smooth surface. This is the discriminating facet against 1D continuum manifold (one coordinate, dimension 1) and against Circle (one cyclic coordinate).
Bounded and non-periodic → contractible. As a bounded 2D region sampled on a lattice it carries no global wrap-around cycle: $H_1 = 0$ . This is the discriminating invariant against the torus ( $H_1 = \mathbb{Z}^2$ , two independent non-contractible loops). “Looks like a 2D sheet of points” does not by itself establish a torus — the wrap-around (periodic boundary) must be demonstrated directly, not inferred from a 2D arrangement. A finite grid with an open boundary is precisely the non-wrapping case.
Flat (zero curvature). A Euclidean square lattice has no intrinsic curvature; any bending seen in a low-dimensional projection is extrinsic.
Graph-spectral structure. The $m\times n$ grid graph is planar and bipartite, and its Laplacian eigenbasis is separable: eigenvectors are outer products of the two path-graph ( $P_m$ , $P_n$ ) eigenvectors, with eigenvalues $\lambda_{ab} = \mu_a + \nu_b$ . This separability underlies spectral-embedding arguments that connect an emergent grid to a small number of leading principal components.

Key evidence

Park, Lee, Lubana, Yang, Okawa, Nishi, Wattenberg & Tanaka (2025, ICLR 2025), In-Context Learning of Representations (arXiv 2501.00070). Semantically unrelated concept tokens are placed on an imposed square grid and random-walk traces over that grid are fed as in-context exemplars. With sufficient context length, the model’s mean token representations projected onto the top principal components reproduce the imposed “square-grid structure” (Fig. 1), overriding the tokens’ pretrained semantic clustering. A Dirichlet- energy metric computed against the ground-truth grid decreases with context size, quantifying the reorganization, and a spectral-embedding argument links the emergent geometry to the top PCA components. The reorganization is causally validated: intervening on concept projections along the identified principal components changes next-node prediction accuracy (App. C.4). Main model Llama-3.1-8B; replicated on Llama-3.2-1B, Llama-3.1-8B-Instruct, Gemma-2-2B and Gemma-2-9B.

Topological precision (why this node, not Torus): the paper describes the shape only as a “grid” / “square grid” / “square-grid structure” — it never uses “torus” or “toroidal” — and notes the emergent grid is “partially irregular” (wider in the central regions, narrower at the periphery), consistent with a bounded, non-periodic lattice rather than a wrapping torus. The same paper’s ring condition instead reproduces a closed loop → a circle; see Circle. See the observation in-context-graph-tracing-reorganizes-geometry.

Imposed vs. discovered: the graph is specified in-context, but the finding is that the model spontaneously reorganizes its representation to match it — an emergent, causally load-bearing reorganization — which is why it is in scope as a discovered structure rather than an architecturally-imposed geometry.

Exercises

Base

In a $4\times4$ grid, how many neighbours do a corner point, an edge (non-corner) point, and an interior point have? What does the existence of points with fewer than four neighbours tell you about periodicity?

Solution

Corner: 2 neighbours; edge: 3; interior: 4. The presence of points with fewer than four neighbours means some points sit on a boundary — there is no wrap-around that would give every point four neighbours. A periodic lattice (torus) would make every point interior-like (four neighbours); the open boundary shows the object is a bounded grid, not a torus.

Why is a $1\times n$ grid not a genuinely 2D object, and what does it reduce to?

Solution

A $1\times n$ grid has only one row: every point is indexed by a single coordinate $j$ , with adjacency only along that one direction. It is a path graph — a discrete 1D chain (a discretised 1D continuum manifold segment), not a 2D lattice. Two independent coordinates (both $m,n \ge 2$ ) are required before the object is genuinely 2-dimensional.

Middle

State why the top principal components of an idealized grid embedding recover the two grid axes, using the separability of the grid Laplacian.

Solution

The $m\times n$ grid Laplacian is $L = L_{P_m}\oplus L_{P_n}$ (a Kronecker sum), so its eigenvectors are outer products $u_a \otimes v_b$ of the path-graph eigenvectors with eigenvalues $\mu_a+\nu_b$ . The two smallest non-trivial eigenvalues correspond to the lowest-frequency mode along each axis (the “coordinate” eigenvectors), so a 2D spectral embedding using those two modes lays the points out along the two grid directions — which is why the top two PCs of the reorganized representation reproduce the grid.

You recover a 2D sheet of points under PCA. Describe a concrete test for whether it is a bounded grid versus a periodic torus.

Solution

Check for wrap-around adjacency directly. For a torus, moving off one edge should return you to the opposite edge (the extreme rows/columns are neighbours, and every point has the same local neighbour count); for a grid, the extreme rows/columns are not adjacent and boundary points have fewer neighbours. Equivalently, estimate $H_1$ of the occupied point set (e.g. via persistent homology): a torus has $H_1=\mathbb{Z}^2$ (two independent loops), a bounded grid is contractible ( $H_1=0$ ). Do not infer “torus” from a merely 2D arrangement — test the periodicity.

Pro

Give the homological statement that distinguishes a bounded $m\times n$ grid region from a torus, and explain why “the representation is two-dimensional” is insufficient to claim either.

Solution

A bounded grid region is contractible: $H_0=\mathbb{Z}$ (connected), $H_1=0$ (no non-contractible loop). A torus has $H_0=\mathbb{Z}$ , $H_1=\mathbb{Z}^2$ (two independent loops), $H_2=\mathbb{Z}$ . Both are 2-dimensional, so dimension alone cannot separate them — dimension is a facet shared by many distinct objects. The load-bearing evidence is the first homology / periodicity: whether the two coordinate directions wrap (torus) or terminate at a boundary (grid). Only after establishing $H_1$ (or, operationally, wrap-around adjacency) is either claim licensed.

Found in (1 observation · 2 families)

Llama

In-Context Learning of Representations (2025)measured

In-context exemplars reorganize concept geometry to a specified graph

Details

Park et al. (ICLR 2025) define an arbitrary graph (grid or ring) over semantically unrelated concept tokens in-context and feed random-walk traces over it as exemplars [park-etal-2025-icl-representations] As context length scales, windowed activations projected onto top principal components suddenly reproduce the specified graph's topology rather than the pretrained semantic clustering of those tokens [park-etal-2025-icl-representations] A Dirichlet-energy metric against the ground-truth graph decreases with context size, quantifying the sudden reorganization, with a spectral-embedding argument linking it to the top PCA components [park-etal-2025-icl-representations] When reference concepts already carry correlated pretrained semantics (e.g. weekdays), the in-context graph appears only in PC3-4 rather than overriding the pretrained ring in PC1-2 [park-etal-2025-icl-representations] Intervening on concept projections along the identified principal components changes next-node prediction accuracy, a causal check beyond the PCA visualization (exact effect sizes not captured here) [park-etal-2025-icl-representations] Tested mainly on Llama-3.1-8B and replicated on Llama-3.2-1B, Llama-3.1-8B-Instruct, Gemma-2-2B and Gemma-2-9B [park-etal-2025-icl-representations]

models: Llama-3.1-8B, Llama-3.2-1B, Llama-3.1-8B-Instruct · method: PCA, Causal interventions (steering)

Gemma

In-Context Learning of Representations (2025)measured

In-context exemplars reorganize concept geometry to a specified graph

Details

models: Gemma-2-2B, Gemma-2-9B · method: PCA, Causal interventions (steering)