Geographic coordinates are represented on a flat affine plane

measured in 1 paper

Latitude and longitude are linearly decodable from a flat 2D affine plane in the residual stream, unlike the cyclic circle used for calendar time in the same models [gurnee-tegmark-2024] Karkada et al. independently derive the same flat-plane geometry analytically as the D=2 open-boundary case of translation-symmetry theory, confirmed in word embeddings and Gemma 2 2B for US cities [karkada-etal-2026] A later analysis fits a spherical rather than flat geometry to the same data, kept as a separate observation since the two disagree on curvature [karkada-etal-2026]

Structure

Affine Subspace

Context

space, geography, world models, translation symmetry, geography representation debate

Confirmed in models

Llama-2-7B · Llama, Llama-2-13B · Llama, Llama-2-70B · Llama, Pythia-6.9B · Pythia, Gemma-2-2B · Gemma, EmbeddingGemma · Gemma, word2vec (trained on Wikipedia) · word2vec

Method

Linear probing, PCA, Spectral analysis of the co-occurrence/PMI matrix, Analytical derivation

Papers

Language Models Represent Space and Time — Gurnee, W., Tegmark, M.2024 · arXiv:2310.02207

Symmetry in Language Statistics Shapes the Geometry of Model Representations — Karkada, D., Korchinski, D. J., Nava, A., Wyart, M., Bahri, Y.2026 · arXiv:2602.15029