MATH · IN · MODELS

Contextual displacement forms a vector field tracking concreteness

measured in 1 paper

Hu, Niu & Varma define a per-word contextual displacement vector phi(w,tau) = r(w,tau) - r(w,tau_0) in activation space, forming a vector field over the vocabulary rather than one shared direction [hu-etal-2026-concept-transformation-geometry] Across 4 models x 3 depths, word density correlates negatively with displacement magnitude (Spearman rho -0.32 to -0.22) [hu-etal-2026-concept-transformation-geometry] Lexical concreteness correlates negatively with directional deviation (rho -0.41 to -0.21): concrete, low-density concepts move less and more consistently under context [hu-etal-2026-concept-transformation-geometry] The field has no dominant direction (leading PC <15% variance) and is more dispersed than uniform-random; no causal intervention is performed [hu-etal-2026-concept-transformation-geometry]

Structure

Linear Direction

Context

contextual displacement vector field, lexical concreteness, word density, spherical Frechet mean, directional deviation, PCA variance ratio, neural population geometry

Confirmed in models

Qwen2.5-0.5B · Qwen, Qwen2.5-1.5B · Qwen, Qwen2.5-3B · Qwen, Qwen-2.5-7B · Qwen, Qwen2.5-14B · Qwen, Qwen2.5-32B · Qwen, Llama-2-7B · Llama, Llama-2-13B · Llama, Llama-3-8B · Llama, Llama-3.1-8B · Llama, Mistral Small 3 (2501) · Mistral, Ministral-3-3B-Base-2512 · Mistral, Ministral-3-8B-Base-2512 · Mistral, Ministral-3-14B-Base-2512 · Mistral, Gemma-2-2B · Gemma, Gemma-2-9B · Gemma, DeepSeek-LLM-7B-Base · DeepSeek, Pythia-410M · Pythia, Pythia-1B · Pythia, Pythia-1.4B · Pythia, Pythia-2.8B · Pythia, Pythia-6.9B · Pythia, Pythia-12B · Pythia

Method

PCA, Geometric analysis

Papers

Language Models Represent and Transform Concepts with Shared Geometry — Hu, Zhimin, Niu, Lanhao, Varma, Sashank2026 · arXiv:2607.04525