Hierarchical concepts occupy domain-specific 150-250D subspaces, steerable

Sakata et al. extend Linear Relational Concepts into a depth- and domain-specific Linear Hierarchical Encoding fit from intermediate hidden states of Llama-3.2-3B, Llama-3.1-8B, Qwen3-8B and Qwen3-14B [sakata-etal-2026-linear-hierarchical-concepts] Hierarchical information is encoded in relatively low-dimensional subspaces (150-250 dimensions for hidden size 3000-5000) [sakata-etal-2026-linear-hierarchical-concepts] The relevant subspace is domain-specific, and domain-specific subspaces show similar hierarchical structure across domains [sakata-etal-2026-linear-hierarchical-concepts] Editing a child representation by adding a scaled concept-direction difference at every layer flips next-token prediction toward the target parent (causality up to 0.93 for Organization/Llama-3.1-8B) [sakata-etal-2026-linear-hierarchical-concepts]