Categorical concepts are represented as simplex polytopes
measured in 1 paperPark et al. show a mutually-exclusive categorical concept (e.g. mammal/bird/reptile/fish) is represented as the convex hull of its values' vector representations, generically a simplex, proven formally and confirmed in Gemma-2B and Llama-3-8B on 900+ WordNet concepts [park-etal-2024] Adding a concept's vector representation to context shifts logits for that concept's members while leaving unrelated concepts unchanged, a causal validation [park-etal-2024] Marinov et al. independently confirm the same simplex shape for language identity via a causal inner product across Qwen3-4B, Mistral-7B-v0.3, and Llama-3-8B, with four same-family languages at the vertices of a 3D simplex [marinov-etal-2026]
Structure
Context
categorical concepts, vector representation, convex hull, simplex, WordNet, linear representation hypothesis, language identity, multilingual
Confirmed in models
Papers
The Geometry of Categorical and Hierarchical Concepts in Large Language Models — Park, K., Choe, Y. J., Jiang, Y., Veitch, V.
When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs — Marinov, B., Sharma, A., Schroeder de Witt, C., Torr, P., Calinescu, A., Yu, J.