Across nine open LLMs, low-resource languages show representational degeneration in final layers, with higher cosine similarity and lower isotropy than high-resource languages

- Meyer & Buys compare hidden-state geometry of 30 languages (five MADLAD-400 data-size tiers, FLORES sentences) in nine open base LLMs (Llama 3.2 1B/3B, Llama 3.1 8B, Gemma 3 1B/4B/12B, Qwen3 1.7B/4B/8B) using average pairwise cosine similarity and IsoScore; lower-resource languages show higher cosine similarity and lower IsoScore (more anisotropic) in many layers. [meyer-etal-2026-the-geometry-of-low-resource-language-representations] - The most consistent effect is in the final layer, the only layer where degeneration correlates with data scarcity across all models and both metrics (Spearman rho negative for cosine similarity and positive for IsoScore); the effect is strongest in Llama, weaker in Qwen, weakest in Gemma, and Amharic is an outlier (Ge'ez script). [meyer-etal-2026-the-geometry-of-low-resource-language-representations] - IsoScore differences are small and mid-layer correlations with data size are inconsistent; absolute degeneration even improves in final layers for some models (e.g. Gemma cosine similarity). [meyer-etal-2026-the-geometry-of-low-resource-language-representations]