MATH · IN · MODELS
methods / Direction Extraction / Language Activation Probability Entropy (LAPE)

Language Activation Probability Entropy (LAPE)

Techniquebeginner

Selects a sparse subset of units — raw neurons or SAE latent features — as language-associated by computing, for each unit, the entropy of its per-language activation-probability distribution and keeping only low-entropy (single-language-dominant) units.

Used in (1 observation)

Language units are script-conditioned; typology decodes with depthmeasured
structure: Linear Separability · models: Llama-3.2-1B, Gemma-2-2B · paper: Multilingual Language Models Encode Script Over Linguistic Structure