MATH · IN · MODELS
methods / Direction Extraction / Language centroid shift

Language centroid shift

Techniquebeginner

Extracts a per-group (e.g. per-language) direction as that group's own empirical mean activation, then tests its role by subtracting it (centering) and measuring the effect on downstream extrinsic tasks — a one-sided special case of diff-in-means, using an absolute group mean rather than a difference between two contrastive groups.

Used in (1 observation)

Subtracting a per-language centroid removes decodable language identity in mBERTmeasured
structure: Linear Direction · models: mBERT (BERT-base, Multilingual Cased), BERT base multilingual uncased · paper: How Language-Neutral is Multilingual BERT?