A centroid-estimated linear grammatical-gender direction in frozen Spanish/French encoders gives a better leakage-reduction vs. occupation-preservation trade-off than SVM or LDA estimators, and controlled templates give cleaner noun-level reduction than natural contexts

- Xiao & Li estimate a "grammatical gender direction" from final-layer contextual embeddings of Spanish and French inanimate nouns in frozen BETO, CamemBERT, mBERT and XLM-R (checkpoint variants not stated in the paper), using centroid-difference, linear-SVM and LDA estimators on controlled-template contexts and natural Wikipedia contexts, and judge each direction by how much it removes gender separability from held-out inanimate nouns versus how much it preserves for occupation pairs (harmonic trade-off score). [xiao-li-2026-grammatical-gender-directions] - The centroid estimator gives the best trade-off, averaged over models, languages, contexts and weightings (harmonic trade-off about 0.87 vs. 0.77 for linear SVM and 0.58 for LDA), which the authors read as grammatical gender behaving "more like a low-rank global direction embedded in noisy high-dimensional representations" than a nonlinear decision surface; this is an interpretation of estimator rankings, not a direct test of low-rankness. [xiao-li-2026-grammatical-gender-directions] - Controlled templates give higher noun-level leakage reduction than natural contexts in every language-model setting reported, but the overall trade-off is not uniformly better (natural-unweighted scores higher than controlled-unweighted for Spanish mBERT and XLM-R); contamination-aware weighting mostly lowers noun reduction while raising occupation preservation slightly. [xiao-li-2026-grammatical-gender-directions]