MATH · IN · MODELS

mBERT own space supports nearest-neighbor cross-lingual retrieval

measured in 1 paper

Pires, Schlinger & Garrette show real pretrained mBERT (104 languages) supports translation-pair retrieval by direct nearest-neighbor search in its own space, with no fitted map [pires-etal-2019-how-multilingual-is-mbert] This indicates mBERT shared geometry already aligns translation-equivalent items across languages [pires-etal-2019-how-multilingual-is-mbert] They show strong zero-shot cross-lingual transfer (fine-tune one language, evaluate another) for NER and POS, best between typologically similar languages [pires-etal-2019-how-multilingual-is-mbert] No causal intervention is performed [pires-etal-2019-how-multilingual-is-mbert]

Context

nearest-neighbor translation retrieval, zero-shot cross-lingual transfer, shared mBERT geometry

Papers

How Multilingual is Multilingual BERT? — Pires, Telmo, Schlinger, Eva, Garrette, Dan2019 · arXiv:1906.01502