Voita et al. (2019) objective-matched Transformers (MT/LM/MLM)
Voita, Sennrich & Titov (2019), EMNLP-IJCNLP
Structures found in this family (1)
By model (3)
Transformer-base trained as a left-to-right language model
Transformer-base trained with a masked-language-modeling objective
Transformer-base trained on machine translation (En-Ru/En-Fr)
Papers
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives (2019)