Published on Fri Mar 05 2021

Hierarchical Transformer for Multilingual Machine Translation

Albina Khusainova, Adil Khan, Adín Ramírez Rivera, Vitaly Romanov
0
0
0
Abstract

The choice of parameter sharing strategy in multilingual machine translation models determines how optimally parameter space is used and hence, directly influences ultimate translation quality. Inspired by linguistic trees that show the degree of relatedness between different languages, the new general approach to parameter sharing in multilingual machine translation was suggested recently. The main idea is to use these expert language hierarchies as a basis for multilingual architecture: the closer two languages are, the more parameters they share. In this work, we test this idea using the Transformer architecture and show that despite the success in previous work there are problems inherent to training such hierarchical models. We demonstrate that in case of carefully chosen training strategy the hierarchical architecture can outperform bilingual models and multilingual models with full parameter sharing.

Tue May 12 2020
NLP
A Framework for Hierarchical Multilingual Machine Translation
Multilingual machine translation has recently been in vogue given its potential for improving machine translation performance for low-resource languages via transfer learning. Empirical examinations demonstrating the successes of existing multilingual machine Translation strategies are limited to experiments in specific language groups. In this paper, we present a framework
0
0
0
Thu Dec 31 2020
NLP
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
XLM-T was inspired by the recent success of language model pre-training. The code will be at https://aka.ms/xlm-t.
0
0
0
Thu Jul 30 2015
NLP
One model, two languages: training bilingual parsers with harmonized treebanks
Parsers can analyze sentences in either of the learned languages, or even those that mix both. The results show that most combinations not only preserve accuracy, but some even achieve significant improvements over the monolingual parsers.
0
0
0
Sat Apr 21 2018
NLP
Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation
The main challenges we identify are the lack of low-resource language data, effective methods for cross-lingual transfer, and the variable-binding problem that is common in neural systems. We build atranslation system that addresses these challenges using eight European language families as our test ground.
0
0
0
Thu Sep 09 2021our pick
NLP
Distributionally Robust Multilingual Machine Translation
Multilingual neural machine translation (MNMT) learns to translate multiple language pairs with a single model. However, the heavy data imbalance between languages hinders the model from performing uniformly across language pairs. In this paper, we propose a new learning objective for MNMT based on distributionally robust optimization.
2
6
21
Thu Apr 09 2020
Machine Learning
On Optimal Transformer Depth for Low-Resource Language Translation
Transformers have shown great promise as an approach to Neural Machine Translation (NMT) for low-resource languages. However, at the same time, transformer models remain difficult to optimize and require careful tuning of hyper-parameters. In this work, we find that the current trend in the field to
0
0
0