Joint Training for Pivot-based Neural Machine Translation

1 Introduction

Recent several years have witnessed the rapid development of neural machine translation (NMT) [Sutskever et al., 2014; Bahdanau et al., 2015], which advocates the use of neural networks to directly model the translation process in an endto-end way. Thanks to the capability of learning representations from training data, NMT systems have achieved significant improvements over conventional statistical machine translation (SMT) across a variety of language pairs [JunczysDowmunt et al., 2016; Johnson et al., 2016].

However, there still remains a major challenge for NMT: large-scale parallel corpora are usually non-existent for most language pairs. This is unfortunate because NMT is a datahungry approach and requires a large amount of data to fully train translation models. Without sufficient training data, NMT tends to learn poor estimates on low-count events. Zoph et al. [2016] indicate that NMT obtains much worse translation quality than SMT when only small-scale parallel corpora are available.

As a result, improving neural machine translation on resource-scarce language pairs has attracted much attention [Firat et al., 2016; Zoph et al., 2016; Johnson et al., 2016].

∗Corresponding Author

Most existing methods focus on leveraging parallel corpora of multiple resource-rich language pairs to improve NMT for resource-scarce language pairs. Firat et al. [2016] propose multi-way, multilingual neural machine translation to achieve direct source-to-target translation even without parallel data available. Zoph et al. [2016] present a transfer learning method that transfers the model parameters trained for resource-rich language pairs to initialize and constrain the translation model training of resource-scarce language pairs. Johnson et al. [2016] introduce a universal NMT model for all language pairs, which takes advantage of multilingual data to improve NMT for all languages involved.

Bridging source and target languages with a pivot language is another important direction, which has been intensively studied in conventional SMT [Cohn and Lapata, 2007; Wu and Wang, 2007; Utiyama and Isahara, 2007; Bertoldi et al., 2008; Zahabi et al., 2013; El Kholy et al., 2013]. Pivot-based approaches assume that there exist source-pivot and pivot-target parallel corpora, which can be used to train source-to-pivot and pivot-to-target translation models, respectively. One of the most representative approaches, the triangulation approach [Wu and Wang, 2007], is to construct a source-to-target phrase table through combining source-topivot and pivot-to-target phrase tables. Another representative approach adopts a pivot-based translation strategy [Wu andWang, 2007]. As a result, source-to-target translation can be divided into two steps: the source sentence is first translated into a pivot sentence using the source-to-pivot model, which is then translated to a target sentence using the pivot-totarget model. Pivot-based approaches have been widely used in SMT due to its simplicity, effectiveness, and minimum requirement of multilingual data. Recently, Johnson et al. [2016] adapt pivot-based approaches to NMT and show that their universal model without incremental training achieves much worse translation performance than pivot-based NMT.

However, pivot-based approaches often suffer from the error propagation problem: the errors made in the sourceto-pivot translation will be propagated to the pivot-to-target translation. This can be partly attributed to the discrepancy between source-pivot and pivot-target parallel corpora since they are usually loosely-related or even unrelated. To aggravate the situation, source-to-pivot and pivot-to-target translation models are trained independently, which further enlarges the gap between source and target languages.

In this work, we propose an approach to joint training for pivot-based neural machine translation. The basic idea is to connect the source-to-pivot and pivot-to-target NMT models and enable them to interact with each other during training. This can be done either by encouraging the sharing of word embeddings on the pivot language or by maximizing the likelihood of the cascaded model on a small source-target parallel corpus. Experiments on the Europarl and WMT corpora show that joint training of source-to-pivot and pivot-to-target models obtains significant improvements over independent training.

2 Background

Given a source language sentence x and a target language sentence y, we use P(y|x; θx→y) to denote a standard attention-based neural machine translation model [Bahdanau et al., 2015], where θx→y is a set of model parameters.

Ideally, the source-to-target model can be trained on a source-target parallel corpus Dx,y = {〈x(s) , y(s) 〉}Ss=1 using maximum likelihood estimation:

where the log-likelihood is defined as:

Unfortunately, parallel corpora are usually not readily available for low-resource language pairs. Instead, one can assume that there exist a third language called pivot with source-pivot and pivot-target parallel corpora available. As a result, it is possible to bridge the source and target languages with the pivot [Cohn and Lapata, 2007; Wu and Wang, 2007; Bertoldi et al., 2008; Zahabi et al., 2013; El Kholy et al., 2013]. Figure 1 illustrates the idea of pivot-based translation.

Let z be a pivot language sentence. The source-to-target model can be decomposed into two sub-models by treating the pivot sentence as a latent variable:

Let Dx,z = {〈x(m) , z(m) 〉}Mm=1 be a source-pivot parallel corpus, and Dz,y = {〈z(n) , y(n) 〉}Nn=1 be a pivot-target parallel corpus. The source-to-pivot and pivot-to-target models can be independently trained on the two parallel corpora, respectively:

where the log-likelihoods are defined as:

Given an unseen source sentence to be translated x, the decision rule is given by:

Due to the exponential search space of the pivot language, the decoding process is usually approximated with two steps. The first step translates the source sentence x into a pivot sentence:

Then, the pivot sentence is translated to a target sentence:

Although pivot-based approaches are widely used for addressing the data scarcity problem in machine translation, they suffer from cascaded translation errors: the mistakes made in the source-to-pivot translation as shown in Eq. (9) will be propagated to the pivot-to-target translation as shown in Eq. (10). This can be partly attributed to the model discrepancy problem: the source-to-pivot and pivot-to-target models are quite different in terms of vocabulary and parameter space because the source-pivot and pivot-target parallel corpora are usually loosely-related or even unrelated. To make things worse, the source-to-pivot model P(z|x; θx→z) and the pivot-to-target model P (y|z; θz→y) are trained on the two parallel corpora independently, which further increases the discrepancy between two models.

Therefore, it is important to reduce the discrepancy between source-to-pivot and pivot-to-target models to further improve pivot-based neural machine translation.

3 Joint Training for Pivot-based NMT

3.1 Training Objective

To alleviate the model discrepancy problem, we propose an approach to joint training for pivot-based neural machine translation. The basic idea is to connect source-to-pivot and pivot-to-target models and enable them to interact with each other during training. Our new training objective is given by:

Note that the training objective consists of three parts: the source-to-pivot likelihoodL(θx→z), the pivot-to-target likelihood L(θz→y), and a connection term R(θx→z, θz→y). The hyper-parameter λ is used to balance the preference between likelihoods and the connection term.

We expect that the connection term associates the sourceto-pivot model θx→z with the pivot-to-target model θz→y and enables the interaction between two models during training. In the following subsection, we will introduce the three connection terms used in our experiments.

3.2 Connection Terms

It is difficult to connect the source-to-pivot and pivot-to-target models during training because the source-to-pivot and pivotto-target models are distantly-related by definition. More importantly, NMT lacks linguistically interpretable language structures such as phrases in SMT to achieve a direct connection at the parameter level [Wu andWang, 2007].

Fortunately, both the source-to-pivot and pivot-to-target models include the word embeddings of the pivot language as parameters. It is possible to connect the two models via pivot word embeddings.

More formally, let V z x→z be the pivot vocabulary of the source-to-pivot model and V z z→y be the pivot vocabulary of the pivot-to-target model. We use w to denote a word in the pivot language and θw x→z ∈ Rd to denote the vector representation of w in the source-to-pivot model. θw z→y ∈ Rd is defined in a similar way.

Our first connection term encourages the two models to generate the same vector representations for pivot words in the intersection of two vocabularies:

where δ(θw x→z , θw z→y ) = 1 if the two vectors θw x→z and θw z→y are identical. Otherwise, δ(θw x→z , θw z→y ) = 0. This constraint requires that θw x→z and θw z→y share the same embeddings of the intersected pivot words.

As word embeddings seem hardly to be exactly identical due to the divergence of natural languages, an alternative is to soften the above hard matching constraint by penalizing the Euclidean distance between two vectors:

The third connection term assumes that there is a small bridging source-target parallel corpus ˜Dx,y = {〈x(l) , y(l) 〉}Ll=1 (Bridging Corpus) available. The connection term is defined as the log-likelihood of the bridging data:

3.3 Training

In training, our goal is to find the optimal source-to-pivot and pivot-to-target model parameters that maximize the training objective:

The partial derivative of J(θx→z, θz→y) with respect to the parameters θx→z of the source-to-pivot model can be calculated as:

The partial derivative with respect to the parameters θz→y can be calculated similarly.

The gradients of the first and second connection terms Rhard(θx→z, θz→y) and Rsoft(θx→z, θz→y) with respect to model parameters are easy to calculate. However, calculating the gradients of the third connection term Rlikelihood(θx→z, θz→y) involves enumerating all possible pivot sentences in an exponential search space (see Eq. (15)).

To alleviate this problem, we follow standard practice to use a subset to approximate the full space [Shen et al., 2016; Cheng et al., 2016]. Two methods can be used to generate a subset: sampling k translations from the full space [Shen et al., 2016] or generating a top-k list of candidate translations [Cheng et al., 2016]. We find that using top-k lists leads to better results than sampling in our experiments.

We use standard mini-batched stochastic gradient descent algorithms to optimize model parameters. In each iteration, three mini-batches are constructed by randomly selecting sentence pairs from the source-pivot parallel corpus Dx,z, the pivot-target parallel corpus Dz,y, and the bridging sourcetarget parallel corpus Dx,y (only available for the third connection term), respectively. After separate gradient calculation in each mini-batch, the gradients are collected to update model parameters.

4 Experiments

4.1 Setup

We evaluated our approach on two translation tasks:

1. Spanish-English-French: Spanish as the source language, English as the pivot language, and French as the target language,
2. German-English-French: German as the source language, English as the pivot language, and French as the target language.

Table 1 shows the statistics of the Europarl and WMT corpora used in our experiments. We use the tokenize.perl script for tokenization. For each language pair, we remove the empty lines and retain sentence pairs with no more than 50 words. To avoid the intersection of the source-pivot and pivot-target corpora, we split the overlapped pivot-language sentences of source-to-pivot and pivot-to-target corpora into two separate parts with equal size and merge them separately with the non-overlapping parts for each language pair.

The Europarl corpus consists of 850K Spanish-English sentence pairs with 22.32M Spanish words and 21.44M English words, 840K German-English sentence pairs with 20.88M German words and 21.91M English words, and 900K English-French sentence pairs with 22.56M English words and 25.00M French words. The WMT 2006 shared task datasets are used as the development and test sets. The evaluation metric is case-insensitive BLEU [Papineni et al., 2002] as calculated by the multi-bleu.perl script.

The WMT corpus is composed of the Common Crawl, News Commentary, Europarl v7 and UN corpora. The Spanish-English parallel corpus consists of 6.78M sentence pairs with 183.01M Spanish words and 166.28M English words. The English-French parallel corpus comprises 9.29M sentence pairs with 227.06M English words and 258.95M French words. The newstest2011 and newstest2012 datasets serve as development and test sets. We use case-sensitive BLEU as the evaluation metric.

We use the attention-based neural machine translation system RNNSEARCH [Bahdanau et al., 2015] in our experiments. For the Europarl corpus in Table 1, we set the vocab- Iterations ulary size of all the languages to 30K which covers over 99% of words for English, Spanish and French and over 97 % for German. We follow Jean et al. [2015] to address rare words. For Spanish-English and English-French corpora from the WMT corpus, due to large vocabulary size, we adopt byte pair encoding [Sennrich et al., 2016b] to split rare words into sub-words. We set the size of sub-words to 43K, 33K, and 43K for Spanish, English, and French, respectively. These sub-words cover 100% of the text.

Figure 2: Learning curves of independent training and joint training on different connection terms.

We set the hyper-parameter λ for balancing between likelihood and the connection term to 1.0. The threshold of gradients is set to 0.1. The bridging source-target parallel corpus contains 100K sentence pairs that are sampled from the training corpus but do not overlap with the training corpus through removing them. We set k to 10 for calculating top-k lists. The parameters for the source-to-pivot and pivot-to-target translation models in the likelihood connection term are initialized by pre-trained model parameters.

4.2 Results on the Europarl Corpus

Table 2 shows the comparison results between our joint training on three connection terms and independent training on the Europarl Corpus. For the source-to-target translation task, we present source-to-pivot, pivot-to-target and source-to-target translation results compared with independent training. In Spanish-to-French translation task, soft connection achieves significant improvements in Spanish-to-French and Spanishto-English directions although hard connection still performs comparably with independent training. In German-to-French translation task, soft and hard connections also achieve comparable performances with independent training.

In contrast, we find that likelihood connection dramatically improves translation performance on both Spanish-to-French and German-to-French corpora (up to +2.80 BLEU scores in Spanish-to-French and up to 2.23 BLEU scores in Germanto-French). The significant improvements for source-to-pivot and pivot-to-target directions are also observed. This suggests that introducing source-to-target parallel corpus to maximize P(y|x; θx→z, θz→y) with z as latent variables makes the source-to-pivot and pivot-to-target translation models improved collaboratively.

Table 3 shows pivot and target translation examples of independent training and our approaches. Apparently, our approaches improve the translation quality of both pivot sentences and target sentences.

According to Eq. (3), the cost of the source-to-target model can be decomposed into the costs of source-to-pivot and pivot-to-target models. As we have a small test trilingual corpus, (Spanish, English, French), we use the English sentence to approximate the latent variables in Eq. (3). Then we calculate the cost of Spanish-to-French on the trilingual corpus. Figure 2 shows the learning curves of the test cost of independent training and joint training on three connection terms. We can find that hard and soft connections learn slower than the independent training. Likelihood connection drives its cost lower after fine-tuning based on pre-trained parameters in just 10K iterations.

4.3 Results on theWMT Corpus

Likelihood connection obtains the best performance in our three proposed connection terms according to experiments on the Europarl corpus. To further verify its effectiveness, we evaluate all the methods on the WMT corpus, which is much larger than Europarl. As shown in Table 4, we find that likelihood connection still outperforms independent training significantly on Spanish-to-English, English-to-French and Spanish-to-French directions (up to +1.18 BLEU scores in Spanish-to-French).

We also compare our approach with Firat et al. [2016]. Although our parallel training corpus is much smaller than theirs, Table 5 shows that our approach achieves substantial improvements over them (up to +4.32 BLEU).

4.4 Effect of Bridging Corpora

As bridging corpora are used in likelihood connection term for "bridging" the source-to-pivot and pivot-to-target translation models, why do not we directly build NMT systems with these corpora?

We train source-to-target models using bridging corpora and show translation results in Table 6 . We observe that performance is much worse than that in Table 2 and Table 4 using the pivot-based translation strategy. It indicates that NMT yields poor performance on low-resource languages and the pivot-based translation strategy remedies the drawback to alleviate data scarcity effectively.

We also investigate the effect of the data size of bridging corpora on the likelihood connection. Table 7 shows that using a small parallel corpus (1K sentence pairs) has made a measurable improvement. When more than 50K sentence pairs are added, the further improvements become modest. This finding suggests that a small corpus suffices to enable the likelihood connection to reach the reasonable performance.

5 RelatedWork

Our work is inspired by two lines of research: (1) machine translation with pivot languages and (2) incorporating additional data resource for NMT.

5.1 Machine Translation with Pivot Languages

Machine translation suffers from the scarcity of parallel corpora. For low-resource language pairs, a pivot language is introduced to "bridge" source and target languages in SMT [Cohn and Lapata, 2007; Wu and Wang, 2007; Utiyama and Isahara, 2007; Zahabi et al., 2013; El Kholy et al., 2013].

In NMT, Firat et al. [2016] and Johnson et al [2016] propose multi-way, multilingual NMT models that enable zeroresource machine translation. They also need to apply pivotbased approaches into NMT to ameliorate its performance. Zoph et al. [2016] adopt transfer learning to fine-tune parameters of the low-resource language pairs using trained parameters on the high-resource language pairs. However, our approach aims to jointly train source-to-pivot and pivot-totarget NMT models, which can alleviate the error propagation of pivot-based approaches. We use connection terms to "bridge" these two models and make them benefit each other.

5.2 Incorporating Additional Data Resources for NMT

Due to the limit in quantity, quality and coverage for parallel corpora, additional data resources have raised attention recently. Gulccehre et al [2015] propose to incorporate targetside monolingual corpora as a language model for NMT. Sennrich, Haddow, and Birch [2016a] pair the target monolingual corpora with its automatic, NMT-generated translations, then merge them with parallel corpora for retraining source-to-target model. Zhang and Zong [2016] propose two approaches, self-training algorithm and multi-task learning framework, to incorporate source-side monolingual corpora. Cheng et al. [2016] introduce an autoencoder framework to reconstruct monolingual sentences using source-to-target and target-to-source NMT models. In contrast to Cheng et al. [2016], the objective of our likelihood connection is to maximize the probability of target-language sentences through pivot sentences given source sentences. We use a small source-to-target parallel corpus to train source-to-pivot and pivot-to-target NMT models jointly.

6 Conclusion

We present joint training for pivot-based neural machine translation. It is appealing to combine source and pivot sentences for decoding target sentences [Firat et al., 2016] or train a multi-source model directly [Zoph and Knight, 2016]. We also plan to study better connection terms for our joint training.

Acknowledgements

This research is supported by the 863 Program (2015AA015407), the National Natural Science Foundation of China (No. 61532001, No. 61522204, No. 61432013), Tsinghua Initiative Research Program grants 20151080475, MOE Online Education Research Center (Quantong Fund) grant 2017ZD203, and gift funds from Huawei and Ant Financial. We sincerely thank the reviewers for their valuable suggestions.