Improving Stylized Neural Machine Translation with Iterative Dual Knowledge Transfer

1 Introduction

Stylized neural machine translation aims to translate sentences of one style into sentences of another style, which is essential for a wide range of NLP applications, such as non-native speaker assistants and child education. Table 1 shows an example of stylized translation. Despite its importance, few efforts have been made to deal with translation with stylistic transformation. Recently, [Wu et al., 2020] introduce a dataset, the Machine Translation Formality Corpus (MTFC), serving as a benchmark dataset for translation formality transfer task, which aims to translate an informal sentence to a formal one while keeping its semantics unchanged.

However, a major challenge in stylized neural machine translation is the scarcity of stylized parallel training data, the generated target sentences are then combined with the source sentences to train an NMT model. After that, we iteratively perform bidirectional formality knowledge transfer with knowledge distillation. In this way, both NMT model and formality style transfer model are expected to continually boost each other. Second, to address the noise in synthetic data, we further enhance the proposed framework by a datarefinement module to regenerate the target formal sentences with higher quality.

∗Equal contribution

†Corresponding author

To verify the effectiveness of our method, we perform experiments on MTFC dataset with informal Chinese translated to formal English. Our method surpasses the existing best model by 5 BLEU points and gets the highest formality accuracy.

The contributions of this work are summarized as follows:

• We propose an iterative dual knowledge transfer framework for improving stylized NMT, which can iteratively perform bidirectional knowledge transfer between NMT model and text style transfer model to boost each other.
• We extend our framework with a data-refinement module to improve the quality of synthetic parallel data generated during knowledge transfer.
• We conduct experiments on two benchmark datasets, MTFC and GYAFC, and achieves state-of-the-art results in producing stylized translation sentences based on both automatic and human evaluation 1 .

2 RelatedWork

Stylized Neural Machine Translation. Previously, the work been made on stylized NMT is limited. [Rabinovich et al., 2016; Michel and Neubig, 2018] take an adaptation approach to personalize MT with gender-specific or speakerspecific data, [Niu et al., 2018] uses multi-task learning to perform formality transfer and translation. Other work like [Niu and Carpuat, 2020] focuses on controlling the formality of NMT. However, the training process of these NMT models usually require translation pairs with the target-style, [Wu et al., 2020] proposes the machine translation formality corpus by extending the Grammarly's Yahoo Answers Formality Corpus (GYAFC) [Rao and Tetreault, 2018], which makes it possible to benchmark stylized machine translation.

Text Style Transfer. Text style transfer is a more general task of changing the style of a sentence while preserving its content. Previous work explores this task as a sequenceto-sequence learning task using paired sentences in different styles, [Jhamtani et al., 2017] adopts an end-to-end neural network with pointer network to transform Modern English text to Shakespearean style English. However, available paired stylized texts are very limited. Some works focus on learning style-independent content representation. Recent works [Li et al., 2018; Sudhakar et al., 2019] propose to separate style information and content representation by directly removing stylistic words, however, the separation is challenging due to the content and style interacting in subtle ways in natural language. [Shen et al., 2017;

1 Code and data are available at https://github.com/mt887/IDKT

Fu et al., 2018] use adversarial network to ensure that the content representation is separated from style. Regarding the tasks, formality style transfer is one of the most wild direction of text style transfer, aiming to change the style of a given sentence from informal to formal. The research of formality style transfer is mainly motivated by the GYAFC dataset, released by [Rao and Tetreault, 2018]. [Xu et al., 2019] uses CNN-based discriminators and cyclic reconstruction objective in a semi-supervised setting. Due to the small size of the parallel corpus, [Wang et al., 2019; Chawla and Yang, 2020] incorporate a pre-trained neural network for formality style transfer, and [Zhang et al., 2020] proposes several data augmentation methods to generate augmented data with various formality style transfer knowledge.

3 Our Method

In this section, we first illustrate the overview of our iterative dual knowledge transfer framework, then describe the iterative knowledge transfer strategy and the architecture of datarefinement module.

3.1 Overview

As shown in Figure 1, our proposed framework consists of three sub-modules: (1) A stylized NMT model to translate sentences from informal to formal style. (2) A text style transfer model to regenerate the sentences from informal to formal style. (3) A data-refinement module aims to correct the mistakes produced by above two models. The NMT model and the style transfer model will iteratively teach each other to enhance their performance. The NMT model and text style transfer model in our study use the Transformer-based encoder and decoder architecture and the parameters of the text style transfer model are initialized using pre-trained BART [Lewis et al., 2019].

In this study, we propose to build a formal NMT model without translation parallel sentences in the target style. Our method leverages two parts of data during training, one is machine translation corpus Dmt = {〈x1, y1〉 , . . . , 〈xn, yn〉}, where n is the number of training samples. The other is text style transfer corpus Dstyle = {〈ys 1 , yt1 〉 , . . . , 〈ysm, ytm〉}, where ys i is the source-style sentence and yt i is the targetstyle sentence. Our stylized NMT model aims to translate sentences into target language while changing the style of the source sentences and preserving their style-independent content.

3.2 Iterative Dual Knowledge Transfer

Figure 1 illustrates our iterative dual knowledge transfer procedure between the translation model and the text style transfer model. At first, we train two sequence-to-sequence models individually via minimizing the negative likelihood on their training corpora Dmt and Dstyle, the NMT model θmt:X -→ ˆ Y translates the source language sentence x into the target language sentence ˆ y, while the style transfer model θstyle: ˆ Y -→ Y transfers an informal sentence ˆ y into a formal sentence y.

Then, we perform iterative dual knowledge transfer between the NMT model and text style transfer model. We introduce sequence-level knowledge distillation [Kim and Rush, 2016] to transfer knowledge between the two models, which is a method for improving the performance of student model by imitating the behaviour of the teacher model. The sequence-level knowledge distillation involves three steps:

1. Randomly initialize a teacher model and train it until convergence on real-world training data.
2. Decode the source-side training data using the teacher model to produce source-target pairs.
3. Randomly initialize a student model and train it until convergence on the synthetic source-target pairs.

During the transfer from θstyle to θmt, the student model is optimized by minimizing the following loss function

where q (yt | ys ) represents the probability outputs of the previous best formality style transfer model. The student model is trained on the synthetic source-target pairs generated by the teacher model.

Although other knowledge transfer methods like finetuning can be effective in this task, it may cause catastrophic forgetting of previously learned knowledge. Sequence-level knowledge distillation is more effective and can alleviate the style difference between NMT model and text style transfer model. Despite the predicted results are not hundredpercent accurate and fluent, the output texts contain taskrelated knowledge which could be helpful for strengthening counterpart model. More importantly, despite the limitation of parallel data, monolingual data is readily accessible, thus refraining our model from the data-scarcity nature of style transfer task.

To better understand our proposed framework, we summarize the training procedure of the framework in Algorithm 1. The whole framework starts with pre-trained NMT model, style transfer model and data-refinement module (Line 1), then a joined training process is carried out to train the models iteratively (Lines 3-13). In each iteration, we first leverage text style transfer model to transfer the target informal sentences Y smt to formal style ˆ Y tmt . In this way, we obtain stylized paired translation synthetic data X∗ = {〈xsmt , ˆ ytmt 〉}, Algorithm 1 Iterative Dual Knowledge Transfer for Improving Stylized NMT

and the max iteration number K.

Output: Stylized NMT model θ∗mt

• 1: θ∗mt ←-TrainModel(Dmt) , θ∗ style ←- TrainModel(Dstyle) θ∗ dr ←- TrainModel(Dstyle and ˆ yt )
• 2: k=0
• 3: for k ≤K do
• 4: Use θ∗ style to build synthetic data X∗ = {〈xsmt , ˆ ytmt 〉}
• 5: Use θ∗ dr to refine ˆ ytmt and build ¯X = {〈xsmt , ytmt 〉}
• 6: θ∗mt ←- TransferKnowledge(θ∗ style , ¯X) based on Eq.1
• 7: Sample sentences {〈xsp , ys p 〉} from Dmt
• 8: Use θ∗mt to build synthetic data Y ∗ = {〈ys p , ˆ ytp 〉}
• 9: Use θ∗ dr to refine ˆ ytp and build ¯ Y = {〈ys p , ytp 〉}
• 10: θ∗ style ←- TransferKnowledge(θ∗mt , ¯ Y )
• 11: Fine-tune θ∗ dr with {〈ys p , ˆ ytp , ytp 〉}
• 12: k=k+1
• 13: end for

which can be applied to train the stylized NMT model (Lines 4-6). Further, a batch of parallel sentences {〈xsp , ys p 〉} are sampled fromDmt (Line 7), then the stylized NMT model is adopted to decode the sampled sentences xsp into target-style translation ˆ ytp . After that, we obtain the synthetic text style transfer data Y ∗ = {〈ys p , ˆ ytp 〉} to further improve text style transfer model (Line 8-10). During this process, we evaluate and save the best model parameters. Specifically, the synthetic parallel data produced by a teacher model will be regenerated by a data-refinement module to filter out the syntactic and semantic noise.

In this way, we enable stylized NMT model to retain the previously learned translation knowledge and absorb the effective style transfer knowledge from text style transfer model. Similarly, we perform knowledge transfer in the inverse direction to enhance the performance of text style transfer model. During this procedure, the knowledge contained in two different models is thoroughly interacted with each other, leading to better single model with limited training data.

3.3 Data-Refinement Module

Since the synthetic data contains noise, we further propose a data-refinement module to improve its quality, which is motivated by the advance of Automatic post-editing (APE [Correia and Martins, 2019] ). APE is to automatically correct the mistakes produced by an NMT model. We design our data-refinement module architecture, following the practices in APE. The module takes the source sentences and the imperfect style transferred sentences as inputs to regenerate target sentences with mistakes corrected.

To fit in our proposed framework, we adopt the Multisource Transformer [Junczys-Dowmunt and Grundkiewicz, 2018] with an encoder-decoder structure for our datarefinement model, as shown in Figure 2. Since pre-trained text representations like BERT are trained on a large corpus, beneficial to correct grammar errors. Following the practice in [Zhu et al., 2020], we combine BERT [Devlin et al., 2018] with a standard transformer to improve the performance of data-refinement module, in which we first employ BERT to extract representations for an input sentence, then the representations are combined with each layer of the encoder in transformer model through attention mechanisms. The data-refinement module is trained via minimizing the negative likelihood of triplets ̂ Y = {〈ys , ˆ yt , yt 〉}, containing the source sentence, the style transferred sentence, and the target sentence respectively.

Given an style transferred sentence ˆ yt = (y1, . . . , yn), where n is its sentence length, yi represent i-th token in ˆ yt . BERT is trained on a large-scale corpus to generate styleindependent and robustness representation, so we first use BERT to encode ˆ yt into representation HB = (b1, . . . , bn). Then the encoder encodes the other input ys into HlE = (hl1 , . . . , hlm), where hli is the i-th representation of the l-th layer in the encoder. Next, the representation of BERT and encoder are combined through attention mechanism. We calculate hli as follows:

where Attn is a self-attention model, which works as follows, note that we leverage the transform in self-attention to ensure HlB and HlE are the same dimensions:

We utilize an attention strategy to incorporate the pretrained model into the original transformer-based text style transfer model, which can adequately leverage the information of the source sentence and the style transferred sentence.

4 Experiments

4.1 Setup

We use two datasets to evaluate our proposed method, the size of each training dataset is presented in Table 2.

MTFC. Machine Translation Formality Corpus (MTFC) is built by selecting bilingual subtitle parallel data and extending the GYAFC dataset. MTFC dataset contains 14 million informal Chinese-English sentence pairs.

GYAFC. We use Grammarly's Yahoo Corpus Dataset (GYAFC) as our parallel data for text style transfer model training.

Settings

We choose Transformer [Vaswani et al., 2017] as the architecture of NMT model and style transfer model, the texts are preprocessed with Byte Pair Encoding (BPE) [Sennrich et al., 2016] with a vocabulary size of 25,000. We used Fairseq [Ott et al., 2019] library to implement our experiments. The dimensionality of all input and output layers is 1024, and that of FFN layer is 4096. Both the encoder and decoder have 6 layers with 8 attention heads. For formality transfer model (a teacher model), we follow the work of [Chawla and Yang, 2020], and use BART-large [Lewis et al., 2019] model pretrained on CNN-DM summarization [Nallapati et al., 2016] data as our initialized parameters. The data-refinement module is pretrained on grammatical error correction (GEC) data and style transfer data. Besides, we expand these parallel data to triplets by adding the noise sentences generated by imperfect style transfer model or GEC model. We train our models with Adam [Kingma and Ba, 2015] optimizer using β1=0.9 β2=0.98 on 2 NVIDIA 2080Ti GPUs.

Baselines

We compare our stylized NMT model with the following models, which is proposed in [Wu et al., 2020], namely:

• Base-model: This method directly uses a transformerbased NMT model to evaluate on stylized test sets.
• Pivot-rule: This method first translates informal Chinese sentences as informal English sentences with NMT model. Then, it rewrites the generation results with several effective rules (Rules include capitalization, lowercase words with all upper characters, etc.).
• Pivot-model: This method first trains an informal NMT model and a formality style transfer model, then the two models decode source sentences in a pipeline manner.
• Teacher-student: For this method, it first trains a formality style transfer model that serves as a teacher model, then the student model is trained on the data generated by the teacher model.

• Back-translation: This method trains two backdirectional models, including a target-style to sourcestyle model and a model that translates source-style English sentences to source-style Chinese ones. Then the synthetic-parallel data are created by using a limited parallel corpus and a large-scale non-parallel corpus.

We also compare our text style transfer model with previous state-of-the-art works:

• Hybrid [Xu et al., 2019]: This method uses classification feedback and reconstruction constraints to train the model.
• Multi-Task [Niu et al., 2018]: This method combines the training data of GYAFC corpus with data from machine translation to train their model with a multi-task learning schema.
• GPT-CAT [Wang et al., 2019]: This method uses a pre-trained GPT2 model and combines source sentences with rule-based processed sentences to train the model.
• BART+tune [Chawla and Yang, 2020]: This method finetunes BART on our data.
• BART+LM [Chawla and Yang, 2020]: This method trains a semi-supervised formality text style transfer model using language model discriminator and mutual information maximization.

4.2 EvaluationMetrics

To evaluate different models, we apply three automatic metrics, including Formality, Fluency and Overall Evaluation, mostly following [Wu et al., 2020].

Formality: We train a BERT-based classifier using the training data of the GYAFC by regarding this problem as a binary classification problem.

Fluency: We evaluate the fluency of translation sentences using a language model, which is trained on GYAFC training data by Kenlm 2 . Each sentence is scored from 0 to 4 by the language model according to the syntactic correctness.

2 https://github.com/kpu/kenlm

Overall Evaluation: We evaluate the overall quality of formality-transferred sentences with BLEU [Papineni et al., 2002].

Human Evaluation: Similar to [Wu et al., 2020], at first, we randomly sample 300 output sentences for human evaluation, then three human annotators are required to score the overall quality of all model outputs from -3 to 3, denoted as: -3: very informal, -2: informal, -1: somewhat informal, 0: neutral, 1: somewhat Formal, 2: formal and 3: very formal.

4.3 Results

We show the results on MTFC corpus in Table 3. As can be seen, our proposed method IDKT+Refine significantly surpasses all previous methods, achieving improvements by up to 5 BLEU points compared to previous best results and gaining best score 0.897 on formality accuracy. The poor performance of the Base-model on test data indicates that a general NMT model is not applicable in this specific task. Although the pivot-based method can significantly improve the accuracy of formality style transfer and overall performance, it still suffers from the problem of error propagation. The teacher-student method achieves high score on formality accuracy, because the NMT model learns style knowledge from the teacher model. Compared to the pivot-based method, our method absorbs the benefit of teacher-student method and generate stylized translation in one model, which can reduce the number of model parameters and accelerate the decoding speed. The back-translation method produces good results in terms of BLEU, but the noise in synthetic-data of backtranslation may harm formality accuracy.

Besides, as shown in Table 4 and Table 5, by utilizing the knowledge from NMT model, our formality style transfer model achieves more than 2 BLEU points improvement compared to the BART+tune baseline. After refining the synthetic parallel data with the data-refinement module, our method achieves consistent improvement in terms of BLEU and formality accuracy. Notably, our method only needs the synthetic data generated by NMT model but can still outperform previous baselines, indicating that the knowledge in NMT model is beneficial to text style transfer model.

We also conduct human evaluation, which reflects the superior performance of our NMT model and text style transfer model. Our method gets the highest scores in human evaluation, which is consistent with BLEU and formality accuracy.

4.4 Discussion

Ablation Study. We implement an ablation study to check the contributions of the key components in our proposed framework. The performance of our method is reported in row (1) of Table 6. Comparing row (1) and row (2), we can see that if the text style transfer model is not initialized with pre-trained BART, the performance would suffer a degradation of 2.1 BLEU points, proving that the style transfer model contains more style information when the model has better performance. Moreover, when we remove the datarefinement module, as shown in row (3), the BLEU score drops by 1.9 points, which shows that the data refinement module can filter out the noise of synthetic data effectively. When only performing one-pass knowledge transfer from text style transfer model to NMT model, as shown in row (4), the performance of the stylized NMT model decreases from 45.58 to 42.83, which indicates that corpora of machine translation and formality style transfer should be repeatedly used to fully utilize the knowledge in these datasets.

Effect of Iteration Number K. We conduct experiments with different iteration number K, on small batches of data that are sampled from MTFC, as shown in Figure 3. It can be observed that the performance of NMT model increases steadily during the first three iterations. However, the model performance does not change significantly when the number of iterations exceeds 3. So we set K=3 in our experiments.

Effect of The Performance of Text Style Model. In this group of experiments, we investigate the impacts of text style transfer model on the performance of stylized NMT model.

As shown in Table 7, the performance of the NMT model is related to the performance of the text style transfer model. When the performance of the latter increases, the performance of the former also increases. A better text style transfer model may contain more style knowledge, which enables the NMT model to achieve better performance after knowledge transfer. In other words, our bidirectional knowledge transfer method is beneficial to both the stylized NMT model and the text style transfer model.

5 Conclusion

In this paper, we propose an iterative dual knowledge transfer framework for producing coherent and stylized translations. Our method can perform bidirectional knowledge transfer between translation model and text style transfer model iteratively to fully exploit the knowledge and reinforce each other. We further leverage a data-refinement module to refine the low-quality synthetic data. Experiments on formality style translation demonstrate the effectiveness of our method, achieving obvious improvements over previous work.

In the future, we will explore how to effectively leverage unpaired monolingual data in our method. We also prepare to apply our framework to other text generation tasks.

Acknowledgments

The research work descried in this paper has been supported by the National Key R&D Program of China (2020AAA0108001) and the National Nature Science Foundation of China (No. 61976015, 61976016, 61876198 and 61370130). The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve this paper.