Neural Network Translation Models for Grammatical Error Correction

1 Introduction

Grammatical error correction (GEC) is a challenging task due to the variability of the type of errors and the syntactic and semantic dependencies of the errors on the surrounding context. Most of the grammatical error correction systems use classification and rule-based approaches for correcting specific error types. However, these systems use several linguistic cues as features. The standard linguistic analysis tools like part-of-speech (POS) taggers and parsers are often trained on well-formed text and perform poorly on ungrammatical text. This introduces further errors and limits the performance of rule-based and classification approaches to GEC. As a consequence, the phrase-based statistical machine translation (SMT) approach to GEC has gained popularity because of its ability to learn text transformations from erroneous text to correct text from error-corrected parallel corpora without any additional linguistic information. They are also not limited to specific error types. Currently, many state-of-the-art GEC systems are based on SMT or use SMT components for error correction [Susanto et al., 2014; Felice et al., 2014;

Junczys-Dowmunt and Grundkiewicz, 2014]. In this paper, grammatical error correction includes correcting errors of all types, including word choice errors and collocation errors which constitute a large class of learners' errors.

We model our GEC system based on the phrase-based SMT approach. However, traditional phrase-based SMT systems treat words and phrases as discrete entities. We take advantage of continuous space representation by adding two neural network components that have been shown to improve SMT systems [Ha et al., 2014; Devlin et al., 2014]. These neural networks are able to capture non-linear relationships between source and target sentences and can encode contextual information more effectively. Our experiments show that the addition of these two neural networks leads to significant improvements over a strong baseline and outperforms the current state of the art.

2 RelatedWork

In the past decade, there has been increasing attention on grammatical error correction in English, mainly due to the growing number of English as Second Language (ESL) learners around the world. The popularity of this problem in natural language processing research grew further through Helping Our Own (HOO) and the CoNLL shared tasks [Dale and Kilgarriff, 2011; Dale et al., 2012; Ng et al., 2013; 2014]. Most published work in GEC aimed at building specific classifiers for different error types and then use them to build hybrid systems [Dahlmeier et al., 2012; Rozovskaya et al., 2014]. One of the first approaches of using SMT for GEC focused on correction of countability errors of mass nouns (e.g., many informations!much information) [Brockett et al., 2006]. They had to use an artificially constructed parallel corpus for training their SMT system. Later, the availability of large-scale error corrected data [Mizumoto et al., 2011] further improved SMT-based GEC systems.

Recently, continuous space representations of words and phrases have been incorporated into SMT systems via neural networks. Specifically, addition of monolingual neural network language models [Bengio et al., 2003; Vaswani et al., 2013], neural network joint models (NNJM) [Devlin et al., 2014], and neural network global lexicon models (NNGLM) [Ha et al., 2014] have been shown to be useful for SMT. Neural networks have been previously used for GEC as a language model feature in the classification approach [Wu et al.,

2014] and as a classifier for article error correction [Sun et al., 2015]. Recently, a neural machine translation approach has been proposed for GEC [Yuan and Briscoe, 2016]. This method uses a recurrent neural network to perform sequenceto-sequence mapping from erroneous to well-formed sentences. Additionally, it relies on a post-processing step based on statistical word-based translation models to replace outof-vocabulary words. In this paper, we investigate the effectiveness of two neural network models, NNGLM and NNJM, in SMT-based GEC. To the best of our knowledge, there is no prior work that uses these two neural network models for SMT-based GEC.

3 AMachine Translation Framework for Grammatical Error Correction

In this paper, the task of grammatical error correction is formulated as a translation task from the language of 'bad' English to the language of 'good' English. That is, the source sentence is written by a second language learner and potentially contains grammatical errors, whereas the target sentence is the corrected fluent sentence. We use a phrase-based machine translation framework [Koehn et al., 2003] for translation, which employs a log-linear model to find the best translation T ⇤ given a source sentence S. The best translation is selected according to the following equation:

where N is the number of features, hi and λi are the ith feature function and feature weight, respectively. We make use of the standard features used in phrase-based translation without any reordering, leading to monotone translations. The features can be broadly categorized as translation model and language model features. The translation model in the phrasebased machine translation framework is trained using parallel data, i.e., sentence-aligned erroneous source text and corrected target text. The translation model is responsible for finding the best transformation of the source sentence to produce the corrected sentence. On the other hand, the language model is trained on well-formed English text and this ensures the fluency of the corrected text. To find the optimal feature weights (λ), we use minimum error rate training (MERT), maximizing the F0▷5 measure on the development set [Junczys-Dowmunt and Grundkiewicz, 2014]. The F0▷5 measure [Dahlmeier and Ng, 2012], which weights precision twice as much as recall, is the evaluation metric widely used for GEC and was the official evaluation metric adopted in the CoNLL 2014 shared task [Ng et al., 2014].

Additionally, we augment the feature set by adding two neural network translation models, namely a neural network global lexicon model [Ha et al., 2014] and a neural network joint model [Devlin et al., 2014]. These models are described in detail in Sections 4 and 5.

4 Neural Network Global LexiconModel

A global lexicon model is used to predict the presence of words in the corrected output. The model estimates the overall probability of a target hypothesis (i.e., a candidate corrected sentence) given the source sentence, by making use of the probability computed for each word in the hypothesis. The individual word probabilities can be computed by training density estimation models such as maximum entropy [Mauser et al., 2009] or probabilistic neural networks [Ha et al., 2014]. Following [Ha et al., 2014], we formulate our global lexicon model using a feed-forward neural network. The model and the training algorithm are described below.

4.1 Model

The probability of a target hypothesis is computed using the following equation:

where S and T are the source sentence and the target hypothesis respectively, and ♣T ♣ denotes the number of words in the target hypothesis. P(ti♣S) is the probability of the target word ti given the source sentence S. P(ti♣S) is the output of the neural network. The architecture of the neural network is shown in Figure 1. P(ti♣S) is calculated by:

where O1 is the hidden layer output, and W2 and b2 are the output layer weights and biases respectively. σi is the element-wise sigmoid function which scales the output to (0↪ 1).

O1 is computed by the following equation:

where φ is the activation function, and W1 and b1 are the hidden layer weights and biases applied on a binary bag-ofwords representation of the input sentence denoted by ˆ S. The size of ˆ S is equal to the size of the source vocabulary ♣Vs♣ and each element indicates the presence or absence (denoted by 1 or 0 respectively) of a given source word.

4.2 Training

The model is trained using mini-batch gradient descent with back-propagation. We use binary cross entropy (Equation 2)

as the cost function:

where ˆ T refers to the binary bag-of-words representation of the reference target sentence, and Vt is the target vocabulary. Each mini-batch is composed of a fixed number of sentence pairs (S↪ T ). The training algorithm repeatedly minimizes the cost function calculated for a given mini-batch by updating the parameters according to the gradients.

4.3 Rescaling

Since the prior probability of observing a particular word in a sentence is usually a small number, the probabilistic output of NNGLM can be biased towards zero. This bias can hurt the performance of our system and therefore, we try to alleviate this problem by rescaling the output after training NNGLM. Our solution is to map the output probabilities to a new probability space by fitting a logistic function on the output. Formally, we use Equation 3 as the mapping function:

where Q(ti♣S) is the rescaled probability and w and b are the parameters. For each sentence pair (S↪ T) in the development set, we collect training instances of the form (x↪ y) for every word t in the target vocabulary, where x = P(t♣S) and y 2 ¶0↪ 1♦. The value of y is set according to the presence (y = 1) or absence (y = 0) of the word t in the target sentence T. We use weighted cross entropy loss function with L2regularization to train w and b on the development set:

Here, M is the number of training samples, p(x) is the probability of x computed by p(x) = 1◁(1+exp(-wx-b)), and c0 and c1 are the weights assigned to the two classes y = 0 and y = 1, respectively. In order to balance the two classes, we weight each class inversely proportional to class frequencies in the training data (Equation 4) to put more weight on the less frequent class:

In Equation 4, f0 and f1 are the number of samples in each class. After training the rescaling model, we use w and b to calculate Q(ti♣S) according to Equation 3. Finally, we use Q(ti♣S) instead of P(ti♣S) in Equation 1.

5 Neural Network Joint Model

Joint models in translation augment the context information in language models with words from the source sentence. A neural network joint model (NNJM) [Devlin et al., 2014] uses a neural network to model the word probabilities given a context composed of source and target words. NNJM can scale up to large order of n-grams and still perform well because of its ability to capture semantic information through continuous space representations of words and to learn non-linear relationship between source and target words. Unlike the global lexicon model, NNJM uses a fixed window from the source side and take sequence information of words into consideration in order to estimate the probability of the target word. The model and the training method are described below.

5.1 Model

The probability of the target hypothesis T given the source sentence S is estimated by the following equation:

where ♣T♣ is the number of words in the target sentence, ti is the ith target word, and hi is the context (history) for the target word ti. The context hi consists of a set of m source words represented by (sai- m-1 2 ↪ · · · ↪ sai+ m-1 2 ) and n - 1 words preceding ti from the target sentence represented by (ti-n+1↪ · · · ↪ ti-1). The context words from the source side are the words in the window of size m surrounding the source word sai that is aligned to the target word ti. The output of the neural network P(ti♣hi) is the output of the final softmax layer which is given by the following equation:

where Ui(hi) is the output of the neural network before applying softmax and Z(hi) is given by following equation:

The output of the neural network before softmax is computed by applying output layer weights W2 and biases b2 to the hidden layer output O1.

O1 is computed by applying weights W1 and biases b1 on the hidden layer input O0 and using a non-linear activation function φ:

The input to the hidden layer (O0) is a concatenated vector of context word embeddings:

where ˆ s and ˆ t are the one-hot representations of the source word s and the target word t, respectively. Similarly, Es and Et are the word embeddings matrices for the source words and the target words.

As we use log probabilities instead of raw probabilities in our GEC system, Equation 5 can be rewritten as the following:

Finally, since the network is trained by Noise Contrastive Estimation (NCE) (described in Section 5.2), it becomes selfnormalized. This means that Z(hi) will be approximately 1 and hence the raw output of the neural network Ui can be directly used as the log probabilities during decoding.

5.2 Training

To avoid the costly softmax layer and thereby speed up both training and decoding, we use Noise Contrastive Estimation (NCE) following [Vaswani et al., 2013]. During training, the negative log likelihood cost function is modified to a probabilistic binary classifier, which learns to discriminate between the actual target word and k random words (noisy samples) per training instance selected from a noise distribution q. The two classes are C = 1 indicating that the word is the target word and C = 0 indicating that the word is a noisy sample. The conditional probabilities for C = 0 and C = 1 given a target word and context is given by:

where, P(ti♣hi) is the model probability given in Equation 6. The negative log likelihood cost function is replaced by the following function.

where ¯ tij refers to the j th noise sample for the target word ti. Z(hi) is required for the computation of the neural network output, P(ti♣hi). However, setting the term Z(hi) to 1 during training forces the output of the neural network to be selfnormalized. Hence, Equation 7 reduces to:

Using Equation 5.2 avoids the expensive softmax computation in the final layer and consequently speeds up decoding.

6 Experiments

We describe our experimental setup including the description of the data we used, the configuration of our baseline system and the neural network components, and the evaluation method in Section 6.1, followed by the results and discussion in Section 6.2

6.1 Setup

We use the popular phrase-based machine translation toolkit Moses 1 as our baseline SMT system. NUCLE [Dahlmeier et al., 2013], which is the official training data for the CoNLL 2013 and 2014 shared tasks, is used as the parallel text for training. Additionally, we obtain parallel corpora from Lang8 Corpus of Learner English v1.0 [Mizumoto et al., 2011], which consists of texts written by ESL (English as Second Language) learners on the language learning platform Lang8 2 . We use the test data for the CoNLL 2013 shared task as our development data. The statistics of the training and development data are given in Table 1. Source side refers to the original text written by the ESL learners and target side refers to the corresponding corrected text hand-corrected by humans. The source side and the target side are sentencealigned and tokenized.

We train the translation model for our SMT system using a concatenation of NUCLE and Lang-8 v1.0 parallel data. The training data is cleaned up by removing sentence pairs in which either the source or the target sentence is empty, or is too long (greater than 80 tokens), or violate a 9:1 sentence ratio limit. The translation model uses the default features in Moses which include the forward and inverse phrase translation probabilities, forward and inverse lexical weights, word penalty, and phrase penalty. We compute the phrase alignments using standard tools in Moses. We use two language model features: a 5-gram language model trained using the target side of NUCLE used for training the translation model and a 5-gram language model trained using English Wikipedia (⇠1.78 billion tokens). Both language models are estimated with KenLM 3 using modified Kneser-Ney smoothing. We use MERT for tuning the feature weights by optimizing the F0▷5 measure (which weights precision twice as much as recall). This system constitutes our baseline system in Table 2. Our baseline system uses exactly the same training data as [Susanto et al., 2014] for training the translation model and the language model. The difference between our baseline system and the SMT components of [Susanto et al., 2014] is that we tune with F0▷5 instead of BLEU and we use the standard Moses configuration without the Levenshtein distance feature.

1 https://github.com/moses-smt/mosesdecoder/tree/fscorer

2 http://lang-8.com/

3 https://kheafield.com/code/kenlm/

On top of our baseline system described above, we incorporate the two neural network components, neural network global lexicon model (NNGLM) and neural network joint model (NNJM) as features. Both NNGLM and NNJM are trained using the parallel data used to train the translation model of our baseline system.

We implement NNGLM using the Theano library 4 in Python in order to make use of parallelization with GPUs, thus speeding up training significantly. We use a source and target vocabulary of 10,000 most frequent words on both sides. We use a single hidden layer neural network with 2,000 hidden nodes. We use tanh as the activation function for the hidden layer. We optimize the model weights by stochastic gradient descent using a mini-batch size of 100 and a learning rate 5 of 10. We train the model for 45 epochs. The logistic regression function for rescaling is trained using the probabilities obtained from this model on the development set. To speed up tuning and decoding, we pre-compute the probabilities of target words using the source side sentences of the development and the test sets, respectively. We implement a feature function in Moses to compute the probability of a target hypothesis given the source sentence using the precomputed probabilities.

To train NNJM, we use the publicly available implementation, Neural Probabilistic Language Model (NPLM) [Vaswani et al., 2013]. The latest version of Moses can incorporate NNJM trained using NPLM as a feature while decoding. Similar to NNGLM, we use the parallel text used for training the translation model in order to train NNJM. We use a source context window size of 5 and a target context window size of 4. We select a source context vocabulary of 16,000 most frequent words from the source side. The target context vocabulary and output vocabulary is set to the 32,000 most frequent words. We use a single hidden layer to speed up training and decoding with an input embedding dimension of 192 and 512 hidden layer nodes. We use rectified linear units (ReLU) as the activation function. We train NNJMwith noise contrastive estimation with 100 noise samples per training instance, which are obtained from a unigram distribution. The neural network is trained for 30 epochs using stochastic gradient descent optimization with a mini-batch size of 128 and learning rate of 0.1.

We conduct experiments by incorporating NNGLM and NNJM both independently and jointly into our baseline system. The results of our experiments are described in Section 6.2. The evaluation is performed similar to the CoNLL 2014 shared task setting using the the official test data of the CoNLL 2014 shared task with annotations from two annotators (without considering alternative annotations suggested by the participating teams). The test dataset consists of 1,312 error-annotated sentences with 30,144 tokens on the source side. We make use of the official scorer for the shared task, M2Scorer v3.2 [Dahlmeier and Ng, 2012], for evaluation. We perform statistical significance test using one-tailed sign test with bootstrap resampling on 100 samples.

4 http://deeplearning.net/software/theano

5 We divide the gradient by the mini-batch size.

6.2 Results and Discussion

Table 2 presents the results of our experiments with neural network global lexicon model (NNGLM) and neural network joint model (NNJM).

We see that the addition of both NNGLM and NNJM to our baseline individually improves F0▷5 measure on the CoNLL 2014 test set by 0.43 and 0.80, respectively. Although both improvements over the baseline are statistically significant (with p < 0▷01), we observe that the improvement of NNGLM is slightly lower than that of NNJM. NNGLM encodes the entire lexical information from the source sentence without word ordering information. Hence, it focuses mostly on the choice of words appearing in the output. Many of the words in the source context may not be necessary for ensuring the quality of corrected output. On the other hand, NNJM looks at a smaller window of words in the source side. NNJM can act as a language model and can ensure a fluent translation output compared to NNGLM.

We also found rescaling to be important for NNGLM because of imbalanced training data. While the most frequent words in the data, 'I' and to', appear in 43% and 27% of the training sentences, respectively, most words occur in very few sentences only. For example, the word 'set' appears in 0.15% of the sentences and the word 'enterprise' appears in 0.003% of the sentences.

By incorporating both components together, we obtain an improvement of 1.17 in terms of F0▷5 measure. This indicates that both components are beneficial and complement each other to improve the performance of the baseline system. While NNGLM looks at the entire source sentence and ensures the appropriate choice of words to appear in the output sentence, NNJM encourages the system to choose appropriate corrections that give a fluent output.

We compare our system to the top 3 systems in the CoNLL 2014 shared task and to the best published results [Yuan and Briscoe, 2016; Susanto et al., 2014] on the test data of the CoNLL 2014 shared task. The results are summarized in Table 4. Our final system including both neural network models outperforms the best system [Yuan and Briscoe, 2016] by 1.85 in F0▷5 measure. It should be noted that this is despite the fact that the system proposed in [Yuan and Briscoe, 2016] uses much larger training data than our system.

We qualitatively analyze the output of our neural networkenhanced systems against the outputs produced by our baseline system. We have included some examples in Table 3 and the corresponding outputs of the baseline system and the reference sentences. The selected examples show that NNGLM and NNJM choose appropriate words by making use of the surrounding context effectively.

Note that our neural networks, which rely on fixed source and target vocabulary, map the rare words and misspelled words to the UNK token. Therefore, phrases with the UNK token may get a higher probability than they actually should due to the large number of UNK tokens seen during training. This leads to fewer spelling error corrections compared to the baseline system which does not employ these neural networks. Consider the following example from the test data:

... numerous profit-driven companies realize the hugh (huge) human traffic on such social media sites ....

The spelling error hugh ! huge is corrected by the baseline system, but not by our final system with the neural networks. This is because the misspelled word hugh is not in the neural network vocabulary and so it is mapped to the UNK token. The sentence with the UNK token gets a higher score and hence the system chooses this output over the correct one.

From our experiments and analysis, we see that NNGLM and NNJM capture contextual information better than regular translation models and language models. This is because they make use of larger source sentence contexts and continuous space representation of words. This enables them to make better predictions compared to traditional translation models and language models. We also observed that our system has an edge over the baseline for correction of word choice and collocation errors.

7 Conclusion

Our experiments show that using the two neural network translation models improves the performance of a phrasebased SMT approach to GEC. To the best of our knowledge, this is the first work that uses these two neural network models for SMT-based GEC. The ability of neural networks to model words and phrases in continuous space and capture non-linear relationships enables them to generalize better and make more accurate grammatical error correction. We have achieved state-of-the-art results on the CoNLL 2014 shared task test dataset. This has been done without using any additional training data compared to the best performing systems evaluated on the same dataset.

Acknowledgments

This research is supported by Singapore Ministry of Education Academic Research Fund Tier 2 grant MOE2013-T2-1150.