Published on Fri Mar 06 2015

Encoding Source Language with Convolutional Neural Network for Machine Translation

Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang, Qun Liu

The recently proposed neural network joint model (NNJM) (Devlin et al., 2014)augments the n-gram target language model. The proposed model can achieve significant improvements over the previous NNJM by up to +1.08 BLEU points on average.

0
0
0
Abstract

The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language model with a heuristically chosen source context window, achieving state-of-the-art performance in SMT. In this paper, we give a more systematic treatment by summarizing the relevant source information through a convolutional architecture guided by the target information. With different guiding signals during decoding, our specifically designed convolution+gating architectures can pinpoint the parts of a source sentence that are relevant to predicting a target word, and fuse them with the context of entire source sentence to form a unified representation. This representation, together with target language words, are fed to a deep neural network (DNN) to form a stronger NNJM. Experiments on two NIST Chinese-English translation tasks show that the proposed model can achieve significant improvements over the previous NNJM by up to +1.08 BLEU points on average

Mon Nov 07 2016
NLP
A Convolutional Encoder Model for Neural Machine Translation
The prevalent approach to neural machine translation relies on bi-directional LSTMs. In this paper we present a faster architecture based on a succession of convolutional layers. This allows to encode the entire source sentence simultaneously.
0
0
0
Tue May 02 2017
Machine Learning
Deep Neural Machine Translation with Linear Associative Unit
Deep Neural Networks (DNNs) have provably enhanced the state-of-the-art Neural Machine Translation (NMT) systems. NMT systems with deep architecture in their encoder or decoder RNNs often suffer from severe gradient diffusion. To address this problem we
0
0
0
Sun Feb 16 2020
NLP
Neural Machine Translation with Joint Representation
The Reformer models offer a new Sequence-to- Sequence modelling paradigm. They outperform the Transformer baseline in IWSLT14 German-English and NIST12 Chinese-English translation tasks.
0
0
0
Wed Dec 06 2017
NLP
Multi-channel Encoder for Neural Machine Translation
Attention-based Encoder-Decoder has the effective architecture for neural machine translation (NMT) NMT typically relies on recurrent neural networks (RNN) to build blocks that will be lately called by attentive reader during decoding. This design of encoder yields relatively uniform composition on source sentence.
0
0
0
Wed Mar 02 2016
Neural Networks
Character-based Neural Machine Translation
The proposed MT scheme provides improved results even when the source language is not morphologically rich. Improvements up to 3 BLEU points are obtained in the German-English WMT task.
0
0
0
Wed Oct 24 2018
Artificial Intelligence
Exploiting Deep Representations for Neural Machine Translation
Advanced neural machine translation (NMT) models generally implement encoder and decoder as multiple layers. We propose to simultaneously expose all of these signals with layer aggregation and multi-layer attention mechanisms. Experimental results on widely-used WMT14 English-German and WMT17 Chinese-
0
0
0