Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-length vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.
核心贡献 · Key contributions
提出注意力机制，允许对每个目标词在源句中进行软搜索。 Proposes an attention mechanism that allows soft-search over source sentence parts for each target word.
消除了编码器-解码器模型中的固定长度向量瓶颈，改善了长句翻译。 Eliminates the fixed-length vector bottleneck in encoder-decoder models, improving long sentence translation.
在英法 WMT'14 任务上达到与基于短语的统计机器翻译相当的性能。 Achieves translation performance comparable to phrase-based SMT on English-to-French WMT'14.
使用双向 RNN 编码器捕捉每个词的前后文信息。 Uses bidirectional RNN encoder to capture both preceding and following context for each word.
通过反向传播在单个神经网络中联合训练对齐和翻译。 Jointly trains alignment and translation in a single neural network via backpropagation.
可视化软对齐符合语言直觉，处理非单调重排。 Visualizes soft-alignments that align with linguistic intuition, handling non-monotonic reordering.
局限 · Limitations
为每个目标词计算所有源词的权重，限制了在超长序列上的可扩展性。 Computes annotation weights for every source word per target word, limiting scalability to very long sequences.
需要固定词表并处理未知词，无法处理罕见或词表外词汇。 Requires fixed vocabulary with unknown word token, failing to handle rare or out-of-vocabulary words.
仅在英法上测试，对其他语言对（如英德）的有效性未验证。 Tested only on English-to-French; effectiveness on other language pairs with different reordering patterns is unverified.
软对齐相比传统统计机器翻译的硬对齐计算开销更大。 Soft alignment is computationally expensive compared to hard alignment used in traditional SMT.
性能依赖大规模平行语料，数据有限时可能下降。 Performance relies on large parallel corpora; may degrade with limited data.