Sequence to Sequence Learning with Neural Networks
打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→深度神经网络（DNN）是强大的模型，在困难的学习任务上取得了优异的性能。尽管 DNN 在拥有大量标注训练集时表现良好，但它们不能用于序列到序列的映射。在本文中，我们提出了一种通用的端到端序列学习方法，该方法对序列结构做了最少的假设。我们的方法使用多层长短期记忆（LSTM）将输入序列映射到一个固定维度的向量，然后使用另一个深度 LSTM 从该向量解码目标序列。我们的主要结果是，在 WMT'14 数据集的英法翻译任务上，LSTM 生成的翻译在整个测试集上达到了 34.8 的 BLEU 分数，其中 LSTM 的 BLEU 分数因词汇外单词而受到惩罚。此外，LSTM 在长句子上没有困难。相比之下，基于短语的 SMT 系统在同一数据集上达到了 33.3 的 BLEU 分数。当我们使用 LSTM 对上述 SMT 系统产生的 1000 个假设进行重排序时，其 BLEU 分数增加到 36.5，接近该任务之前的最佳结果。LSTM 还学习了合理的短语和句子表示，这些表示对词序敏感，并且对主动语态和被动语态相对不变。最后，我们发现反转所有源句子（但不是目标句子）中的单词顺序显著提高了 LSTM 的性能，因为这样做引入了源句子和目标句子之间的许多短期依赖关系，使得优化问题更容易。
Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequences to sequences. In this paper, we present a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure. Our method uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector. Our main result is that on an English to French translation task from the WMT'14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8 on the entire test set, where the LSTM's BLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did not have difficulty on long sentences. For comparison, a phrase-based SMT system achieves a BLEU score of 33.3 on the same dataset. When we used the LSTM to rerank the 1000 hypotheses produced by the aforementioned SMT system, its BLEU score increases to 36.5, which is close to the previous best result on this task. The LSTM also learned sensible phrase and sentence representations that are sensitive to word order and are relatively invariant to the active and the passive voice. Finally, we found that reversing the order of the words in all source sentences (but not target sentences) improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.