Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem. Previous work addresses the translation of out-of-vocabulary words by backing off to a dictionary. In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open-vocabulary translation by encoding rare and unknown words as sequences of subword units. This is based on the intuition that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration), compounds (via compositional translation), and cognates and loanwords (via phonological and morphological transformations). We discuss the suitability of different word segmentation techniques, including simple character n-gram models and a segmentation based on the byte pair encoding compression algorithm, and empirically show that subword models improve over a back-off dictionary baseline for the WMT 15 translation tasks English-German and English-Russian by 1.1 and 1.3 BLEU, respectively.
核心贡献 · Key contributions
提出将稀有词和未知词编码为子词单元，实现开放词汇的神经机器翻译，避免使用回退字典。 Proposes encoding rare and unknown words as subword units for open-vocabulary NMT, avoiding back-off dictionaries.
将字节对编码（BPE）应用于词分割，用固定大小的符号集实现开放词汇表示。 Adapts byte pair encoding (BPE) for word segmentation, enabling open-vocabulary representation with a fixed-size symbol set.
在 WMT 2015 英德和英俄任务上，相比回退字典基线分别提升 1.1 和 1.3 BLEU。 Achieves BLEU improvements of 1.1 (EN-DE) and 1.3 (EN-RU) over back-off dictionary baselines on WMT 2015 tasks.
联合 BPE 编码提高源语言和目标语言的分割一致性，提升翻译质量。 Joint BPE encoding improves segmentation consistency between source and target, enhancing translation quality.
表明减少子词词汇量可因稀疏性降低而提升性能。 Demonstrates that reducing subword vocabulary size can improve performance due to less sparsity.
局限 · Limitations
改进主要集中在低频和未见词上，整体 BLEU 提升有限。 Gains are mainly on rare and unseen words; overall BLEU improvements are modest.
BPE 分割可能产生不透明的单元，无法独立翻译，可能损害质量。 BPE segmentation can produce opaque units that are not translatable independently, potentially harming quality.
效果因语言对和字母表而异；不同文字的音译可能需要额外处理。 Effectiveness varies across language pairs and alphabets; transliteration for different scripts may need extra processing.
子词分割导致序列长度增加，提高训练和推理时间。 Increased sequence length from subword segmentation raises training and inference time.
BPE 合并次数是超参数，需要针对每个任务调整。 The number of BPE merge operations is a hyperparameter that requires tuning per task.