This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps character embeddings to mel-scale spectrograms, followed by a modified WaveNet model acting as a vocoder to synthesize timedomain waveforms from those spectrograms. Our model achieves a mean opinion score (MOS) of $4.53$ comparable to a MOS of $4.58$ for professionally recorded speech. To validate our design choices, we present ablation studies of key components of our system and evaluate the impact of using mel spectrograms as the input to WaveNet instead of linguistic, duration, and $F_0$ features. We further demonstrate that using a compact acoustic intermediate representation enables significant simplification of the WaveNet architecture.
核心贡献 · Key contributions
提出 Tacotron 2，一种全神经 TTS 系统，结合序列到序列的梅尔频谱预测与改进的 WaveNet 声码器。 Proposes Tacotron 2, a fully neural TTS system combining sequence-to-sequence mel spectrogram prediction with a modified WaveNet vocoder.
达到 4.53 的平均意见得分，与专业录音的 4.58 相当。 Achieves a MOS of 4.53, comparable to 4.58 for professionally recorded speech.
证明使用梅尔频谱作为 WaveNet 的条件输入可显著简化 WaveNet 架构。 Demonstrates that mel spectrograms as WaveNet conditioning input enable significant simplification of the WaveNet architecture.
表明当以梅尔频谱为条件时，仅 12 层、感受野 10.5 毫秒的浅层 WaveNet 即可生成高质量音频。 Shows that a shallow WaveNet with 12 layers and 10.5 ms receptive field can generate high-quality audio when conditioned on mel spectrograms.
通过后处理网络和条件特征的消融研究验证设计选择。 Validates design choices through ablation studies on post-net and conditioning features.
无需复杂特征工程，实现接近自然人类语音的最先进音质。 Achieves state-of-the-art sound quality close to natural human speech without complex feature engineering.
局限 · Limitations
系统偶尔产生发音错误，尤其对于域外文本如人名。 The system occasionally produces mispronunciations, especially for out-of-domain text like names.
韵律建模仍不完美，某些情况下出现不自然的强调或音高。 Prosody modeling remains imperfect, with unnatural emphasis or pitch in some cases.
端点预测在极长输入句子上失败。 End-point prediction fails on very long input sentences.
训练需要匹配的数据分布；在域外文本上性能下降。 Training requires matched data distribution; performance degrades on out-of-domain text.
两个组件分开训练，使用真实特征训练 WaveNet 在推理使用预测特征时质量下降。 The two components are trained separately, and using ground truth features for WaveNet training degrades quality when inference uses predicted features.