FastSpeech: Fast, Robust and Controllable Text to Speech
打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→基于神经网络的端到端文本到语音合成（TTS）显著提高了合成语音的质量。主流方法（如 Tacotron 2）通常先由文本生成梅尔频谱，再通过声码器（如 WaveNet）合成语音。与传统拼接和统计参数方法相比，基于神经网络的端到端模型存在推理速度慢、合成语音鲁棒性差（如跳词或重复）以及缺乏可控性（如语速或韵律控制）等问题。本文提出一种基于 Transformer 的新型前馈网络，用于并行生成梅尔频谱。具体地，我们从基于编码器-解码器的教师模型中提取注意力对齐，用于音素时长预测，并通过长度调节器扩展源音素序列以匹配目标梅尔频谱序列的长度，从而实现并行梅尔频谱生成。在 LJSpeech 数据集上的实验表明，我们的并行模型在语音质量上与自回归模型相当，几乎消除了困难情况下的跳词和重复问题，并能平滑调整语速。最重要的是，与自回归 Transformer TTS 相比，我们的模型将梅尔频谱生成速度提升了 270 倍，端到端语音合成速度提升了 38 倍。因此，我们将该模型命名为 FastSpeech。
Neural network based end-to-end text to speech (TTS) has significantly improved the quality of synthesized speech. Prominent methods (e.g., Tacotron 2) usually first generate mel-spectrogram from text, and then synthesize speech from the mel-spectrogram using vocoder such as WaveNet. Compared with traditional concatenative and statistical parametric approaches, neural network based end-to-end models suffer from slow inference speed, and the synthesized speech is usually not robust (i.e., some words are skipped or repeated) and lack of controllability (voice speed or prosody control). In this work, we propose a novel feed-forward network based on Transformer to generate mel-spectrogram in parallel for TTS. Specifically, we extract attention alignments from an encoder-decoder based teacher model for phoneme duration prediction, which is used by a length regulator to expand the source phoneme sequence to match the length of the target mel-spectrogram sequence for parallel mel-spectrogram generation. Experiments on the LJSpeech dataset show that our parallel model matches autoregressive models in terms of speech quality, nearly eliminates the problem of word skipping and repeating in particularly hard cases, and can adjust voice speed smoothly. Most importantly, compared with autoregressive Transformer TTS, our model speeds up mel-spectrogram generation by 270x and the end-to-end speech synthesis by 38x. Therefore, we call our model FastSpeech.