This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.
核心贡献 · Key contributions
提出 WaveNet，一种用于原始音频波形的深度自回归模型。 Introduces WaveNet, a deep autoregressive model for raw audio waveforms.
使用扩张因果卷积高效实现大感受野。 Uses dilated causal convolutions to achieve large receptive fields efficiently.
在英语和中文文本转语音中达到最先进的自然度。 Achieves state-of-the-art text-to-speech naturalness for English and Mandarin.
单个模型可通过条件化说话人身份生成多种声音。 Single model can generate multiple voices by conditioning on speaker identity.
在音乐数据上训练时生成新颖且逼真的音乐片段。 Generates novel and realistic music fragments when trained on music data.
适应音素识别等判别任务并取得强结果。 Adapts to discriminative tasks like phoneme recognition with strong results.
局限 · Limitations
有限感受野（如 300 毫秒）导致自由形式语音缺乏长程连贯性。 Limited receptive field (e.g., 300 ms) causes lack of long-range coherence in free-form speech.
文本转语音的韵律在没有显式 F0 条件化时可能不自然。 Text-to-speech prosody can be unnatural without explicit F0 conditioning.
音乐生成在数秒内缺乏长程一致性。 Music generation lacks long-range consistency across seconds.
256 量化值的 Softmax 相比 16 位限制了音频保真度。 Softmax over 256 quantized values limits audio fidelity compared to 16-bit.
对于极长序列，训练和推理计算成本高。 Training and inference are computationally expensive for very long sequences.