Deep Speech 2：英语和普通话的端到端语音识别

Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

达里奥·阿莫迪 Dario Amodei · Baidu · 2015-12-08 · arXiv:1512.02595 ↗ · 被引 3172

摘要 · Abstract

我们展示了一种端到端的深度学习方法，可用于识别英语和普通话这两种截然不同的语言。由于用神经网络取代了整个人工设计的组件流水线，端到端学习使我们能够处理多样化的语音，包括嘈杂环境、口音和不同语言。我们方法的关键是应用高性能计算技术，相比之前的系统实现了 7 倍的加速。由于这种效率，以前需要数周的实验现在只需几天即可完成。这使我们能够更快地迭代，以识别更优的架构和算法。因此，在多个案例中，我们的系统在标准数据集上的表现与人工转录员相当。最后，通过使用数据中心 GPU 的批量调度技术，我们展示了系统可以在线低成本部署，在服务大规模用户时提供低延迟。

We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech--two vastly different languages. Because it replaces entire pipelines of hand-engineered components with neural networks, end-to-end learning allows us to handle a diverse variety of speech including noisy environments, accents and different languages. Key to our approach is our application of HPC techniques, resulting in a 7x speedup over our previous system. Because of this efficiency, experiments that previously took weeks now run in days. This enables us to iterate more quickly to identify superior architectures and algorithms. As a result, in several cases, our system is competitive with the transcription of human workers when benchmarked on standard datasets. Finally, using a technique called Batch Dispatch with GPUs in the data center, we show that our system can be inexpensively deployed in an online setting, delivering low latency when serving users at scale.

核心贡献 · Key contributions

端到端深度学习用神经网络取代手工设计的 ASR 流水线，处理多样化语音。
End-to-end deep learning replaces hand-engineered ASR pipelines with neural networks, handling diverse speech.
HPC 技术实现相比先前系统 7 倍加速，将训练从数周缩短至数天。
HPC techniques achieve 7x speedup over prior system, reducing training from weeks to days.
RNN 的批归一化改善深层网络的收敛和泛化。
Batch Normalization for RNNs improves convergence and generalization in deep networks.
SortaGrad 课程学习通过按长度排序样本稳定训练。
SortaGrad curriculum learning stabilizes training by ordering examples by length.
系统在多个基准上匹配或超越人类转录准确率。
System matches or exceeds human transcription accuracy on several benchmarks.
GPU 批量调度实现大规模低延迟在线部署。
Batch Dispatch with GPUs enables low-latency online deployment at scale.

局限 · Limitations

系统需要大量标注数据（英语 11,940 小时，普通话 9,400 小时）。
System requires large labeled datasets (11,940 hours English, 9,400 hours Mandarin).
双向 RNN 阻碍低延迟流式处理；单向替代可能损失准确率。
Bidirectional RNNs hinder low-latency streaming; unidirectional alternatives may lose accuracy.
在真实噪声数据上的性能仍落后于人类水平。
Performance on noisy real-world data still lags behind human level.
语言模型集成需要外部文本语料和调参。
Language model integration requires external text corpora and tuning.
GPU 内存限制长话语的模型深度和批大小。
GPU memory constraints limit model depth and batch size for long utterances.

论文章节 · Sections（共 31）

摘要 Abstract
1 引言 1 Introduction
2 相关工作 2 Related Work
3 模型架构 3 Model Architecture
3.1 预备知识 3.1 Preliminaries
3.2 深度 RNN 的批归一化 3.2 Batch Normalization for Deep RNNs
3.3 SortaGrad 3.3 SortaGrad
3.4 简单 RNN 与 GRU 的比较 3.4 Comparison of simple RNNs and GRUs
3.5 频率卷积 3.5 Frequency Convolutions
3.6 步长 3.6 Striding
3.7 行卷积与单向模型 3.7 Row Convolution and Unidirectional Models
3.8 语言模型 3.8 Language Model
3.9 针对普通话的适配 3.9 Adaptation to Mandarin
4 系统优化 4 System Optimizations
4.1 可扩展性与数据并行 4.1 Scalability and Data-Parallelism
4.2 CTC 损失函数的 GPU 实现 4.2 GPU implementation of CTC loss function
4.3 内存分配 4.3 Memory allocation
5 训练数据 5 Training Data
5.1 数据集构建 5.1 Dataset Construction
5.2 数据增强 5.2 Data Augmentation
5.3 数据规模扩展 5.3 Scaling Data
6 结果 6 Results
6.1 英语 6.1 English
6.2 普通话 6.2 Mandarin
7 部署 7 Deployment
7.1 批量调度 7.1 Batch Dispatch
7.2 部署优化的矩阵乘法核 7.2 Deployment Optimized Matrix Multiply Kernels
7.3 束搜索 7.3 Beam Search
7.4 结果 7.4 Results
8 结论 8 Conclusion
致谢 Acknowledgments

阅读逐段中英对照全文 →