We introduce Jukebox, a model that generates music with singing in the raw audio domain. We tackle the long context of raw audio using a multi-scale VQ-VAE to compress it to discrete codes, and modeling those using autoregressive Transformers. We show that the combined model at scale can generate high-fidelity and diverse songs with coherence up to multiple minutes. We can condition on artist and genre to steer the musical and vocal style, and on unaligned lyrics to make the singing more controllable. We are releasing thousands of non cherry-picked samples at https://jukebox.openai.com, along with model weights and code at https://github.com/openai/jukebox
核心贡献 · Key contributions
提出 Jukebox，一种在原始音频域生成带歌声音乐的生成模型。 Introduces Jukebox, a generative model for music with singing in raw audio domain.
使用多尺度 VQ-VAE 将原始音频压缩为离散编码。 Uses multi-scale VQ-VAE to compress raw audio into discrete codes.
用自回归 Transformer 对压缩编码建模，实现长程连贯性。 Models compressed codes with autoregressive Transformers for long-range coherence.
支持以艺术家、流派和未对齐歌词为条件，实现可控生成。 Enables conditioning on artist, genre, and unaligned lyrics for controllable generation.
展示高保真、多样化的歌曲生成，连贯性可达数分钟。 Demonstrates high-fidelity and diverse song generation with coherence up to multiple minutes.
发布模型权重、代码及数千个样本，确保可复现性。 Releases model weights, code, and thousands of samples for reproducibility.
局限 · Limitations
生成的歌曲缺乏传统大型音乐结构，如重复的副歌。 Generated songs lack traditional large-scale musical structures like repeating choruses.
音频质量在小时间尺度上存在噪声和刺耳感。 Audio quality suffers from noise and scratchiness at small time scales.
模型仅用英语歌曲训练，限制了语言和风格多样性。 Model trained only on English-language songs, limiting language and style diversity.
采样和上采样过程缓慢（每分钟音频需数小时）。 Slow sampling and upsampling process (hours per minute of audio).
歌词对齐在快节奏流派（如嘻哈）中仍不完美。 Lyrics alignment remains imperfect for fast-paced genres like hip-hop.
论文章节 · Sections（共 21）
摘要Abstract
1 引言1 Introduction
2 背景2 Background
2.1 VQ-VAE2.1 VQ-VAE
3 音乐 VQ-VAE3 Music VQ-VAE
3.1 嵌入的随机重启3.1 Random restarts for embeddings
3.2 分离自编码器3.2 Separated Autoencoders
3.3 频谱损失3.3 Spectral Loss
4 音乐先验与上采样器4 Music Priors and Upsamplers
4.1 艺术家、流派和时序条件4.1 Artist, Genre, and Timing Conditioning