This report presents Wan, a comprehensive and open suite of video foundation models designed to push the boundaries of video generation. Built upon the mainstream diffusion transformer paradigm, Wan achieves significant advancements in generative capabilities through a series of innovations, including our novel VAE, scalable pre-training strategies, large-scale data curation, and automated evaluation metrics. These contributions collectively enhance the model's performance and versatility. Specifically, Wan is characterized by four key features: Leading Performance: The 14B model of Wan, trained on a vast dataset comprising billions of images and videos, demonstrates the scaling laws of video generation with respect to both data and model size. It consistently outperforms the existing open-source models as well as state-of-the-art commercial solutions across multiple internal and external benchmarks, demonstrating a clear and significant performance superiority. Comprehensiveness: Wan offers two capable models, i.e., 1.3B and 14B parameters, for efficiency and effectiveness respectively. It also covers multiple downstream applications, including image-to-video, instruction-guided video editing, and personal video generation, encompassing up to eight tasks. Consumer-Grade Efficiency: The 1.3B model demonstrates exceptional resource efficiency, requiring only 8.19 GB VRAM, making it compatible with a wide range of consumer-grade GPUs. Openness: We open-source the entire series of Wan, including source code and all models, with the goal of fostering the growth of the video generation community. This openness seeks to significantly expand the creative possibilities of video production in the industry and provide academia with high-quality video foundation models. All the code and models are available at https://github.com/Wan-Video/Wan2.1.
核心贡献 · Key contributions
开源 Wan 发布了 1.3B/14B 视频基础模型及代码;14B 模型在基准评测上优于已有的开源与商业解决方案。 Open-source Wan releases 1.3B/14B video foundation models and code; the 14B model outperforms prior open and commercial solutions on benchmarks.
设计了新型时空 VAE(127M),采用 4×8×8 压缩与特征缓存,在保证高视觉质量的同时显著提升重建速度。 A novel spatio-temporal VAE (127M) with 4×8×8 compression and feature cache achieves faster reconstruction and high visual quality.
通过数十亿图像/视频和高达 14B 参数的训练,展示了视频生成中的缩放定律。 Demonstrates scaling laws for video generation by training through billions of images/videos and up to 14B parameters.
消费级效率:1.3B 模型仅需 8.19 GB 显存,在文本到视频任务上优于更大的开源模型。 Consumer-grade efficiency: the 1.3B model requires only 8.19 GB VRAM while beating larger open-source models in text-to-video.
提供全面的下游应用:图生视频、指令引导编辑、个性化生成、镜头控制、实时视频、音频以及中英双语视觉文本生成。 Provides comprehensive applications: image-to-video, instruction-guided editing, personalization, camera control, real-time video, audio, and bilingual visual text.
文本编码限制为 512 个 umT5 token,影响较长或高度细化提示条件的处理。 Text encoding is limited to 512 umT5 tokens, restricting long or highly detailed prompt conditioning.
1.3B 模型仅支持 480p 分辨率;高质量生成依赖于资源消耗较大的 14B 模型。 The 1.3B model supports only 480p resolution; high-quality generation relies on the resource-heavy 14B model.
基准评测多为内部设计或自动计算,更广泛的人类偏好证据有限。 Benchmark evaluations were largely internally designed or automatically computed; broader human preference evidence is limited.
实时生成和音频生成模块的架构细节较少,可能仍处于初步阶段。 Real-time and audio generation modules are presented with less architectural detail and may be preliminary.
扩散缓存与量化优化旨在实现无损性能,但未必能原样迁移到未来模型版本。 Diffusion cache and quantization optimizations target lossless performance but may not transfer unchanged to future model versions.
论文章节 · Sections(共 23)
AbstractAbstract
1 引言1 I NTRODUCTION
2 相关工作2 R ELATED W ORK
3 数据处理流水线3 DATA P ROCESSING P IPELINE
3.1 预训练数据3.1 P RE - TRAINING DATA
3.2 后训练数据3.2 P OST- TRAINING DATA
3.3 密集视频字幕3.3 D ENSE V IDEO C APTION
4.1 时空变分自编码器4.1 S PATIO - TEMPORAL VARIATIONAL AUTOENCODER
32 Step Video 460M32 Step Video 460M
4.2 模型训练4.2 M ODEL T RAINING
4.3 模型 Scaling 与训练效率4.3 M ODEL S CALING AND T RAINING E FFICIENCY