LeJEPA 何时能学习世界模型？

When Does LeJEPA Learn a World Model?

杨立昆 Yann LeCun · NYU · 2026-05-25 · arXiv:2605.26379 ↗ · 被引 3

打开互动全文版（逐段中英对照 + 图/公式 + 论文问答）→

摘要 · Abstract

一种打乱世界真实自由度的表示无法支持可靠的规划或组合泛化。我们证明，在潜在变量遵循平稳加性噪声演化的广泛世界类别中，LeJEPA（对齐加高斯正则化）能从非线性观测中线性恢复世界的潜在变量，这一性质称为线性可识别性。我们的主要结果是：在所有此类世界中，高斯分布是唯一能保证这一性质的潜在分布。正向方向依赖于谱分解，其中每个非线性度都受到对齐的严格惩罚，使得线性映射成为最优；反向方向排除了所有非高斯替代方案。我们进一步证明了近似可识别性结果，其中保证会优雅地退化，并表明线性正交可识别性能够实现最优潜在空间规划。我们通过从二维示例到 1024 维潜在变量的实验验证了该理论，包括分布消融和基于像素的机器人控制。我们的理论将经验上成功的配方转化为数学保证，为构建能够可靠恢复世界结构的世界模型奠定了基础。

A representation that scrambles the true degrees of freedom of the world cannot support reliable planning or compositional generalization. We prove that LeJEPA (alignment plus Gaussian regularization) linearly recovers the world's latent variables from nonlinear observations, a property known as linear identifiability, in a broad class of worlds where latents evolve under stationary, additive-noise transitions. Our main result is that among all such worlds, the Gaussian is the unique latent distribution for which this guarantee holds. The forward direction rests on a spectral decomposition in which each degree of nonlinearity is strictly penalized by alignment, making the linear map the optimum; the converse rules out every non-Gaussian alternative. We further prove an approximate identifiability result where the guarantee degrades gracefully, and show that linear, orthogonal identifiability enables optimal latent-space planning. We validate the theory with experiments ranging from 2D examples to 1024-dimensional latents, including distributional ablations and pixel-based robotic control. Our theory turns an empirically successful recipe into a mathematical guarantee, providing the foundation for building World Models that provably recover the structure of the world.

核心贡献 · Key contributions

• 证明 LeJEPA（对齐+高斯正则化）在高斯潜在变量世界中实现线性可识别性。
  Proves that LeJEPA (alignment + Gaussian regularization) achieves linear identifiability for Gaussian latent worlds.
• 确立高斯分布是在平稳加性噪声转移下保证线性可识别性的唯一潜在分布。
  Establishes Gaussian as the unique latent distribution guaranteeing linear identifiability under stationary additive-noise transitions.
• 推导出近似可识别性界限，该界限随对齐和白化误差优雅退化。
  Derives an approximate identifiability bound that degrades gracefully with alignment and whitening errors.
• 表明线性正交可识别性使得旋转不变成本下的最优潜在空间规划成为可能。
  Shows linear orthogonal identifiability enables optimal latent-space planning for rotation-invariant costs.
• 在 2D 到 1024 维潜在变量、分布消融和基于像素的机器人控制中验证理论。
  Validates theory across 2D to 1024-dimensional latents, distributional ablations, and pixel-based robotic control.
• 为联合嵌入预测架构（JEPA）提供了首个可识别性结果。
  Provides the first identifiability result for Joint-Embedding Predictive Architectures (JEPAs).

局限 · Limitations

• 假设编码器输出维度等于真实潜在维度；未分析维度不匹配的情况。
  Assumes encoder output dimension equals true latent dimension; mismatched dimensions not analyzed.
• 总体水平结果；未涉及有限样本和优化动态。
  Population-level result; finite-sample and optimization dynamics not addressed.
• 现实世界的潜在变量可能非高斯；理论优雅退化但实际违反情况存在。
  Real-world latents may not be Gaussian; theory degrades gracefully but practical violations occur.
• 未涵盖动作条件转移学习；仅保证状态恢复。
  Action-conditioned transition learning not covered; only state recovery is guaranteed.
• 要求平稳加性噪声转移；未考虑非平稳或乘性噪声。
  Requires stationary additive-noise transitions; non-stationary or multiplicative noise not considered.

论文章节 · Sections（共 20）

• 摘要 Abstract
• 1 引言 1 Introduction
• 2 相关工作 2 Related Work
• 3 世界与学习者 3 The World and the Learner
• 3.1 世界 3.1 The World
• 3.2 学习者：LeJEPA 3.2 The Learner: LeJEPA
• 4 世界的谱分析 4 Spectral Analysis of the World
• 5 理论 5 Theory
• 5.1 正向：LeJEPA 学习世界模型 5.1 Forward Direction: LeJEPA Learns the World Model
• 5.2 反向：高斯分布的唯一性 5.2 Converse Direction: The Gaussian is Unique
• 5.3 近似可辨识性 5.3 Approximate Identifiability
• 5.4 最优潜在规划 5.4 Optimal Latent Planning
• 6 实验 6 Experiments
• 6.1 正向：高斯潜变量的线性可辨识性 6.1 Forward: Linear Identifiability from Gaussian Latent Variables
• 6.2 反向：非高斯潜变量破坏线性可辨识性 6.2 Converse: Non-Gaussian Latent Variables Break Linear Identifiability
• 6.3 近似界与损失预测可辨识性 6.3 Approximation Bound and Loss Predict Identifiability
• 6.4 线性可辨识性实现潜在空间规划 6.4 Linear Identifiability Enables Latent-Space Planning
• 7 局限性 7 Limitations
• 8 讨论 8 Discussion
• 致谢 Acknowledgments

阅读逐段中英对照全文 →