国际象棋是人工智能历史上研究最广泛的领域。最强的程序基于复杂搜索技术、领域特定调整和人类专家数十年来精心设计的评估函数的结合。相比之下，AlphaGo Zero 程序最近通过从自我对弈游戏中进行的无先验知识强化学习，在围棋中实现了超人类表现。在本文中，我们将这种方法推广为一个单一的 AlphaZero 算法，该算法可以在许多具有挑战性的领域中通过无先验知识学习达到超人类表现。从随机对弈开始，除了游戏规则外没有任何领域知识，AlphaZero 在 24 小时内就在国际象棋、将棋（日本象棋）以及围棋中达到了超人类水平，并在每种情况下令人信服地击败了世界冠军程序。
The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.
核心贡献 · Key contributions
提出 AlphaZero，一个通过从零开始的强化学习在象棋、将棋和围棋中实现超人类水平的单一算法。 Proposes AlphaZero, a single algorithm achieving superhuman play in chess, shogi, and Go via tabula rasa reinforcement learning.
使用深度神经网络和蒙特卡洛树搜索，除游戏规则外无需领域特定知识。 Uses deep neural networks and Monte-Carlo tree search without domain-specific knowledge beyond game rules.
在 100 局比赛中击败世界冠军程序 Stockfish、Elmo 和 AlphaGo Zero，象棋中零败。 Defeats world-champion programs Stockfish, Elmo, and AlphaGo Zero in 100-game matches with zero losses in chess.
证明 MCTS 在象棋和将棋中随思考时间的扩展效果优于 alpha-beta 搜索。 Demonstrates MCTS scales more effectively with thinking time than alpha-beta search in chess and shogi.
在自我对弈训练中独立发现并下出常见的人类象棋开局。 Discovers and plays common human chess openings independently during self-play training.
提供无需超参数调优即可适用于多种游戏的通用框架。 Provides a general framework applicable to multiple games without hyper-parameter tuning.
局限 · Limitations
需要游戏规则的完美知识用于 MCTS 模拟和输入编码。 Requires perfect knowledge of game rules for MCTS simulations and input encoding.
神经网络架构可能不适合具有不对称规则或长程交互的游戏。 Neural network architecture may not suit games with asymmetric rules or long-range interactions.
训练需要大量计算资源（5000 个 TPU 用于自我对弈，64 个用于训练）。 Training requires massive computational resources (5000 TPUs for self-play, 64 for training).
未整合领域特定增强，如开局库或残局数据库。 Does not incorporate domain-specific enhancements like opening books or endgame tablebases.
在平局或非二元结果的游戏中，由于期望结果优化，性能可能下降。 Performance may degrade in games with draws or non-binary outcomes due to expected outcome optimization.