Progress on provable inference for large language models
打开互动全文版(逐段中英对照 + 针对本篇提问)→本文介绍 FlashProve——一个用于可证明推理的开源原型,它利用 zkSNARKs 让模型提供方对特定模型作出承诺,并证明每个响应均由该模型生成,同时不泄露其权重。作者指出,用户目前无法验证某个响应是来自所声称的模型,还是来自被替换或被篡改的模型,而密码学证明可以弥补这一缺口。对于一个 400B 参数的混合专家模型(Llama 4 Maverick,4 比特量化),单块 H100 GPU 生成证明的速度超过每秒 100 个 token,每 1024 token 请求的证明大小约为 19 MB,其 GPU 耗时约为在 vLLM 上运行模型本身的 70 至 95 倍,而此前的开源系统则要慢上数个数量级。该发布包含可执行代码、一个 C++ 验证器、一个经 Lean 验证的验证器,以及针对顶层安全保证的 Lean 证明,其既定目标是加速可证明推理研究,并证明 zkSNARKs 在此用途上具有实用性。作者总结称,这一进展对把控 AI 发展节奏、为客户提供完整性保证,以及为审计人员和监管者提供工具都具有意义。
This article presents FlashProve, an open-source prototype for provable inference that uses zkSNARKs to let a model provider commit to a specific model and prove that each response was produced by that model, without revealing its weights. The authors argue that users currently cannot verify that a response came from the claimed model rather than a substituted or tampered one, and that cryptographic proofs address this gap. For a 400B-parameter mixture-of-experts model (Llama 4 Maverick, 4-bit quantized), a single H100 GPU generates proofs at over 100 tokens per second with a proof size of roughly 19 MB per 1,024-token request, about 70-95 times the GPU time of running the model itself on vLLM, while prior open-source systems are orders of magnitude slower. The release includes executable code, a C++ verifier, a Lean-verified verifier, and Lean proofs for the top-level security guarantees, with the stated aim of accelerating provable inference research and demonstrating that zkSNARKs are practical for this purpose. The authors conclude that this progress has implications for pacing AI development, providing integrity guarantees to customers, and enabling tooling for auditors and regulators.
* 03 AI 扩散中的信任挑战
* 03 AI 扩散中的信任挑战
* 05 通过形式化验证实现可信性
* 05 通过形式化验证实现可信性
_Roy Rinberg(Anthropic)、Alex Allain(Anthropic);Alessandro Chiesa(EPFL、StarkWare)、Miranda Christ(MIT、RESI)、Rahul Dalal(Anthropic STEM Fellows Program、Succinct)、Giacomo Fenzi(EPFL)、Ben Mayer(Anthropic Fellows Program、MIT)、Guy Weissenberg(EPFL)_
_Roy Rinberg(Anthropic)、Alex Allain(Anthropic);Alessandro Chiesa(EPFL、StarkWare)、Miranda Christ(MIT、RESI)、Rahul Dalal(Anthropic STEM Fellows Program、Succinct)、Giacomo Fenzi(EPFL)、Ben Mayer(Anthropic Fellows Program、MIT)、Guy Weissenberg(EPFL)_
当你与朋友面对面交谈时,有很多方式可以确认对方真的是你的朋友:你可以看到他们、听到他们、与他们握手。而当你与 AI 模型对话时,你几乎没有任何方式可以确认。如今,用户查询模型时无法验证响应是否来自开发者声称的特定模型,而不是路由服务为了节省成本而秘密替换的模型,甚至也不是攻击者篡改了 token 流后返回的响应。仅凭模型的输出进行判断是不可靠的,即使它产生了高质量的响应。推理的密码学证明解决了这个问题:它们让模型提供者对一个模型做出承诺,然后证明每个响应都是由该模型生成的。这种证明可以支持从消费产品到内部安全的各种应用,并有助于构建一个用于验证是否符合 AI 节奏要求的工具包。
When you talk to a friend in person, you have many ways to know it is really them: you can see them, hear them, shake their hand. When you talk to an AI model, you have almost none. A user querying a model today cannot check that the responses came from the specific model that a developer claims is responding, as opposed to one secretly substituted in by a routing service for cost reasons, or even by an attacker who has tampered with the token stream. Judging from the model’s output alone is unreliable, even if it produces high-quality responses. Cryptographic proofs of inference solve this problem: they let a model provider commit to a model and then prove that each response was produced by that model. Such proofs could enable applications ranging from consumer products to internal security, and could contribute to a toolkit for verifying compliance with AI pacing requirements.
具体来说,我们在一种称为“zkSNARK”的强大密码学证明类型上取得了进展,它特别适合**可证明推理**(由于其强大的隐藏性和效率特性)。在本文中,我们主要通过一个名为 FlashProve 的系统来分享部分进展。简要的技术总结如下:
Specifically, we have made progress on a powerful type of cryptographic proof known as a “zkSNARK”, which is uniquely fit for **provable inference** (owing to its strong hiding and efficiency properties). In this post, we share some of that progress, primarily through a system we call FlashProve. A brief technical summary is as follows:
1. Claude 在用于推理的 zkSNARK 上取得了改进,构建了一个比先前开源系统快得多的系统。
1. Claude has produced improvements on zkSNARKs for inference, building a system that is substantially faster than prior open-source systems.
1. 对于大型模型(Llama 4 Maverick,400B 参数的专家混合模型),单个 H100 GPU 在 4 位量化权重下,使用预填充中的整数算术,生成证明的速度约为 110 个 token/秒,每 1,024 个 token 请求的证明大小约为 19 MB。这大约是模型本身在 vLLM(一个常见的开源库)上运行所需 GPU 时间的 70-95 倍。
1. For a large model (Llama 4 Maverick, 400B-parameter mixture of experts), a single H100 GPU generates proofs at around 110 tokens/second, with a proof size of ~19 MB per 1,024-token request, when in 4-bit quantized weights, using integer arithmetic in prefill. This is ~70-95x the GPU time of running the model itself on vLLM, a common open-source library.
2. 先前的开源系统要慢几个数量级,正如我们在第 6 节中讨论的那样。Claude 找到了新颖的方法来应用密码学和安全文献中的最先进工具,同时依赖于密码学证明系统的成熟设计原则。对模型的承诺以及随后的推理证明隐藏了模型的权重;隐藏模型的架构不是本系统的目标。
2. Prior open-source systems are orders of magnitude slower, as we discuss in section 6. Claude found novel ways to apply state-of-the-art tools in the cryptography and security literature, while relying on well-established design principles for cryptographic proof systems. The commitment to the model, and subsequent inference proofs, hide the weights of the model; hiding the architecture of the model is not a goal of this system.
2. Claude 擅长优化,但常常做出难以评估的决策,尤其是在大型复杂系统中。我们通过在 Lean(一种形式化验证语言)中为所声称的安全保证提供顶层可审计的陈述来应对这一问题。
2. Claude is good at optimization, but often makes decisions that are hard to evaluate, especially in large complex systems. We counter this by providing top-level auditable statements in Lean (a formal verification language) for the claimed security guarantees.
* 当前的博客文章,概述了我们的系统及其潜在应用;
* The current blog post, which outlines our system and its potential applications;
* 一份解释我们系统细节的技术报告;
* A technical report explaining the details of our system;
* 一份(由 Claude 撰写的)协议规范文档;
* A (Claude-written) protocol specification document;
* 可执行代码,包括:一个证明器和两个验证器,一个用 C++ 编写的快速验证器,以及一个在 Lean 中可证明正确的验证器;以及
* Executable code that includes: a prover and two verifiers, a fast verifier in C++, and a provably correct verifier in Lean; and
* 针对顶层数学陈述的 Lean 证明,以及一个用于辅助审计的 Lean 定理交互式可视化工具。
* Lean proofs for the top-level mathematical statements, along with an interactive visualizer of the Lean theorems to aid with auditing.
我们以 Apache 2.0 许可证发布我们的代码、Lean 证明和文档。1 本研究文章旨在分享阶段性进展,而非一份已完成的研究议程。我们明确分享这些内容,目的是让他人审视我们的工作、采纳并加以改进。
We release our code, Lean proofs, and documentation under the Apache 2.0 license.1 This research post is meant to share intermediate progress, and not a completed research agenda. We are sharing this explicitly with the goal that others look at what we have done, take it, and improve it.
当用户向 AI 模型发送请求时,如今他们无法确定所选模型就是生成响应的那个模型。对于日常使用,用户主要关心输出,而不一定关心是哪个具体模型产生的,因此这个问题并不紧迫。但在几种情况下,这种保证会很有用。例如,在发布前后测试 AI 模型的独立评估者需要知道他们评估的模型正是所服务的模型。通过第三方路由服务使用模型的客户希望得到保证,令牌来自他们付费的模型。而如果网络攻击者冒充给定模型修改令牌流,用户仅从输出无法检测到。
When a user sends a request to an AI model, today they have no way of knowing for certain that the model they selected is the one that generated the response. For everyday uses, in which users mostly care about the output and not necessarily which specific model produced it, this is a less pressing concern. But there are several cases where such assurance would be useful. For example, independent evaluators who test AI models before and after release need to know that the model they are evaluating is the model being served. Clients who use a model through a third-party routing service want assurance that the tokens came from the model they paid for. And if a cyberattacker modifies the token stream while pretending to be a given model, a user has no way to detect it from the output alone.
然而,实现模型响应的验证具有挑战性。大多数前沿模型的权重是私有的,即使用户能够访问权重,检查给定响应是否由具有这些权重的模型产生也需要运行模型,这所需的计算资源远超典型用户(甚至评估者)所能承受的范围。
However, enabling verification of model responses is challenging. Most frontier models’ weights are kept private, and even if the user had access to the weights, checking that a given response was produced by a model with those weights would involve running the model, which requires computational resources far beyond what a typical user (or even evaluator) can afford.
_可证明推理_(PI)的目标是使计算资源有限且对权重一无所知的用户能够验证响应来自所请求的模型。PI 的动机既包括增强客户对 AI 提供商的信任,也因为它可以向独立评估者提供保证,并允许提供商证明他们满足某些承诺和法律义务,所以它对控制 AI 发展前沿和 AI 监管也有影响。
The goal of _provable inference_ (PI) is to enable a user with limited computational resources and without any information about the weights to verify that a response came from the requested model. PI is motivated both by a desire to increase customer trust in AI providers, and—because it can be used to provide assurances to independent evaluators and allows providers to demonstrate that they are meeting certain commitments and legal obligations—it also has implications for pacing the frontier of AI development and for AI regulation.
实现 PI 的一种方法是通过一种称为_零知识简洁论证_的密码学对象:模型提供商在声称的响应之外发送一个简短的_证明_(例如几兆字节),用户可以_快速_验证(理想情况下在 CPU 上几秒钟内)。如果声称的响应偏离指定的模型,用户的检查将失败。重要的是,密码技术确保证明_不_泄露关于模型权重的_任何_信息。
One way to implement PI is via a cryptographic object called a _zero-knowledge succinct argument:_ the model provider sends, along with the claimed response, a short _proof_ (say, a few megabytes) that the user can _quickly_ verify (ideally, in seconds on a CPU). If the claimed response deviates from the specified model, the user’s check will fail. Importantly, cryptographic techniques ensure that the proof reveals _nothing_ about the model’s weights.
今天,我们发布 FlashProve:一个**开源的、高性能的可证明推理原型**。对于一个 400B 参数的混合专家模型(Llama 4 Maverick,4 位量化),我们展示单个 H100 GPU 以超过 100 令牌/秒的速度生成证明,每个 1,024 令牌请求的证明大小约为 19 MB(约为常见开源库 vLLM 处理相同长度提示所需 GPU 时间的 70-95 倍)。
Today, we release FlashProve: an **open-source performant prototype for provable inference**. For a 400B-parameter mixture-of-experts model (Llama 4 Maverick, 4-bit quantization), we show that a single H100 GPU generates proofs at over 100 tokens/second, with proof size of ~19 MB per 1,024-token request (~70-95 times the GPU time that vLLM, a common open-source library, takes to process prompts of the same length).
本次发布包含可执行代码、Lean 形式化以及协议的技术描述。我们的目标有两方面:(i) 加速并促进可证明推理的发展;(ii) 证明零知识简洁论证对于可证明推理是实用的。
The release comes with executable code, Lean formalization, and a technical description of the protocol. Our aim is twofold: (i) accelerating and facilitating the development of provable inference; and (ii) demonstrating that zero-knowledge succinct arguments are practical for provable inference.
我们相信这一进展对安全具有重要影响,包括**把控 AI 发展节奏**、**为 AI 客户提供完整性保证**、**为审计人员和监管机构提供工具支持**等。
We believe that this progress has meaningful implications for security, in **pacing AI development**, in **providing AI customers with integrity guarantees**, in **facilitating tooling for auditors and regulators**, and more.
人工智能的发展及其在我们的数字基础设施与经济中日益重要的作用,带来了 AI 提供方必须回应的信任挑战。借助**可证明推理(provable inference, PI)**工具,有助于应对这些及其他挑战。
The development of artificial intelligence and its growing role in our digital infrastructure and economy raises trust challenges that AI providers must answer. Addressing these and other challenges is facilitated by relying on tools for _provable inference (PI)._
PI 工具包括硬件方法、软件方法,或两者的结合。在此,我们聚焦于纯软件方法:一种称为“zkSNARK”(**零知识简洁知识论证**)的密码学工具。通俗地说,zkSNARK 是一个系统,允许**证明者**通过一份简短的“证书”使**验证者**相信某个给定计算被正确执行;关键在于,该证书是**零知识**的,即除计算正确性所隐含的信息外不泄露任何信息,并且是**简洁**的,即与所证明的计算相比,验证速度快指数级。
Tools for PI include hardware approaches, software approaches, or a combination of both. Here, we focus on a software-only approach: a cryptographic tool known as a "zkSNARK" (_zero-knowledge succinct argument of knowledge_). Informally, a zkSNARK is a system allowing a _prover_ to convince a _verifier_, by way of a short "certificate", that a given computation was executed correctly; crucially, the certificate is _zero-knowledge_ in that it reveals no information beyond what is implied by the computation's correctness and is _succinct_ in that it's exponentially faster to verify compared to the proved computation.
下面我们概述若干挑战,以及 zkSNARK 如何帮助应对这些挑战。
Below we outline several challenges and how zkSNARKs help in addressing them.
一些监管和调节 AI 发展节奏的提案依赖于监控硬件及其使用方式。例如,可能要求提供商将其硬件专用于推理,而不是训练强大的新模型。提供商可能试图通过提供更便宜的模型或以意想不到的方式使用算力来作弊,从而释放资源用于训练。监管机构如何可靠地确保硬件用于推理?
Some proposals to regulate and pace AI development rely on monitoring hardware and how it is used. For example, a provider might be required to devote its hardware to inference rather than to training a powerful new model. The provider might try to cheat by serving cheaper models or using compute in an unexpected way, freeing up resources for training. How can a regulator reliably ensure that the hardware is used for inference?
**zkSNARKs 如何提供帮助:** 监管机构可以要求提供商提供涵盖给定时间段内所服务的推理请求的证书。证书必须表明这些请求是由所声称的模型生成的。恶意提供商无法满足此要求,除非其硬件确实在服务正确的模型。使用 zkSNARK,监管机构可以验证提供商的声明是否有效,而无需提供商可能拥有的复杂硬件设置,同时提供商也得到保证,监管机构不会获知专有模型信息。我们注意到,尽管这验证了硬件正在服务推理,但它并不能确定硬件没有(例如)在进行旁路训练。
_How zkSNARKs can help:_ The regulator can ask the provider for certificates covering the inference requests served in a given period. The certificates must show that the requests were generated by the claimed model. A malicious provider cannot satisfy this request unless its hardware was genuinely serving the right model. Using a zkSNARK, the regulator can check that the provider's claims are valid without needing the elaborate hardware setup that the provider may have, and at the same time the provider has the guarantee that the regulator does not learn proprietary model information. We note that although this verifies that the hardware is serving inference, it does not establish that the hardware is not, e.g., training on the side.
随着 AI 模型变得更有价值,运行它们的服务器成为主要目标。攻击者或恶意智能体可能入侵推理服务器,以窃取权重、向用户提供恶意响应(例如向用户计算机传递恶意软件)等,同时隐藏在真实响应中。对手不一定是外部的。随着模型能够自我改进和部署,一个更先进的模型可能取代被授权的模型运行,并伪装成较弱的模型,以避免适用于它的分类器和监控。在这两种情况下,问题是一样的:公司如何确保其服务器上实际运行的是_哪些权重_?在我们的前沿安全路线图中,Anthropic 设定了一个公开目标,即制作一个验证推理流量的原型:“_我们将在 2026 年 9 月 30 日前开发出一个可证明推理的原型,这是一种可靠、可证明地‘签名’AI 模型输出的技术,使其可归因于特定的一组模型权重。_”正如我们在 RSP 更新中所述,我们通过一个独立但相关的项目实现了该目标,该项目对我们的推理流量进行签名,并在受信任的服务器上重新计算推理;本博客文章中的工作通过不同的方法追求相同的目标。3
As AI models grow more valuable, the servers that run them become prime targets. An attacker or rogue agent may compromise an inference server to exfiltrate the weights, serve users malicious responses (such as delivering malware to the user’s computer), and more, all while hiding among genuine responses. The adversary need not be external. As models become capable of improving and deploying themselves, a more advanced model could run in place of the one that was authorized and present itself as the weaker model, so as to avoid the classifiers and monitoring that would apply to it. In both cases the question is the same: how can a company be sure of _which weights_ are actually running on its servers? In our Frontier Safety Roadmap, Anthropic set a public goal to produce a prototype for verifying inference traffic: “_We will develop a prototype by September 30, 2026 of provable inference, a technique for reliably, provably ‘signing’ AI model outputs in a way that makes them attributable to a specific set of model weights._” As stated in our RSP updates, we met that goal through a separate but related project, which signs our inference traffic and recomputes inference on a trusted server; the work in this blog post pursues the same objective by a different method.3
_zkSNARK 如何提供帮助:_ 提供商可以要求服务器对每个响应提供有效的 zkSNARK。证明绑定到权重的特定承诺,因此它不仅证明_某个_模型生成了响应,而且证明是_所承诺的_模型生成的。入侵服务器的攻击者,或替换自身为另一个的模型,必须为预期权重生成有效证明以避免检测,而这是他们无法做到的。用户也可以接收证明,因为证明不泄露内部信息(通过零知识)且检查成本低(通过简洁性);这限制了攻击者通过推理 API 所能实现的目标。
_How zkSNARKs can help:_ The provider can require a valid zkSNARK from the server on every response. A proof is bound to a specific commitment to the weights, so it certifies not just that _some_ model produced the response but that the _committed_ model did. An attacker who compromises a server, or a model that substitutes itself for another, must then produce valid proofs for the expected weights to avoid detection, which they cannot do. The user may receive the proofs as well, since they leak no internal information (by zero knowledge) and are cheap to check (by succinctness); this limits what an attacker can accomplish through the inference API.
人们可能希望证明在理解/分析某事物上投入了相当多的精力。例如,开发者可能声称其软件库已由前沿网络安全模型扫描过漏洞,或者公司可能声称其商业计划已获得适当模型的批准。用户如何验证这些声明?用户可能没有资源自行运行模型(或者可能没有权限这样做)。即使公开聊天记录也不能提供绝对保证;开发者理论上可以伪造记录,不是用前沿模型而是用更便宜的模型或降级版本生成。一般来说,用户如何验证他们正在由所选模型提供服务?
One may want to prove that considerable effort went into understanding/analyzing something. For example, a developer might claim that their software library was scanned for vulnerabilities by a frontier cybersecurity model, or a company might claim that its business plan was given the go-ahead by an appropriate model. How can a user verify these claims? The user might not have the resources to run the model itself (or they might not have the clearance to do so). Even publishing the chat logs gives no absolute guarantees; a developer might in theory fake them, generating them not with a frontier model but instead with a cheaper one or a degraded version. In general, how can users verify that they are being served by their chosen model?
_zkSNARKs 如何提供帮助:_ 提供者可以在每个响应中附加一个 zkSNARK,证明输出是由所声称的模型在给定输入上产生的。简洁性让用户无需专用硬件即可快速检查此证书,而零知识性向提供者保证用户无法获知专有权重。这补充了现有的验证哪个模型产生响应的机制。
_How zkSNARKs can help:_ The provider can append, along with each response, a zkSNARK proving that the output was produced by the claimed model on the given input. Succinctness lets the user check this certificate quickly, without specialized hardware, and zero-knowledge assures the provider that the user learns nothing about the proprietary weights. This complements existing mechanisms for verifying which model produced a response.
例如,在漏洞利用频繁被发现的环境中,模型提供者可以在发布新代码之前向代码开发者提供一份失败证明(Proof-of-Failure),证明模型提供者在特定模型上花费了至少 X 个 token,并且未达到特定结果(例如,该特定模型在尝试至少 100 万 token 后,未在特定补丁中发现任何漏洞)。
For example, in a world where exploits are being found regularly, prior to releasing new code a model provider could provide the code-developer a Proof-of-Failure that the model-provider spent at least X tokens on a specific model, and did not achieve a specific result (for example, this specific model did not find any bugs in a specific patch, after trying for at least 1M tokens).
监管可能要求提供商仅提供经批准为安全的模型,而非被视为网络安全或安全风险的模型。提供商可能有动机规避这一要求,因为高风险模型可能表现优于竞争对手,或提供服务的成本更低。监管者如何确保仅提供经批准的模型?
Regulation might require providers to serve only models approved as safe, rather than models deemed a cybersecurity or safety risk. A provider might have an incentive to circumvent this, since risky models might outperform competitors or be cheaper to serve. How can a regulator ensure that only approved models are served?
_zkSNARK 如何提供帮助:_ 监管者可要求提供商在其响应中附加 zkSNARK,以强制使用经批准的模型。该证明将每个请求与所使用的模型绑定,因此若提供商使用未经批准的模型,证明将无法通过验证。得益于简洁性,监管者可以轻量级方式检查这一点,而无需重新执行每个请求。
_How zkSNARKs can help:_ The regulator can require the provider to attach a zkSNARK to its responses, enforcing the use of approved models. The proof binds each request to the used model, so it will not verify if the provider is using an unapproved model. The regulator can check this in a lightweight way thanks to succinctness without requiring to re-execute each request.
_关于来自可信硬件的 PI 的说明。_ 先前工作常聚焦于_可信执行环境_来实现 PI。这些是基于硬件的解决方案,提供隐私与证明,依赖于制造商生产防篡改硬件。在本研究中,我们聚焦于纯软件的密码学解决方案,其安全性与隐私依赖于计算问题的困难性。即使面对被攻陷的环境(硬件、操作系统等),对手也无法在不破解计算上不可行的问题的情况下生成无效证明。可信硬件与密码学是互补的方法,具有不同的保证与权衡。两者都将在应对上述信任挑战中发挥作用。
_A note on PI from trusted hardware._ Prior work often focuses on _trusted execution environments_ to achieve PI. These are hardware-based solutions that provide privacy and attestation that rely on the manufacturer producing tamper-resistant hardware. In this research, our focus is on software-only cryptographic solutions whose security and privacy relies on the hardness of computational problems. Even in the face of compromised environments (hardware, operating system and more) an adversary cannot produce an invalid proof without breaking a computationally infeasible problem. Trusted hardware and cryptography are complementary approaches with different guarantees and tradeoffs. Both will play a role in addressing the above trust challenges.
FlashProve 是一个用于大型混合专家(MoE)语言模型可证明推理的系统。更多细节见我们的技术报告;这里我们概述该系统。
FlashProve is a system for provable inference of large Mixture-of-Experts (MoE) language models. More details are in our technical report; here we outline the system.
FlashProve 涉及两方:模型提供方(_证明者_)和用户(_验证者_)。它分为两个阶段:
FlashProve involves two parties: the model provider (the _prover_) and a user (the _verifier_). It is split into two phases:
* 在**离线阶段**,证明者发布对模型权重的_承诺_,这是一个简短的指纹,用于确定_一个_确切的权重选择,而权重本身保持隐藏。后续的推理声明将根据该承诺进行检查,即在此阶段承诺的权重模型。
* In the **offline phase**, the prover publishes a _commitment_ to the model weights, which is a short fingerprint to pin down _one_ exact choice of weights while the weights themselves remain hidden. The later inference claims will be checked against that commitment, i.e., the weight model committed to in this phase.
* 在**在线阶段**,根据验证者的请求(即提示和相应的输出词元),证明者生成一个证明文件,证明以下陈述:
* In the **online phase**, upon requests from the verifier (i.e., prompts and the corresponding output tokens), the prover produces a proof file proving the following statement:
_所承诺的模型在评估时产生这些输出词元。_
_The committed model produces these output tokens when evaluated._
任何持有承诺、提示词和回复的人,无需模型权重,且只需有限算力,即可对其进行验证,并确信所承诺的权重在该提示词上运行后恰好产生这些 token。
Anyone who holds the commitment, the prompt, and the reply can check it, without the weights and with limited compute power, and be convinced that the committed weights, run on this prompt, produce exactly these tokens.
一个证明文件满足以下“完备性”和“可靠性”性质:
A proof file satisfies the following “completeness” and “soundness” properties:
* **如果证明文件是诚实的,则它被接受(完备性)。** 诚实验证者的证明会被验证者接受。
* **A proof file is accepted if it is honest (completeness).** An honest prover's proof is accepted by the verifier.
* **证明文件不可伪造(可靠性)。** 实际上不可能(即等同于破解广泛相信的密码学假设)为错误陈述生成有效证明,因此验证者不必信任证明者。
* **A proof file cannot be forged (soundness).** It is effectively impossible (i.e., tantamount to breaking widely believed cryptographic assumptions) to produce a valid proof of a false statement, so the verifier does not have to trust the prover.
仅完备性和可靠性可以通过一个知道模型权重的计算能力强大的验证者来实现(在提示上评估模型并检查结果是否与响应匹配)。接下来的两个性质才是使系统非平凡的原因。
Completeness and soundness alone can be achieved by a computationally powerful verifier that knows the model weights (evaluate the model on the prompt and check whether the result matches the response). The next two properties are what makes the system non-trivial.
* **证明文件除陈述外不泄露任何信息(零知识)。** 验证者_不会_获知关于权重、计算的中间值或每一层运行了哪些专家的任何信息。模型架构(不包括权重)不被隐藏。
* **A proof file reveals nothing beyond the statement (zero-knowledge).** The verifier learns _nothing_ about the weights, the intermediate values of the computation, or which experts ran at every layer. The model architecture (excluding the weights) is not hidden.
* **证明文件小且校验成本低(简洁性)。** 被证明的计算是在一块 H100 GPU 上运行一个 400B 参数(量化版本约 232 GB)的模型,而验证者收到的证明文件约为 19 MB,在 24 核 CPU 上只需数秒即可完成校验。审计者或硬件条件远逊于提供方的用户,无需访问复杂的基础设施即可校验证明文件。
* **A proof file is small and cheap to check (succinctness).** The attested computation is a run of a 400B-parameter (about 232 GB for a quantized version) model on an H100 GPU, while the verifier receives a proof file of about 19 MB that takes seconds to check on a 24-core CPU. An auditor, or a user with far less hardware than the provider, can check proof files without access to elaborate infrastructure.
我们在 Llama 4 Maverick(一个 400B 参数的专家混合模型,每个 token 使用约 17B 参数)上使用公开可用的 4 位权重测量了协议的一个实现。每个验证器请求是一个 1,024 个 token 的窗口,每个证明检查一个 token。证明器运行在配备单个 H100 和 24 个 CPU 核心的服务器上,验证器运行在 CPU 上。
We measured an implementation of the protocol on Llama 4 Maverick (a 400B-parameter mixture of experts, with ~17B parameters used per token), using the publicly available 4-bit weights. Each verifier request is a window of 1,024 tokens, and each proof checks one token. The prover runs on a server with a single H100 and 24 CPU cores, and the verifier on a CPU.
* **证明器。** 证明器运行在配备单个 H100 和 24 个 CPU 核心的服务器上。_离线阶段_ 对每个模型运行一次,耗时约 39 分钟。(由于我们在技术报告中讨论的技术原因,生成的承诺大约可用于 1600 万个请求,之后提供者必须重新承诺。)_在线阶段_,对于一批 24 个请求,证明器大约需要 223 秒,即每秒 110 个 token,每个证明为 18.78 MB。
* **Prover.** The prover runs on a server with a single H100 and 24 CPU cores. The _offline phase_ runs once per model and takes about 39 minutes. (For technical reasons we discuss in the technical report, the resulting commitment is good for about 16 million requests, after which the provider has to commit again.) The _online phase_, on a batch of 24 requests, takes the prover about 223 seconds, which is 110 tokens per second, and each proof is 18.78 MB.
* **验证器。** 验证器运行在客户端的 CPU 上。它不需要 GPU,并且只能访问权重的承诺。客户端下载承诺(总共约 6.5 MB),然后每个请求下载一个证明。检查一个证明在 24 个线程上需要 17 秒(在单线程上需要 21 秒),包括离线阶段的检查。
* **Verifier.** The verifier runs on the client's CPU. It needs no GPU and only has access to a commitment to the weights. The client downloads the commitment, ~6.5 MB in all, and then one proof per request. Checking a proof takes 17 seconds on 24 threads (and 21 seconds on a single thread), including the offline phase's checks.
与普通推理的比较:我们在八块 H100 上使用广泛使用的推理引擎 vLLM 运行了同一模型的 FP8 版本,发现每块 GPU 上,证明一批 24 个请求比 vLLM 一次处理相同的 24 个请求所需时间长约 72 倍(多次运行在 66 到 77 倍之间),而当 vLLM 在我们尝试的最高负载下运行时,长约 94 倍。
Comparison with plain inference: We ran vLLM, a widely used inference engine, with the FP8 release of the same model on eight H100s, and found that, per GPU, proving a batch of 24 requests takes about 72 times longer than vLLM takes to process the same 24 requests at once (66 to 77 across runs), and about 94 times longer when vLLM runs at the highest load we tried.
**系统内部。** FlashProve 背后的底层机制是一种 zkSNARK,类似于部署在点对点系统(例如,以太坊等区块链的 Starknet 或 Mantle 等 L2 网络)中的那种。我们的 zkSNARK 按照一套成熟的方法构建:一个称为 _交互式预言机证明_(IOP)的底层信息论对象通过依赖哈希函数被编译成密码学证明(并且甚至提供后量子安全性)。我们的 IOP 由最先进的构建模块组装而成,使用了诸如 _带线性约束的代码邻近性测试_ 和 _查找论证_ 等工具,这些工具基于编码理论和复杂性理论的技术。将这些部分组合成一个为推理定制的 IOP 是一项精细的任务;关于构造的更多细节可以在技术报告中找到。
**Inside the system.** The underlying mechanism behind FlashProve is a zkSNARK of the kind deployed in peer-to-peer systems (e.g., L2 networks such as Starknet or Mantle for blockchains like Ethereum). Our zkSNARK is built according to a well-established recipe: an underlying information-theoretic object called an _interactive oracle proof_ (IOP) is compiled into a cryptographic proof by relying on hash functions (and offers even post-quantum security). Our IOP is assembled from state-of-the-art building blocks using tools such as _code proximity tests with linear constraints_ and _lookup arguments_, which are based on techniques from coding theory and complexity theory. Combining these pieces into a bespoke IOP for inference is a delicate task; more details about the construction can be found in the technical report.
FlashProve 受益于我们所做的架构决策,这些决策在技术报告中有详细讨论。这里我们重点介绍几点:
FlashProve benefits from architectural decisions that we made, and which are discussed in detail in the technical report. Here we highlight a few:
* 证明器的工作量与实际运行的专家成正比。在 MoE 模型中,一个 token 使用 400B 参数中的约 17B,而证明器每个 token 的工作量与这约 17B 参数成正比,而不是与所有参数成正比(证明仍然隐藏了选择了哪些专家)。
* The prover's work is proportional to the experts that actually ran. In a MoE model a token uses ~17B of the 400B parameters, and the prover's work per token is proportional to those ~17B instead of all parameters (the proof still hides which experts were chosen).
* 请求在事后批量进行证明。证明器以一次并行传递(如同预填充)读取已完成的文本,并且每批只打开一次权重承诺,而不是每个请求打开一次。
* Requests are proved in batches, after the fact. The prover reads finished text in one parallel pass (as in prefill), and it opens the commitment to the weights once per batch instead of once per request.
* 系统设计为适合单个 GPU。每一层的模型权重被分割成段,并一次证明一段层。权重保存在普通内存中并流式传入,因此单个 80 GB GPU 加上约 290 GB 的主机内存就足够了。
* The system is designed to fit on a single GPU. The model weights at each layer are split to segments and are proven one segment of layers at a time. The weights are kept in ordinary memory and streamed in, so a single 80 GB GPU, with ~290 GB of host memory beside it, suffices.
在将证明机制集成到推理服务器时,存在一个选择:在线证明(in-line proving,即与推理同步进行证明)与事后证明(post-hoc proving,即事后针对一部分 token 进行证明)。我们的系统采用后者。在线方案预期速度极快,并可能为效率而牺牲安全性;而事后方案允许较慢,且可能不必承受这种权衡。另一方面,事后证明也有局限:重计算带来的额外复杂度、获得关于流量的保证存在延迟,以及固有的“非确定性”。我们在技术报告中详细阐述事后证明的动机与影响。
When integrating proving mechanisms into inference servers there is a choice: in-line proving (i.e., proving alongside the inference) versus post-hoc proving (i.e., proving after-the-fact, for a subset of tokens). Our system follows the latter. In-line solutions are expected to be ultra-fast and may sacrifice security for efficiency, whereas post-hoc solutions are allowed to be slower and may not suffer such a tradeoff. On the other hand, post-hoc proving has limitations: added complexity of recomputation, the delay in getting guarantees about your traffic, and inherent "non-determinism". We elaborate on the motivation and implications of post-hoc proving in the technical report.
密码学代码以脆弱和易错著称。用户如何能信任我们今天发布的代码?
Cryptographic code is notoriously brittle and error-prone. How can users trust the code we're releasing today?
答案是**形式化验证**:我们使用 Lean 证明助手来规约协议以及期望的安全性质。我们以机器可检验且保证正确(在编译器无误的前提下)的形式证明这些性质。我们针对三个顶层性质:
The answer is **formal verification**: we use the Lean proof assistant to specify the protocol and the desired security properties. We prove these properties in a form that is machine-checkable and guaranteed to be correct (up to compiler errors). We target three top-level properties:
1. **完备性**:若陈述正确,则生成的证明会被验证者接受。
1. **Completeness**: if the statement is correct, the resulting proof is accepted by the verifier.
2. **可靠性**:任何伪造的证明都会被验证者拒绝(除可忽略概率外)。
2. **Soundness**: any fake proof is rejected by the verifier (except with negligible probability).
3. **零知识**:证明不会泄露模型权重的任何信息,超出验证者从响应本身所能推断的范围。
3. **Zero knowledge**: the proof reveals no information about the weights of the model, beyond what the verifier can infer from the response itself.
这些性质是在所谓的随机预言机模型中陈述的,这是一种基于信息论的方法,用于分析依赖哈希函数的密码协议。
These properties are stated in the so-called random oracle model, an information-theoretic way to analyze cryptographic protocols that rely on hash functions.
形式化验证留下了两个重要的信任缺口。首先,必须审计形式化陈述以确保它们符合意图。这必须手动完成(尽管这比审计证明要轻松得多);我们提供了一个交互式可视化工具来辅助这一过程。
Formal verification leaves two important trust gaps. First, the formal statements must be audited to ensure that they match their intent. This has to be done manually (though it is dramatically less onerous than auditing proofs); we provide an interactive visualizer to assist with this.
其次,必须审计可执行代码以确保其与所证明安全性质的形式化定义协议一致。如何最好地辅助这一点取决于性质,因为可靠性仅依赖于验证者,零知识性仅依赖于证明者,而完备性依赖于它们的交互。
Second, the executable code must be audited to ensure it matches the formally-defined protocol for which the security properties are proven. How to best assist with this depends on the property, because soundness depends solely on the verifier, zero-knowledge depends solely on the prover, and completeness depends on their interaction.
* 对于可靠性,我们只需要验证者代码是正确的(拥有“证明”的全部价值在于:如果你信任验证者,你就可以完全不必信任证明者)。我们提供了一个可执行的 Lean 验证者,以及 C++ 验证者。Lean 验证者运行较慢,但 Lean 证明了它只接受我们的可靠性定理中验证者所接受的内容。尽管如此,日常使用中应使用 C++ 验证者:它不携带形式化保证,但在我们测试的每个证明上都与 Lean 验证者一致。
* For soundness, we only need the verifier code to be correct (the entire value of having “proofs” is that you can fully distrust the prover if you trust the verifier). We provide an executable Lean verifier, alongside the C++ verifier. The Lean verifier runs more slowly, but Lean proves that it accepts only what the verifier in our soundness theorem accepts. Nevertheless, the C++ verifier is the one to use day to day: it carries no formal guarantee, and it agrees with the Lean verifier on every proof in our tests.
* 对于完备性,威胁模型不假设对抗性攻击(证明者和验证者是诚实的)。因此,Lean 验证者让我们通过标准软件测试获得信心,即诚实的证明者会产生可接受的证明:在正确的陈述上运行证明者,然后在所得证明上运行 Lean 验证者。
* For completeness, the threat model does not assume adversarial attacks (the prover and verifier are honest). Therefore, a Lean verifier lets us gain confidence through standard software testing that the honest prover produces accepting proofs: run the prover on a correct statement, and then the Lean verifier on the resulting proof.
* 对于零知识而言,我们只需要证明者(prover)是正确的。然而,与验证者(verifier)不同,证明者是一个开销高昂的过程,运行 Lean 证明器并不现实。我们在此不提供便利措施:我们期望,任何希望隐藏其所服务模型权重的证明者,都拥有足够的资源来进行细致的智能体式和/或人工安全审计,以确保与 Lean 证明器等价。
* For zero knowledge, we only need the prover to be correct. However, unlike the verifier, the prover is an expensive procedure, and it would be impractical to run a Lean prover. We do not provide facilitations here: we expect that any prover that wants to hide the weights of a model they are serving has access to enough resources for careful agentic and/or human security audits to ensure equivalence to the Lean prover.
实现可证明推理是一系列工作的目标,这些工作被通俗地称为“zkML”。早期工作,如 SafetyNets(2017)验证了一个两层卷积的 MNIST CNN 和一个小型全连接语音模型,zkCNN(2021)在每次证明 88.3 秒内达到了 VGG-16(1500 万参数),而 Kang 等人(2022)在全分辨率 ImageNet 上证明了 MobileNet v2。
Achieving provable inference is the objective of a line of work colloquially termed “zkML”. Early works, such as SafetyNets (2017) verified a two-conv-layer MNIST CNN and a small fully connected speech model, zkCNN (2021) reached VGG-16 (15M parameters) at 88.3 s per proof, and Kang et al. (2022) proved MobileNet v2 on full-resolution ImageNet.
随后出现了 Transformer 的证明:ZKML 的 GPT-2 对于 1.17 亿参数每个 token 耗时约 159 秒(≈0.006 tok/s),而 Modulus Labs 的 15 亿参数 GPT-2 XL 需要超过 90 小时。EZKL(2023)证明了一个 100 万参数的 GPT 风格 transformer,每个 token 约 966 秒(≈0.001 tok/s)。性能逐步提升:zkLLM(2024)在 GPU 上约 15 分钟内证明了 2048 个 token 输入的 LLaMA-2-13B,即每个新 token 约 0.001 tok/s,或批处理中约 2.5 tok/s。2025 年,zkGPT 将 GPT-2 降至 21.8 秒(≈0.05 tok/s),zkPyTorch 在单个 CPU 核心上以每个 token 约 150 秒(≈0.007 tok/s)证明了 Llama-3 8B,而 ZKTorch 在约 20 分钟内证明了 GPT-J(6B)(≈0.001 tok/s)。VerInf 证明了 Llama 4 Maverick(与我们证明的相同的 400B 参数 MoE 模型)达到约 0.02 tok/s。DeepProve 是当前同行评审的最先进技术,通过单次前向传播认证整个生成的序列。它在一台机器上对 GPT-2 达到约 2.9 tok/s,对 Gemma 3 达到 1.4 tok/s,分布式达到 31 tok/s。SASH 的证明器(2026)在单个 H100 上对 Llama 3.1 8B 达到约 1.25 tok/s。
Transformer proofs followed: ZKML’s GPT-2 took about 159 s per token for 117M parameters (≈0.006 tok/s), and Modulus Labs’ 1.5B-parameter GPT-2 XL needed over 90 hours. EZKL (2023) proved a 1M-parameter GPT-style transformer in about 966 s per token (≈0.001 tok/s). Performance progressively improved: zkLLM (2024) proved LLaMA-2-13B on a 2,048-token input in about 15 minutes on GPU, which is ≈0.001 tok/s per new token, or ~2.5 tok/s in the batch. In 2025, zkGPT reduced GPT-2 to 21.8 s (≈0.05 tok/s), zkPyTorch proved Llama-3 8B at ~150 s per token on one CPU core (≈0.007 tok/s), and ZKTorch proved GPT-J (6B) in about 20 minutes (≈0.001 tok/s). VerInf proved Llama 4 Maverick (the same 400B-parameter MoE model we prove) at ≈0.02 tok/s. DeepProve, a current peer-reviewed state of the art, certifies an entire generated sequence with a single forward pass. It reaches about 2.9 tok/s for GPT-2 and 1.4 tok/s for Gemma 3 on one machine, and 31 tok/s distributed. SASH's prover (2026) reached ≈1.25 tok/s on Llama 3.1 8B on a single H100.
最近商业努力已达到令人印象深刻的数字,Attestable 在单个 H100 上以 53–77 tok/s 证明了 Gemma 4 31B;在撰写本文时,Attestable 尚未发布相应的开源库。
Recently commercial efforts have reached impressive numbers, with Attestable proving Gemma 4 31B at 53–77 tok/s on a single H100; at the time of writing, Attestable has not released a corresponding open source library.
上述数字说明了过去几年可证明推理的急剧进展。它们应该更新我们对零知识证明可能性的先验。
The above numbers speak to the precipitous progress of provable inference in the past years. They should update our priors on what it is possible to prove in zero-knowledge.
* 2024 年,zkLLM 在单个 A100 上以约 2.5 tokens/秒证明了 13B 模型。我们所知的 Llama 4 Maverick 的唯一早期证明 VerInf 报告在 DGX Spark 上 1000 个 token 需要 14.3 小时,证明大小为 93.6 GB,验证耗时近一天。
* In 2024, zkLLM proved a 13B model at about 2.5 tokens/second on one A100. The only earlier proof of Llama 4 Maverick that we know of, VerInf, reports 14.3 hours for 1,000 tokens on a DGX Spark, with a 93.6 GB proof which took almost a day to verify.
* FlashProve 在单块 H100 上以 110 tokens/second 的速度为同一模型生成证明,证明大小为 19 MB,可在数秒内完成验证。Attestable 独立报告称,对于稠密的 31B 模型,其速度为 53–77 tokens/second。
* FlashProve proves the same model at 110 tokens/second on a single H100, with 19 MB proofs that can be verified in seconds. Attestable independently reports 53–77 tokens/second for a dense 31B model.
我们相信,这一进展应当能够迁移到更多模型和架构上,从而产生一个灵活而强大的工具,其相对于原生计算的额外开销约为 2 个数量级(在我们的测量中约为 70 至 95 倍),远低于此前的预期。
We believe that the progress should transfer to more models and architectures, yielding a flexible and powerful tool with overhead over native computation of roughly 2 orders of magnitude (about 70 to 95 times in our measurements), much less than previously expected.
本发布版本是一个研究原型,具有以下技术局限性:
This release is a research prototype, and has the following technical limitations:
* **对快速可执行代码没有形式化保证。** 我们提供了协议的 Lean 实现(包括一个可执行的“慢速”Lean 验证器),以及在 Lean 中证明的安全属性。论文和快速可执行代码均源自 Lean 协议,但我们不保证三者(Lean、论文、代码)彼此完全一致。
* **No formal guarantees on fast executable code.** We provide a Lean implementation of the protocol (including an executable “slow” Lean verifier), along with security properties proved in Lean. The paper and the fast executable code are derived from the Lean protocol, but we do not guarantee that all three (Lean, paper, code) match each other.
* **证明针对的是贪心解码下的整数模型。** 我们证明了已发布权重的精确整数版本,并且证明断言每个被检查的 token 都是该模型的贪心选择。来自浮点实现或采样解码的回复可能在某个位置发生偏离,从而不再与所证明的计算完全匹配。通常预期并经过经验测试,整数版本在一般情况下与真实浮点推理的误差率很小(即使考虑到浮点计算的非确定性),但这并不覆盖对抗性设计的提示。
* **The proof is about an integer model under greedy decoding.** We prove an exact integer version of the published weights, and a proof asserts that each checked token is that model’s greedy choice. A reply from a floating-point implementation, or sampled decoding, could depart at some position and no longer exactly match the computation proved. The integer version is generally expected and empirically tested to be within some small error rate of true floating-point inference generically (even taking into account non-determinism of floating-point computations), though this does not cover adversarially designed prompts.
* **架构隐藏。** 我们的系统假设所服务的模型使用混合专家(MoE)架构,其中每个 transformer 层的前馈块被多个专家子网络和一个 _路由器_ 取代,路由器为每个 token 仅激活一小部分专家,并且我们的系统利用 MoE 结构来提高效率。其他架构(如稠密模型)也被广泛使用,其描述可能包含模型提供者希望保密的智力财产。实际上,还可以设想一个进一步隐藏模型架构的系统:证明者通过隐藏承诺同时承诺权重和架构(即推理的“可执行代码”),而验证者的声明是关于检查该承诺的可执行代码在承诺模型上的执行。
* **Architecture hiding.** Our system assumes the served model uses a Mixture-of-Experts (MoE) architecture, in which each transformer layer’s feed-forward block is replaced by multiple expert sub-networks and a _router_ that activates only a small subset of experts for each token and our system uses the MoE structure for efficiency gains. Other architectures, such as the dense models, are also widely used and its description may contain intellectual-property the model provider would like to keep secret. Indeed, one could also imagine a system that further hides the architecture of the model: the prover commits to both the weights and the architecture (i.e., the “executable code” for inference) via a hiding commitment and the verifier’s claim is about checking the execution of that committed executable on the commitment model.
* **侧信道攻击。** 该协议当前的保证是证明本身不会泄露承诺模型的权重信息。然而,“侧信道攻击”,例如测量生成证明所需的时间,可能潜在地泄露关于权重的信息。我们认为这超出了本发布版本的范围。
* **Side channel attacks.** The current guarantee of the protocol is that the proof itself reveals nothing about the weights of the committed model. However, “side channel attacks”, such as measuring the time it takes to produce a proof, could potentially reveal information about the weights. We consider this out of scope for this release.
* **更长的上下文。** 该系统目前能够对最长 1,024 个 token 的上下文进行推理验证。与聊天场景中累积的上下文相比,这一长度相当短。理想情况下,该上下文长度可以提升至前沿模型的完整上下文窗口(前沿模型的上下文窗口通常为数十万至数百万个 token)。
* **Longer contexts.** The system currently proves inference over contexts of up to 1,024 tokens. This is quite short in comparison to the contexts accumulated in chat settings. Ideally, this context length could be increased to full context windows for frontier models (which regularly have context windows of 100s of thousands to millions of tokens).