Anthropic is working on an open source system to verify which AI model presents a given output, as per a blog post published Tuesday. The system, called FlashProve, is built to offer what researchers call “provable inference” — a cryptographic proof, mainly a mathematical assure, that a given output came from a particular model. It’s intended to let users confirm that the answer they obtained really came from the model they think they’re using.
Similar to cryptographic email signatures that let recipients confirm a message really came from its claimed sender, FlashProve might be sturdy against impostors attempting to disguise one model as another, even if the impostor is the lab that constructed and runs the model. In main, the researchers note that the system could be used to prevent an AI company from secretly swapping in a cheaper model to cut costs, or a more expensive model to inflate its performance results.
FlashProve may also be used to verify claims by third parties that a specific tasks was accomplished by using a advanced model. “A developer might claim that their software library was scanned for vulnerabilities by a frontier cybersecurity model, or a company might claim that its business plan was given the go-beforehand by the proper model,” the researchers write. “How can a user confirm those claims?”
To work, the system needs AI labs to publish a “fingerprint” of each model they launch which could later be matched to precise outputs and verified cryptographically. Crucially, the fingerprints don’t disclose a model’s weights, the internal numbers that define how it behaves and which closed-model makers like Anthropic and OpenAI keep private. That means labs could publish the fingerprints without giving up their most valuable asset, allowing the system to work with commercial models, too. The system is also designed to be light-weight, so verification could be performed on a local device.
As a part of the declaration, researchers launched a detailed technical specification and working code for numerous elements of the system, that’s publicly available under an open source license on GitHub. Research in the system is ongoing, and there are no plans to deploy it at Anthropic or some other lab.
“This research research is supposed to share intermediate progress, and not a completed research agenda,” researchers explain in the declaration post. “We are sharing this explicitly with the goal that others look what we’ve done, take it, and improve it.”
