Safety protections built into some of the world’s most broadly used artificial intelligence (AI) models can be stripped away with alarming ease, as per a brand new international study.
The research team, led by the University of Waterloo and FAR.AI, a nonprofit AI security research group, fastidiously tested 21 of the most popular open-weight large language models (LLMs) and discovered they could all be tampered with no matter their built-in safeguards. The research team also included members from the Massachusetts Institute of Technology, ETH Zurich and the University of Toronto.
A paper on its work, TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering, was into currently presented at the ACM Conference on Knowledge Discovery and Data Mining in South Korea.
The holes in even the first-class protections recently available increase concerns that open-weight models could be used to wage mass disinformation campaigns, generated sophisticated e-mail scams or generate step-by-step instructions to make hazardous chemicals.
“When the safety guardrails are stripped out of a capable model, it can be used at scale for harm in ways a single person could never manage manually,” mentioned Dr. Sirisha Rambhatla, a professor of management science and engineering at Waterloo.
Open access brings added risk
LLMs are advanced AI systems that may understand and generate human language to carry out tasks such as drafting emails, writing computer code and conversing with users.
Unlike closed proprietary models which include ChatGPT and Gemini, open-weight LLMs are publicly available for download and fine-tuning by everyone from individual software developers to private companies and public organizations like hospitals.
Rambhatla said the “sobering” outcomes of testing by the team—which includes members in Canada, the US and Switzerland—should serve as a wake-up call to worldwide researchers about the requirement to develop stronger security systems.
“The leading open-weight models are often not too a far behind the best closed models,” stated Rambhatla, director of the Critical Machine Learning Lab at Waterloo. “As they develop more powerful, the potential results of someone stripping out their protection features develop with them.”
TamperBench tests real-world attacks
While the study detected significant vulnerabilities, Rambhatla cited that the weaknesses may not be precise to open models. “Open-weight models stay important to AI research and accountability,” Rambhatla said. “This openness is a part of how we make sure the models people use work for everyone.”
To test a cross section of open-weight AI models, the research team first built an open-source tool known as TamperBench, a standardized way to simulate a few one of a different attacks. The hope is that other researchers will now assist refine and improve it.
“The defenses available today don’t yet appear sturdy sufficient to assure that a publicly launched model will stay safe once it is in the hands of anyone who chooses to modify it,” stated Saad Hossain, a researcher in the lab who led the study.
“And as governments increasingly depend on AI in health care, fraud detection, education and other public services, the assessment of models and their procurement must be more rigorous and grounded in evidence.”
