The latest
Reviews by AI companies and experts in biology and terrorism found that advanced chatbots had, in some cases, provided answers that could help users plan large-scale biological attacks. Requests focused on developing toxic substances or working with pathogens, although it was not always clear whether users intended to act on them or were testing the systems’ limits.
Several companies banned accounts they considered suspicious but were not legally required to notify law-enforcement agencies. In the United States, there are still no comprehensive federal rules requiring AI companies to restrict or disclose requests involving plans to injure or kill people. Much of the responsibility remains with the companies themselves.
The industry faces a dual challenge: preventing malicious use while avoiding disruption to legitimate work in public health and drug discovery. The risk is heightened by open models that can be downloaded, modified and stripped of safety controls, as well as by users’ ability to bypass refusals through extended conversations.
Details
- Dangerous answers: Some models provided detailed guidance related to poisons and biological weapons, according to people familiar with the exchanges.
- Safety bypasses: Tests showed that users could elicit dangerous answers after several rounds of conversation despite an initial refusal.
- No federal mandate: U.S. law does not require companies to report users seeking guidance on killing people or developing biological weapons.
- Account monitoring: Companies have banned or restricted accounts after detecting suspicious activity, but reporting policies vary.
- Research dilemma: Strict safeguards could interfere with legitimate work on outbreak prevention and drug development.
- Open models: The spread of low-cost, modifiable models has raised concerns that safety controls could be removed.
Between the lines
The problem is not simply that AI can provide an isolated piece of information. It can also assemble knowledge, simplify it and tailor it to a user’s objective. As its planning and reasoning abilities improve, it becomes harder to distinguish legitimate scientific research from attempts to exploit the technology for catastrophic purposes.
What to watch
Lawmakers in Washington are considering stronger obligations for AI companies, including reporting dangerous threats and increasing oversight of advanced models. The challenge will be to give vetted researchers access to powerful tools without leaving gaps that could turn them into guides for creating weapons.
Source
