AI-generated report nearly triggered US boarding of Chinese ship

The latest:

US military aircraft were airborne and armed personnel were preparing to board a Chinese ship in the Middle East before officials discovered the intelligence behind the operation had been generated with artificial intelligence and was wrong, four sources told CNN. A chatbot used by a special operations command analyst had misidentified the vessel’s cargo as nuclear weapons components. The operation was called off.

Details:

• The trigger: An intelligence report circulated across the US military this spring, during the war with Iran, asserted that a Chinese ship in the Middle East was carrying components of a nuclear weapons program. Four sources familiar with the episode told CNN the military moved immediately to plan an interception.
• How far it went: Two sources said armed US personnel were preparing to board the ship, and military planes were already in the air, according to one of them and another source familiar with the incident. Officials examined the underlying report only just before the operation was to begin.
• The AI chain: The analyst queried a chatbot about intelligence on the ship’s manifest that originated with US Special Operations Command Pacific in Hawaii. The bot fused open-source material with classified signals intelligence, then the analyst used AI again to package the findings into a standard report and disseminate it.
• The assessment: One source described the report as entirely false and said it “almost started a war.” CNN noted any US operation against a Chinese vessel risked spiraling into armed conflict between the two nuclear-armed powers, and was unable to learn what the misidentified cargo actually was.
• Unanswered questions: It was not established whether the chatbot involved was a commercially available product or a US government tool. US Special Operations Command Pacific and the Pentagon did not respond to CNN’s request for comment on the episode.
• The policy push: Defense Secretary Pete Hegseth released an Artificial Intelligence Acceleration Strategy in January, pledging to unleash experimentation and eliminate bureaucratic barriers. An accompanying memo set a goal of putting AI models directly in the hands of three million civilian and military personnel at all classification levels.
• No common standard: Multiple US officials told CNN the rollout is decentralized, with different parts of government using different tools under different orders and safety standards. There is no single standard for verifying information these systems generate, and reliability varies widely across the military and intelligence community.
• Not isolated: One source said hallucinations of this kind have recurred across the intelligence community since the tools began proliferating. A former senior US official familiar with the systems said internal government tools are largely copies of commercial products with cosmetic changes.
• Targeting risk: Sources said the military is rapidly turning to AI for targeting. One source familiar with current policies said there is no real guidance on how keeping a human in the loop would prevent civilian casualties or fratricide, an area where errors are fatal.
• Inside the workforce: Several sources said AI has increased pressure on analysts to produce and disseminate intelligence faster, opening the door to mistakes. Younger analysts, they said, are native users of these tools and more likely to trust their outputs uncritically.

Background:

US officials argue rapid AI adoption is necessary to speed battlefield decisions on targeting and force movements, warning Washington cannot fall behind China or another adversary that might otherwise stay a step ahead.

Between the lines:

Washington’s AI debate has focused on warnings from Silicon Valley engineers about systems escaping human control. This episode points to a nearer-term failure mode: humans acting decisively on inaccurate machine-generated output. The decentralized rollout, absent verification standards, and the analyst’s double use of AI — once to reach the conclusion, once to format it as a trusted report — describe a chain in which no checkpoint caught the error until aircraft were airborne.

What’s next

Watch for any Pentagon or Special Operations Command response, congressional scrutiny of the Artificial Intelligence Acceleration Strategy, and whether the department imposes uniform verification standards for AI-generated intelligence used in targeting.