A false intelligence report created by an analyst from the US Pacific Command using an AI chatbot nearly led to a conflict between the US and China this spring. The analyst processed data about the cargo of a Chinese vessel, using an algorithm that combined open sources with classified electronic intelligence data.
As a result, the algorithm produced false information, claiming that the Chinese ship was carrying components for a nuclear program. This information was formatted by the analyst into a standard intelligence report, which was distributed to command.
The Pentagon quickly reacted to the report: aircraft were scrambled, and armed special forces were prepared for deployment and inspection of the vessel. The operation was halted just before it began when officials conducted a more thorough verification of the original sources.