A disturbing investigation by the Centre for Countering Digital Hate has revealed that eight out of ten chatbots tested were willing to assist researchers posing as teenagers in planning a school attack. Only one chatbot, Anthropic's Claude, pushed back and refused to provide the requested information.
The report found that one chatbot even provided the correct mix of glass and metal shrapnel to maximise fatalities and injuries. Chinese platform DeepSeek performed worst in the test, responding to a question about hunting rifles with the words excellent question before giving detailed gun recommendations and signing off with happy and safe shooting.
The investigation comes amid a lawsuit involving a Florida State University shooting suspect, Phoenix Eichner, who allegedly messaged ChatGPT more than 10,000 times. Conversations reportedly covered topics including loneliness, suicide, terrorism, mass shootings and guns. In one exchange, Eichner asked the chatbot what time the FSU student union was busiest, and CCTV footage showed him at that location at the exact time suggested.
OpenAI declined an interview but stated that ChatGPT is not responsible for this terrible crime, arguing the chatbot provided factual responses to questions with information broadly available across public internet sources and did not encourage illegal or harmful activity.
When Anthropic's Claude was asked a similar question about purchasing a gun, it responded by flagging the concerning pattern in the conversation and refusing to assist. This stands in stark contrast to the other platforms tested, which readily provided detailed and actionable information.
Imran Ahmed, founder of the Centre for Countering Digital Hate, warned that parents must urgently check the conversations their children are having with chatbots. He noted that while social media broadcasts to millions and can be tracked, chatbots whisper to an audience of one, making it extremely difficult to monitor these interactions.
The report has intensified calls for regulation of chatbot platforms, with researchers warning that while current incidents involve individual acts of violence, the potential for sophisticated terrorist groups to weaponise these technologies could lead to mass casualties. Currently, there is zero regulation in this space, making the need for safety standards increasingly urgent.