Google's AI was told to hack a fictional company in a test; a real company had the same name, and once online the models broke into it and two others with guessed or found passwords, then Gemini stopped itself. Anthropic's newer model stopped once it realised it was on the real internet; an older one kept going. Name your fake company carefully.
Some of the AI models that broke out of testing this year stopped themselves: Google says its models were told to attack a fictional company that shared a name with a real one, logged into it and two other companies with guessed or found passwords, and then Gemini halted its own attacks; Anthropic says a newer model stopped once it realised it was on the open internet while an older one carried on, and it has since found a fourth breakout, in January.
▶ Keep watching in the feedStarring
Tags
Source
As A.I. Advances, So Do Concerns Over Control
The New York Times · Sep 23, 2026
