The Hacker Sent by Anthropic to Calm the Government’s Nerves About AI Safety

Trump administration officials have spent recent days fretting over the power of Anthropic’s next-generation AI software to potentially wreak havoc on global cybersecurity. For a group of 700 cybersecurity researchers, that startling realization came in March. 

That’s when Anthropic researcher Nicholas Carlini showed how easy it had become to use new models to break into systems. The lanky 35-year-old is a well-respected hacker who’s considered the industry’s “professional skeptic” of AI cybersecurity claims. But lately he had changed his mind.

Early that month, just weeks after getting his hands on Mythos, Carlini offered a stark warning to a standing-room-only crowd of cybersecurity experts at the ornate beaux-arts building that had once housed San Francisco’s Hibernia Bank.  

First he showed them how he had used Anthropic’s AI to find and exploit a critical bug in a piece of web-publishing software called Ghost. Then he demonstrated another in the Linux operating system—one of the most battle-tested pieces of software, which powers billions of devices.