Everyone is navigating real-time AI security, even Google

I recently had the opportunity to sit down with Francis De Souza, COO of Google Cloud, behind the scenes at an event in Los Angeles. Amid the noise around us, De Souza, who speaks in the calm, measured manner of a university professor, offered helpful advice for companies navigating the AI ​​security moment we’re all in, noting that “there will be a transition period, and then I think we’ll get to that better place.”

He wasn’t talking about Google at that moment, but apparently even Google is still figuring things out.

De Souza’s key message was one that security professionals have been trying to convince executives for years, and which has now become urgent thanks to AI: security cannot be an afterthought. “As companies embark on this AI journey, they need to take a platform approach,” he said. “Security is not something you can take advantage of later, and it is not something you can leave to employees to do on their own.” He specifically warned against “shadow AI” – where employees turn to consumer tools without regulatory oversight – and said companies need to demand security, governance and auditability from their platforms from the start. “There is no such thing as an AI strategy without a data strategy and a security strategy. They must go hand in hand.”

It’s worth noting that he wasn’t promoting Google Cloud alone. When I noticed that his advice sounded like a Google ad, he dismissed it. He said Google is committed to a multi-cloud approach, and explained that companies that think they are running on a single cloud are almost certainly not. “Even if they choose one cloud, relying on SaaS applications, there are business partners who may be using different clouds,” he said. “It is important that companies have a consistent security posture across clouds and across models.”

He also explained that the threat landscape has changed so radically that old defensive models have become too slow. He noted that the average time between the initial breach and handover to the next stage of the attack decreased from eight hours to 22 seconds, and that the attack surface expanded beyond the perimeter of a traditional network. “In addition to your usual properties, you have models now. You have data pipelines that are used to train models. You have agents, you have claims. It all needs to be protected.”

One threat De Souza pointed out that doesn’t get enough attention: Agents moving through a company’s internal systems can expose forgotten repositories of data that no one thought about in years. “A lot of organizations have legacy SharePoint servers [and access controls] It wasn’t really updated, but it didn’t matter because no one really knew where it was. But agents roaming your organization will find these data assets and expose the data on them.

The answer, from his point of view, is to match the speed of the machine with the speed of the machine. “We are now seeing the emergence of a fully operational AI-driven defence, where organizations can run agents leading their defence,” he said. “Instead of having a human-led defense or even a human in the loop, you can now have humans overseeing an entire agent defense.” He added that this has become a leadership issue, not just a technology issue. “This is a board-level problem and an executive team problem. It’s not just a security team problem.”

But even as AI takes on more of the defense workload, the number of people qualified to oversee it is lacking — and the vulnerabilities introduced by AI itself are multiplying faster than security teams can address them. “We’re going to need people to deal with the bug apocalypse,” Leah Kesner, LinkedIn’s chief information security officer, told the New York Times this week, adding that she doesn’t expect the industry to understand AI security in any long-term sustainable way for at least several years.

Which brings us back to the platform providers themselves. The Register has published a series of reports over the past few weeks documenting a wave of Google Cloud developers being hit with five-figure bills after unauthorized API calls to Gemini Forms — services many of them had never used or had intentionally enabled. The cases followed a familiar pattern: API keys originally published for Google Maps, which had been made public according to Google’s own instructions, quietly became able to reach Gemini after Google expanded their scope without clearly disclosing the change.

Rod Danan, CEO of interview preparation platform Prentus, said his bill reached $10,138 in about 30 minutes after attackers exploited his compromised API key. Isoro Fonseka, a Sydney-based developer whose account was similarly hacked, woke up to nearly AU$17,000 in charges despite believing he had a maximum spending limit of $250. What neither of them knew was that Google’s automated systems had upgraded their billing levels based on account history, raising the actual caps to as much as $100,000 without explicit approval.

Google refunded the whistleblower after The Register published its initial report. However, Google told The Register that it has no plans to change its automatic tier upgrade policy, saying it prioritizes preventing service outages over enforcing users’ stated budget preferences.

Meanwhile, there’s a separate question about what happens when a developer tries to stop things. This week, The Register reported on research by security firm Aikido, which found that even developers who catch a compromised key and immediately delete it may not be safe. According to Aikido’s findings, it appears that attackers can continue to use this key for up to 23 minutes because Google’s revocation gradually spreads across its infrastructure. During that period, success rates are unpredictable — some minutes, more than 90% of requests are still authenticated — and attackers can use the time to filter cached files and conversation data from Gemini, aikido researcher Joseph Lyon told The Register.

Leon also noted that Google’s newer credential formats don’t seem to suffer from the same problem: Service Account API credentials are revoked in about five seconds, and Gemini’s newer key format that starts with AQ takes about a minute. “Both operate on the scale of Google,” he wrote in a related Aikido paper. “Both suggest that this is technically solvable for Google API keys as well.” In short, according to Leon, the 23-minute period is not an engineering constraint but a matter of priorities for the company.

This is something worth keeping in mind when reading de Souza’s advice, which is sound advice and should be taken seriously. He’s not wrong, but there is currently a gap between the platforms that prescribe drugs and how quickly they adapt themselves, and it’s good to be aware of that as well.