The era of the all-access AI agent has arrived

For years, The cost of using “free” services from Google, Facebook, Microsoft and other big tech companies has been the equivalent of handing over your data. Uploading your life to the cloud and using free technology brings convenience, but it also puts personal information in the hands of giant corporations who are often looking to monetize it. Now, the next wave of generative AI systems will likely want more access to your data than ever before.

Over the past couple of years, generative AI tools — such as OpenAI’s ChatGPT and Google’s Gemini — have moved beyond the relatively straightforward text chatbots that companies initially rolled out. Instead, major AI companies are increasingly building and pushing for adoption of agents and “assistants” who promise their ability to take actions and complete tasks on your behalf. The problem? To get the most out of them, you’ll need to give them access to your systems and data. While much of the initial controversy around large language models (LLMs) was about blatant copying of copyrighted data online, access by AI agents to your personal data is likely to raise a new set of issues.

“In order to have their full functionality, to be able to access applications, they often need access to the operating system or operating system level of the device you’re running them on,” says Harry Farmer, a senior researcher at the Ada Lovelace Institute, whose work has included studying the impact of AI assistants and found that they may pose a “profound threat” to cybersecurity and privacy. As for customizing chatbots or assistants, Farmer says, there can be data trade-offs. “All of these things, in order to be successful, need a lot of information about you,” he says.

Although there is no strict definition of what an AI agent actually is, it is often viewed as a generative AI system or LLM that has been given a certain level of autonomy. Currently, agents or assistants, including AI-powered web browsers, can control your device and browse the web for you, book flights, conduct research, or add items to shopping carts. Some can complete tasks that involve dozens of individual steps.

While current AI agents are flawed and often unable to complete the tasks they are hired to do, technology companies are betting that the systems will radically change the jobs of millions of people as they become more capable. A major part of their usefulness likely comes from access to data. So, if you want a system that can provide you with your schedule and tasks, it will need access to your calendar, messages, emails, and more.

Some of the most advanced AI products and features provide a glimpse into how much access agents and systems can provide. Some agents developed for businesses can read code, emails, databases, Slack messages, files stored in Google Drive, and more. Microsoft’s controversial Recall product takes screenshots of your desktop every few seconds, so you can dig into everything you’ve done on your device. Tinder has created an artificial intelligence feature that can search through photos on your phone to “better understand the interests and personalities” of users.

Most of the time, consumers have no real way to check whether AI or technology companies are handling their data in the ways they claim, says Carissa Velez, an author and associate professor at the University of Oxford. “These companies are very illicit with the data,” Velez says. “They have shown that they have little respect for privacy.”