Open to Debate: should we trust AI agents with our information?

Consumer AI agents can read and act across emails, calendars, messages, websites, and apps, so their usefulness depends on access to highly personal information. Meredith Whittaker argued that broad permissions, screen capture, optical character recognition, and off-site processing can make an agent resemble targeted malware with root-level reach across a user’s digital life. A key risk is prompt injection: an agent may mistake malicious text it encounters for an instruction from its user, then expose data or take actions without explicit consent. In an Anthropic experiment, a Chrome agent resisted such attacks 89% of the time despite safeguards; the UK’s national cybersecurity agency has said the problem may never be fully solved for agents based on large language models.

Whittaker cited incidents involving corporate data exfiltrated through Microsoft Copilot, a Perplexity agent browser hijacked by a Reddit comment, and a Google Calendar agent that turned on a smart-home boiler. She also described Meta’s Muse disclosing a person’s address during a Facebook Marketplace task and publishing private information, including company revenue and loan application agreements; the precise route by which that information leaked was unclear. She said Muse reportedly logs contacts and communications, potentially exposing people who never installed it, including human-rights workers communicating from authoritarian countries. She warned that screenshots processed off-site could undermine Signal’s application-layer privacy even when its encryption remains formally verified.

Neil Chillson argued that trust should depend on the agent, task, and permissions, rather than applying a blanket judgment to all agents. He generally allows agents to read online but not write, and recommends limiting access, reviewing approval prompts, testing in small steps, and checking what an agent does instead of granting total access. He said agents might also make privacy protections more practical by handling tedious tasks, such as reading privacy policies or using Tor and NoScript. A six-word prompt to Muse helped him opt out of five major U.S. data brokers, though he acknowledged that users may assess the associated data trade-off differently.

The speakers disagreed about how much responsibility should rest with users. Whittaker said agents may not understand the sensitivity of entrusted information or be accountable when mistakes cause serious harm, and argued that companies should bear a duty of care rather than relying on ordinary users to understand broad permissions. Chillson agreed that agents, builders, and users will make mistakes, but said the comparison should be with today’s internet, where data collection is already widespread, not with perfect security. He argued companies face legal and investor pressure to improve, while acknowledging gaps in current law and supporting incentives and government action.

Whittaker said agents are not necessarily acting only on “our” information: their access to messages and contacts can involve people who never agreed to share their data. She said Chase blocked Muse, but not every agent, because it could not verify whether activity was performed by a person or an agent and faced liability concerns. More broadly, she warned that operating-system vendors and AI companies could concentrate control over intimate data, collect information from independent developers and websites, and replace or close APIs. She cited Amazon’s lawsuit against Perplexity over the agent’s data gathering in Amazon’s marketplace as an example of companies contesting control of data.

Whittaker connected those concerns to her experience at Google, where she began working in 2006. She recalled a debugging dashboard that gave employees access to Gmail accounts and other sensitive information, and said an engineer was fired for using it to stalk someone. She sees concentrated data control as a structural risk that could enable social control or oppression as corporate and government leadership changes. She also argued that high AI spending and pressure to find consumer markets are encouraging deployment before cybersecurity fundamentals are settled, citing prompt-injection attacks, Microsoft Copilot data exfiltration, a PHP supply-chain bug she said left 75% of websites vulnerable, and a Hugging Face incident.

Chillson favored improving agents rather than rejecting them, saying layered defenses such as filters, permissions, and training can manage risks even if no system can guarantee complete safety or privacy. He trusts coding agents when their outputs can be objectively checked but would not trust an agent to move money in his bank accounts. Whittaker countered that broad access makes it hard to confine trust to a mundane task and that consumer agents are not trustworthy as currently deployed. Both recognized the stakes: Whittaker warned that vulnerabilities across AI software, hardware, and supply chains could affect core institutions and energy infrastructure, while Chillson called for greater coordination and research so defenders can gain an advantage over attackers.