Yesterday saw two reports about Apple software development. The first said that the company was aiming to give Siri more agentic capabilities “as soon as next year,” while the second said that CEO John Ternus is pushing Craig Federighi to speed up software development.
If accurate, it seems near-certain the two reports are connected. But if Ternus really is pushing for an acceleration in Siri’s ability to act as an AI agent, that comes with three potentially huge risks …
Dots, Muse and Grok can keep track of goals, determine what needs to happen next and continue pursuing objectives without the user initiating every individual step. Apple ultimately wants to go in this direction, I’m told, and I believe the company could make Siri more like an agent as early as next year.
The report about Ternus pushing Federighi for faster software development mentions “moving beyond necessary improvements to Siri,” of which agentic capabilities would be the obvious candidate.
But agentic capabilities are among the most dangerous forms of AI given the multitude of things that could go very, very wrong.
The most obvious is simply failing to carry out the task. Depending on the importance of it, the consequences could range from annoyance through to significant harm.
Technology has long failed to live up to one of its biggest promises: relieving us of mundane work in order to devote our time to more fulfilling things. For example, obtaining tickets to a popular gig or other event, which currently involves scheduling time in order to repeatedly refresh a page, perhaps wait through a queuing system, and finally carry out the transaction. It would be great to be able to hand this off to an AI.
But even on something relatively unimportant like this, if we delegated the task and the AI failed to obtain the tickets, especially if this was due to a silly error, this could prove enormously frustrating. If the task were more important, the results could obviously be far worse.
Potentially even worse is achieving the task but doing so in inappropriate ways. We’ve already seen high-profile examples of this, like the Hugging Face hack caused by an AI model seeking so hard to carry out instructions that it used underhand methods to bypass crucial safeguards in the process.
Another huge risk is to privacy. Just today we learned that Claude Code exposed over 2,000 real secrets to GitHub, as Cybernews reports.
Over two thousand real secrets, including passwords, API tokens, SSH root logins, and even admin-level cloud keys, were found leaking on GitHub after developers clicked “Yes, and don’t ask again” in Claude Code […] When a developer grants Claude Code permission to proceed with “Yes, and don’t ask again,” it saves the exact command, including secrets, to a file, .claude/settings.local.json, inside the project folder. This file should never be exposed publicly [but] Apiiro, an application security company, found 104,185 public copies of such files on GitHub across 103,617 repositories.
This was just a few days after Anthropic started encouraging Claude users to connect their bank and finance card accounts to the AI chatbot. (My own answer is “Hell, no.”)
When we delegate a task to an AI system that requires access to any of our personal data, and we’re not expected to check on it again until the task is complete, the potential for disaster here is enormous.
It’s worth prefixing this with a note that the reliability of the paired reports is far from certain. In particular, in a closed-door meeting between Ternus and Federighi, it’s hard to see how the content could have leaked unless one of the two men chose to do this. If Ternus leaked it, that suggests a very high level of frustration on his part. But leaving that aside …
Agree; but glad they recognize this. Otherwise, with AI tools available I could see people switching out of the hardware if it’s not native and working at least at baseline levels
We’re currently witnessing an AI arms race, with each company aiming to beat its competitors to next-generation capabilities, even while acknowledging a non-trivial possibility that the pace of development could put humanity’s future at risk.
In my view, the very last thing we need is Apple joining this race. I know it can be frustrating when Apple lags behind other companies in the introduction of some types of new tech, but this is one field in which I would be very happy to see Apple proceed very much more cautiously than others.
Right now, AI needs at least one player that prioritizes safety and privacy over speed to market – in that way, we’d get a choice. If we want agentic capabilities sooner rather than later, and are prepared to run the risks involved in this, then we can turn to companies like Anthropic and Meta. If we are prepared to wait for safer systems, then Apple should provide that alternative.
What’s your view on this? Please share your thoughts in the comments.
FTC: We use income earning auto affiliate links. More.
Check out 9to5Mac on YouTube for more Apple news:
Breaking news from Cupertino. We’ll give you t…
Ben Lovejoy is a British technology writer and EU Editor for 9to5Mac. He’s known for his op-eds and diary pieces, exploring his experience of Apple products over time, for a more rounded review. He also writes fiction, with two technothriller novels, a couple of SF shorts and a rom-com!
