Five days after initial reports that there was an AI-model shrinking technology by PrismML suitable for iPhones, the company has confirmed it is talking to Apple over the use of its technology.
One of the major problems with AI processing is the need to manage massive models that most normal computers and smartphones cannot easily handle alone. On July 9, the start-up PrismML surfaced with a solution to shrink down the models to a more manageable size.
A few days later, on July 14, the startup confirmed the talks were underway. Speaking to CNBC, startup CEO Babak Hassibi said that Apple and other companies are evaluating its models.
"They're really evaluating our technology right now," he said of Apple. Hassibi didn't go into detail about what the talks are about, but that they are very early and are "progressing nicely."
These talks could range from simply licensing the technology from PrismML to outright offering to buy the startup completely.
For Apple, the technology that PrismML has developed could be extremely advantageous. As Apple is keen to keep as much AI processing on-device as possible, it is limited in what it can do at the moment due to size constraints.
PrismML's work allows it to shrink the size of models down considerably. The work appears to make it feasible to get it to a size where it could run in an iPhone's memory, without needing cloud access.
An example is the Qwen model which it released on Tuesday to show its capability. The 27-billion-parameter model normally weighs in at 54 gigabytes in size, making it usable only on Macs with 64 gigabytes of memory or more.
However, using PrismML's technology, the same model can be reduced down to under 4 gigabytes. This is much more easily handled by the 8GB of memory in the iPhone 15 Pro and Pro Max.
Since the model can be theoretically handled by the iPhone completely, it opens the door to more AI processing being handled on-device. The current alternative is the use of cloud servers to deal with tougher AI processing tasks.
On-device processing is preferred by Apple, in part because of the privacy and security elements, but also eliminating the need for an Internet connection.
There is a trade-off to this shrinking process. Hassibi confirmed that the models do lose a few percentage points in performance versus the full-fat alternative.
The smaller model is also apparently weaker when dealing with factual reasoning, as well as mathematics and coding.
The compactness and the potential for on-device processing will have implications for the ongoing AI infrastructure buildout. The one that's causing the ongoing memory and component crisis.
The rising price of hardware and components has been caused by AI companies needing to build more data centers. This has consumed the existing supply lines for memory and chips, which won't be solved until more production lines are built.
By bringing processing to devices, especially memory-constrained items like smartphones, that would reduce the need to do AI processing in the cloud. In turn, that means less reliance on the AI infrastructure in general.
That's not just for a better Siri on an iPhone or Mac. That can enable more locally-driven AIs to be put to work on other hardware, so long as they can perform some level of AI processing.
