Apple has been accused of "aiding and abetting" the use of "nudify" AI apps on the iPhone, with San Francisco City Attorney David Chiu demanding they be removed from the App Store in a new cease-and-desist letter sent to the company.
A Wired report notes that both Apple and Google have been told to prevent the sale of AI apps that can be used to create deepfake images. They must also "sever" business relationships with all of the developers responsible for the apps.
So-called "nudify" apps have been around for some time. They allow users to create non-consensual intimate images of others, often minors, without the knowledge of the subject.
Such apps can be used to remove the clothes of subjects. Some allow the subjects to then be placed into specific scenarios of the user's choosing.
In a statement provided to Wired, Chiu pointed out that generating deepfake imagery is "illegal, harmful, and completely unacceptable." Chiu also noted that Apple and Google have likely made considerable sums from the sale of, or subscriptions relating to, such "nudify" apps.
Chiu also made it clear that Apple and Google have a responsibility to ensure that their platforms aren't being used to create such content. His legal letters also pointed out to both companies that California law prohibits supporting services that can create deepfakes.
Apple is yet to comment on the legal letter it has received. Google says that it investigates apps that are believed to be used to create deepfake content. It added that it "takes swift action" when required.
Apple previously removed a number of similar generative AI apps from the App Store. The company previously told AppleInsider that nudification apps aren't allowed in the App Store, with no exceptions.
App Store reviewers appear to give developers some benefit of the doubt. Developers are first notified if their apps are found to be used to create "nudify" images.
Violations that aren't addressed generally result in the offending app being removed from the App Store. It's not clear if any of the apps in question have gone through this process, or where they are in it.
Apple also says that it actively blocks App Store search terms for such apps.
As for Chiu, he intends to continue to pursue both companies. For now, he wants Apple and Google to remove the offending apps and strengthen the tools that they use to screen new apps before they are made available for download.
