Apple announced details of a scan system to find child sexual abuse material (CSAM) on customers’ devices.
Before an image is stored onto iCloud Photos, the technology will search for matches of already known CSAM.
Apple said that if a match is found a human reviewer will then assess and report the user to law enforcement.
However there are privacy concerns that the technology could be expanded to scan phones for prohibited content or even political speech.
Experts worry that the technology could be used by authoritarian governments to spy on its citizens.
Apple said that new versions of iOS and iPadOS will have “new applications of cryptography to help limit the spread of CSAM online, while designing for user privacy”.
The system works by comparing pictures to a database of known child sexual abuse images compiled by the National Center for Missing and Exploited Children (NCMEC) and other child safety organisations.
Those images are translated into “hashes”, numerical codes that can be “matched” to an image on an Apple device.
Apple says the technology will also catch edited but similar versions of original images.
The company claimed the system had an “extremely high level of accuracy and ensures less than a one in one trillion chance per year of incorrectly flagging a given account”.