A photo of a toddler’s groin infection on a father’s Android smartphone has taken an unexpected turn. According to the American daily New York TimesGoogle shut down this parent’s accounts, filed a report with the National Center for Missing and Exploited Children (NCMEC) in the United States, prompting a police investigation.
Another relative, called Mark by the New York Times, received the same treatment from Google. He had sent a photo of his child’s swollen genital area to a nurse, at her request, in February 2021, at the height of the COVID-19 pandemic, before a video consultation with a doctor.
The Californian giant sent him a notification two days after the events, telling him that his accounts had been locked for
serious violation of Google policies due to content that
could be illegal.
Mark lost access to his emails, contacts, photos and phone number (he was using Google Fi mobile service). His request to appeal to Google was denied.
A San Francisco police investigation was opened, and the investigator concluded that the incident
did not meet the elements of a crime and that no crime had been committednote it NYT.
Useful AI, but not without shortcomings
It is precisely the nightmare that worries us allsaid Jon Callas, director of technology projects for the Electronic Frontier Foundation (EFF), which fights for the privacy of Internet users.
The organization convinced Apple to step on the brakes following an announcement in 2021 of a plan to protect minors. The giant wanted to scan the images of the owners of its devices before they were uploaded to Apple’s iCloud, in order to compare them to the NCMEC database.
According to the EFF, this was a step back on user privacy.
In addition to privacy gaps, this story sheds light on the difficulty for an AI to distinguish a medical photo from potential abuse, if an image of a naked child is stored in the library. Internet user’s digital or on the cloud (clouds).
Like Google, social networks Facebook and Twitter, as well as the Reddit forum, use the same tool, Microsoft’s PhotoDNA, to detect potential abuse.
How it works: It does a hash match with the PhotoDNA tool to analyze uploaded images and detect matches to known child sexual abuse material.
In 2012, technology helped arrest a registered sex offender who was using Gmail to send images of a young girl.
Google has also had its own toolbox since 2018 that uses more advanced AI technology, going so far as to identify even illicit material never used before.
Requested by the New York TimesGoogle has indicated that it only scans the personal images of its users when
positive measures were taken. Backing up photos to Google Photos, for example, could be considered as such. The company is required by US federal law to report potential abuse to NCMEC.
” Child sexual abuse material is hateful, and we are committed to preventing its dissemination on our platforms. »
Our team of child safety specialists review flagged content for accuracy and consult with pediatricians to ensure we are able to identify instances where users may seek medical advice.added in a press release the spokesperson for Google.
In 2021, Google reported 621,583 cases of child sexual abuse material to CyberTipLine, of the National Center for Missing and Exploited Children. Of these, 4,260 potential victims who have been reported to authorities, according to the New York Times.