Google changes appeals process for suspected child abuse images

When Google notified a mother in Colorado that her account had been disabled, it was as if her house had burned down, she said. In an instant, she lost access to her wedding photos, videos of her son growing up, emails dating back a decade, tax documents and everything she had kept in what she thought it was the safest place. She had no idea why.

Google declined to reconsider the decision in August, saying its YouTube account contained harmful content that may be illegal. It took him weeks to find out what happened: his 9-year-old son finally confessed that he used his old smartphone to upload a YouTube short film of himself dancing naked.

Google has an elaborate system, involving algorithmic monitoring and human review, to prevent child exploitation images from being shared and stored on its platforms. If a photo or video uploaded to the company’s servers is considered sexually explicit content featuring a minor, Google will deactivate the user’s account, on all Google services, and report the content to an organization nonprofit that works with law enforcement. Users have the ability to challenge Google’s action, but in the past they had no real ability to provide context for a naked photo or video of a child.

Now, after a New York Times report, Google has changed its appeals process, giving users accused of the heinous crime of child sexual exploitation the chance to prove their innocence. Content found to be abusive will still be removed from Google and flagged, but users will be able to explain why it was in their account – stating, for example, that it was a thoughtless prank by a child.

Susan Jasper, Google’s head of trust and safety operations, said in a blog post that the company would “provide more detailed reasons for account suspensions.” She added: “And we will also be updating our appeals process to allow users to submit even more context on their account, including to share more information and documentation from independent professionals or bodies responsible for law enforcement to help us understand the content detected in the account. .”

Over the past few months, The Times, reporting on the power tech companies wield over the most intimate parts of their users’ lives, has drawn Google’s attention to several instances where its previous review process appeared to have gone wrong.

In two separate cases, dads took nude photos of their toddlers to help with medical treatment. An algorithm automatically flagged the images, then human moderators deemed them to be in violation of Google’s rules. Police determined that the fathers had committed no crime, but the company deleted their accounts anyway.

The fathers, one in California and the other in Texas, were stymied by Google’s previous appeal process: they were at no time able to provide medical records, communications with their doctors or police documents absolving them of wrongdoing. San Francisco’s father eventually recovered six months of his Google data, but on a flash drive from the police department, which had obtained it from the company with a warrant.

“When we find child sexual exploitation material on our platforms, we remove it and suspend the corresponding account,” Google spokesman Matt Bryant said in a statement. “We take the implications of suspending an account seriously, and our teams are constantly working to minimize the risk of an improper suspension.”

Tech companies that offer free services to consumers are notoriously bad at customer support. Google has billions of users. Last year it deactivated more than 270,000 accounts for breaking its rules against child pornography. In the first half of this year, it deactivated more than it did in all of 2021.

“We don’t know what percentage of those are false positives,” said Kate Klonick, an associate professor at St. John’s University School of Law who studies internet governance issues. Even just 1% would result in hundreds of calls per month, she said. She predicted that Google would need to expand its trust and safety team to handle disputes.

“It looks like Google is doing the right thing,” Ms. Klonick said, “in judging and resolving false positives. But it’s an expensive proposition.

Evelyn Douek, an assistant professor at Stanford Law School, said she would like Google to provide more details on how the new appeals process will work.

“Just having a process in place doesn’t solve everything. The devil is in the details,” she said. “Is the new review significant? What’s the timeline?”

It took the Colorado mother, who asked that her name not be used to protect her son’s privacy, four months to get her account back. Google reinstated it after The Times brought the matter to the company’s attention.

“We understand how upsetting it would be to lose access to your Google Account and the data stored there, due to the wrong circumstance,” Bryant said in a statement. “These cases are extremely rare, but we’re working on ways to improve the appeals process when people come to us with questions about their account or think we’ve made the wrong decision.”

Google did not tell the woman that the account was active again. Ten days after her account was reactivated, she learned of the decision from a Times reporter.

When she logged on, she found that everything had been restored beyond the video her son had made. A message appeared on YouTube, with an illustration of a referee whistling and saying that his content violated community guidelines. “Because this is the first time, this is just a warning,” the message read.

“I wish they had just started here,” she said. “It would have saved me months of tears.”

Jason Scott, a digital archivist who wrote a memorably profane blog post in 2009 warning people not to trust the cloud, said companies should be legally obligated to give users their data, even when an account has failed. been closed for violation of the rules.

“Data storage should be like tenant law,” Scott said. “You shouldn’t be able to keep someone’s data and not give it back.”

The mother also received an email from “The Google Team”, sent on December 9.

“We understand that you have attempted to appeal several times and we apologize for the inconvenience caused,” he said. “We hope you understand that we have strict policies to prevent our services from being used to share harmful or illegal content, especially egregious content like child sexual abuse material.”

Many companies other than Google monitor their platforms in an attempt to prevent the widespread sharing of child sexual abuse images. Last year, more than 100 companies sent 29 million reports of suspected child exploitation to the National Center for Missing and Exploited Children, the nonprofit that serves as a clearinghouse for such material and transmits reports to law enforcement for investigation. The nonprofit does not track how many of these reports represent actual abuse.

Meta sends the largest volume of reports to the national center – more than 25 million in 2021 from Facebook and Instagram. Last year, the company’s data scientists analyzed some of the reported material and found examples that were deemed illegal under federal law, but “not malicious.” In a sample of 150 reported accounts, more than 75% “did not exhibit malicious intent,” the researchers said, giving examples that included a “meme of a child’s genitals bitten by an animal” that was shared with humor and teenagers sexting. .