People at the center of an effective digital defense

Therefore, content moderation – UGC monitoring – is essential for online experiences. In his book Custodians of the Internet, sociologist Tarleton Gillespie writes that effective content moderation is necessary for digital platforms to function, despite the “utopian notion” of an open Internet. “There is no platform that does not impose rules, to some extent not doing so would be simply untenable,” he wrote. “Platforms must, in one form or another, moderate: both to protect one user from another, or a group from their antagonists, and to remove the offensive, vile or illegal – as well as to present their best face to new users, their advertisers and partners, and the general public.

Content moderation is used to address a wide range of content, across industries. Skillful content moderation can help organizations keep their users safe, their platforms usable, and their reputation intact. A best practice approach to content moderation relies on increasingly sophisticated and precise technical solutions while supporting these efforts with human skill and judgment.

Content moderation is a rapidly growing industry, essential for all organizations and individuals who congregate in digital spaces (i.e. over 5 billion people). According to Abhijnan Dasgupta, Director of Trust and Security (T&S) Specialist Practice at Everest Group, the industry was valued at around $7.5 billion in 2021 – and experts predict that number will double by 2024. Gartner research suggests that nearly a third (30%) of large enterprises will consider content moderation a top priority by 2024.

Content moderation: more than social media

Content moderators delete hundreds of thousands of problematic content every day. Facebook’s Community Standards Enforcement Report, for example, says that in the third quarter of 2022 alone, the company removed 23.2 million incidents of violent and graphic content and 10.6 million incidents of hate speech, in addition to 1.4 billion spam messages and 1.5 billion fake accounts. But while social media is the most widely reported example, a slew of industries rely on UGC — everything from product reviews to customer service interactions — and therefore require content moderation. .

“Any site that allows the entry of information that is not produced in-house needs content moderation,” says Mary L. Gray, senior researcher at Microsoft Research, who is also on the Luddy faculty. School of Informatics, Computing, and Engineering at Indiana University. Other industries that rely heavily on content moderation include telehealth, gaming, e-commerce, and retail, as well as public sector and government.

In addition to removing offensive content, content moderation can detect and eliminate bots, identify and remove fake user profiles, address bogus reviews and ratings, remove spam, control misleading advertising, mitigate predatory content ( especially one that targets minors) and facilitate the safety of two. two-way communications
in online messaging systems. Fraud is an area of ​​great concern, especially on e-commerce platforms. “There are a lot of bad actors and scammers out there trying to sell fake products, and there’s also a big problem with fake reviews,” says Akash Pugalia, Global President of Trust and Safety at Teleperformance, which provides non-blatant content moderation support for international brands. “Content moderators help ensure products meet platform guidelines, and they also remove banned products.”

Download the report.

This content was produced by Insights, the custom content arm of MIT Technology Review. It was not authored by the editorial staff of MIT Technology Review.