Musk’s ‘Twitter Files’ offer insight into the raw, complicated and thankless task of moderation – TechCrunch

New Twitter owner Elon Musk is feverishly promoting his “Twitter Files”: select internal company communications, painstakingly tweeted by sympathetic amanuenses. But Musk’s obvious belief that he posted a partisan kraken is misguided — far from conspiracy or systemic abuse, the files are valuable insight behind the curtain of large-scale moderation, hinting at the Sisyphus works undertaken by every social media platform.

For a decade, companies like Twitter, YouTube and Facebook have performed an elaborate dance to keep the details of their moderation processes equally out of the reach of bad actors, regulators and the press.

Revealing too much would expose processes to abuse by spammers and scammers (who in effect profit from every detail leaked or published), while revealing too little leads to damaging reports and rumors as they arise. lose control of the story. In the meantime, they must be prepared to justify and document their methods or face censorship and fines from government authorities.

The result is that even if everyone knows a little on exactly how these companies inspect, filter and organize the content published on their platforms is just enough to be sure that what we are seeing is just the tip of the iceberg.

Sometimes there are exposes of the methods we suspected – hourly entrepreneurs clicking violent and sexual images, a heinous but seemingly necessary industry. Sometimes companies overstate their hands, such as repeated claims about how AI is revolutionizing moderation and subsequent reports that AI systems for this purpose are impenetrable and unreliable.

What almost never happens – typically companies don’t unless forced to – is that the actual large-scale content moderation tools and processes are exposed unfiltered. And that’s what Musk did, perhaps at his peril, but surely for the great benefit of anyone who’s ever wondered what moderators actually do, say, and click when making decisions that can affect millions of people.

Pay no attention to the honest and complex conversation behind the curtain

Chain emails, Slack conversations, and screenshots (or rather screenshots) posted over the past week provide insight into this important and misunderstood process. What we see is a bit of the raw material, which is not the partisan illuminati some expected – although it is clear from its very selective presentation that this is what we are meant to perceive. .

Far from it: the actors are in turn cautious and confident, pragmatic and philosophical, frank and accommodating, showing that the choice to limit or prohibit is not made arbitrarily but according to an evolving consensus of opposing points of view.

(Update: Moments after posting this, a new thread started, it’s more or less the same – serious discussions on complex issues in coordination with experts, law enforcement and others.)

Before the choice to temporarily restrict the Hunter Biden laptop story – probably at this point the most controversial moderation decision in recent years, behind Trump’s ban – there is no partisanship or insinuated conspiracy. by the explosive packing of documents.

Instead, we find serious, thoughtful people trying to reconcile conflicting and inadequate definitions and policies: what constitutes “pirated” content? How confident are we in this or that assessment? What is a proportionate response? How to communicate it, to whom and when? What are the consequences if we do, if we don’t limit? What precedents are we setting or breaking?

The answers to these questions are not at all obvious, and are the sort of things usually chopped up over months of research and discussion, or even in court (legal precedents affect legal language and repercussions). And they had to be done quickly, before the situation somehow got out of control. Dissent from inside and outside (from a US Representative, no less – ironically, doxxed in the thread with Jack Dorsey in violation of the same policy) was considered and honestly incorporated.

“This is an emerging situation where the facts remain unclear,” said former Trust and Safety chief Yoel Roth. “We have chosen to include a disclaimer and prevent amplification of this content.”

Some question the decision. Some question the facts as presented. Others say it’s not supported by their reading of the policy. One says they need to be very clear about the ad hoc basis and scope of the action, because obviously it will be looked at as a partisan action. Assistant General Counsel Jim Baker asks for more information, but says caution is advised. There is no clear precedent; the facts are at this stage absent or unverified; some of the material is obviously non-consensual nude imagery.

“I think Twitter itself should restrict what it recommends or puts in trending news, and your policy against QAnon groups is good,” concedes rep Ro Khanna, while saying the action in question is a not too far. “It’s a difficult balance.”

Neither the public nor the press were aware of these conversations, and the truth is that we are as curious and largely in the dark as our readers. It would be incorrect to call the published documents a complete or even accurate depiction of the whole process (they are obviously, though ineffectively, selected and chosen to fit a narrative), but even as they are , we are more informed than we were before .

Tools of the trade

Even more directly telling was the following thread, which featured screenshots of the actual moderation tool used by Twitter employees. While the thread dishonestly attempts to equate the use of these tools with the shadow ban, the screenshots do not show any nefarious activity, nor do they need to be interesting .

On the contrary, what is shown is convincing for the simple reason that it is so prosaic, so simply systematic. Here are the various techniques that every social media company has repeatedly explained they use, but whereas previously we’ve expressed it in the cheerful diplomatic tone of PR, it’s now presented without commentary: “Trends Blacklist”, “High Profile”, “DO NOT TAKE ACTION” and the rest.

Meanwhile, Yoel Roth explains that actions and policies need to be better aligned, that more research is needed, that plans are underway to improve:

The assumption underlying much of what we have implemented is that if exposure to, for example, misinformation directly causes harm, we should use remedial measures that reduce exposure, and limit spreading/virality of content is a good way to do that… we’re gonna have to make a stronger case for building this into our repertoire of policy remedies – especially for other policy areas.

Again, the content belies the context in which it is presented: it is not the deliberations of a secret liberal cabal going after its ideological enemies with a ban hammer. It’s an enterprise-level dashboard as you can see for lead tracking, logistics, or accounts, discussed and iterated by sober people working within practical limits and aiming to satisfy multiple stakeholders.

As it should be: Twitter, like its other social media platforms, has been working for years to make the moderation process efficient and systematic enough to work at scale. Not only to keep the platform free of bots and spam, but also to comply with legal frameworks such as FTC orders and GDPR. (Which “expanded and unfiltered access” to the illustrated tool may well constitute a violation. Relevant authorities told TechCrunch that they are “engaging” with Twitter on this matter.)

A handful of employees making arbitrary decisions without rubric or oversight is no way to effectively moderate or meet such legal requirements; nor (like the resignation of many on Twitter Trust & Safety Council testifies today) is automation. You need a large network of people cooperating and working under a standardized system, with clear boundaries and escalation procedures. And that’s certainly what the screenshots that Musk had posted seem to show.

What the documents don’t show is some sort of systematic bias, which Musk’s replacements insinuate but fail to quite prove. But whether or not it fits the narrative they want, what’s published is of interest to anyone who thinks these companies should be more open about their policies. It’s a win for transparency, even if Musk’s opaque approach accomplishes it more or less by accident.