Ninjas Mapping the Rabbit Hole

April 14, 2016 9:34 AM

THE SECRET RULES OF THE INTERNET: The murky history of moderation, and how it’s shaping the future of free speech [Potentially NSFW - language & content]
posted by chavenet (13 comments total)

This sounds like a job, not unlike cops on sex or hate crime squads, that should be rotated, otherwise people's worldviews will get utterly warped.
posted by jonmc at 10:32 AM on April 14, 2016 [1 favorite]

Let's just get this out of the way early -- in the US, Freedom Of Speech has nothing to do with what a private entity (like Facebook or YouTube) will allow you to but on their platform. It has to do exclusively with government censorship.

That's strictly true, but American "freedom of speech" laws were invented at a time before the overwhelming majority of communications forums were privately owned, and that is no longer the case. This is not to say that people who bleat "Freedom of Speech" to avoid responsibility for or the consequences of their actions are in the right, just that (in my opinion) there's a nuanced argument worth making about the nature of free speech in modernity.
posted by mhoye at 10:39 AM on April 14, 2016 [5 favorites]

Let's just get this out of the way early -- in the US, Freedom Of Speech has nothing to do with what a private entity (like Facebook or YouTube) will allow you to but on their platform.

Has nothing to do with the First Amendment (which, yeah, also has exceptions in reality)- doesn't mean it has nothing to do with the concept of freedom of speech. But the web provides lots of examples of communities that require participants to submit to certain restrictions in order for the whole thing to work, so as long as access to web hosting is fairly accessible and open the existence of different venues moderated as befits their users and their purpose is a good thing. I mean, as always, I'm writing this on MeFi not whatever-chan.
posted by atoxyl at 10:49 AM on April 14, 2016

I can't even begin to express how sick it makes me feel that obscenely wealthy companies are outsourcing the screening of some of the most unspeakably monstrous content in existence to vulnerable people who are willing to work for a pittance.
posted by Panthalassa at 10:51 AM on April 14, 2016 [3 favorites]

I can't even begin to express how sick it makes me feel that obscenely wealthy companies are outsourcing the screening of some of the most unspeakably monstrous content in existence to vulnerable people who are willing to work for a pittance.

Yeah, but think of the surge of importance they feel every time they delete a comment. That's gotta count for something.
posted by entropicamericana at 11:02 AM on April 14, 2016

I'm gonna read every comment in this thread as thinly-veiled subtweeting about Metafilter.
posted by beerperson at 11:14 AM on April 14, 2016

Unacknowledged legislators of the online world: "[C]ustomer content management constitutes the quiet transnational transfer of free-speech decisions to the private, corporately managed corners of the internet where people weigh competing values in hidden and proprietary ways."
posted by MonkeyToes at 11:18 AM on April 14, 2016 [3 favorites]

This sounds like a job, not unlike cops on sex or hate crime squads, that should be rotated, otherwise people's worldviews will get utterly warped.

When I worked at [photosharing website], the folks who worked the moderation queues got free psychotherapy as part of their comp.
posted by murphy slaw at 1:12 PM on April 14, 2016 [1 favorite]

That's strictly true, but American "freedom of speech" laws were invented at a time before the overwhelming majority of communications forums were privately owned

What non-privately-owned communications forums that existed in the 1700s but don't exist now are you thinking of?
posted by straight at 1:13 PM on April 14, 2016

This article is playing really interestingly in my head with the previous one about toxic game culture, and steps being taken to curb it.

I have a hard time with this issue, because there really is content out there I feel doesn't have a place in the world (bigotry, hate speech, calls for violence based on same being my particular lines in the sand, there). I'm also well aware that there's private forums for those exact things. On another hand, part of me wants these things drawn firmly into the light and publicized widely (and hopefully shown as unacceptable, but my faith in that is a little weaker these days).
posted by Archelaus at 2:26 PM on April 14, 2016

I can't even begin to express how sick it makes me feel that obscenely wealthy companies are outsourcing the screening of some of the most unspeakably monstrous content in existence to vulnerable people who are willing to work for a pittance.

Well it's one of those things where there are people who would love to have the job - but they are probably not the people you want doing it.
posted by atoxyl at 2:53 PM on April 14, 2016 [2 favorites]

Wow, that was a great read. Long but worth it. The authors touched on a lot of complexities of moderation ranging from the fact that there are sweatshops of mostly female and 2nd world moderators staring at horrifying images all day, to the dangers of an entirely secretive and privatized system of moderation across the various major internet corporations, to the true ambiguities of what types of images should be available to see and not see.

Saying that the first amendment does not apply to Twitter is a simplification as what gets to appear on twitter can have an enormous impact on political and social movements all the way down to individual mental health. It's too easy to end the conversation with the first amendment.

My biggest takeaway is that most of these companies are not taking moderation seriously as demonstrated by keeping their policies totally private and by not dedicating engineering staff or senior managerial time to addressing these issues.

Also, it's scary that the young men who run the internet are the ones deciding (or choosing to ignore) these major issues that do impact both free expression on a wide scale, the success or failure of social and political movements (Black Lives Matter to the Syrian Civil War), and safety for society's most vulnerable individuals.

Also, Pintrest seems to once again be the more mature and thoughtful member of the massive internet corporate players.

Great link, thanks for posting.
posted by latkes at 4:10 PM on April 14, 2016

Mora-Blanco and her colleagues ultimately agreed to keep the video up. It was fueling important conversations about free speech and human rights on a global scale and was quickly turning into a viral symbol of the movement. It had tremendous political power. They had tremendous political power. And the clip was already available elsewhere, driving massive traffic to competing platforms.

The Policy team worked quickly with the legal department to relax its gratuitous violence policy, on the fly creating a newsworthiness exemption.

This sounds to me like it is not only a free speech issue, it is a means to give free speech to people around the world, to, in fact, empower the disempowered. It routes around the control of local dictators and corrupt goverments, etc.
posted by Michele in California at 2:37 PM on April 16, 2016