"What you do only matters to the decision makers when Things Go Wrong."

October 17, 2023 3:18 PM

Trust & Safety Tycoon is a free browser-based game exploring the difficult choices and tradeoffs involved in managing a trust and safety team. From the folks who brought you Moderator Mayhem (previously). More about the creation of this game.
posted by jessamyn (31 comments total)

I lolsobbed at Yoel Roth's comment about the game

I especially enjoyed the part of this game where the CEO threatened to fire me because I banned someone and then I had to testify in front of congress. 10/10, fun experience, would recommend.

(It's funny because Roth briefly ran Trust and Safety for Elon Musk at Twitter. Musk then sicced a violent mob on him with a false claim of child sexual abuse.)
posted by Nelson at 3:43 PM on October 17, 2023 [17 favorites]

Some of this stuff really felt like it was pulled directly from either TwitterX or MetaFacebook which is not surprising. I totally failed my first round in Year One because moderation was "too slow" (guilty!) and then figured out what the path to the finals was like (long game) and wound up being able to retire at the end with four stars and 1911 points.
posted by jessamyn at 3:57 PM on October 17, 2023 [8 favorites]

I'm no Jessamyn, but given that I'm not, I'm pretty pleased with my 1894 point four star outcome.
posted by aubilenon at 4:51 PM on October 17, 2023 [4 favorites]

This fixes one of the problems that I previously complained about in Moderator Mayhem: it's no longer speed-based. But it replaces it with a much worse problem, which we might dub "thumb-on-the-scale". You decide what to do, but Mike Masnick decides what the result of your actions are. Masnick is smart and I generally agree with him on the issues. But I don't think it's that easy to predict the effects of any given policy on the health or popularity of a community. Those might turn out to be opposing goals (the Toxoplasma of Rage hypothesis) -- especially over the short term.

There are three policies that I would consider that I didn't get offered: 1. Charging five bucks to join; 2. requiring a referral from an existing member to join and 3. Twitter-style community notes. The latter requires some explaining (Sorry , long post with opinion mixed in, but the explanation is useful).

Finally, there is some of the same nebulosity problem that I complained about last time and have since turned into a mini-game.

Oh, and getting fired for standing up for my principles is considered a "loss", but I don't see why it should be. Better that than keep a job that is doing things that you oppose.
posted by novalis_dt at 5:37 PM on October 17, 2023 [5 favorites]

Definitely a tight-rope to walk. Took the IPO exit at 4* and 1788 points. Interestingly, the CEO always totally believed in me.

Since I was going for pro-social behaviour at the cost of engagement and ad-revenue, it was an entirely unrealistic scenario, one that could never take place in real life.
posted by seanmpuckett at 5:59 PM on October 17, 2023 [7 favorites]

pulled directly from either TwitterX or MetaFacebook

I finished with **** the first playthrough, and my decisions were heavily weighted by what I know about MetaFacebook's complicity in the Rohingya genocide.
posted by MengerSponge at 6:04 PM on October 17, 2023 [4 favorites]

Managed to walk the tightrope of slow moderation and win with 1819, but had to cheat and undo one choice when the EU tried to bury me under moderation explanations. Like all games this one oversimplifies but I think it does what it's trying to do well. I'd love to use this in a classroom with students.
Novalis I managed to avoid getting fired for standing on principal because I had high ceo trust from other choices earlier. Grabbing some of the cheap verification techs with no immediate payoff seems to help there when you avoid potential problems later.
posted by Wretch729 at 6:07 PM on October 17, 2023

Given that I am part of a T&S team (though mostly from the security perspective), I'm reasonably pleased with my 1954 4-star IPO score. I tried to not use knowledge of what other companies have done, but what I would generally hope my company would do.
posted by jferg at 6:11 PM on October 17, 2023

Yes, the one I got fired for (before I undid it) was publicly revealing the secret NSL, which was obviously going to get me fired but still maybe the right thing to do. No level of CEO trust was going to let one survive that. I think that aspect of the game is fine from a simulation perspective, but I didn't appreciate having scoring like 250 for it. If I had done it in real life, I might be in jail but likely Masnick would think me a hero.
posted by novalis_dt at 6:18 PM on October 17, 2023

740-ish and I blame a moment of madness at year 5 when I managed not to spend any of my budget on improvements by mistake.

Would have gotten away with slow moderation if only I’d been able to add a crouton garden……
posted by inflatablekiwi at 6:42 PM on October 17, 2023 [3 favorites]

Managed 1962 on my second go, after my first crashed and burned for slow moderation.
posted by juv3nal at 7:07 PM on October 17, 2023 [1 favorite]

There are three policies that I would consider that I didn't get offered: 1. Charging five bucks to join;

Xitter is actually considering something like that as a trial, the reason given to combat bots, although they're doing it, as a trial, as a $1 yearly charge in a couple of areas.
posted by JHarris at 8:12 PM on October 17, 2023

Four stars, 1808 points, and happy to have done that well. I enjoyed identifying the parallels between the scenarios in the game and various historical content moderation controversies.

If you’re also a huge nerd about this topic, I really enjoy the Moderated Content podcast from Stanford Law. Generally a lot snarkier than one would expect from a trust and safety podcast.
posted by learning from frequent failure at 9:18 PM on October 17, 2023 [1 favorite]

1941 on my first try. I will be adding this to my resume, citing this statistic in every MeTa comment, etc
posted by Tom Hanks Cannot Be Trusted at 10:48 PM on October 17, 2023 [6 favorites]

I lost due to slow moderation although, honestly, at no time did I get the option to slow user growth, so the problem is the 'growth at all costs' mentality at the expense of creating safe and valuable communities.

Living in a country where my Disney+ channel for kids carries, for example, the French Dispatch with Léa Seydoux completely nude, I wanted to disagree with the anti-nudity policy at the start but its presented as an 'obvious' policy.
posted by vacapinta at 4:22 AM on October 18, 2023 [2 favorites]

Struck that porn is not one of the standard conversational topics.

Died on my first try due to slow moderation, retired the second time to a puppy.
posted by praemunire at 7:13 AM on October 18, 2023

First play, died in less than a year after going all-in on earthquake relief.

Second play, won via IPO with a score of 1980.
posted by box at 7:44 AM on October 18, 2023

I testified before Congress, won via IPO, finished with 1949, and left to advise the government. On the side, I started a site that specializes in "Oh, I deserve a good spanking, Mr Senator" videos, where I track user IDs to see which senators I can count on for their support.
posted by pracowity at 1:16 PM on October 18, 2023 [1 favorite]

There are three policies that I would consider that I didn't get offered: 1. Charging five bucks to join;

Xitter is actually considering something like that as a trial, the reason given to combat bots, although they're doing it, as a trial, as a $1 yearly charge in a couple of areas.

I don't know how it's working for Metafilter, but Denise, the co-founder of Dreamwidth, who's been doing T&S since before it was called T&S, unfortunately says that it doesn't work.

In the game, I was fired in Year 3 for standing for my principles while keeping my team's morale high and I'll take that badge of honor. I found it interesting how with some situations, there are no good solution. All of them are bad, the most you can do is mitigate the fallout.

(Also, the breastfeeding/visible nipple incident comes from LiveJournal, I still remember that particular clusterfuck.)
posted by snakeling at 2:10 PM on October 18, 2023

This was fun and interesting. I got 2018 ⭐⭐⭐⭐⭐ which I achieved mostly with a strategy of "what would old Twitter do", with notable exceptions for notorious historical mistakes by Twitter or others (such as ignoring the Rohingya genocide).
posted by grouse at 2:16 PM on October 18, 2023 [2 favorites]

Denise may well say that, but I can't see it -- on Firefox mobile, I just get a login prompt. Yet another reason to be depressed about bsky. (The major reason is that it's supposed to eventually be federated but it isn't. And people are joining on the promise that it will offer user autonomy, and that is somehow not bsky's top priority, or seemingly a priority at all. Haven't we learned not to trust promises like this?)

(I recognize that there are reasons why it is an imperfect solution; the question is whether it could be part of a larger solution.).
posted by novalis_dt at 2:24 PM on October 18, 2023 [2 favorites]

(Also, the breastfeeding/visible nipple incident comes from LiveJournal, I still remember that particular clusterfuck.)

Don't forget the tumblr male vs. female-presenting nipple insanity.
posted by praemunire at 2:25 PM on October 18, 2023 [2 favorites]

Yeah, Bluesky posts are not visible without a login. Would be nice if folks posting them could also post at least an excerpt of the text.

Haven't we learned not to trust promises like this?

Especially when it's Jack Dorsey making them.
posted by We had a deal, Kyle at 5:22 PM on October 18, 2023 [2 favorites]

wrt the game, I kept getting bounced in year 2 or 3 for slow moderation; at least until I discovered that you can go back from the game-over page to the decision that got you bounced and try a different -- and what usually seemed to me a less principled -- choice.

I found it a generally depressing experience; the overwhelming lesson was that people are shitty and are always trying to game your systems and you'll inevitably end up pandering to some of them. Maybe that was the point.
posted by We had a deal, Kyle at 5:29 PM on October 18, 2023

1982, ⭐⭐⭐⭐, IPO, left to advise the govt

i struggled with nipples, generally told governments to fuck off, and always went 110% to the mat for my moderation team. moderation speed was a constant challenge because doing the right thing is hard
posted by glonous keming at 6:00 PM on October 18, 2023 [1 favorite]

moderation speed was a constant challenge

I felt that way too but I also felt it was kind of arbitrarily made that way. Like, if you can blow millions fighting something in court then you should be equally able to staff up and pay for more/better training but the way moderation speed always takes a hit it seems an implicit assumption that either that wouldn't help at all or it's not an option.
posted by juv3nal at 11:36 PM on October 18, 2023

Yeah, Bluesky posts are not visible without a login. Would be nice if folks posting them could also post at least an excerpt of the text.

Ah yes, my bad. Here you go:

Your regular reminder that payment based verification does absolutely nothing to reduce the amount of spam a site gets because most spam comes from sophisticated operations that have access to an infinite number of stolen cards and the rest is so profitable $1 is a tiny cost of doing business

posted by snakeling at 11:22 AM on October 19, 2023 [1 favorite]

Payment may not reduce spam, but it might increase the quality of the conversation in other ways (for instance, by changing users' beliefs about what it means to be a member).

(And I wonder if there are clever strategies to be better at taking payments -- like, don't let the user post until a week has gone by without a chargeback, or accept Zelle which is less likely to be stolen. And for legit cards, have an outrageous "cleaning fee" that gets charged for rule violations.)

Like, Metafilter charges and doesn't have a spam problem (I am aware that occasionally spam gets posted, but it's nothing like Twitter). So maybe there is some effect.
posted by novalis_dt at 5:50 PM on October 20, 2023

I'm no quitter - 1843, four stars, won via IPO and stayed on to grow Yapper. I did my best to resist government non-warrant requests and protect my team as much as possible, so the moderation speed was always somewhat slow, but I guess the whole point is that you can't win by trying to keep everyone happy. Definitely didn't matter which of the pressing moderation issues you dealt with yourself or handed to someone else - sometimes, you just cant win.
posted by dg at 10:18 PM on October 22, 2023

Like, Metafilter charges and doesn't have a spam problem...So maybe there is some effect.

I think there's probably something to the amount that is being charged versus the desirability of access to the platform. Like "$1 is a tiny cost of doing business" for what you get on twitter maybe, but it's certainly not equally true of $5 for a much, much smaller set of eyeballs, right?
posted by juv3nal at 11:24 PM on October 22, 2023

I think the real benefit of the fee is not the money but the fact that it's much harder to make a thousand accounts without the mods being able to tell they're the same person. (whether all at once or over and over after each one gets banned)
posted by aubilenon at 11:59 PM on October 22, 2023