As I slowly shrink and transform into a GPT-2 model

December 31, 2019 11:26 AM

Bot technology may finally have advanced to the point where it can seamlessly replace Weird Twitter. Can you tell the difference between dril and an AI pretending to be dril?

Back in 2013, beloved Twitter account @horse_ebooks, described on Metafilter as "a Russian spam account that communicates entirely through snippets of ebooks and is more hilarious, confusing and philosophically poetic than any non-spambot on the internet," was revealed to be... just some guy. Fans, many of whom had been inspired to make poetry and art based on @horse_ebooks tweets, expressed a sense of betrayal; they'd been led to believe that a glorious, transcendent surrealism could arise from randomness, and it was just Weird Twitter all along. We already had Weird Twitter!

But with the 2019 release of GPT-2, a text-generating AI system trained on huge swathes of the internet, it may be possible for the bots to overtake Weird Twitter for real. The Dril Turing Test measures your ability to distinguish between an AI and just some guy shitposting, and folks: it isn't easy. Seven years after the unmasking of @horse_ebooks, bots may finally be ready to take (sorry) the reins.

The test presents you with a tweet that's either from Weird Twitter mainstay @dril, or from @dril_gpt2, a human-curated feed of AI-generated tweets from a GPT-2 model trained on dril. Unlike earlier text-generating AIs, GPT-2 is powerful enough to kinda remember what it's talking about from the beginning of a tweet to the end (for more on this read Janelle Shane's book You Look Like a Thing and I Love You), and dril isn't exactly a paragon of coherence, so it's challenging. For instance, is this a real dril tweet?

my ass has become highly immunized against fungal infections, by ingesting fantastic amounts of kfc

What about this?

once had a car accident with 100 pelicans because they all had gunpowder embedded in them and started shooting at me

Or this?

market research firm YouGov estimates that 37% of youtubers use YouTube to make seriously bad opinions on subjects. 0% use it to make good opinions on subjects

Answer: no, no, and no. But they could be.

(Warning that the quiz will just keep going if you keep playing, and it's loaded with about 3200 tweets from each feed, so if you want to announce a score you will have to decide when you're done.)

posted by babelfish (11 comments total)

Eponysterical, or is it?
posted by Glomar response at 11:33 AM on December 31, 2019 [1 favorite]

I'm 10/10 so far...
posted by Cpt. The Mango at 11:38 AM on December 31, 2019 [1 favorite]

Generally dril’s tweets are funnier, but I don’t know why
posted by Doleful Creature at 12:43 PM on December 31, 2019 [2 favorites]

The real dril tweets often contain an oblique reference to something, whereas the AI dril tweets are more like just plain context-free weird.
posted by doop at 12:48 PM on December 31, 2019 [7 favorites]

The AI has better spelling than dril.
posted by thatwhichfalls at 1:14 PM on December 31, 2019 [2 favorites]

I keep seeing more and more of these "have fun while training my GPT-2 instance" projects out there.
posted by BS Artisan at 2:22 PM on December 31, 2019

I think the model has already been trained using dril's tweets as the dataset; it would be cool if the "Turing test" results were fed back in somehow to continually refine the model, but it's not obviously doing so. There's also a human curation step between the output from the model and the website that might mess with trying to build a feedback loop like that.
posted by Kadin2048 at 2:33 PM on December 31, 2019 [4 favorites]

Growing up, my family subscribed to Reader's Digest for a time, and that magazine had one section specifically for jokes (Laughter is the Best Medicine), and numerous other sections for funny anecdotes that supposedly happened in real life (Humor in Uniform, Life in these United States). While the content was all fairly corny, it taught me of the distinction between fictional comedy and reality-based (allegedly) comedy. The latter lacks the polish that fiction controls over its subject, yet when the universe randomly produces funny content, there's an extra level of hilarity that comes with the coincidence.

That's how I feel about drilbot vs. real dril. While these tweets may be curated, the random happenstance that generated them gives them an added amount of funny, and endearment. The jokes aren't as sophisticated as dril's best work, but the ones posted match his voice and mood very well. And dril outputs a lot of unfunny meaningless white noise tweets (which disjointed spammy nature makes me suspect that Weird Twitter accounts may be easier to impersonate with GPT-2), while the curation cuts out that noise. An optimized dril.
posted by Apocryphon at 1:00 PM on January 1, 2020 [1 favorite]

I was about 50/50 when I quit, but then I've only found dril occasionally mildly funny.
posted by bongo_x at 6:55 PM on January 1, 2020

But, the ones that were actually funny I knew were real, so there's that.
posted by bongo_x at 6:56 PM on January 1, 2020

I got 9 out of 10 correct and didn’t find it very hard to tell the difference. It still takes a human touch to make random-seeming shit funny.
posted by zipadee at 3:54 PM on January 6, 2020