The Most Human Human

February 15, 2011 11:16 AM

"During the competition, each of four judges will type a conversation with one of us for five minutes, then the other, and then will have 10 minutes to reflect and decide which one is the human. Judges will also rank all the contestants—this is used in part as a tiebreaking measure. The computer program receiving the most votes and highest ranking from the judges (regardless of whether it passes the Turing Test by fooling 30 percent of them) is awarded the title of the Most Human Computer. It is this title that the research teams are all gunning for, the one with the cash prize (usually $3,000), the one with which most everyone involved in the contest is principally concerned. But there is also, intriguingly, another title, one given to the confederate who is most convincing: the Most Human Human award."

In 1995, Charles Platt wrote in Wired about his experience as a "confederate" in The Loebner Prize in Artificial Intelligence, the much criticized but highly publicized annual competition of computer programmers who attempt to pit their AI chatbots against a kind of human control group, attempting to fool judges into thinking their programs are "more human" than the human participants. Platt reveals that by "being moody, irritable, and obnoxious" he is crowned "the Most Human Human," an award given to the participant most likely to be judged a person.

Brian Christian, author of the above excerpt from a forthcoming book, documents how he himself tries to win the same award. After much consideration, he settles on a slightly different tactic to convince others of his humanity: typing a lot.

Previously about the Loebner Prize (1, 2, 3) and the Turing test

posted by jng (35 comments total)

(Solid 18 carat, notgold-plated like the Olympic "Gold" medals)

When this is the first thing on the page, you know this is a classy prize.
posted by DU at 11:21 AM on February 15, 2011 [1 favorite]

How I Failed The Turing Test, by Jason Striegel
posted by Plutor at 11:24 AM on February 15, 2011 [1 favorite]

I wish I could get someone to certify me as human. Or, at least, human enough to fool a panel.
posted by GenjiandProust at 11:25 AM on February 15, 2011

My mother . . . let me tell you about my mother . . .
posted by chaff at 11:27 AM on February 15, 2011 [4 favorites]

A critique and discussion about the article, prize, and AI in general over at Mother Jones. Some CS professors chime in in the comments.

In case anyone was wondering how prestigious the award is:

Ugh. The Loebner Prize again. Sponsored by the owner of a factory that makes brass fittings and luggage carts, this scam embarrasses the AI community yearly by confusing real academic progress in AI topics with silly games for chatbots based around a fundamentally brain-damaged misunderstanding of what the Turing Test was about. The Loebner prize is to AI much as Scientology is to neuroscience.
posted by jng at 11:27 AM on February 15, 2011

A friend's son did the following test for a high school project. Experiment was to have one person sit in a chair, curtain divider up, and have a conversation with the person in the other chair that is out of sight. People would have conversations with the other person, but the other person would remain silent. People were sad, people got angry. At the end he asked them questions such as "did you speak with a male or female?," "did the other person seem upset?" and "what do you think the other person thought of you?"

Most subjects thought the other person was a jerk, shy, etc. About 10% suspected there was no person. The 10% were correct. The other person was a toaster.
posted by Mister Fabulous at 11:31 AM on February 15, 2011 [7 favorites]

Additionally, if a computer ranks higher than any of the humans, it will be given the Voight-Kampff Award, and declared "More Human Than Human."
posted by explosion at 11:32 AM on February 15, 2011 [1 favorite]

@GenjiandProust - I'm afraid you appear to be a rodent.
posted by zeoslap at 11:33 AM on February 15, 2011

About 10% suspected there was no person. The 10% were correct. The other person was a toaster.

Hey now, we don't refer to Cylons as "toasters," in this house, Mister!
posted by explosion at 11:34 AM on February 15, 2011 [5 favorites]

I am the psychotherapist. Please describe your problems. Each time you are finished talking, type RET twice.
posted by Joey Michaels at 11:43 AM on February 15, 2011 [1 favorite]

Um, so what's the difference between a sockpuppet and a MeFite? What if we're all dangling on the ends of modfingers waving at each other?

On preview, RET RET
posted by infini at 11:46 AM on February 15, 2011

"gold" "medals" to be awarded to the "winners".

explosion: "Additionally, if a computer ranks higher than any of the humans, it will be given the Voight-Kampff Award, and declared "More Human Than Human.""

This prize will be a solid gold origami swan.
posted by boo_radley at 12:15 PM on February 15, 2011 [1 favorite]

Having interacted with members of the AI community at various points, and noting that the judges seem to hail from same, I wonder what, if any, bias this introduces into the competition. Awarding the Loebner prize to the program that appears the most human to a group of AI researchers is a different criteria than awarding the prize to the most human program.

Not a knock against the social skills of AI researchers, mind (although I have met some...whooboy), merely noting that having a high level of sophistication in one's thinking about AI may cause one to perceive programming "tricks" that approximate human capabilities as being cues that the entity on the other end is just a program. "Well, clearly that can't be a person; it's simply using a rapid Bayesian approximation backed up by a large knowledge base and some cleverly chosen priors to infer context and meaning in order to generate an appropriate response which meets a given confidence threshold."
posted by logicpunk at 12:36 PM on February 15, 2011

Tell me more about that, Greg Nog.
posted by nickmark at 12:39 PM on February 15, 2011 [2 favorites]

Most subjects thought the other person was a jerk, shy, etc. About 10% suspected there was no person. The 10% were correct. The other person was a toaster.

And that other person never responded throughout the whole test? How did they convince the person sitting down that the other person even existed? Did they just tell him he was there? I'm very temped to call shenanigans here.

In fact, shenanigans.
posted by JHarris at 12:41 PM on February 15, 2011

Not a knock against the social skills of AI researchers, mind (although I have met some...whooboy), merely noting that having a high level of sophistication in one's thinking about AI may cause one to perceive programming "tricks" that approximate human capabilities as being cues that the entity on the other end is just a program. "Well, clearly that can't be a person; it's simply using a rapid Bayesian approximation backed up by a large knowledge base and some cleverly chosen priors to infer context and meaning in order to generate an appropriate response which meets a given confidence threshold."

Have you read the transcripts? This is an excerpt from one of the conversations from the latest "winner":

bot: I have a coffee in my garden.
judge: Coffee beans, or just coffee?
bot: My favorite spice flavoring is vanilla. I love it in cookies and ice cream.
judge: But what about your garden?
bot: Oh, nothing.
bot: A man walks into a doctor's office with a cucumber up his nose, a carrot in his ear and a banana in his right ear. "what's the matter with me?" he asks the doctor. The doctor replies, "you're not eating properly."
judge: Well, I believe that is worth a "LOL".
bot: Should I know that?
judge: Possibly, it is used widely in popular culture nowadays.
bot: Glitterati are just silly things to care about.
judge: Glitterati? Are you talking about celebrity culture?
bot: What about Glitterati?
judge: Nevermind.

You don't need to know anything about machine learning or neural nets or whatever to realize when your conversation partner is spewing vaguely-related gibberish rather than actually answering your questions in any recognizable way.
posted by burnmp3s at 1:04 PM on February 15, 2011 [1 favorite]

Come, come, elucidate your thoughts.
posted by orthogonality at 1:05 PM on February 15, 2011 [3 favorites]

In sum: the results of these tests will be very different if the human you get as the conversational control is, say, William F. Buckley or Snooki.
posted by one_bean at 1:08 PM on February 15, 2011

I imagine the conversation between William F. Buckley and Snooki would end with each of them convinced the other is a barely-functional AI.
posted by Pickman's Next Top Model at 1:10 PM on February 15, 2011 [15 favorites]

You don't need to know anything about machine learning or neural nets or whatever to realize when your conversation partner is spewing vaguely-related gibberish rather than actually answering your questions in any recognizable way.

You haven't read some of my students' essays, then.
posted by joe lisboa at 1:11 PM on February 15, 2011 [5 favorites]

I imagine the conversation between William F. Buckley and Snooki would end with each of them convinced the other is a barely-functional AI.

What if its two educated, widely read, beanplating overthinkers, what would happen then?
posted by infini at 1:15 PM on February 15, 2011

You don't need to know anything about machine learning or neural nets or whatever to realize when your conversation partner is spewing vaguely-related gibberish rather than actually answering your questions in any recognizable way.

Sounds like a presidential debate.
posted by MtDewd at 1:27 PM on February 15, 2011 [1 favorite]

Um, so what's the difference between a sockpuppet and a MeFite? What if we're all dangling on the ends of modfingers waving at each other?

Does that question interest you?
posted by Joey Michaels at 1:28 PM on February 15, 2011 [1 favorite]

I think the secret of a human-sounding AI might just be complaints about health issues or financial woes. Sprinkle a few,"Yeah, I should see a doctor about my back but I can't afford it just now" remarks in there and you would at least sound like an American human nowadays.

Categorized under: Yet Another Thing Learned from AskMe, which could make up an entire book.
posted by misha at 1:30 PM on February 15, 2011

"Sir, are you classified as human?"

"Negative. I am a meat popsicle."
posted by brundlefly at 2:42 PM on February 15, 2011 [3 favorites]

The Turing Test is stupid. Or rather, it's stupid that they don't always choose judges who understand how to tell the difference. A reasonably smart programmer should never be fooled except by a true intelligence. Why? Because emotions and casual babble and all that are not the unique part of humanity. The unique part is our logical, literal, abstract reasoning ability. That isn't the romantic part, but it's the part machines don't do (and also the part that animals don't do).

So I would just ask a direct, logical question whose answer requires a little explanation, and then sphexishly (but as politely as possible) insist that that question be answered before continuing the conversation. I would even explain that a chatbot would be evasive and that's why I was asking. As soon as I got a direct answer, I would know it was a human. There's no way for a computer to fool that kind of judge. The only way it fails is if the confederate absolutely refuses to cooperate despite all explanation and pleading.
posted by Xezlec at 3:00 PM on February 15, 2011

The Most Human Human

The contest was so much better back in the '60s and '70s, when Bert Parks would sing to the winner.
posted by PlusDistance at 3:11 PM on February 15, 2011

The Harvard psychologist Daniel Gilbert says that every psychologist must, at some point in his or her career, write a version of what he calls “The Sentence.” Specifically, The Sentence reads like this: The human being is the only animal that ______.

Well, that's easy. The human being is the only animal that I am willing to make the sex with.

Done and done.
posted by evidenceofabsence at 3:37 PM on February 15, 2011 [1 favorite]

Well, that's easy. The human being is the only animal that I am willing to make the sex with.

So... for you, a vibrator passes the Turing Test?
posted by Xezlec at 3:41 PM on February 15, 2011

PlusDistance:...when Bert Parks would sing to the winner.

Don't you mean Rob Zombie?
posted by evidenceofabsence at 3:41 PM on February 15, 2011

The human being is the only animal that thinks it's not like the other animals.
posted by norm at 3:45 PM on February 15, 2011 [1 favorite]

A favorite part of the Atlantic piece:

In May 1989, Mark Humphrys, a 21-year-old University College Dublin undergraduate, put online an Eliza-style program he’d written, called “MGonz,” and left the building for the day. A user (screen name “Someone”) at Drake University in Iowa tentatively sent the message “finger” to Humphrys’s account—an early-Internet command that acted as a request for basic information about a user. To Someone’s surprise, a response came back immediately: “cut this cryptic shit speak in full sentences.” This began an argument between Someone and MGonz that lasted almost an hour and a half. (The best part was undoubtedly when Someone said, “you sound like a goddamn robot that repeats everything.”)

Returning to the lab the next morning, Humphrys was stunned to find the log, and felt a strange, ambivalent emotion. His program might have just shown how to pass the Turing Test, he thought—but the evidence was so profane that he was afraid to publish it.

Humphrys’s twist on the Eliza paradigm was to abandon the therapist persona for that of an abusive jerk; when it lacked any clear cue for what to say, MGonz fell back not on therapy clichés like “How does that make you feel?” but on things like “You are obviously an asshole,” or “Ah type something interesting or shut up.” It’s a stroke of genius because, as becomes painfully clear from reading the MGonz transcripts, argument is stateless—that is, unanchored from all context, a kind of Markov chain of riposte, meta-riposte, meta-meta-riposte. Each remark after the first is only about the previous remark. If a program can induce us to sink to this level, of course it can pass the Turing Test.

Once again, the question of what types of human behavior computers can imitate shines light on how we conduct our own, human lives. Verbal abuse is simply less complex than other forms of conversation. In fact, since reading the papers on MGonz, and transcripts of its conversations, I find myself much more able to constructively manage heated conversations. Aware of the stateless, knee-jerk character of the terse remark I want to blurt out, I recognize that that remark has far more to do with a reflex reaction to the very last sentence of the conversation than with either the issue at hand or the person I’m talking to. All of a sudden, the absurdity and ridiculousness of this kind of escalation become quantitatively clear, and, contemptuously unwilling to act like a bot, I steer myself toward a more “stateful” response: better living through science.

posted by AceRock at 3:47 PM on February 15, 2011 [11 favorites]

Xezlec- Vibrators aren't animals. Well, at least, vibrators shouldn't be animals.
posted by evidenceofabsence at 4:01 PM on February 15, 2011

After reading through the Reddit thread on Cleverbot, I was surprised it didn't win. It is so clever!
posted by isnotchicago at 5:34 PM on February 15, 2011

The human being is the only animal that thinks it's not like the other animals and thus is not.
posted by joe lisboa at 11:19 AM on February 16, 2011 [1 favorite]