"We have not yet developed chair-tossing technology."
June 5, 2015 1:19 PM Subscribe
Computer science professor Jordan Boyd-Graber is currently working on a National Science Foundation grant for "Bayesian Thinking on Your Feet: Embedding Generative Models in Reinforcement Learning for Sequentially Revealed Data." At first glance, this might not sound like fun, but in the paper, Besting the Quiz Master, Boyd-Graber showed how machine learning could be used to create a quiz bowl version of the Terminator that can take all human comers. This weekend, that proposed machine finally played a nervewracking 200-200 tie game against a team of four Jeopardy! champions (Kristin Sausville of single contestant Final Jeopardy fame, teacher tournament winner Colby Burnett, professional poker player Alex Jacob, and underdog Tournament of Champions winner Ben Ingram).
That video was lots of fun. Fairly generous powers, but that makes sense if it was a high school packet (as you would expect at HSNCT). Lots of vultures from the human team, but it's not clear how much actual Quiz Bowl experience the players have, and one of the vultures was a power.
Incidentally there was a pretty good personal essay about quiz bowl that just recently came out. And man, I was never a great player but reading the linked Chicago Open packet was brutal. Never even heard of a lot of the tossup answers. CO was known as the hardest tournament out there but I don't remember it being that impossible. Of course I've forgotten tons of stuff since I played too.
posted by kmz at 2:05 PM on June 5, 2015 [1 favorite]
Incidentally there was a pretty good personal essay about quiz bowl that just recently came out. And man, I was never a great player but reading the linked Chicago Open packet was brutal. Never even heard of a lot of the tossup answers. CO was known as the hardest tournament out there but I don't remember it being that impossible. Of course I've forgotten tons of stuff since I played too.
posted by kmz at 2:05 PM on June 5, 2015 [1 favorite]
A distinct disadvantage that the human team has is that it can't pool its collective knowledge through telepathy.
posted by codacorolla at 2:08 PM on June 5, 2015
posted by codacorolla at 2:08 PM on June 5, 2015
A distinct advantage the human team has is that they can read and reason, rather than approximate them via probabilistic algorithms. I would like to see the algorithm play against a team of experienced quiz bowlers (not that the Jeopardy! champions aren't smart). I bet they'd wipe the floor with it.
I skimmed the paper and it looks to me like their algorithm is interesting but not a particularly surprising application of techniques in NLP and classification. One shortcoming of these approaches is the heavy dependence on the training corpus. The paper seems to indicate that the algorithm's effectiveness is highly dependent on the size and quality of the training data; any question lying outside the subject matter covered by the training data will likely produce nonsense answers from the computer player. I wonder if the questions used in the live demo were chosen from the traning corpus or not. I didn't watch the entire video but some of the wrong guesses (Monet for Manet, for example) seemed to be less nonsensical than others.
posted by axiom at 2:38 PM on June 5, 2015 [1 favorite]
I skimmed the paper and it looks to me like their algorithm is interesting but not a particularly surprising application of techniques in NLP and classification. One shortcoming of these approaches is the heavy dependence on the training corpus. The paper seems to indicate that the algorithm's effectiveness is highly dependent on the size and quality of the training data; any question lying outside the subject matter covered by the training data will likely produce nonsense answers from the computer player. I wonder if the questions used in the live demo were chosen from the traning corpus or not. I didn't watch the entire video but some of the wrong guesses (Monet for Manet, for example) seemed to be less nonsensical than others.
posted by axiom at 2:38 PM on June 5, 2015 [1 favorite]
« Older Migrant Worker Employment Standards at Patagonia | Evan Young gets his chance Newer »
This thread has been archived and is closed to new comments
posted by Wolfdog at 1:50 PM on June 5, 2015