Wednesday, February 16, 2011

Watson and the Future of Everything

The world changed Monday night. Perhaps you missed it. Perhaps you were eating dinner or watching the news. Perhaps you were taking a walk or making love. Perhaps you were watching Jeopardy and did not fully appreciate the world changed before your very eyes.

On Monday, the two best Jeopardy players in the world squared off against IBM's Watson computer, and the world changed.

Let me begin by introducing the contestants.

On your far left is Ken Jennings. Ken won 74 Jeopardy matches in a row, and earned $2,520,700 in the process. He also earned $2,000 in his 75th game when he came in second to Nancy Zerg, who came in third during her next game. Jennings then won $500,000 when he placed second in the Jeopardy Ultimate Tournament of Champions. That's a grand total of $3,022,700.

It is almost statistically impossible that Ken Jennings won that many games in a row. Prior to Jennings' streak of 74 games in a row, the longest streak was a pitiful (only by comparison) 8 games in a row by Tom Walsh. Jennings streak was more than 9 times longer than Walsh's streak. By comparison, Joe DiMaggio's 56 game hitting streak, considered by many to be the greatest feat ever in baseball, is only 1.3 times as long as Pete Rose's 44 game streak, second longest in the modern era.

On your far right is Brad Rutter. Brad Rutter has never lost a Jeopardy game to another human. When Rutter first appeared on Jeopardy in 2000, contestants were limited to five games in a row. (That rule was eliminated in 2003, one year before Ken Jennings began his remarkable run.) Rutter, however, later won each of the three Tournaments of Champions in which he was invited to play.

In the third of those tournaments, the Ultimate Tournament of Champions, Rutter beat Ken Jennings and took home $2,100,000 for his efforts. In all, Rutter has earned $3,255,102 for his initial five Jeopardy games and his three Jeopardy tournaments, making him the top Jeopardy earner of all time.

In the center is Watson, a massively parallel IBM supercomputer.


Born in 2007, Watson has (at the hands of some amazing teachers) learned to understand the subtleties of natural language. Consider Watson's rapid fire questions to some of Jeopardy's ambiguous, nuanced questions in the practice session recorded before the formal contest.


Since Jeopardy might present any of an infinite number of convoluted answers to the contestants, there was no way Watson's teachers could simply program it to select the right question from a database of question/answer pairs. They had to teach Watson to understand natural language. That was the challenge. That's why they created Watson. It was to be the first computer to understand the intricacies and subtleties of natural language well enough to compete on Jeopardy.

Watson's knowledge is kept in much the same way as Google stores all its data. When faced with a Jeopardy answer, Watson uses keywords, much as Google does, to collect a long list of possible answers from its stored information.

It is at this point, however, that IBM has gone beyond Google and put itself back into the lead as the world's foremost computing company. Watson is programmed to use its natural language skills to decide which of the many possible answers is most likely correct, and to answer only if it is confident beyond some threshold value.

On its web site, IBM provides a 22 minute video explaining how sophisticated Watson's decision-making can be. Consider the Jeopardy answer "In May 1898 Portugal celebrated the 400th anniversary of this explorer's arrival in India." The graphic below represents one possible result Watson might quickly find. Based on a keyword search, Watson finds a match for "celebrated", "may", "anniversary", "Portugal", and "India". Watson also finds that the link is associated with some guy named Gary.

As it turns out, "Who is Gary?" is unlikely to be the question Jeopardy was searching for.

The link segment reads "In May, Gary arrived in India after he celebrated his anniversary in Portugal."  As humans, we can easily see Gary is not who we are looking for, because we understand natural language. No computer, other than Watson, shares that understanding.

For Google, the Gary link would be a terrific match. For Watson, however, it would be but one of many to be rejected as improbable. Instead, Watson would home in on a Vasco de Gama link, which reads "On the 27th of May in 1498, Vasco de Gama landed in Kappad Beach."

The astonishing point here is that the Vasco de Gama link has only one matching keyword, and that keyword is the commonly used and ambiguous word "may." Holy linguistics, Batman! There are millions and millions of documents out there with the word "may" in them. Why did Watson decide to consider this one?

Watson picked up on this link because Watson has been programmed, as have we humans, to apply temporal reasoning, geospatial reasoning, and statistical paraphrasing.

While the Vasco de Gama link doesn't actually mention 1898 as being the 400th anniversary of anything, it does mention the year 1498. Watson knows that 1898 minus 1498 is equal to 400, and  therefore recognizes two very strong clues in the link, strong clues missing from the Gary link.

Furthermore, while the Vasco de Gama link doesn't actually mention India, Watson knows that Kappad Beach is in India. It knows that from a few of the other millions of links it has in its memory. So that's another big clue.

Finally, Watson understands statistical paraphrasing. It knows that "arrival in" is a good match of "landed in."

Watson combines all the information at hand and assigns a high probability that Vasco de Gama is the correct result. No other answer it considers has a higher probability. Since the Vasco de Gama probability is greater than the confidence threshold, it buzzes in and says, in the best computerized voice to date, "Who is Vasco de Gama?"

When Watson finds a reasonably probable answer, it typically buzzes in (using a mechanical finger) more quickly than its human competitors. This too is impressive, given all the mental work it has to do. I learn from the video that if Watson used my home computer as its processor, it would need several hours to decide each question. Watson, however, uses thousands of processors working at the same time, working in parallel.

So how did all this pan out in the actual game? As I said in the beginning, the world changed on Monday night. Watson finished the first game (played over two nights) with $35,734. Rutter had $10,400. Jennings had $4,800.

After three days and two games, the totals were:
Watson: $77,147
Jennings: $24,000
Rutter: $21,600

In my next post, I'll discuss what this might mean to the quarter million people we have wrongfully incarcerated in America.