Computers learn language (and world domination) by reading the manual
By Darren Quick
July 13, 2011
Researchers at MIT's Computer Science and Artificial Intelligence Lab have been able to create computers that learn language by doing something that many people consider a last resort when tackling an unfamiliar task - reading the manual (or RTBM). Beginning with virtually no prior knowledge, one machine-learning system was able to infer the meanings of words by reviewing instructions posted on Microsoft's website detailing how to install a piece of software on a Windows PC, while another was able to learn how to play Sid Meier's empire-building Civilization II strategy computer game by reading the gameplay manual.
Without so much as an idea of the task they were intended to perform or the language in which the instructions were written, the two similar systems were initially provided only with a list of possible actions they could take, such as moving the cursor or performing right or left clicks. They also had access to the information displayed on the screen and were able to gauge their success, be it successfully installing the software or winning the game. But they didn't know what actions corresponded to what words in the instructions, or what the objects in the game world represent.
Predictably, this means that initially the behavior of the system is pretty random, but as it performs various actions and words appear on the screen it looks for instances of that word in the instruction set as well as searching the surrounding text for associated words. In this way it is able to make assumptions about what actions the words correspond to and assumptions that consistently lead to good results are given greater credence, while those that consistently lead to bad results are abandoned.
Using this method, the system attempting to install software was able to reproduce 80 percent of the steps that a person reading the same instructions would carry out. Meanwhile, the system playing Civilization II ended up winning 79 percent of the games it played, compared to a winning rate of 46 percent for a version of the system that didn't rely on the written instructions.
What makes the results even more impressive for the Civilization II-playing system is that the manual only provided instructions on how to play the game.
"They don't tell you how to win. They just give you very general advice and suggestions, and you have to figure out a lot of other things on your own," said Regina Barzilay, associate professor of computer science and electrical engineering, who took the best-paper award at the annual meeting of the Association for Computational Linguistics (ACL) in 2009 for the software installing system.
"Games are used as a test bed for artificial-intelligence techniques simply because of their complexity," says graduate student S. R. K. Branavan, who along David Silver of University College London applied a similar approach to Barzilay in developing the system that learned to play Civilization II. "Every action that you take in the game doesn't have a predetermined outcome, because the game or the opponent can randomly react to what you do. So you need a technique that can handle very complex scenarios that react in potentially random ways," Branavan said.
Although the main purpose of the project was to demonstrate that computer systems that learn the meanings of words through exploratory interaction with their environments is a promising area for future research, Barzilay and Branavan say that such systems could also have more near-term applications.
Most computer games that let a player play against the computer require programmers to develop strategies for the computer to follow and write algorithms that execute them. Systems like those developed at MIT could be used to automatically create algorithms that perform better than the human-designed ones.
With such machine-learning systems also having applications in the field of robotics, and Barzilay and her students at MIT have begun to adapt their meaning-inferring algorithms to this purpose. Let's just hope they don't take the lessons learned playing Civilization II and try for the world domination win in the real world.
Source: MIT News