PC Games Teach Soldiers Foreign Tongues
March 9, 2004 -- Military computer simulations are teaching soldiers how to triumph in battle against foreign combatants. But winning the peace afterwards may require a new computer game — one that helps train fighting forces to have friendly chats with the natives.
Computer science professors at the University of Southern California, with funding from DARPA, have been working on a simulation program designed to help military personnel perform a more prevalent — and difficult — task in the international war on terrorism: communicating peacefully and correctly with foreigners in their own native tongues.
By knowing the local lingo, "peace-keeping" and "nation-building" tasks — locating suspected terrorists, helping to rebuild roads and local infrastructure, gaining the trust of local leaders, and so forth — would be a lot easier to accomplish in places such as Iraq and Afghanistan. And the idea, says Lewis Johnson, director of the Center for Advanced Research in Technology for Education (CARTE) at USC, was that computer games, programmed with artificially intelligent "agents" could help soldiers develop those much needed linguistic abilities.
"We proposed that to DARPA," says Johnson. "They were interested in supporting new training technology and recognized that [foreign] language skills was a big gap right now."
For over a year, Johnson and his team along with input from instructors at the U.S. Military Academy at West Point, N.Y., have worked on developing the idea of a high-tech language simulator. The result: The Tactical Language Training System.
Speak to Me Software
The program is based on the graphics capabilities of Unreal Tournament, a consumer computer game that has been popular with game players for its team-based approach to virtual combat.
But, Johnson and his team of researchers have tweaked the game by adding a "speech recognition" engine and their own "intelligent agents," software code that "reacts" to how a user speaks and what he says.
The first part of the game, says Johnson, acts as basically an "intelligent tutoring" program. Players work on their Arabic language skills, for example, by speaking words and phrases into a computer's microphone. The software then corrects them on pronunciation and other mistakes.
Once the soldiers are comfortable with their basic linguistic skills, the second half of the game puts their knowledge to the test with several "missions."
"Players are given objectives they must achieve… ask a local for directions, or communicate that they are there to remove a land mine — if they can be told where the bombs are located," says Johnson. "Soldiers speak into a microphone, and the software's speech recognition program analyzes what the soldier's saying to provide [the appropriate] response."
Learning What Soldiers Know
But what makes the program really "intelligent" are the computer-generated and -controlled characters, such as a virtual village leader and a virtual "team member" that acts as an in-game guide. These game characters are programmed to react in ways that are unique to each individual user.
"As you run the program, it develops a knowledge of what you know, what kind of help you get from virtual guide, also an effect on difficulty," says Johnson. Beginner students, for example, might go into a town in the game and the local people pretty cooperative to the soldier's questions. "But once they master that, the locals are more suspicious and ask a lot of question and [soldiers] have to do a better job of allaying [the virtual villagers'] fears."
When students get stumped or are having trouble making headway, the virtual guide may step in and provide clues on what a user should say and how to word the proper questions.
"Every point along the way, they get immediate feedback on learning," says Johnson. "One-on-one speaking is clearly the best way to learn [foreign language skills]. But very few get an opportunity to do that. Classrooms don't do that because it's one teacher to many students. Even in the real world, it's not a great environment. You can talk to people, but they won't tell your mistakes."
Keeping Cadets Engaged
Col. Stephen LaRocca, director of the Center for Technology-Enhanced Language Learning at the U.S. Military Academy, says USC's prototype language system could be an extremely useful tool for the military.
LaRocca says that computer tools have been tried before to help bolster the foreign language skills of cadets during their four-year course of studies at the academy. A rudimentary multimedia teaching tool called "Microworld" had used some of the basic elements of speech recognition and computer graphics to help students grasp the proper way to speak Arabic. But the USC program goes much further than anything they've tried before.
"They did a nice job, a difficult job of software integration and a [speech] recognizer," says LaRocca. "It measures when a student is not succeeding and will slow down or change [the lesson] to something different."
And during an impromptu testing period at the military academy, the game aspect really kept cadets interested in the difficult language tasks.
"We videotape one cadet using it and noted that — as many have hypothesized — that motivation is key to learning," says LaRocca. "And there is something motivating about the video game format. "
Nothing Beats a Human?
LaRocca and Johnson's USC team plan on conducting further tests with more cadets at West Point later this month.
And while LaRocca is confident that the tool could be a great aid for future soldiers, the software won't replace human teachers and classroom instruction anytime soon. Part of the reason is that the program, so far, is very limited in scope.
"What this program aspires to do is isolate a task — request directions, understand what you've been told and then executing on what you know," says LaRocca. "It's rehearsing reactions in interacting."
"This is exciting in the possibility it brings us," says LaRocca. "But we haven't ever yet seen [a program] as good as a human."