After each session Pagulayan analyzes the data for patterns that he can report to Bungie. For example, he produces snapshots of where players are located in the game at various points in time — five minutes in, one hour in, eight hours in — to show how they are advancing. If they're going too fast, the game might be too easy; too slow, and it might be too hard. He can also generate a map showing where people are dying, to identify any topographical features that might be making a battle onerous. And he can produce charts that detail how players died, which might indicate that a particular alien or gun is proving unexpectedly lethal or wankishly impotent.
The lab also records video footage of every testing session and hyperlinks these clips to the individual progress reports. If the design team wonders why players are having trouble in a particular area, they can just pull up a few test games to see what's going wrong. Take what happened last March: A report noted an unusual number of "suicides" among players piloting the alien Wraith tank in an upper level. After watching dozens of archived test games, Griesemer spotted the problem. The players were firing the tank's gun when its turret was pointed toward the ground, attempting to wipe out nearby attackers. But the explosion ended up also killing (and frustrating) the player. To prevent this, Griesemer reprogrammed the tank so that the turret couldn't be lowered beyond a certain point. The Wraith suicides stopped.
A similar report showed that in the game's first level, called Jungle, players often ran out of rounds for their rifles. This was a mystery, because the designers had been careful to leave more than enough ammunition lying around. The team checked Pagulayan's video records and found that people were firing at the aliens when they were too far away, misjudging the range of the weapon and wasting bullets.
At first the designers couldn't figure out how to fix this problem. But then Griesemer stumbled on an elegant hack: He made the targeting reticule turn red when enemies were in range, subtly communicating to players when their shots were likely to hit home. It worked.
The ideal in gameplay, the goal every developer aims for, is an experience that keeps players in a "flow" state — constantly surfing the edges of their abilities without bogging down. Modern videogames are often compared to Hollywood movies, but the comparison, many Bungie designers will tell you, is inaccurate. A movie is static. "You sit there and absorb it all in a single two-hour shot, and it's perfectly linear," says Frank O'Connor, one of the writers tasked with scripting the story line in Halo 3.
Creating a game, in contrast, is like a combination of architecture — constructing environments that influence the behavior of people inside them — and designing a new sport. Gamemakers have to devise a system of rules and equipment that gives players a few basic goals and then allows them to find their own ways of achieving those goals. The flow comes from constantly discovering innovative ways to solve these open-ended problems.