Data, Commander

Section: 

Fifty-one hours and 34 minutes into Just Cause 2, I’ve completed 55.16% of the game. Of the 369 locations I’ve discovered throughout Panau—that’s all of them, by the way—I’ve fully completed 111. I’ve assassinated 26 out of 50 colonels, collected 1,131 of 2,700 resource items, and killed 2,571 enemies. Oh, and I’ve destroyed 134 gas pumps.

Somewhere in there, I think I finished the story.

In my 22 hours and 45 minutes with Assassin’s Creed: Brotherhood, I’ve managed to rebuild 67% of Rome. Although my total sync progress is still only a paltry 23.76%, I have destroyed nine of 12 Borgia Towers, synchronized 18 of 24 viewpoints, and collected 44 of 144 treasures. If I want to take the coveted top spot on the Courtesan’s Guild board, I’ll need to commandeer 10 guards’ horses without killing their owners. Apparently Rome’s prostitutes have a pressing interest in illicit equestrianism.

Pick a game for a current-gen system, and more than likely it’ll be able to spit out a ream of statistics from its pause menu. Time played. Percentage of objectives completed. Portion of map discovered. There’s a relentless, almost fetishistic focus on making gameplay data explicit for the player. It’s not uncommon to see, having collected an item, a little box pop up in the corner of the screen informing you exactly which of the 523 total items you’ve just found. The game interrupts itself to deliver tracking data. It’s like a teacher hovering over your shoulder mid-test, announcing the impact of every right or wrong answer on your GPA.

From a developer perspective, it makes perfect sense for games to track player behavior in such minute detail. Gameplay metrics can be enormously useful tools for analyzing what’s gone well and what’s gone poorly. In this era of The Patch, The DLC and The Sequel, that’s not necessarily a bad thing. Where else should developers look for ideas on how to improve their games? As a player, I want gameplay data to be the starting point. Cut the features that aren’t getting used, and bolster the ones that are.

And why not make at least some those metrics explicit in-game? They can be an efficient way of showing the player the wealth of things to do in the game world. Especially in open-world games, it’s crucial to make the player aware of all the potential objectives. Apart from the rare phenomenon like Minecraft, “freedom” only takes you so far. Measurable goals make that prized sense of progression less nebulous, and, if implemented well, can encourage players to more fully explore the game world. To some degree, the focus on data is the logical extension of the High Score; it’s an incentive to master the game as thoroughly as possible.

But here’s the thing about data: It’s seductive. The more information you have at your disposal, the better you can maintain the feeling of understanding, of influence, of control. But no matter how facile our access to information has become, we still face the problem of figuring out what it means. And what to do with it.

If the 21st century has taught us anything so far, it’s that information—and the ease with which it can be stored, analyzed and shared—is the key to power. In a year in which Twitter is helping topple regimes and WikiLeaks is dominating the foreign-policy conversation, it’s easy to see why people are so invested in collecting and sharing data. It’s democratic. It’s empowering. (Case in point: my mother-in-law signing up for Facebook. Talk about your Shadow Brokers.)

Just as in the political sphere, though, an abundance of data in a game does not equate to understanding or value. When you pack your game with hundreds of side-quests and sub-objectives—and especially when you display the mechanisms for tracking progress against these goals prominently in the user interface—you run the risk of devaluing your narrative. You could be awarding priority to behaviors that are not necessarily essential to the core experience. You’re in danger of sending the player mixed messages on what your game is really “about.” If you look at my gameplay stats for the original Assassin’s Creed, for example, you’d wonder why they decided to give that strange title to a game about collecting flags.

More importantly, by smothering players with metrics, you’re endangering your value proposition. At a certain point, the allure of the potential fun ahead symbolized by those incomplete objectives turns to indifference or antipathy. By overwhelming players with data on all the objectives they’ve failed to accomplish (or could not accomplish without an additional hundred hours of play), you’re risking making players feel the opposite of mastery. Players are not necessarily hardwired to filter out what data matter; some even feel guilty about not achieving a perfect completion percentage. In those cases, filling in the gaps in the data set becomes the goal, not the actual game experience itself. Like teachers who give every classroom activity a point value, you’re confusing the reward system with the reward. The reward for working hard in school is not the grade, which is an inherently meaningless thing. It’s the gains in knowledge and skill that the grade reflects. At least, that’s how it should work.

Naturally, it’s not quite that simple. Red Dead Redemption is an interesting example. Without the incentive of the various “challenges” (hunting, collecting flowers, etc.), I wouldn’t have explored the game world nearly as thoroughly as I did. I wouldn’t have bothered to watch the sun rise in golden splendor over New Austin if I wasn’t out in the hills picking flowers. My favorite parts of RDR were those moments that transcended the story, that let me experience this beautifully rendered, fictional landscape for what it was, without needing some arbitrary goal to work toward. Ironically, it was the arbitrary goal that got me there in the first place. Sort of like how I only came to realize grades weren’t important after I’d spent twelve years killing myself for straight A’s.

Just as we do with other metrics, we have to learn to read performance data in games as a reflection of value, not the other way around. As a reformed completionist, I sometimes struggle with separating data from meaning. But it’s getting easier as I force myself to think more critically about what matters to me in each game I play. While it’s interesting to know my accuracy with the Springfield rifle was 81.8% in RDR, it’s more meaningful to know I’d gotten better at shooting by the time I acquired that weapon. My accuracy was a reflection of my growing mastery, not an end in itself.

Come to think of it, though, there was at least one meaningful statistic in that game: I apparently harvested 8,641 pounds of meat. Suck on that, Oregon Trail.

Comments

Welcome, new kid!

There's a lot to talk about here, but I will have to collect my notes before venturing forth.

Great article! I really enjoy your punchy style.

I feel guilty, because I am one of those gamers that developers are pandering to with their stat tracking. I really like knowing that I have cast 5 of the 15 possible spell combination in Fable 3, and when I get them all - PLOCK! Achievement unlocked.

That said - it is a refreshing change when a game does away with rewarding (or in my case, forcing) exploration. I think that is one of the things I liked about Enslaved so much - the game was more about the experience than the gameplay, and I didn't need to worry about the achievements.

Excellent observation. I've personally learned to filter that stuff out if it's a game I don't care about. For example, in Mass Effect, I didn't care about doing the side missions. Similarly, in Red Dead Redemption, I didn't bother doing those extra challenges. Whereas you give me a platformer like Mario Galaxy or Donkey Kong Country Returns and you better believe I'll be tracking down all 743 hidden widgets and completing every level backwards with one life. It's just certain types of games that drive me to be a completionist.

One example of data leading the player in interesting directions comes from Deadly Premonition (I loved your blog posts on that game, by the way). The game presents players with a list of side missions, when they'll be available, and who will trigger them.

Like the challenges in Red Dead Redemption, the side missions in Deadly Premonition push the player toward content they might not otherwise experience. What's different here is that the player's gains in Deadly Premonition are often not the quest rewards themselves, or even the satisfaction of having completed all of the side quests (there's no Achievement), but the additional character development and world building built into the quests. The back stories of not only the side characters around town but the main characters are fleshed out by the largely inconsequential errands you're asked to run for them—errands that you could easily miss if not for the game making you explicitly aware of them.

Similarly, some of the most interesting content in Dante's Inferno comes by way of the Damned, characters you're not required to seek out but are encouraged to by the game's stat tracking (and Achievements).

Welcome JP. Great piece -- lots to think about, like I said behind the veil. I'll stop back later to see what people took hold of.

DorkmasterFlek wrote:

Excellent observation. I've personally learned to filter that stuff out if it's a game I don't care about. For example, in Mass Effect, I didn't care about doing the side missions. Similarly, in Red Dead Redemption, I didn't bother doing those extra challenges. Whereas you give me a platformer like Mario Galaxy or Donkey Kong Country Returns and you better believe I'll be tracking down all 743 hidden widgets and completing every level backwards with one life. It's just certain types of games that drive me to be a completionist.

I find this really interesting, as Nintendo is the one major platform that doesn't do achievements / trophies.

Great piece. Love the reference to Oregon trail! I think I may have to go over to my must play pile and start a Red Dead Redemption game now...

One of the main reasons I kept playing Stargate Resistance long past the flat line announced, was a fan made stat site that kept track of my head shots, staff weapon battery kills, etc.

As much as I hate achievements, sometimes they drive you to push a bit harder while playing and result in that sun set you refered to.

Clemenstation wrote:

I find this really interesting, as Nintendo is the one major platform that doesn't do achievements / trophies.

They don't have achievements/trophies in any sort of centralized system, but Mario Galaxy 2 does a fantastic job of letting you know how much of the game you've played and offering incentives to continue. They also prompt you to explore the game's mechanics and locations in much the same way that achievements do, but rather than awarding you with an achievement for a particular accomplishment, that accomplishment is built into a course that rewards you with one of the game's gold stars.

For example, an achievement you might be unsuprised to see in a 360 game might be for beating a boss or completing a level without taking any damage. Mario Galaxy 2 has optional challenge courses that require you to beat a boss or play through a level with only one hit point, which is functionally the same task. For finishing that challenge level, you get a gold star.

Likewise, after you've gotten all the gold stars, you unlock the green stars which are hidden throughout the games various levels. They're hidden in parts of the level geometry you didn't know you could access and/or require you to use some of the game's mechanics in unexpected ways. Where another game might give you an achievement for finding the hidden doodad in a level, Mario Galaxy 2 makes that doodad and the reward for finding it the same thing.

Your star count essentially becomes your gamer score, but it's more holistically integrated into the game than achievements and trophies typically are.

Welcome! Wonderful read, and something I thought long and hard about almost 1 year ago. It was on a grand day in December when I decided I didn't care about completionist achievements. But I will admit, your example of finding rare moments in a game you wouldn't have seen otherwise if you hadn't been running the completionist route is food for thought.

Different strokes. I've played almost exactly the same amount of hours in Just Cause 2, but have around half your completion rate, probably because I spent most of the time trying to drive motorbikes down from the highest mountain as fast as possible without crashing.

Great first article, welcome to the fold!

The obsession with measurables outside of sports games and high score chases has begun to strike me as a good idea co-opted by cynics and turned into something crass and ugly. The constant numbers and comparisons flashing on the screen seem to be reaching beyond my frontal lobe and hitting the lizard part of my brain that associates progress with survival. I'm becoming desensitized to the numbers though -- making them more of an annoying encumbrance than anything that gives me much pleasure.

Even Bulletstorm, a game that tightly integrates flashing numbers and totals into the design in ways Assassin's Creed never has, doesn't quite sit well with me. Like we're not willing to be creative with our problem solving unless there's a steady reward IV drip drilled into our arms.

Certis wrote:

The constant numbers and comparisons flashing on the screen seem to be reaching beyond my frontal lobe and hitting the lizard part of my brain that associates progress with survival. I'm becoming desensitized to the numbers though -- making them more of an annoying encumbrance than anything that gives me much pleasure.

This is where Assassin's Creed Brotherhood finally lost its hold on me to compel my play with stats. It was tracking my completion percentage to four significant digits—that's more precise than an IGN review score. And then substantial progress in the main story would barely tick that number any higher! At that point the stat became meaningless. I finished the story and liberated 100% of Rome and my total sync is only 50.74% (or whatever it is)? That might as well be 100—or zero—for all I care. Or more importantly, for all I actually accomplished in the game and felt good about.

Good article.

Well said throughout! I really like the idea of developers doing more meaningful things with stats, especially showing player improvements over time, rather than just raw percentages/numbers. As an example, I've started playing Marvel Pinball lately and would feel a much better sense of accomplishment from it if I could track the progress of my own scores on each table over time, rather than against friends or world leader boards (on which I know I'll never be able to compete at a meaningful level).

But I suspect this kind of stat tracking over time multiplies the code complexity and processing required, so may not be worth the investment to most developers.

As to the broader issue of "information literacy" - I wholeheartedly agree! Most people have no idea how to properly interpret and synthesize the information surrounding them every day into meaningful knowledge. In democratic, information-heavy societies like the US and UK, this kind of skill is crucial. This is part of the reason why, as a college librarian, I have a keen interest in gaming as a learning tool. Research continues to show that well crafted games like WoW, etc can enable an environment where people WANT to learn about the context of the information and outcomes they are seeing and are even willing to go to long lengths to master that information and solve problems. This is a learning environment most teachers would love to be able to tap into.

Again, thanks for the great post!

I guess I am the only person confused about the intent of your article. Ou start out by saying that data sulleys the value prop, then go on to say how it has informed your play and opened up new experiences. You are neither for nor against.

Data like this I can remember having access to all the way back in Fallout. It was there, but there was no ding and graphic to show me as I progressed. If it's the alerts that bother you, turn 'em off. Or ignore them. Have a little self control. If a narrative doesn't speak to you, it's prolly not just because you have gamer OCD.

TheWanderer wrote:

I guess I am the only person confused about the intent of your article. Ou start out by saying that data sulleys the value prop, then go on to say how it has informed your play and opened up new experiences. You are neither for nor against.

Data like this I can remember having access to all the way back in Fallout. It was there, but there was no ding and graphic to show me as I progressed. If it's the alerts that bother you, turn 'em off. Or ignore them. Have a little self control. If a narrative doesn't speak to you, it's prolly not just because you have gamer OCD.

What I take from the article is the disconnect between what is directly in the game, and a bunch of numbers in a menu or as a HUD pop-up. Because games are full of little systems and data collection, it must seem easy to make a few of them of them visible to the player, but a number doesn't really mean much to you as you're playing.

I suppose you could compare it to money. Money can be just a number on a page or screen, but it also has a meaning, how much your employer values your services, a roof over your head, food on the table, being able to entertain yourself with games.

Actions in a game need to do more than increase a number (see also: achievements), they need to have meaning within the game. If I shoot someone, are they gone forever or is there just more cannon fodder going to come out of the clown car spawn closet? If I level the military base in JC2, does it affect the region or just increase that counter?

I enjoyed the article! Good job.

Certis wrote:

Great first article, welcome to the fold!

The obsession with measurables outside of sports games and high score chases has begun to strike me as a good idea co-opted by cynics and turned into something crass and ugly. The constant numbers and comparisons flashing on the screen seem to be reaching beyond my frontal lobe and hitting the lizard part of my brain that associates progress with survival. I'm becoming desensitized to the numbers though -- making them more of an annoying encumbrance than anything that gives me much pleasure.

I'm surprised nobody's talked about the giant feedback in fighting games. You get combo count numbers as big as your character, over-the-to animations, and across the screen another character-sized block of text screaming "Awesome!" You almost have to remind yourself to look at the top of the screen to see what's really going on.

disobedientlib wrote:

As to the broader issue of "information literacy" - I wholeheartedly agree! Most people have no idea how to properly interpret and synthesize the information surrounding them every day into meaningful knowledge. In democratic, information-heavy societies like the US and UK, this kind of skill is crucial. This is part of the reason why, as a college librarian, I have a keen interest in gaming as a learning tool. Research continues to show that well crafted games like WoW, etc can enable an environment where people WANT to learn about the context of the information and outcomes they are seeing and are even willing to go to long lengths to master that information and solve problems. This is a learning environment most teachers would love to be able to tap into.

This is a huge issue, and I'd like to talk more about it, but I don't have much to add other than my agreement and a stack of links and literature to back you up.

TheWanderer wrote:

I guess I am the only person confused about the intent of your article. Ou start out by saying that data sulleys the value prop, then go on to say how it has informed your play and opened up new experiences. You are neither for nor against.

It's both. All that extra data can be a very useful tool, but it can also be overused or used improperly, to the detriment of the game.

Thanks so much for the warm welcome, all! Thanks, also, for not complaining that this article wasn't actually about Star Trek. (But speaking of which...Wordy, let's talk.)

I do want to respond to all your thoughtful comments, but I won't have much time until later this weekend. I'll save the brunt of my responses for later, but I do want to add a few quick thoughts while I have a moment:

Certis wrote:

The obsession with measurables outside of sports games and high score chases has begun to strike me as a good idea co-opted by cynics and turned into something crass and ugly.

I'm glad you mentioned this. There's an element to this trend that reeks of marketing. Check out this RDR one-sheet (linking to it because the image is gigantic). Can't help but wonder if some of the metrics tracked in games are inserted just so marketing departments can have fuel for collateral. Not sure otherwise why it should matter to me that I traveled 13.92 miles by stagecoach.

Certis wrote:

Like we're not willing to be creative with our problem solving unless there's a steady reward IV drip drilled into our arms.

Yeah, that's where things get tricky for me. I appreciate what The Wanderer is saying about simply ignoring the data; I usually do. If I get curious, it doesn't ruin my game experience to sneak a peek at the stats page. The problem I'm trying to articulate is more along the lines of what Certis says here, and what Sally's Enslaved example above was getting at. What's really valuable about the experience? Does data confuse that? Often, I think it does.

I think Just Cause 2 pulls off data-as-experience pretty elegantly, actually. Because its core mechanics are so fun, because its environments are so pretty, and because the game makes no pretense of a meaningful narrative, I'm perfectly comfortable just tooling around Panau. In this case, I dig the stat tracking, because quantifying the already frivolous experience doesn't diminish it any further. It almost legitimizes it, if that makes any sense.

Clemenstation wrote:
DorkmasterFlek wrote:

Excellent observation. I've personally learned to filter that stuff out if it's a game I don't care about. For example, in Mass Effect, I didn't care about doing the side missions. Similarly, in Red Dead Redemption, I didn't bother doing those extra challenges. Whereas you give me a platformer like Mario Galaxy or Donkey Kong Country Returns and you better believe I'll be tracking down all 743 hidden widgets and completing every level backwards with one life. It's just certain types of games that drive me to be a completionist.

I find this really interesting, as Nintendo is the one major platform that doesn't do achievements / trophies.

Somehow Nintendo makes it fun. I enjoyed all 240 stars in Mario Galaxy and all 240 in the sequel. But I loathed dozens of the repetitive sidequests and item hunting in the first Mass Effect for 100% achievements. If there wasn't an achievement I never would have visited every damn planet in ME1. But I do so in Galaxy because the gameplay is varied on each world and a new challenge awaits around every star.

As a lifetime sports fan who loves stats, digging into them, calculating them, everything about them.... I enjoy stats in my games too. Achievements I still have a love/hate relationship with, depending on how ludicrous they are.

Certis wrote:

Even Bulletstorm, a game that tightly integrates flashing numbers and totals into the design in ways Assassin's Creed never has, doesn't quite sit well with me. Like we're not willing to be creative with our problem solving unless there's a steady reward IV drip drilled into our arms.

Some games (like Assassin's Creed, and Dead Space) situate the player's user interface into the fiction of the game world. AC has the memory machine, and Dead Space has health and kinesis levels on Isaac's backpack.

Bulletstorm is ludicrous, but there's a narrative reason for all those numbers -- albeit a sparse one. When Gruff Drunken Protagonist finds a leash early in the game, the UI kicks in and he says something shortly after along the lines of: "Hey! This stupid f*cking thing is evaluating my performance!" Amusing and eye-rollingly meta, but given the already 'cranked to 11' enthusiasm of the game, it actually works.

Later you find that there's a pretty ridiculous explanation for why earned points convert into upgrades/ammo:

Spoiler:

The super-soldiers who use the leashes are subject to a draconian system where those who excel are permitted to re-arm, while those who falter and fail are left to die. It is never explained why blasting someone up into the air and incinerating them is 'better' than a headshot, in any way -- but maybe I'm asking too much.

So under these circumstances, and given that the entire game's premise revolves around finding new and bizarre ways to kill folks, the feedback isn't unwarranted. I can certainly see how all those pop-up numbers might come across as overtly crass, however.

I find it weird that Civ 5 has gone the other way with the data. There seems to be even less stats than ever before. (Admittedly, it's been a while since I played a Civ game.) I really miss all the graphs and stats about how my civ fared compared to others throughout the game. Now all I get is best/worst/average stat. Very boring.

article wrote:

More importantly, by smothering players with metrics, you’re endangering your value proposition. At a certain point, the allure of the potential fun ahead symbolized by those incomplete objectives turns to indifference or antipathy. By overwhelming players with data on all the objectives they’ve failed to accomplish (or could not accomplish without an additional hundred hours of play), you’re risking making players feel the opposite of mastery.

This is exactly what happened to me with Batman:AA. The game emphasizes distractions to the point where they become the focus of the game. Completion is measured by how many hidden side-items you found in dead end ducts, rather than how many bad guys you beat up with your awesome batman skills. In fact, most of the rewards in this game come from collecting these irrelevant side items. Unfortunately, they are not straight forward to get. Many of them are in places inaccessible to you until you progress in your skills as Batman. So they keep rubbing in how you're not yet good enough to get things that you need to get to feel accomplished. It really got to me after a while. I completely lost taste for that game, no matter how well produced the main story line may have been, or how revolutionary the fighting system was. The game completely marginalized those aspects in my eyes and they won't be what I remember about that game.

MoonDragon wrote:

Completion is measured by how many hidden side-items you found in dead end ducts, rather than how many bad guys you beat up with your awesome batman skills. In fact, most of the rewards in this game come from collecting these irrelevant side items. Unfortunately, they are not straight forward to get. Many of them are in places inaccessible to you until you progress in your skills as Batman. So they keep rubbing in how you're not yet good enough to get things that you need to get to feel accomplished.

That sounds like a Metroid game. Have you played many of those (including similar titles like Shadow Complex and Symphony of the Night)? Did they make you feel the same way?