Mitch Gitelman is not a happy camper.
The lead Shadowrun developer has been making the rounds on the podcast circuit, discussing in great detail why he feels his game is getting the short end of the stick. Gitelman lays a lot of the blame on the press, claiming that reviewers are being too harsh on his game and holding the title up to unfair standards. On the Official Xbox Magazine podcast this past Monday he points to middle-of-the-road review scores as keeping people from realizing that this title is an instant classic. Game reviews are broken, he claims, and it's screwing over developers who want to take risks. Ballsy stuff, especially coming from a guy who isn't hiding the fact that he wants to sell more copies of his game.
Thing is, Mitch has a point.
Just so we're clear, this is not a review of Shadowrun. I haven't played the game, and I don't know if it's good or not.
On the OXM podcast, Senior Editor Ryan McCaffrey discusses the reasons he gave Shadowrun a 7.0, stating that he enjoyed the multiplayer experience, but he, "didn't think there was enough variety within the maps, and I just felt like it was hard to justify $60 for the experience that it gave." Gitelman asks him if he thought the game was worth playing, and McCaffrey says yes, going so far as to explain just how much the OXM crew has been playing. Gitelman asserts that OXM should have scored the game at an 8.5 and mentioned in the review that maybe there should be more content, but what's presented is top notch. Gitelman is, of course, as objective as possible in this case.
7.0, 8.5. Who cares? Would a point and a half really sell a million more copies? Taken on their own, these numbers tell us nothing about the game.
According to OXM's scale, a 7.0 indicates a game that has a lot of things going for it, but still has a few major issues, or something that limits its appeal. That's fine for OXM, but an arbitrary value like that means different things to different people. I don't think a 7.0 is a game that deserves my attention, and I'm pretty positive that most people feel the same way. The Xbox 360 version of Shadowrun is hovering around 70.3% on review score aggregators like GameRankings.com, and that number means radically different things to developers, reviewers, and consumers.
Part of the problem with these scales is that they're top heavy. Most games are scored no lower than a 6 on a 10-point scale, with the lower scores reserved for reviews of games that are comically bad. If all the values were equal, a 6 would be a pretty above average title. It's higher than 5, and 5 is square in the middle. The way these scores are implemented, however, adds weight to the 8s and 9s, marking anything below an 8.5 as cursed and branding the title as a missed opportunity in the eyes of the reviewer. No one rushes out to buy a game that 1up thinks is a 7, not while we could be spending our scratch on the games that show up in the top end of the graph. For $60, we want blockbusters, not a game that's just barely above average.
Why not abolish review scores entirely? GWJ abandoned writing "reviews" in the traditional sense, and instead focused on Perspectives. A semantic difference? Sure. But one which is at least symbolic in its assertion that each person brings with them their own unique point of view. The problem with not using scores, however, is that no one actually reads reviews. Many of them read like shoddy instruction manuals, detailing gameplay and mechanics in far too much detail. Others are poorly written, with no flash or style. And if people aren't reading your boring reviews, that means they're probably not picking up your magazine. So editors assign a score to give people a chance to determine the value of the game without all that difficult "reading comprehension" business. Even if the value means something different to the reader than the author.
Reviews just aren't important anymore. The Internet and its gazillion ways to communicate have replaced the need for magazines to tell us what deserves our attention. Your online friends probably bought The Darkness already. Do they like it? Chances are there's already a thread with their impressions. Your fellow forum dwellers may not have the "journalistic integrity" of the gaming press, but you know what their tastes are. Many of these communities dove head first into Shadowrun, setting up games night after night, and many of those guys didn't care what number IGN used to judge the game. They played the beta and the demo, decided it was the game for them, and talked their friends into their purchases.
Reviews didn't help Shadowrun, but they certainly didn't kill it either. Unfortunately for Mitch, Shadowrun had perception problems in the community long before the final product shipped, and an 8.5 review at OXM probably wouldn't have eased the pain much. Ryan judged the game on its merits and gave it a score that fit his opinion. It's too bad the number alone is meaningless, and it's worse still that the number is all the industry cares about.