Daily Elysium: The Score

Unburdened by such malignities as inimical viruses or even an itchy rash, I'd like to talk for a moment about one of the most common reasons people read game sites: the review.  Of course every reviewer across the pallid landscape of the ever smartly written internet has to undergo extensive training before carefully penning his own gaming critique.  And certainly every review is objectively written with a wide swath of gaming styles in mind and a careful eye placed on the accurace of reportage.  Less, what would the value of amateur review writings be, if not erudite independent thoughts built on the foundation of unbiased reporting.

In the sarcastic words of Edna Crabapple, "Hah!"

More often than not even professionally written reviews are a mish-mash of half related thoughts as written by people with their own unavoidable biases and arbitrary values.  And, should you think otherwise, I happily indict many of my own attempts at the craft.  The question in many ways still left largely unanswered is, what should a review be?  Should it be personal or strive for universality, should it be a recommendation or a caveat, and how does one pass final judgment?

These really are the kinds of questions I ask myself before I put fingertip to keyboard and write a review.  Honestly, I think my process is probably a lot more introspective than most peoples, which is not necessarily a good thing.  However, I read a lot of reviews out there, and wonder if the reviewer is remotely aware that other people with different experiences and expectations actually exist.  All too often an internet review is barely more than a personal diatribe, a critical indictment of the 'flaws' that interfered with the reviewers own enjoyment of the game rather than a critical look at the mechanics of the software itself.  Or, worse still, a blind proclamation of greatness even in the face of overwhelming evidence to the contrary.  Too many reviews are little more than expressions of personal bias with a target.  I always wonder how someone who begins a review for an RPG by saying 'I don't really like RPGs' expects to maintain credibility when complaining about the pace, the combat system, or even the graphics.  But, are these supposedly bad examples just being more honest with their articles than the rest of us who pretend detachment? 

I once read an article by Greg Kasavin of Gamespot that dictated his personal review style.  In the article Greg talked about how a review is not a personal opinion, should all but eliminate the actual preferences of the individual reviewer and instead look at the software from a wholly objective point of view.  I thought this was a very nice, if entirely unrealistic thing to say.  At the risk of becoming unnecessarily philosophical on the matter, there's simply no way to make an intrinsically subjective commentary from a completely subjective perspective wholly objective.  One can be objective about math because the results of a mathematical equation are not subject to a mathematician's experience.  Reviewing games is not math, nor is it science. 

So, it occurs to me that there is simply no way to extract the reviewer from the review, that as a reader or writer of reviews, one must accept that bias and interpretation are intrinsic to the process.  Is that a flaw in the design?  Possibly, and particularly if you've not become used to a given reviewers style and preferences. 

I further recall Gabe or Tycho of Penny-Arcade (if one is to believe they are individuals and don't join in a very Voltron style when it comes time to make their comics) speaking once about the same issues.  As I recall they came to the conclusion that a reader could only find value in a review if they had come to know and trust a certain writer's recommendations of the past, and even then it was still hit and miss.  Consider, after all, how often even your best friend will assume you will like a given song, television show, or movie, leaving you reeling in disgust at the recommendation, wondering if your friend has any clue who you actually are.  If your best friend is wrong about you part of the time, how the hell am I supposed to know whether you'll like a game or not?

Which really begs the question of, what is the point of a review?  Most people seem to be of the mind that a review should tell them whether they will enjoy a certain piece of software.  I contend that this is impossible, and to try and write a review on that premise is doomed to failure.  What a review can do is make a reader aware of the game's mechanics, can extrapolate on the reviewers subjective experience playing the game, and can make people aware of technical, mechanical, or fundamental flaws in the game itself. 

Which, as it happens, are precisely the parts of the review people are most likely to skim.

I'm as guilty as anyone else when it comes to reading reviews.  I see a page of carefully detailed thoughts on the game I'm considering, and really who has the time to read all that?  I jump straight to the good stuff, the score.  And, of course, these scores are scientifically extracted from complicated formulas that balance objective values in weighing subtly conflicting variables across a statistically described spectrum of intertwining ... oh come on!  Scores are derived pretty much the way anyone might expect, they are pulled from the air like a mime climbing a rope. 

Can someone tell me precisely what the difference between a game that ranks 8.4 and one that ranks 8.5 is?   What is the mathematical value of that tenth?  Certainly scores have some purpose in the review, as they roughly describe a general sense the reviewer had of how he values the game as a whole, but without the context of the actual text what is the point of that value?  Isn't the real worth of a review the elements that approach a more real sense of objectivity, which random numbers are, as a rule, not a set member of?

And after all, how often have you read a review where, by and large, the reviewer panned the software and yet still ultimately rewards the game with an apparently bloated score?  Or vice versa?  It's been my experience that, when written by someone with both hemispheres of their brain functioning simultaneously, a review's text usually describes the experience of the game far more accurately than the quick and easy score.  So why do we keep the scores around?

Unfortunately the answer is a matter of simplicity and acquiescence on the part of most sites.  I think you'd find a substantial number of reviewers who would readily admit that a quick score doesn't begin to accurately describe the story of a given game, and yet they keep them because more often than not, it's all readers seem to want.  Scores get hits, or sell magazines, or keep people watching.  Scores give way to rank, make specific and uncomplicated statements, and they take a stand.  People don't want to hear that your enjoyment of a given game depends on various factors, and they don't want to get muddied in the subtleties.  They want direct and accountable statements.  They want to be able to say that game X scored higher than Half-Life or lower than Panty Raider, and when they think you're full of sh*t - as they often do - they haul out that score like they're presenting evidence at Nuremberg. 

But, in the end, the score devalues the review.  It unnecessarily simplifies the complex, and codifies the subjective.

Which, as some of you may have noticed, is why we haven't been putting scores on our reviews lately.  I'm not sure how it's going to play out, but for now we're hanging our random number generator on the shelves, and forcing people to make a choice.  If they want quick easy and arbitrary numbers, then they'll have no trouble finding them, but if they want a serious and hopefully even-handed consideration of a game, then we'll try and provide that.  At least you can take heart that we've thought long and hard on the issue. 

- Elysium

Comments

You bastard!  You made me put numbers in my Planetside review!  Ok, ok so I didnt put up much resistance.  It is sad how confused most of the forum members are from that "other" site.  They are panning your SWG review.  A legitimate gripe that was raised was that MMORPGs shouldnt be held to different standards.  An incomplete product shouldnt be reviewed more favorably on a promise of content and bug fixes.

Maybe MMORPGs shouldnt be reviewed at all.  I mean they are so contingent on server populations and the dynamics change constantly.  Plus the dynamics are level or experience or time dependent.

They should be "reviewed" as more of a perpetual preview.

Barring a complete AO or SWG launch f*ckup, if you play MMORPGs, you should play most MMORPGs because their is enough gameplay in that initial month trying to discover how to play the game and building relationships to justify the initial box fee.  From there, you can then decide on whether to continue paying and playing.

You should have heard his scepticism when I first suggested we stop posting scores. It was like when I told him there wasn't a Santa Clause, it totally blew his mind.

I give this article a 4.5 out of 5.

I couldn't resist.

True.

Here's a suggestion.  How 'bout having 2 people review the same game and give it either a thumbs up or thumbs down and why, ala Siskel & Eb... well, just Siskel.  2 oppinions for the price of one.  Twice the work, twice the fun.

That was actually one of the reasons why we started the Conference Call reviews.  Our most recent of which, did not include a score.  Great minds think alike BrokenCrayon

- Elysium

Don't forgot the ultra-rare two thumbs "way up"!.

How I found this site was following a link to a review by one of you all, and I was so impressed by both mindset and content of the review that I decided to stay.  Well okay I also like name/premise of the site as well.

I like reviews to do exactly what you said, tell me enough about mechanics and how well they are achieved so that I can get some idea if I would enjoy it.  What I normally hate about reviews is that they assume I've played every game since Pong, and that all of their allusions will make instant sense to me.   Most especially reviews of games that are "sort of sequals" when the last version was on a different platform (ps1 vs ps2).  I want to know about *this* game.  Have a separate section entitled, "if you've played X".  that's cool, but I just want to know about the game in question, even if it's Final Fantasy X (which I think can be described on its own merits.  it rocked. )

I'd just like to say that I was one of the first on board with this policy. When I wrote my WineX review, it took me a good 30 minutes after posting it to figure out, "hey I don't have a score on this thing! Oh sh*t!" Of course, it was premeditated all along, I knew what was going down and there were no panicked edits that added scores and removed them arbitrarily, no siree.

Since there's no such a thing as constantly reviewing an MMORPG I think it simply should be reviewed twice. 1 or 2 weeks after launch and 5-6 months later again. I'm not a fan of reviews under the "there's potential, lots of thing to be featured some day"-premise. When thinking about purchasing a software I'd rather like to know what it features right now, and that's what the focus should be put on. Really nice to know that the devs promise upgrades and additions here and there, but those might be delayed or turn out to be disappointing.

Eric & Chet Wolpaw are the only game reviewers I've run across who often do a great job of reviewing games.

I'm not sure that I agree with the idea a reader can only find value in a review if they have come to know and trust a certain writer's recommendations. A good review should have enough information about the reviewer's ideas and opinions to give a reasonably perceptive reader a decent idea of how they personally would react to the game.

I think game reviewing is fundamentally the same skill as literary or film criticism. These are tough subjects. And many of the people attracted to the gaming world aren't necessarily those with the most developed and refined lit-crit skills.

Yeah, I have never bought that whole "but someday it won't suck so give it 10 starts today" argument. The game was not purchased based upon some far future feature it set, it was purchased and played as it is today and that is what should be reviewed. Theoretical potential fun in some far off day is all very well and good and should be mentioned maybe, but a review is snap shop of of an experience taken at a certain time and should both reflect that and be taken as such by a reader.

-Griffon

It unnecessarily simplifies the complex, and codifies the subjective.

Yet Douglas Adams could come up with a numerical answer to Life the Universe and Everything. You know I think if Douglas Adams were alive today he would call you lazy Elysium. After all shouldn't the result of every review naturally be 42?

Hmmm. A well-reasoned article, but I'm not sure that I entirely agree. Numbers have their place, if they're used properly.

I'm a fan of the glory days of Amiga Power, and I remember clearly their reviewing system: average games got 50%. Above average games got more, depending on how good they were, and below average games got less, depending on how bad they were. I've never seen any other game place use this methodology - GameSpy, in particular is terrible at it: an average game, by their own admission, is 65% plus!

Hey Elysium, have you been hitting the Thesaurus again?  Malignities?  Inimical?  Pallid?  All these in the first paragraph alone!  Good job.  I can come here to keep up with the latest gaming commentary and get an education too.  In my defense, I'm a verbally challenged math person but I do find your articles both interesting, insightful and usually fun to read as I usually haphazardly guess at what some the words mean.  Keep up the good work.  It's one of the reasons I keep coming back to this site.  Now please allow me to return to my corner while donning my dunce cap.

And many of the people attracted to the gaming world aren't necessarily those with the most developed and refined lit-crit skills.

That's why I like this site! What, there's something on my nose?

Putting Greg Kasavin -- or any other Gamespot flack -- next to "wholly objective point of view" makes my stomach do funny things like the first time I played a first-person shooter. Gamespot reviews are worthless because if you advertise on Gamespot you are pretty much guaranteed at least an 8. For God's sake they gave Civ III a great score and never has there been a more offensive bugged-out AI-cheating piece-of-puppy-dung foisted on the strategy game community. I will always remember wiping out all the roads around an enemy city prior to taking it over so that I could not be attacked the next turn, only to be reamed the next turn by about 20 Marines all miraculously generated in one turn by a bankrupt opponent running at triple the speed they would run if they were my Marines. Yup that's a superb game. And let's not forget Black & White, a 9.x game if there ever was one, if you like clicking on hundreds of individual villagers all too stupid to quit praying when they are about to starve while trying to keep your pet gorilla from eating its own scat. The only rationale I can think of for high ratings for these travesties is the beaucoup advertising dollar buying all the Twinkies Kasavin and co. have been eating.

And that's why this site rocks. Straight-up dope, no b.s. Plus the dude that published the viable strategy for getting the ladies to play games.

 

"Plus the dude that published the viable strategy for getting the ladies to play games."

Yeah, he's pretty much a hero to us all now.