Numbers Lie

Mitch Gitelman is not a happy camper.

The lead Shadowrun developer has been making the rounds on the podcast circuit, discussing in great detail why he feels his game is getting the short end of the stick. Gitelman lays a lot of the blame on the press, claiming that reviewers are being too harsh on his game and holding the title up to unfair standards. On the Official Xbox Magazine podcast this past Monday he points to middle-of-the-road review scores as keeping people from realizing that this title is an instant classic. Game reviews are broken, he claims, and it's screwing over developers who want to take risks. Ballsy stuff, especially coming from a guy who isn't hiding the fact that he wants to sell more copies of his game.

Thing is, Mitch has a point.

Just so we're clear, this is not a review of Shadowrun. I haven't played the game, and I don't know if it's good or not.

On the OXM podcast, Senior Editor Ryan McCaffrey discusses the reasons he gave Shadowrun a 7.0, stating that he enjoyed the multiplayer experience, but he, "didn't think there was enough variety within the maps, and I just felt like it was hard to justify $60 for the experience that it gave." Gitelman asks him if he thought the game was worth playing, and McCaffrey says yes, going so far as to explain just how much the OXM crew has been playing. Gitelman asserts that OXM should have scored the game at an 8.5 and mentioned in the review that maybe there should be more content, but what's presented is top notch. Gitelman is, of course, as objective as possible in this case.

7.0, 8.5. Who cares? Would a point and a half really sell a million more copies? Taken on their own, these numbers tell us nothing about the game.

According to OXM's scale, a 7.0 indicates a game that has a lot of things going for it, but still has a few major issues, or something that limits its appeal. That's fine for OXM, but an arbitrary value like that means different things to different people. I don't think a 7.0 is a game that deserves my attention, and I'm pretty positive that most people feel the same way. The Xbox 360 version of Shadowrun is hovering around 70.3% on review score aggregators like GameRankings.com, and that number means radically different things to developers, reviewers, and consumers.

Part of the problem with these scales is that they're top heavy. Most games are scored no lower than a 6 on a 10-point scale, with the lower scores reserved for reviews of games that are comically bad. If all the values were equal, a 6 would be a pretty above average title. It's higher than 5, and 5 is square in the middle. The way these scores are implemented, however, adds weight to the 8s and 9s, marking anything below an 8.5 as cursed and branding the title as a missed opportunity in the eyes of the reviewer. No one rushes out to buy a game that 1up thinks is a 7, not while we could be spending our scratch on the games that show up in the top end of the graph. For $60, we want blockbusters, not a game that's just barely above average.

Why not abolish review scores entirely? GWJ abandoned writing "reviews" in the traditional sense, and instead focused on Perspectives. A semantic difference? Sure. But one which is at least symbolic in its assertion that each person brings with them their own unique point of view. The problem with not using scores, however, is that no one actually reads reviews. Many of them read like shoddy instruction manuals, detailing gameplay and mechanics in far too much detail. Others are poorly written, with no flash or style. And if people aren't reading your boring reviews, that means they're probably not picking up your magazine. So editors assign a score to give people a chance to determine the value of the game without all that difficult "reading comprehension" business. Even if the value means something different to the reader than the author.

Reviews just aren't important anymore. The Internet and its gazillion ways to communicate have replaced the need for magazines to tell us what deserves our attention. Your online friends probably bought The Darkness already. Do they like it? Chances are there's already a thread with their impressions. Your fellow forum dwellers may not have the "journalistic integrity" of the gaming press, but you know what their tastes are. Many of these communities dove head first into Shadowrun, setting up games night after night, and many of those guys didn't care what number IGN used to judge the game. They played the beta and the demo, decided it was the game for them, and talked their friends into their purchases.

Reviews didn't help Shadowrun, but they certainly didn't kill it either. Unfortunately for Mitch, Shadowrun had perception problems in the community long before the final product shipped, and an 8.5 review at OXM probably wouldn't have eased the pain much. Ryan judged the game on its merits and gave it a score that fit his opinion. It's too bad the number alone is meaningless, and it's worse still that the number is all the industry cares about.

Comments

TheGameguru wrote:
Y yo. Even if I HAD wanted to pay $60 for a Quake-3-with-sprinkles that pisses all over its source material (FASA does need the money, I guess), I've no plans to roll Vista. Maybe Mitch should have thought that over before accepting the filthy Vista-exclusive-lucre.

You do realize they really didnt have any choice or say in the matter?

You do realize that if I did, my comment would have not been so worded.

Wizkids, at some point, had to have signed over rights for MS/FASA Int. to do whatever they like. I can only assume that such a waiver meant more $$ for the the signer, since MS wants to push Vista. During negotiations, someone could have said, "We'd like this to stay within X and Y plot parameters, and to be playable on XP too, so we can sell more people on Shadowrun". So, ultimately, they did have a say, but sold it.

I really need to wield that, "You do realize..." a bit more; it gives me jollies to sound so full-of-myself! ^.^

TheGameguru wrote:
Y yo. Even if I HAD wanted to pay $60 for a Quake-3-with-sprinkles that pisses all over its source material (FASA does need the money, I guess), I've no plans to roll Vista. Maybe Mitch should have thought that over before accepting the filthy Vista-exclusive-lucre.

You do realize they really didnt have any choice or say in the matter?

Where did you get that idea? If you listened to the Major Nelson podcast with FASA studio interview, it was their (FASA's) idea to do the cross platform thing locking them into Vista.

http://www.majornelson.com/archive/2...

I check metacritic when I'm looking at a slightly older game or one that didn't/doesn't get discussed here. The community here doesn't play every game and has its own mores, which I am generally in step with but sometimes we diverge.
I mainly check to make sure the game isn't a lemon though - if a score is in the 60s (right below Shadowrun's 70) then that's a big warning sign. A 70 means I have to do a lot more looking. 80 or above means it's probably a safe bet, though I will stick click through to read actual reviews and find out more on my own. But a low score could mean a pass without giving it too much more thought.

The trick with reviews, as we've discussed here before, is that you have to find a reviewer whose taste you agree with. A review is going to be inherently subjective, so you want to match well with the reviewer and understand where he is coming from.

I have a personal rating system as well, it reflects value, urgency, and anticipation.
In theory it started out as a 1-5 scale, but it was heavily weighted with 4s. Then I realized that these were all games I wanted to play - they had passed the "interest" test and the "not sucking/getting reamed in reviews/worth of mouth" test.

I still use the scale, but it consists entirely of 3, 4, 4.5, and 5.
5 means it's a day 1 or thereabouts purchase, and I'll probably pay retail (with a coupon or gift card I hope, but I'm still willing to pony up what's being asked).
4.5 is similar.
4 means I'm really looking forward to it and might jump on it when it becomes available.
3 means I can wait for some kind of deal to come around. It's more likely to get tagged for a rental, and the sort of game I'd scoop up in a Buy 1 Get 1 or Buy 2 Get 1 deal.

I still want to play all the games on that list though. The problem for a magazine/review site is that their scale has to include games they wouldn't recommend playing, and frankly most of the time they don't have the balls to say as much, which is why the bottom half of the scale doesn't get used much. So I pretty much agree with you guys on the thumbs up/thumbs down scale, or the stoplight, or a similar system. There should be a definite recommendation, and then there should be qualifications, and it should be something like:

Buy this. Don't argue.
Buy it. Here's why.
Don't buy it. Here's why.
Don't go anywhere near it! Here, have a laugh.

If the reviewer has credibility and stock with you then you'll evaluate how interested you are.

I take the cost and divide it by the number of hours I will enjoy it.

I would pay $60 for a 2 hour game if it's one of the best 2 hours I've ever spent. I didn't mind at all back then when I spent $50-60 for Mario, Sonic, etc, basically fun, quick little games. I guess our standards are just different now, but length != quality.

edit: How much fun you think you'll have is different than how long a game is. It's hard to measure the former, and the industry nowadays pushes how long their game is. Sure, an RPG could be 80 hours long, but how much of that is mindless grinding and random battles?

I usually think of the price of games and other activities relative to the price of a movie ticket.

You can usually see a 2 hour movie for about $8. Given that, a $60 game really only needs to provide me with about 8 hours of "fun" to make it worth it.

With a solid multiplayer game, you'll probably pass that threshold in a week.

Bazarov wrote:
TheGameguru wrote:
Y yo. Even if I HAD wanted to pay $60 for a Quake-3-with-sprinkles that pisses all over its source material (FASA does need the money, I guess), I've no plans to roll Vista. Maybe Mitch should have thought that over before accepting the filthy Vista-exclusive-lucre.

You do realize they really didnt have any choice or say in the matter?

You do realize that if I did, my comment would have not been so worded.

Wizkids, at some point, had to have signed over rights for MS/FASA Int. to do whatever they like. I can only assume that such a waiver meant more $$ for the the signer, since MS wants to push Vista. During negotiations, someone could have said, "We'd like this to stay within X and Y plot parameters, and to be playable on XP too, so we can sell more people on Shadowrun". So, ultimately, they did have a say, but sold it.

I really need to wield that, "You do realize..." a bit more; it gives me jollies to sound so full-of-myself! ^.^

You do realize, don't you, that the "Interactive" in "FASA Interactive" means they hold the digital rights to FASA properties?

Hey, you're right! I feel like a condescending dick using that rhetorical flourish. I FEEL LIEK POWER!

My minor correction is that FanPro (the German outfit) has the Shadowrun RPG rights. WizKids has, um, everything else, I guess? Take Crimson Skies, for instance. They do the minis, FASA Interactive does the computer & Xbox game. (Er, they did do the Xbox game, right?)

FASA has been a wholly-owned studio (or "bitch", in industry parlance) of Microsoft for several years now. Their decision-making is going to be a tug-of-war with the business unit responsible for games.

When rating girls, I believe the standard calibration is that 5 = "Maybe if I was wasted." I'm sure we can find some similar way to calibrate 10-point scales.

That said, the $ ranking sounds like a good idea.

McChuck wrote:
Farscry wrote:

Really, as far as scoring goes, I still favor the simplest type of all: thumbs up or thumbs down, with a review explaining why.

When it comes to media, I've always preferred the stoplight over the thumbs because it allows for a degree of exposure instead of buy or don't buy. Something like this:

Green = Buy
Yellow = Try Before Buy (Rent, Borrow, or even Download)
Red = Avoid

I may be misinterpreting your use of the thumbs, though. Do they represent a purchase recommendation or a play recommendation? If it's play, the stoplight doesn't work as well.

Purchase recommendation. When getting into the possibility of playing without buying (i.e. worth a rental), then yeah, the three-tier system is better. The problem is that the gaming media, in an attempt to try to prove that it's something more than it is, wants to emulate film/theatre/culture critics. Or should that be critiques?

In other words, they want to be fancier, more "legitimate", try to put on a refined or educated air. "Why yes, Raphael, I'll have the minced legumes and caviar. I daresay, this game is not quite a 10; while there's a certain joie de vivre exuded by the protagonist and further developed by the metaphors inherent in the varied locales, the control is rather coarse and not quite suited for high society." Or something of the sort.

The point being, that game "critics" are generally attempting to do two things in a review: rate a game for their audience, and show off their knowledge of gaming and thus their ability to "objectively" rate a game based upon their vast repository of gaming experiences.

You know what? Most of us could care less. We want to know three things: is it fun (and why), does it work right, and thus, is it worth our money?

That answer varies from person to person, no matter how objectively great a game allegedly is.

So put simply, all we really need from a reviewer is to know if they feel it's worth buying or not (or, in a three tier system, if it's only worth a rental), and why they feel that way. Everything else is extraneous, because that'll tell us if it's worth it for us (sometimes by virtue of why they dislike a game, I may learn that I'll like it, or vice versa).

Bazarov wrote:

I really need to wield that, "You do realize..." a bit more; it gives me jollies to sound so full-of-myself! ^.^

Bazarov, meet Gameguru; Gameguru, Bazarov.

Scaphism wrote:

The trick with reviews, as we've discussed here before, is that you have to find a reviewer whose taste you agree with. A review is going to be inherently subjective, so you want to match well with the reviewer and understand where he is coming from ... If the reviewer has credibility and stock with you then you'll evaluate how interested you are.

Thanks for saving me the trouble of typing this again That is exactly how professional criticism is SUPPOSED to work. You read Anthony Lane long enough, you know how he thinks... you know what he likes, and you know if his opinions differ from your own. If you agree with Ebert, then Ebert's reviews have value to you. If you find yourself disagreeing with him most of the time, you need to find someone else to read, rather than bitch about how he's "clueless" or whatever.

The problem with game reviews, IMHO, is that readers very rarely get this opportunity to understand how critics think. Maybe it's because games are so much longer and more involved than movies or music, but the reviewers themselves tend to rotate more than a barber pole in a hurricane.

Look at IGN, or Gamespot, or whatever... how many different people review games? Since reviews are so subjective, one person's "8" could be another person's "6." Even worse is something like X-PLAY. Sessler USED to actually write his own reviews, way back when... but now? Who the hell knows what he'll think about a game. You can't trust a thing he says, because random interns put the words in his mouth. In these kinds of cases, a number system is completely pointless... sort of like taking the temperature with 10 different types of thermometers.

Farscry wrote:

In other words, they want to be fancier, more "legitimate", try to put on a refined or educated air. "Why yes, Raphael, I'll have the minced legumes and caviar. I daresay, this game is not quite a 10; while there's a certain joie de vivre exuded by the protagonist and further developed by the metaphors inherent in the varied locales, the control is rather coarse and not quite suited for high society." Or something of the sort.

The point being, that game "critics" are generally attempting to do two things in a review: rate a game for their audience, and show off their knowledge of gaming and thus their ability to "objectively" rate a game based upon their vast repository of gaming experiences.

In other news, a game reviewer once killed Farscry's dog.

I use reviews to details about a game and fora to find out whether it's worth buying.

SommerMatt wrote:
Farscry wrote:

In other words, they want to be fancier, more "legitimate", try to put on a refined or educated air. "Why yes, Raphael, I'll have the minced legumes and caviar. I daresay, this game is not quite a 10; while there's a certain joie de vivre exuded by the protagonist and further developed by the metaphors inherent in the varied locales, the control is rather coarse and not quite suited for high society." Or something of the sort.

The point being, that game "critics" are generally attempting to do two things in a review: rate a game for their audience, and show off their knowledge of gaming and thus their ability to "objectively" rate a game based upon their vast repository of gaming experiences.

In other news, a game reviewer once killed Farscry's dog.

Sparky was such a good boy, too!

All I want from a professional game reviewer is to give me a FEEL of how the game plays. I don't want your

  • filters
  • preconceptions
  • multi-paragraph recitation of the game's entire cockneyed storyline of how the Dark Lord Zulu has enslaved the Kingdom of Mushrooms and only you and your Holy Rocket Jeep can save it. Please... just don't.

All I want is to get the FEEL of how it PLAYS. No less, and perhaps slightly more. You sat down with the game ... what happened ? How did you two interact ? How has it affected you ?

There's plenty of REAL creativity that can be involved in this process. REAL writing skills.

If you're incapable of delivering that experience, then move over to the DS review section and start working on your Pokemon Almond reviewing skills. God knows kids don't give a crap, they just look for words "awesome" and "lame" and in some cases, "pimpin'".

I hate reviews without scores.

Why? Because my time is valuable, and if a game is plainly bad, I don't want to have to read paragraphs of text to get to that point.

There are too many games, albums, movies, TV shows, etc, for me to possibly experience all the ones I would like to. I need help filtering, and the last thing I want is a filter that becomes a time sink of its own.

Yes, a score alone isn't particularly meaningful, but it helps filter out the first 50-75% of stuff that I certainly would not want to bother with. The actual text of a review takes care of informing me on the rest.

It doesn't matter to me if review scales are overly slanted in the high number direction, though I would prefer more balance. But as long as I know what's going on, I can adjust my interpretations accordingly.

Demiurge wrote:

Reviews just aren't important anymore.

Disagree! Yes, there are more ways than ever to find out about good games and stuff, but reviews remain my first line of filtering. This is true on games, but FAR more true on music, which is flat out impossible to truly keep up on without a steady dose of reviews (or just consigning yourself to being the last person to hear about anything good). Games are, at this point, low enough in number where I could probably hear about most of what I play through forums and such, but even still, I get a lot of good info out of reviews.

shihonage wrote:

All I want from a professional game reviewer is to give me a FEEL of how the game plays. I don't want your

  • filters
  • preconceptions
  • multi-paragraph recitation of the game's entire cockneyed storyline of how the Dark Lord Zulu has enslaved the Kingdom of Mushrooms and only you and your Holy Rocket Jeep can save it. Please... just don't.

All I want is to get the FEEL of how it PLAYS. No less, and perhaps slightly more. You sat down with the game ... what happened ? How did you two interact ? How has it affected you ?

There's plenty of REAL creativity that can be involved in this process. REAL writing skills.

What you want to read, I guess, is a press release. Criticism is completely subjective. If you don't wan't a reviewer's opinion, why bother reading them at all?

edit... sorry, a bit flippant in my reply. It seems like you just want an onjective report about "what happened" when someone turned the game on. I can sort of understand your complaints above, but again, I don't think I've ever read a review of anything that is like the one you want. A review/opinion is formed by the thousands of other games and things people have played before they pick up the new one. How can it be anything other than subjective?

SommerMatt wrote:

What you want to read, I guess, is a press release. Criticism is completely subjective. If you don't wan't a reviewer's opinion, why bother reading them at all?

I don't know how you possibly came to that conclusion from what he wrote.

*Legion* wrote:
SommerMatt wrote:

What you want to read, I guess, is a press release. Criticism is completely subjective. If you don't wan't a reviewer's opinion, why bother reading them at all?

I don't know how you possibly came to that conclusion from what he wrote.

I edited my post while you were posting yours...

SommerMatt wrote:

What you want to read, I guess, is a press release. Criticism is completely subjective. If you don't wan't a reviewer's opinion, why bother reading them at all?

shihonage wrote:

All I want is to get the FEEL of how it PLAYS. No less, and perhaps slightly more. You sat down with the game ... what happened ? How did you two interact ? How has it affected you ?

In 22 years that I've been gaming, I've not seen a single press release that would fit the above description. Have you ?

A review/opinion is formed by the thousands of other games and things people have played before they pick up the new one. How can it be anything other than subjective?

Seriously, even the paragraph I quoted above implies a healthy degree of subjectivity. More so if you actually read and comprehend the entirety of my post.

SommerMatt wrote:

edit... sorry, a bit flippant in my reply. It seems like you just want an onjective report about "what happened" when someone turned the game on. I can sort of understand your complaints above, but again, I don't think I've ever read a review of anything that is like the one you want. A review/opinion is formed by the thousands of other games and things people have played before they pick up the new one. How can it be anything other than subjective?

So give me an example of what you're talking about. I already backed down from those comments quoted above.

SommerMatt wrote:

So give me an example of what you're talking about.

If something that I wish existed has already existed in numbers sufficient for me to find an example on moment's notice, I probably wouldn't bring up the fact that it is sorely lacking in the first place.

As it stands, I have to leave work now. Maybe I'll find something later.

I give Shadowrun a $29.99.

I think the $$ review system works. After all, that's the only review publishers are thinking about!

Are you the Shadout Mapes, or some other Shadout?

Its where the name (or title) is stolen from at least

One thing that should at least always accompany number ratings are a short text explaining why it did get this exact rating (Both for the lazy bastards not reading the whole review, but also to give the numbers a bit actual meaning). And more ratings than just an Overall score helps too obviously, to get the numbers closer to whats actually said in the review.

Demiurge wrote:
souldaddy wrote:
Really, as far as scoring goes, I still favor the simplest type of all: thumbs up or thumbs down, with a review explaining why.

That makes all kind of sense. You know if the reviewer was positive or negative on the game, and have to read more to find out why. Might be a good system to adopt at GWJ.

Do you think GWJ needs abstract ratings in its reviews? I'm intrigued.

A declaritive statement in the form of a question? It's like a pleasant form of rebuttal

Esoteric arguments aside, Gamers With Jobs might impliment a rating system for the pratical, pragmatic reasons - it would bring more people to the website. GWJ reviews could be listed on aggregate sites like Gamerankings. A graphic at the top of the article, "Thumbs Up" or "Thumbs Down" might entice transients and skimmers to read your great work.

souldaddy wrote:

A declaritive statement in the form of a question? It's like a pleasant form of rebuttal :-P

I try.

Esoteric arguments aside, Gamers With Jobs might impliment a rating system for the pratical, pragmatic reasons - it would bring more people to the website.

Good content brings people to the site all the time, all without a review score or an abstract guide to what we do or don't like.

GWJ reviews could be listed on aggregate sites like Gamerankings.

Speaking only for myself, I think I'm comfortable not being on Gamerankings. You won't find a lot of people in the industry who are big fans of review aggregate sites.

A graphic at the top of the article, "Thumbs Up" or "Thumbs Down" might entice transients and skimmers to read your great work.

Transients and skimmers are looking for reasons to not read our work. Why bother reading the 1000 words I just wrote about Overlord if they can just check if I gave it an A+? The argument against rankings and scales is that people only read the number, not the reasons why the game got the number, and the number means nothing without that context. The Legions of the world are too busy for the prose, they just want to know if the product is "good" or "bad."

souldaddy wrote:

Esoteric arguments aside, Gamers With Jobs might impliment a rating system for the pratical, pragmatic reasons - it would bring more people to the website. GWJ reviews could be listed on aggregate sites like Gamerankings. A graphic at the top of the article, "Thumbs Up" or "Thumbs Down" might entice transients and skimmers to read your great work.

The above considerations have certainly entered into discussions about how GWJ handles review-type content, and in the time I've been here the issue has been revisited more than once. In the above post Demi nails some of the primary reasons we've steered away from that sort of thing, I think. I've always been happy with the decision. I think it makes for much better writing, of the type that brings people here. The fact that it allows us to remain disengaged from these Gitelman-type entanglements is a side bonus.

Shadowrun could've gotten a 10/10 and I could've won the lottery and still wouldn't have bought it. To me, there's nothing interesting about it at all.

As far as reviews go, I tend to read through them sometimes but I hardly ever listen. I get the impression that they've played the game but have no idea what they're talking about half the time.

I listen to Word of Mouth from people I know because they are honest opinions without the excess bullsh*t.

I think we should give review scores... But have them be completely irrelevant to the context of the review. That is the true way to buck the game rankings pile of crap. Liked Overlord across the board and think it fits among the classics? Give it a 7.1826 out of 11. Can't believe they released another Postal with even more B-list cameos? Give it a 99.5/100.

I mean just the fact that a lot of ranking systems come with instructions and disclaimers is reason enough to give them the boot. As a matter of fact, i think I'd give game ranking systems a 3.1415927 out of -b+/-(b^2-4ac)/2a)^1/2 radians.

Eoin wrote:

Reviews for 'long' games (i.e. games with start, middle & end game sections) never go all the way through.

Why don't publishers furnish reviewers with a copy of the game that allows easy access to the various sections of the game, to facilitate quickly and easily including all of those sections in the game's review?

For a single-player game, this access could be in the form of save game files or perhaps a build of the game with a level-select cheat code included; for MMO-type games, this could be an account with characters of multiple levels already included in the account.

Eoin wrote:

The NY times weighs in with
this

LobsterMobster wrote:

*cough*

;)

I've got you both beat with one cough and it's from this site in August 2003, 6 months before this manifesto that the NY Times mentions. In fact, it is the post that states why there are no ratings on this site for game reviews.

WolverineJon wrote:

Why don't publishers furnish reviewers with a copy of the game that allows easy access to the various sections of the game, to facilitate quickly and easily including all of those sections in the game's review?

If you give out 'snippets' of your game to reviewers, what's stopping you from only giving the only the 'good' parts of a game? I think the full released game should be reviewed (not patched; patches should be reviewed later and could be covered by something like the 'classic' game reviews done here) but in our 'I need it, NOW!' world partial reviews are what we get since someone wants to feel warm and fuzzy before/after their purchase of shovelware v666. Bring into this, the marketing and 'exclusive' pre-reviews and you've got some big pressure to probably not score the game craptacular so you get the next exclusive as well.

quite frankly, some games are just too long to finish before review... at least for a "professional" paid critic. Can you imagine having to slog trough 100% of any given game before commenting? Certainly not a good use of your time, IMHO.

This is sort of the same problem with the ESRB-- you have game companies giving the review board a "highlight video" of their game in question... the ESRB doesn't have anyone actually play the game through, which is why they occasionally are surprised with something that the publisher didn't tell them about.

As for review scores, I understand the point of wanting people to actually READ the brilliantly scripted analysis, but the reality is that most people... most USERS of reviews... want a quick idea of the quality of the product. Not giving people what they want is a sure-fire way to make sure they move on to someone who will (see: the rose of Rottentomatoes and Metacritic). Giving both a great analysis and a final sumative score seems like it will be the best of both worlds -- Those who just want to glance at the number will do so, and those who wish to read more will do so. Granted, having sumative scores is sort of meaningless when you have a rotating review team... even so, if thats what the review consumer wants, I don't think it means they are "stupid," deficient in some way, or incapable of reading.

(not saying anyone here has said as such, but it's easy to infer sometimes from these kinds of arguments)