Part Of The Conversation

Something was bothering me yesterday as I trolled Metacritic for PSP game scores, trying to refresh my memory on what's actually worth buying for the system. I have the infinite wisdom of the GWJ community to draw from, but even our collective memory may fail under the strain of nearly two years worth of games. With this is mind, the oft derided aggregate sites we disdain for reducing the quality of a game to a mere number serves a valuable function. If I want to know if Metal Gear Solid Portable Ops is good, I can punch it in and see that most reviewers seem to like it. No spoilers, no outright bias or flaming, just a raw number denoting quality so I can feel safe making the order.

What bothered me was that these scores were missing something. Our input. Not one aggregate site includes the Gamers With Jobs perspective on games because we don't score them. I'm starting to wonder if we should.

We dislike the common review structure so much we call our articles "Perspectives" so that we can do our own thing. While it suits our own purposes, I wonder if we're doing a disservice to the wider audience who may benefit from our perspective but will never be exposed to it because we're unwilling to play ball.

Putting aside the old debate of assigning a number to an experience, there are some broader issues at play that concern me as someone running an independent gaming site. For starters, a number would mean we appear on aggregate sites next to Gamespot, IGN, 1Up and the dozens of others sites that score their games. People who look up a title might see "Gamers With Jobs" and read something they would never have been exposed to otherwise. While we primarily write for out own amusement and our community, I like to think that the internet at large could use more of what we're cooking here.

The second thing that springs to mind is access to games before release. Like it or not, calling your articles "Perspectives" and refusing to assign a score is a total turn-off for a lot of PR people when they're deciding who should get early review code. If you're not going to contribute to the Metacritic pool -- which can determine whether or not they get paid a bonus -- they may be less inclined to offer you coverage opportunities. There are hundreds of gaming sites out there and early code (or even final boxed software) is often limited. It's a tough gig wading through the countless fan sites vying for your attention. On the surface, we're a pretty enigmatic site to understand already.

We also have our writers to think of. As a creative cat herder, I'm more likely to get a review from writers if I can spare them the expense of buying the game themselves. We're blessed with all the great content we get from such talented people, but as paid opportunities come up and personal lives get busy or financially shaky, every little bit can help.

Is getting a Gamers With Jobs review on the podcast or the front page before the release date of a game something readers want, even if it means making our review content a little more in line with what most sites are doing? I don't think assigning a point value to a game suddenly means the content of the articles has to change, but the fear of people skipping to the end of a review is a real one. No one wants their hard work reduced to a final paragraph, a number and a few screen shots.

With all this in mind, I open the floor to you, dear reader. Would we be selling our souls by scoring games? Is there some rating system that both satisfies the intellectual and the more mainstream reader? Maybe we need our own internal aggregate with a final score that reflects the views of many on this site. Help me out!

Comments

wordsmythe wrote:

Actually and now that I think about it, adding "I give it a 9" to the end of your Columbine review would be hilarious.

In fact, every game should get a 9. It's the scale used that would change!

Al wrote:
Certis wrote:

If we went with scoring, community involvement would be a must.

Gamerankings won't go for this at all. They want a score that isn't a moving target. The best you can do is a reviewer's score and a community score similar to what Gamespot does. I'm not sure about Metacritic but I'd bet they'd have the same problem.

Just wait until a week after the game ships to submit the community review. Would that work for those sites or do they only collect review scores for a certain period of time?

I'd also agree with, I think it was Mystic, that you should go by how long a person has been registered and not by post count. As for how long they should have been here, I'd say that's up to you guys. I just hate the idea of requiring someone to have 1,000 posts or whatever and excluding people who have been on the site for two years but don't post too often from giving their input.

Like others I'd also prefer a very defined scoring system too. If you're going to do 1-10 then give that 10 score a definite meaning so people can't go around claiming that you think it's the perfect game.

I think that posting frequency should determine everything on this site.

I shall be king!

Something I saw mentioned a couple of times on the previous pages that goes along with the scoring. I'm a dirty skimmer, forgive me if it was mentioned above.

If you have problems getting reviews, would you consider reviews from the GWJ community at large? I'm not suggesting opening the floodgates to unsolicited reviews, as a lot of us don't have the time or writing skills to produce something to the standards of the site, but you might find a hidden gem or two. You could delegate some editorial duties to current staff writers, so you don't have to read each one. Save yourself for the second pass on the article. This might encourage some people who were intimidated during the call for talent contests to actually submit something. You would probably get a broader scope of titles covered too, adding to the depth of the Perspectives library.

Just an idea.

I'd like to echo what Danjo said back on page 2. Beyond that, I have a few additional comments.

  • Reviews Generally
    While I appreciate the time, care, and talent that goes into writing a review, for me, the real value is in the discussion. Sometimes a front page review is the seed of that discussion, other times it's an impressions or catch-all thread.
  • Scored Reviews
    The Fly wrote:

    I think scores do lend a slightly more authoritative air to reviews...

    That might be true, but why is that important? Who needs impressing?

    Certis wrote:

    If we went with scoring, community involvement would be a must.

    The GWJ collective is a nuanced tapestry. Applying a community score would be dunking it in a vat of dye the color of the average thread. I feel it would diminish the contributions of the individuals.

  • Scoring Systems
    There is a fundamental problem in a system of rating based on whether someone should rent/buy/skip a game. It introduces a whole set of personal valuations that are not relevant to the game, and they might not be made explicit in the review. A 40 hour game might be rentable to someone with lots of free time, but not to the player who only plays a few hours a week. Opinions on replayability might hinge on the reviewer's preference for or against multiplayer play. There is enormous variation on individuals' disposable income and how they value their free time. Even if all of these valuations are made explicit in the review a summary rating is useless without reading the details of the review.

    A numeric score doesn't fare much better, but for somewhat different reasons. As mentioned earlier, there could be a temptation to write a review in a way that defends a score. An apologetically written review would have a different feel than those that we've come to enjoy here. It would also invite caviling on how individuals value cutting edge graphics relative to gameplay and the like.

  • How We Got Here and Where We're Going
    To answer Certis's question about finding GWJ, it was recommended to me by another member: jonnypolite.

    Lastly, I feel lucky to be a part of this community and I've made some great friends. I've made my preference known, but I'll be here either way.

Would it be a good idea to try and do a "community review" thread of a game that's coming out soon?

wordsmythe wrote:

Would it be a good idea to try and do a "community review" thread of a game that's coming out soon?

That's what we call a catch-all impressions thread. We're not even close to deciding where we're going to go with reviews in the future, so I'd say just stick with the usual for now.

Lots of good thoughts in here guys, I've read every post and you've certainly given us plenty to consider. I'm going to lock this one up now before someone suggests rating games based on the cover art as viewed by hobos.