For now I want to leave the scope pretty-wide, as there are a lot of angles to this discussion. My own biases clearly have me being interested in how these sites damage art and artists, but there's a lot to talk about in terms of fandom wars, the age old "are numerical scores useful" question, and the dark & ugly side of gaming culture. For now everything is fair game. Thanks!
I've never been a fan of review aggregator sites.
There are many reason for this. A review is not supposed to be about finding a consensus - it is supposed to be about an individual reacting to a creative work and the possible deeper dialogue the stems from a well articulated point of view. Averages can be profoundly misleading (particularly on a site like Rotten Tomatoes.) They can also create strife and conflict where none should exist.
Chief among all of the many reasons I don't like aggregators though is that I find numerical values for artistic endeavors completely anathema to how creative works actually function. It feels very much like a hangover of late-era capitalism. An attempt to assign a specific, concrete value to a thing which has a primarily esoteric value.
There is such a vast range of possible scopes for a project that there can't possibly be a way lumping all of them together and assigning a rigid numerical score can reveal anything useful at all. Some things are shooting to be big stupid fun and nothing more. Others are digging at heady questions and exploring complex issues. Then you have to look at the sheer number of individual threads that go into achieving the object of the creative work. In film, there are individual performances, collective ensemble chemistry, design work from lights to sound to music to costumes and beyond. You've got cinematography and editing doing their thing in tandem with a script - a thing which itself is a complicated beast with a lot happening on the hood. If it is a game instead of film, you're adding graphics, UI, mechanics, player reactivity, and host of additional factors.
All of that said, I understand that people enjoy Metacritic and Rotten Tomatoes as a quick shorthand to see if something is worth spending their time and money on. In many ways though, this makes their shortcomings particularly troubling, and I think a case can be made that they are training us to be less nuanced audience members.
For further context, the idea from this thread has been percolating for awhile, but upon hearing that Jim Sterling's website had suffered a DDOS attack for the simply crime of rating a game a 7/10 instead of a 10/10 I figured I would go ahead and pull the trigger.
On the one hand, you're right, but on the other hand, every experience of a creative work involves a self-contained subjective assessment of that value of that work.
"I just saw the new Avengers movie."
"Yeah, any good?"
"Nah, was just an excuse to blow sh*t up for no reason, didn't make a lick of sense."
That right there is my qualitative assessment. Is your argument that the leap from qualitative to quantitative is the problem? Because I can point you towards the community GOTY thread, where we, as a community, bandy about our quantitative assessments of the last year of gaming. Is that problematic, or is it the widespread (like, entire internet-wide) democratization of that process that's the problem?
I disagree. It's trivial for me to tell you whether I enjoyed Mass Effect 3 more than Gone Home or not, despite their vastly different scopes. To a large extent, the scope is baked into that assessment - there's swathes of Mass Effect that I found tedious, while the tighter scope of Gone Home sidestepped the problem of maintaining interest for 60+ hours. So Gone Home burns that much more brightly in my memory.
Brigades of arsehole keyboard-warriors seems like a distinct issue from review aggregation. Let's be honest, they would have found something else to get their fedoras in a twist about had Jim given BotW a glowing 10/10.
You've never known true joy until you've shaken a lich stick at someone.
Assessment is fine. I just think that an assessment that boils something down to a number and a scale - especially something as complicated a film, TV show, or game - isn't really an assessment. Art isn't a pop quiz where you get 7 out 10 questions right. It is a complicated thing that deserves deeper feedback. And I get that people don't always have time to go into nuance - but even when people do fly-by assessments, they don't speak in numbers. For example:
"Did you like the Avengers move?"
"I enjoyed it."
That simple exchange reveals more than this:
"Did you like the Avengers move?"
"7 out of 10."
The latter is gibberish, because numbers don't make much sense as a reaction to creative work. It doesn't mean anything. As for the former, despite only being three words, there is at least a subjective sense of something in "I enjoyed it." Which itself is vaguer than your assessment, which tells me way more about what you think than any number could.
That's part and parcel with my point though - the fact that the scope of those two things is so wildly different further renders any comparative numerical score irrelevant.
Maybe, but also maybe not? There is a reason twitter continues to be a hotbed of harassment - it makes harassment incredibly easy and doesn't provide a whole lot of tools with which to deter harassment. RT and MC are set up in such a fashion as to encourage flame wars and bad fan behavior.
In general this kind of argument - "there will always be people who are jerks" - doesn't work for me. Yes, of course there will always be jerks. But with the right kind of systems in place it is harder for the jerks to dominate the conversation.
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
I think we're going to be talking past each other here, and here's why:
"Did you like the Avengers movie?"
"I enjoyed it."
"How much did you enjoy it?"
"I give it a 7 out of 10."
Those are two different answers to ultimately the same question. The only difference is in the digital vs analog nature of the answer given. You seem to be suggesting that it's nonsensical to assess quantity/quality of enjoyment in any way that isn't strictly yes/no, and I think that's gibberish.
For instance, I enjoyed the sandwich I had for lunch, and I enjoyed witnessing the birth of my child. The scope of those things couldn't be more different, but you can probably guess which one I "enjoyed more". The upshot of your argument seems to be that you can't compare subjective experiences, to which I would rebutt that they are the only experiences you can compare.
You've never known true joy until you've shaken a lich stick at someone.
I agree that with a lot of what you said. Where I disagree is that the number means anything at all. In fact, it diminishes the other things you say about an experience because it assigns a fixed value that flattens the impact of context (as it tries to pull everything towards some non-existent playing field where all things are numerically measurable regardless of the wildly different context they might exist in.)
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
(It's the sandwich, right? For sheer enjoyment, I mean?
And what started as a tongue-in-cheek comment is now making me think that I didn't particularly enjoy watching childbirth. It was fascinating and important and emotional and yadda yadda yadda, but not particularly enjoyable.
Watching childbirth: 6/10)
The number provides some level of emphasis. It's an alternate shorthand for verbal modifiers. "I really enjoyed that movie" vs "I didn't enjoy that movie" -> 9/10 vs 3/10. Is the number an actual quantifiable anything? Of course not, no more that "really" or "kinda" or "didn't" are quantifiable. But they do provide a relative framework by which you can interpret my reaction.
If you rule like Megatron, you're going to raise Starscream.
I understand what they intend to do. But they create a whole host of problems (particularly when aggregated.)
"Really" "kinda" and "didn't" are all fine and good and say the same thing 9, 6, and 3 intend to say - only they allow for context to matter more because I'm not comparing a 9/10 sandwich to a 7/10 becoming a parent.
Rotten Tomatoes dumbs down the cultural conversation about a film. If you remove review scores but keep the same words in reviews, I bet Sterling doesn't get a DDOS attack. There's a lot more interesting things you can do with an aggregator site than average things out. What words do reviewers tend to use about a film? Show me that graph where the most common ones are larger. What reviewers have similar taste to me, and what did they think about this thing? You can collect the same information and utilize it in a way that's a lot more informative and acts as conversation driver instead of a conversation stopper.
What I guess I'm saying is I'm not against aggregator sites per se, I just want to see one that is more about being a nexus that can connect people to critical voices they might appreciate and furthering the conversations while still fulfilling the stated goal of being useful shorthand.
Maybe its time to look into forming a start-up...
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
Isn't context baked into the number anyway? My 6/10 for The Force Awakens already takes into account the context of my personal history with the SW franchise, what kind of mood I was in walking into the theater and the sum total impact that the hype machine has had on my expectation.
One thing I'll absolutely get behind - my number means nothing when compared to your number. In the context of a review aggregator, that's key, and I agree that it seems to make the aggregated number less meaningful, as my 2/10 movie may be your 9/10 movie.
But my inner engineer wants to point out that that exact problem has a name, and that problem is called "noise". The solution to a noisy signal is to aggregate it with other, equally noisy sources. The noise roughly cancels out and what you're left with is mostly signal. And I'd warrant that it mostly works. Take a look at all the movies/games that scored in the 90s last year, and compare with those that scored in the 70s. Which set did you enjoy more?
You've never known true joy until you've shaken a lich stick at someone.
Context is baked into a discussion, but numbers move you further away from it because they represent a consistant value system. My 4 and your 4 don't mean the same thing even though they are the same number.
Our inner engineers like the order numbers can provide, but I think our inner artists would like to know more about the more ambiguous things - emotional impact, resonance. Those sorts of things. Because it isn't just about "enjoyment." Is the experience unique? Is it polished? Is it empty? Is it messy? Is it exciting? Is it boring but captivating?
Numbers are an interesting data point, and I could see some utility there, but they don't belong at the front of the discussion, IMO.
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
Which discussion?
The consumer discussion? i.e. should I go see this movie?
The critical discussion? i.e. how does this movie stack up against the last 100 years of cinema?
The cultural discussion? i.e. what does this movie say about the society in which it was made?
Because if you're not talking about option A, then why are we talking about review aggregators at all, which are explicitly designed to BE that conversation.
You've never known true joy until you've shaken a lich stick at someone.
A-ha - that articulates pretty sharply why I have a distaste for aggregator sites. Because the consumer discussion the least important of those discussions. In an ideal world, that discussion is rolled into the cultural discussion - a small part of it. Not dominating the entire discussion. Not causing strife or being a thing upon which people's bonuses are hinging on.
This is why I'd love to see an aggregator that is actually speaking to the experience of a thing, not rating it or trying to put it into market terms.
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
Personally I like both qualitative (the lawyer in me) and quantitative (the accountant in me) reviews.
Typically when I walk out of a film I've watched with my family, the discussion has sounded like "I (dis)like(d) it, x/10. Yada Yada here's why...".
From a content creator or service provider perspective, it sucks to see your efforts boiled down and reduced to a number. As if the homogenisation caused your content or service to be indistinguishable from any other.
That said, benchmarking via aggregation may be useful but it depends on the legitimacy of the individual reviews. For example, Malor in our community will almost always 0/10 something with DRM; but taken out of context and without that background knowledge we would only see a 0/10 rating. My criticism of metacritic lies in how user reviewers tend to rate excessively in either direction, and sometimes simply to counterweight a particular movement (eg ME3 down votes to 0 vs up voting to 10/10 in the context of fan base dissatisfaction with the warscore and tricolour ending mechanic). Thing is, scores of 1-3 or 9-10 should be surpassingly difficult to achieve - simply from the law of averages. But they are handed out very frequently on aggregation sites.
Once upon a time, I knew the guy who originally made MRQE, back in... uh... some time in the late 90s. I think the philosophy he had is a good one:
Collecting reviews and scores in one place is useful. Giving an aggregate score is less useful. You want to provide people with enough information to get an overall sense of what opinions of the movie is, but then let them look at the individual reviews based on whose reviews they trust to fit their own opinions. (In fact, at least at one point MRQE would watch which review links you clicked regularly and move those up the list towards the top for you, so you could more easily see scores and get to the full reviews from your preferred reviewers.
The current site (I don't know if he's still involved--I'm pretty sure someone may have paid him a lot of money for it at some point) does provide aggregate scores, but does so along-side a graph showing the distribution of reviews. More information means more of a sense of how it's getting reviewed.
In short: Distilling things down to a single average score is rubbish. Show the distribution. Encourage people to drill down to individual reviews.
And things like bonuses pegged to average review scores is beyond rubbish to utter bullsh*t.
slender Aphrodite has overcome me
with longing for a girl.
Oh my goodness, a D&D topic I can actually weigh in on and not feel like I just made a damn fool of myself after. WHOO!
Firstly, in regards to people being assholes, I try to keep reminding people that when Sir Arthur Conan Doyle originally killed Sherlock Holmes, fan response was pretty damn drastic. I could have sworn he also received death threats but I cannot find a proper citation for it. Even so, I know death threats for fictional universes are older than the Internet.
Writing "You brute!" certainly has more class than the vulgarities of today, but honestly, the only real difference from then and now is the avenues with which people can express their displeasure. We can discuss whether people are worse now than ever before, but I think that goes beyond the scope of this thread.
Now then, in regards to reviews, scores, and aggregates, I actually have spent a lot of time thinking about this, and what I have concluded is that the separation of purpose is along the right track.
I view it as criticism for academic purpose and criticism as consumer advice. Criticism as consumer advice is always going to be rough going because it's trying its best to objective, and while I don't scoff at the notion of trying to achieve as objective a perspective as you can (to try and divorce your biases from the quality of the product in question), the fact is that it's impossible for multiple reasons. The first is in the context in which a product is being presented to the audience itself.
A toaster may rate highly for your average at-home consumer, but it may be lacking in features of someone that owns a diner and needs to be making ten slices of toast ever few minutes. The toaster that diner owner is going to need is going to be overkill for most consumers. What you then create is a divide in specialized tastes.
The score itself is not the problem so much as removing it from the context in which it is presented, which is where aggregators themselves can be a problem. Context is removed. I think the goal of Rotten Tomatoes – percentage of positive reviews – is actually a good one, but if you take a horror film and throw all reviews together, you're combining specialized horror aficionados with people that may have a complete distaste for horror. Even then, you have horror aficionados who favor these sets of horror sub-genres while another favors these sets.
You can get pretty damn specialized. So does that make Rotten Tomatoes worthless?
Not necessarily. Consider someone that is in the mood for a new movie. They got a free evening and want to watch something. So they're scrolling through Netflix and see a flick with an interesting description. They can:
1) Just go in blind and find out for themselves
2) Search for an individual review or set of reviews to see what someone thinks, a move that may risk spoilers and may require a lot of reading and searching depending on whether they have a stable of writers they follow
3) Check Rotten Tomatoes and see the percentage of reviewers that responded positively.
Now I know Rotten Tomatoes determines what is "fresh" and not based on some flawed metrics, but it is accurate enough of the time that the average consumer is best served in this scenario by choosing option 3. Option 2 could help them discover reviewers and writers with similar opinions and informative insights in the long term, but it could also eat up an evening they would have otherwise spent watching a damn movie.
Personally, I'd rather people read reviews and get the text as well, but man, even I don't have time for that. I try to read other gaming websites when I can, but most of the reading I do is one site during lunch break.
As someone that loves to discuss and analyze games in a more critical/academic/cultural/however-you-wanna-view-it manner, I've had to learn that I don't have a very large audience. The largest audience is, in fact, on YouTube, which is its own can of worms. But people just don't make the time to read.
If you take away the aggregates and the scores, people aren't going to learn to read reviews. They're just going to post on Facebook or post a Tweet asking what friends thought. So maybe social media really is to blame, but honestly, was more intelligent writing valued by that many beforehand? Or was there always that stereotype of the critic being snooty and disliking everything?
YouTube | Twitch | Blog
SW-3864-9679-7572
I'm very much in agreement with Hyp, who I think articulated what I've been trying to say better than I did using significantly less words.
I do think it is within scope to discuss if fan behavior is worse than it has been. The Doyle point is interesting, but I think it is something of an anomaly for its time. Simply put, it wasn't as easy for trolls to lash out at artists or cultural figures. Importantly, if someone might have had the impulse to do so, they weren't seeing the behavior modeled successfully.
This is where I find metacritic and the like troubling. They are not only ripe for trolling - Zelda fans artificially giving 0 to Horizon despite not even playing it because they want to "win" the score - they don't have a good toolset for discouraging the behavior. If Doyle was alive today, he most certainly would have received death threats from his fandom - in volumes and intensity that I would be willing to bet would make the letters he received in his actual day pale in comparison. Because not only do people have more tools with which to reach him, they could also reach him in a very public way and see other people exhibiting the same behavior without any social or legal repercussion.
To me this is all wrapped in the conflation of capitalism and art & entertainment. Yes, entertainment is - in our society - a product. But it isn't a toaster. A toaster is purely a product. When we look at reviews for a toaster we want to know how well it works. It has no cultural impact. It does not reflect the life we lead. It burns bread. A number or a binary (thumbs up - it works, thumbs down - it doesn't) is fine shorthand for that.
Art and entertainment need more than a number. They aren't purely about being functional, they can fail at their function and still succeed on certain levels. They are better served with words than numbers.
Like Hyp said, a single average is rubbish. Something that got 3 reviews a 10/10, 9/10, and 1/10 tells a significantly different story than something that got 7/10, 7/10, 6/10. But an average doesn't tell any of that story.
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
Otherwise known as Hyp's superpower.
"What forest are you talking about?! I can't see anything with all these trees in the way!" ~Farscry
Here's an interesting article from Owen Good on the Sterling / Zelda situation. It goes into some interesting territory on the capitalism / consumer angle that dovetails nicely with some of things I've said.
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
I'm entertained by this analogy, primarily because your main point is that the reason a review number, and more specifically an aggregate of review numbers, doesn't work is because it isn't nuanced enough - and then you offer an example with zero nuance that would work for a lot of people but not for people looking for more information. (What if I only toast bagels? How long does toasting take? How accurate is the "toasted-ness" setting? Can it handle different bread sizes and thicknesses? etc)
Shorthands are useful, but not the be-all end-all. In general, a score aggregator answers the question, "How do most people seem to feel about this thing?" It isn't the right tool if you, as a consumer, want more than that, but a lot of people want a thumbs-up/thumbs-down quick take on the gestalt reaction.
(Side note: thoughts on Siskel and Ebert's thumbs-up/thumbs-down dichotomy as it relates to lack of nuance?)
EDIT: an aggregate of three scores is silly, and is a good example of a false equivalency. If you had 100 scores, and 66 of them were 10/10 and 33 of them were 1/10, then you're on to something. Normally, though, the outliers are just that - outliers. (And you're much more likely to have 90 10/10s and 10 (or less) 1/10s.) The aggregate is actually helpful in this case, as it reduces the impact of noise.
If you rule like Megatron, you're going to raise Starscream.
But I think you can get those answers about how do most people feel about a thing without a number and still have that utility. The number is the language of capitalism, and it does a disservice to the artists and the artistic endeavor (see also getting scores tied to bonuses.)
If Sterling's short hand review was "Zelda is beautiful and fun, but also extremely frustrating" (which is the the thrust of his review) instead of 7/10 I don't think he wouldn't have been DDOS attacked.
I used 3 scores as an example because it was easy to do. I figured if people needed to they could round up, (like you did.) The point remains the same. And outliers actually DO matter in art in a way that they don't in products.
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
I feel like you're taking where two separate concepts intersect and treating it as a whole. A review aggregator has purpose to consumers, yes. But there are consumers that identify very heavily with the products they consume and are unable to remove themselves from that product. I wouldn't so much call this capitalist as I would consumerist culture, as we live in a country where the vast majority of people are informed that they need to consume. Food, drink, media, what have you, we need to consume. Why? Because we're the Pepsi Generation, bro! Just Do It and Obey Your Thirst, etc. etc.
This also, I believe, ties into this whole "Let Your Geek Flag Fly". I know I identified as a geek and gamer when I was younger because it was a way to own the things that made me a loser and outcast in everyone else's eyes. This meant I took ownership of the things I loved. "Here, friend, let me show you this amazing film called Ghost in the Shell, and after that we can sit slack-jawed as we stare at Akira". Hell, when I switched schools in eighth grade, my first friends in the new school were made because they saw me playing Pokemon Red. Games, and Pokemon specifically, become part of my identity.
But through this method of defining who I am, it caused me to respond aggressively to what I was not. Rap music? Trash. Pop starlets? No talent sluts! Teen drama television? Technicolor vomit! I don't know if it was myself, my brother's influence, or a naturally occurring self-defense mechanism, I became certain that I was superior to these idiot masses, and that the things I enjoyed were enjoyed because I was smarter and better than them and their pedestrian tastes!
I grew up, but to this day I still have friends that look at the mainstream place games and comics and such have been taking and they feel like they're surrounded by people superficially imitating geekdom. "They didn't earn their stripes!" they call, because it's easier than ever to be a cool kid that can admit to liking comics. The irony is that these people Female Doggo about identity politics infecting College campuses and other such things.
The problem is how people allow these things to identify them, and that runs much, much deeper than review scores.
This is also a separate problem from companies determining bonuses and pay-outs on metacritic or rotten tomatoes scores. This is especially under-handed of companies given how easily users can go in and try and f*ck with the numbers, or how many websites with absolutely 0 prestige can get in and add their own review. MetaCritic in particular presents a problem once you consider people using five star reviews, and estimating that up to a 10 or 100 point system.
But, what is it we are trying to determine here? Are review aggregates a useful tool? Do review scores have a place? Or should we get rid of them because they can be misused by businesses or malcontents? Because removing the scores or aggregators aren't going to solve the latter two problems.
YouTube | Twitch | Blog
SW-3864-9679-7572
I don't think we should get rid of review scores simply because they can be misused - I think we should get rid of them because they actively impede our ability to engage with art & entertainment in a meaningful way. The sole benefit that they give us - shorthand - can be attained through other techniques that aren't as problematic and diminishing of the form.
My argument is that the language of capitalism doesn't do a great job at interpreting art. It has a place at the table for sure, but certainly not at the head of the table. It has us looking at the wrong things and dismissing some of the most important things as it stands now.
On a personal level, I'd love to determine the way an aggregate site might operate that isn't based on averaging out a score. What could that look like? What are the other possibilities, even if you are a person who enjoys the way MC and RT function now?
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
I don't disagree with the sentiment here. Obviously I don't, because my writings are about how I engage with my art, and overall try to encourage people to engage with art in ways they may not have otherwise considered.
But I don't want to be bothered to sit down and learn the basics of engineering.
Which may seem unrelated, but at the end of the day, engaging with art is an interest. Some people want to do it, but I don't think the majority really have the inclination. Their engagement is dedicated to other things. Other hobbies and interests. My sister is too busy with her job, fixing up her house, and raising her kids to want to think about the television or film she's watching. That's not what she uses entertainment for. Entertainment is the one thing that she can not think about.
On the other hand, I love to think about the entertainment I absorb, but would rather not put in the effort into growing a garden that she does. To her, growing a garden has benefits. To me, it's just another chore I gotta worry about interfering with my hobbies.
My experience is that people don't want to think about their entertainment all that much. Close friends of mine learn to value my thoughts, but typically people want something simple. "Is it good or bad?" Which leads to:
Yes, it's just as diminishing. People want to know if something is "good" or "bad", which, again, is the general conceit Rotten Tomatoes is built off of. Percentage saying "good". It's reinterpreting the data numerically, but it's the general populace's desired answer. Concise and easy to interpret.
Once you try to make it more complicated than "good" or "bad", you're losing hold of what people actually want, which is a simple recommendation. And even then, let's take your summary of what Jim Sterling said:
Simple summary, right? But... wait, how can frustration be fun? That doesn't make any sense, and I need to read his review to know more.
Which is what you want people to do, yes. And I want people to do it, too. But, it doesn't mean that'll happen. You know what's more likely, and it's something I myself am guilty off? "I heard, based on this small snippet with barely any context, that Breath of the Wild was actually pretty frustrating."
Do you know how many times I have to make corrections with friends because they only read headlines and are going off of information without context? Or are basing something off of the knee-jerk response of a YouTuber? And, again, this is something that even I'm guilty of, and I'm certain a lot of people here are. Even the smartest people can be lazy idiots.
Who says scores and aggregators are capitalism talking? I would argue even award shows aren't necessarily capitalism, though they've certainly got their marketing value. It's trying to reinterpret art through metrics and data. Statistics and mathematics.
Which, in a lot of ways, I think is an inevitability when you're trying to analyze art. Think about how people talk about techniques in editing or cinematography, or even aspects of game design. It's often a sort of scientific approach to understand what methods get what responses from the brain. These techniques are then imitated or taught in schools, as if they can simply be replicated. And in some ways, they can be! You can make a perfectly serviceable product by simply imitating what others have done by breaking it down into a rigid, cold, scientific understanding. "If I implement X, Y, and Z techniques, I will generate response B and D in the audience/player".
This isn't capitalism, though it can lead to a lot of assembly line product. It's just what happens when you stop for a moment and ask yourself "Wait a minute, why did that work?"
Aggregators are similarly trying to take something and make it mathematical. Trying to approach something from a scientific perspective. But the thing about business is that, if you go and get a business degree, the only thing they're not going to teach you is about the human element of creation. So business people view everything without that human element.
Honestly, I feel like you need a way to convey likelihood of an audience enjoying something based on interest metrics (once again, cold, hard mathematical numbers). Let's once again take the intent of Rotten Tomatoes, where the percentage should be the odds that the audience will enjoy something. Even if something is at 15%, that means 15% of the people that see that movie will find it enjoyable.
This is by no means an infallible system, what with being based on subjective human opinion, but if we're removing the concept of quality from the score, then we're not declaring the movie to be good or bad. We're simply saying there's a smaller audience for something than something else.
But the biggest problem with this system is who you have writing about movies. A lot of movie writers are, unlike their audience, people that engage more deeply with their films. Michael Bay's Transformers score low, and yet the sales numbers continue to be high. So now, instead of determining the percentage chance members of the audience will enjoy the film, we're determining its quality regardless of popularity.
So what do we want to do? Summarize the quality or provide useful consumer advice? Personally, I think reviews as consumer advice are best left to specialized publications or websites. Have horror aficionados review horror, sci-fi review sci-fi, etc. But even that comes with baggage, because in the end, you're trying to apply one person's experience to a broad concept: what will people think as opposed to person.
And this is why I love discussing this topic: because it continues to allow me to recognize the flaws of the review concept as a whole and see if there's a way to tackle it.
YouTube | Twitch | Blog
SW-3864-9679-7572
At the risk of turning this into yet another "Are Games Art" conversation, I can't help but feel that your argument rests upon the supposition that they are.
Because you seem pretty insistent that it's the aggregators that are preventing us from engaging with games in meaningful ways, where Occam's Razor points towards the much simpler answer that, for many (dare I say, most) games, they're not designed to be engaged with in meaningful ways. They're usually not artist's meditations on the individual's place in society, they're cool ways to blow sh*t up.
You've never known true joy until you've shaken a lich stick at someone.
There's a lot there, but to respond to part of it...
Removing scores doesn't mean removing dealing with the concept of quality. I think there are more imaginative ways you can indicate quality that don't boil down to a numerical shorthand. Some thoughts (off the top of my head) - those word graphs where the more a word is used the larger it appears. A few bullet points about common comments made from a variety of reviewers. Perhaps you can create your own profile and input things you enjoy and things you don't, which matches reviewers with similar taste to you.
I get that some people prefer thoughtless entertainment at certain times, but I have a few thoughts on that. For starters, I do think we have a culture that isn't comfortable with ambiguity. I've talked about this at length in many places (even on your podcast I think!) but a lot of that is indirectly the result of things like numerical ratings. We're trained to want the surface read and to be skeptical of anything existing under that surface. We're quick to call something "pretentious" if it is trying to engage with complicated ideas.
I'd make the case that the desire for us to "not think" about our entertainment doesn't exist. Yes, there are people who don't want complicated or "challenging" entertainment at the end of a hard day - but that's a different thing. They want to turn their brain off, but the thing that allows them to turn their brain off still needs to be thoughtfully executed and competent. It needs to grab their attention and imagination enough to jettison the daily anxieties that are wearing them down. A lot of so called "mindless" entertainment is itself smartly executed. There's good trash, and there's trash trash.
This is probably a little outside the scope of the thread but I'd also make a case that a healthy cultural diet involves the ability to enjoy all of these things at various times (even trash trash.) The fact that your sister is so zonked out from her life that she can't even begin to access a healthy cultural diet feels to me like a sign of a deeper flaw in our entire societal landscape. I don't think she's alone in that type of situation at all. And this is how all of this ties back to capitalism, and the problems with capitalism. Much like everyone should have access to clean water and healthy food, everyone should also have access to a wide array of art & entertainment. But in capitalist systems there are people who can afford - either time wise, money wise, or "energy" wise - to get fast food. And they can only afford to get the cultural equivalent thereof. Obviously that's a massive systemic issue (and certainly outside of thread scope) but it does bring me back to capitalism.
Review scores ARE the language of capitalism, because they are about assigning a value measured in units to a thing - and in some cases that unit DIRECTLY impacts pay. For games, they come from rating electronic products on how well they perform (prior to games becoming widely understood to be an artistic medium on par with books, TV, and film.) But as games are a form of art, their impact can (and should) be assessed in other ways. What's the resonance? How does it affect a general audience? How about a specific community? Is it successful at letting me "turn my brain off" after a hard day at work? Does it inspire me, or light up my imagination? Does it change the way I think about a thing? Did it make me uncomfortable? Was that discomfort ultimately positive? Some of those things are contrary, and numbers do a disservice to that.
From the Owen Good article I referenced upthread, relevant to the capitalism angle here:
This is why I want to try and envision what a culture hub might be like that does away with numerical value. I want something that can give people the quick and dirty version while also encouraging more spaces for people to access the more ruminative or thoughtful side of things. I want something that opens up the possibility of more people getting healthier cultural diets, not something that codifies an existing problematic system. And even if someone truly only wants one thing and that thing is trash trash, I want this system to also be able to help them along the way there too. That's what I'd be interested in hearing about.
If you like numbers and don't mind the system as is, I'm still interested in hearing what some non-number solutions might look like to you - as a thought experiment at the very least!
EDIT TO ADD - Jonman, that's why I'm specifying art AND entertainment. (And there is still artistic craft in entertainment.)
LastSuprise: Destroyer of Wallets would be an excellent tag and/or third handle for you.
https://twitter.com/theharpomarxist
So one of the many YouTube people I subscribe to had some interesting thoughts on the matter.
Curiously, I wrote about similar situations myself, but coming from a different angle. I titled the piece Dragon's Deadline and suggested that some of the features considered "flaws" may have seemed less so if the reviewer didn't have to worry about a due date and could just play through the game at their own pace.
In the end, I think both angles are true. The deadline can positively or negatively impact your view of the game. What I also agree with is that a truly good analysis won't really happen until some time down the line, after the game has become a lot more familiar.
YouTube | Twitch | Blog
SW-3864-9679-7572
I like gamerankings. Don't like the part where they took out the pie charts that showed you the score distribution at a glance (years ago), and the fact that the site sorely needs a redesign to bring its looks to the modern era, but oh well. It's a good tool to have all the reviews listed, and people are unavoidably going to want a summary of that.
It's also something I would point a non-gamer to to explain the quality of a game – many reviewers felt this way about it.
In terms of What I Think Other People Should Do, I wish people's numerical literacy was a bit higher to understand 78% means "more 8s than 7s" and significance, like 98% might not be a significantly higher score than 96.7%, so chill out.
Also it bothers me how many decimal places boardgame geek leaves on its review averages. The second and third ones mean nothing, just take them out.
Edit: My magical solution to this problem is to change all review scores to a word. You get one word.
><( ^_/)?
The only rankings system I ever liked was Crispy Gamer - Buy it, Try it, Fry it. Succinct and expressive and encourages closer reading and consideration.
Struggling to sleep? Try my sleep podcast! Sleepy Time Tales.
Need Podcast editing? Go to Bright Vox Audio
Wow. Crispy Gamer. Now there's a website name that takes me back.
YouTube | Twitch | Blog
SW-3864-9679-7572
I miss this little guy.
Steam: http://steamcommunity.com/id/biginja... | XBOX/PSN: BigInJapanGWJ