Who Is to Blame for the State of Game Scores?

[tmeacham]

Article by Travis Meacham.

Whether or not to include a numerical score in a game review is a constant debate on forums across the Internet. Tom's Games column Side-Quest addresses the issue placing blame on both readers and reviewers alike for the current state of the scoring system.

http://www.tomsgames.com/us/2008/08/12/column_gamescores/

 

[jimr9999us]

Until gaming gains widespread acceptance by the general media, there will be an inherant bias in gaming reviews. A game review on a gaming website that exists because of developer/publisher ad dollars will be held to a higher threshold of criticism; the reviewer will need concrete reasons why they didn't score the game well. I certainly don't blame the reviewer or the website for this bias, no more than I would blame Variety for lavishing praise on a film that cost $200 million to make but left audiences unhappy: gaming sites are basically trade publications, more useful for news and entertainment than criticism or insight.

Once this generation of young gamers grows up, and gets degrees in journalism, and starts reading newspapers, there will be a shift to objective, mainstream criticism of games. Once this occurs, we will see games more like "Braid" or "Mass Effect", games that could be described as art as well as entertainment. The industry cannot continue to innovate indefinately through the stewardship of a few key developers. I look forward to being able to choose between embarrassing a friend online in a frag-fest, or sipping on a beer playing a character who maybe gets the girl, but doesn't save the world.

The future of gaming is bright indeed, so long as we continue to be malcontents.

 

[SEALBoy]

It's going to very hard to change the "7-10" scale. This scale isn't the result of inflated grades or paid reviewers. Everything we did in school and college was graded on a "7-10" scale and below that was fail. It's similar with games. 7 is average, 6 is below average and below 6 is fail. There is no reason to change this scale, all you'll do is confuse everyone.

 

[robwright]

It's going to very hard to change the "7-10" scale. This scale isn't the result of inflated grades or paid reviewers. Everything we did in school and college was graded on a "7-10" scale and below that was fail. It's similar with games. 7 is average, 6 is below average and below 6 is fail. There is no reason to change this scale, all you'll do is confuse everyone.

Huh?

SealBoy, I'm curious: where do you hail from? I don't ever remember being graded with numbers in grade school or college here in the U.S. Then again, I only went to a few schools, so what do I know.

In any event, you may be correct in your claim that using a true 1-10 review scale will confuse people here at Tom's Games that are used to seeing 7-10s and consider a 6 to be a complete failure. We plan on clarifying our scale and review system in the very near future, prior to the slew of games lined up for this fall.

 

[San Pedro]

^ Um, haven't you heard of someone telling you there Grade Point Average? It's presented in a number.

I think that he is talking about percentages though, and it has quite a ring of truth to it, at least until college where they use curves much more often. Anyhow, it is common in schools for grades to break down like so:
100-90 = A
89-80 = B
79-70 = C
69-60 = D
Everything under is fail, and even a D is pretty much a fail in some instances, like in regards to your major.

 

[robwright]

^ Um, haven't you heard of someone telling you there Grade Point Average? It's presented in a number.

I think that he is talking about percentages though, and it has quite a ring of truth to it, at least until college where they use curves much more often. Anyhow, it is common in schools for grades to break down like so:
100-90 = A
89-80 = B
79-70 = C
69-60 = D
Everything under is fail, and even a D is pretty much a fail in some instances, like in regards to your major.

I'm flabbergasted. I'm totally serious, I never knew about this. And I'm not making light of it. It's just that it still doesn't make sense to me. If game review sites/mags want to make 6 a D or or an F...then why not just use a 1-5 scale? I mean, you're only using a few numbers with the 7-10 scale, so just ditch the rest of the spectrum.

 

[jimr9999us]

I'm flabbergasted. I'm totally serious, I never knew about this. And I'm not making light of it. It's just that it still doesn't make sense to me. If game review sites/mags want to make 6 a D or or an F...then why not just use a 1-5 scale? I mean, you're only using a few numbers with the 7-10 scale, so just ditch the rest of the spectrum.

Sorry to say Rob, but Metacritic dictates the use of a 1-10 scale. Metacritic also proves game scores are inflated as well.

Hmmm, doesn't cnet own metacritic?

Doesn't cnet also own Gamespot? ... Uhoh.

 

[engrpiman]

As long was we are at it lets create a Gaussian Distribution curve (bell curve) that way most games fall with in one standard deviation and only the really bad games and really good games stand out. that's the way it should be done.

 

[SEALBoy]

I'm flabbergasted. I'm totally serious, I never knew about this. And I'm not making light of it. It's just that it still doesn't make sense to me. If game review sites/mags want to make 6 a D or or an F...then why not just use a 1-5 scale? I mean, you're only using a few numbers with the 7-10 scale, so just ditch the rest of the spectrum.

Here's a basic example of what happens. In school you take a test of 100 questions. You get 30 wrong, that's 70. Below 70 (and in some case 60) is classified as fail, even if you got more right than you did wrong. That system has translated to game reviews. I'm not saying that there's no inflation, but even without inflation games worth playing would be rated on a 7-10 scale. It's just the way it works. There are a lot more "fail" grades than "pass" grades. A 5-6 is by no means a complete failure, but, strictly speaking, it is a failure nonetheless.

My advice: do NOT change this. No matter how much you clarify your position on this, you will be going against EVERYONE'S mentality. There's nothing wrong with the system, you've just mistaken inflated reviews to represent an inflated scale.

 

[robwright]

Here's a basic example of what happens. In school you take a test of 100 questions. You get 30 wrong, that's 70. Below 70 (and in some case 60) is classified as fail, even if you got more right than you did wrong. That system has translated to game reviews. I'm not saying that there's no inflation, but even without inflation games worth playing would be rated on a 7-10 scale. It's just the way it works. There are a lot more "fail" grades than "pass" grades. A 5-6 is by no means a complete failure, but, strictly speaking, it is a failure nonetheless.

My advice: do NOT change this. No matter how much you clarify your position on this, you will be going against EVERYONE'S mentality. There's nothing wrong with the system, you've just mistaken inflated reviews to represent an inflated scale.

Well, now I know why some people freaked when I gave Assassin's Creed a 5. I thought 5, being in the middle of 1-10, was the literal definition of average. But according to everyone else, a 5 is god awful/inhumanly bad.

I understand your point now, SealBoy, but I don't agree with keeping the system the same way. I think it's better to use the entire spectrum of numbers at your disposal in a 1-10 scale rather than limit to just half that amount. I think you can get a bit more granular and detailed with your opinions, too, rather than being stuck with 9=great, 8=good, 7=average, 6=sucks.

 

[Chazwuzzer]

My advice: do NOT change this. No matter how much you clarify your position on this, you will be going against EVERYONE'S mentality. There's nothing wrong with the system, you've just mistaken inflated reviews to represent an inflated scale.

Yeah.... Not so sure that I agree with your reasoning there. You condemn "going against EVERYONE'S mentality", but caving into the expectations of the lowest common denominator is exactly what is causing ridiculous inflation. It is an endless poisonous feedback loop. Your fallback position appears to be that the game scores scale to the A-F grading system, but that is not acceptable because (1) school grades are undergoing crazy inflation and (2) game reviews are even worse!

A perfect score or even near perfect should be associated with a product that gamers will be playing for years - A game that you keep that old console around for.

I don't expect things to get any better - how many reviewers gave a perfect score to GTA4? That cherry is totally popped and it will be that much easier to go there next time. Sooner than later we'll be on a 9.5-10.0 scale with idiotic drooling fanboys whining when their overhyped game does not get a 10.

Bah!

 

[San Pedro]

I'm flabbergasted. I'm totally serious, I never knew about this. And I'm not making light of it. It's just that it still doesn't make sense to me. If game review sites/mags want to make 6 a D or or an F...then why not just use a 1-5 scale? I mean, you're only using a few numbers with the 7-10 scale, so just ditch the rest of the spectrum.

Wasn't trying to discount the thesis of the piece. In fact, I think 1-5 scale would make much more sense (I think the show on G-tv does this, but they hardly ever do PC games). I never really see any game with a score under 6, or 7 for that matter, but I guess one could argue that most games have some form of enjoyment from them. I haven't played anything that I would think would be under 5 on a 10 point scale, but I never played any Barbie games either. . .

 

[infornography42]

I can see both sides on this.

On the one hand, the standard 6 or 7 to 10 scale is what most people are used to, on the other hand a larger scale would provide more useful information.

Even with the standard old 7 to 10 scale though, you will still occasionally see games getting 4 or below. This could be seen as an exaggeration of how badly the game sucked but pretty much any game getting scores that low do indeed suck and below a certain level of suck the opinion of it's utter lack of quality will be close to unanimous.

The 7 to 10 scale therefore may actually be beneficial as a reminder that while the game was not absolutely superb (9 or 10), it was still a good game (7 or 8) and if certain aspects of it appeal to you, buy it. At least that is how I always interpreted it.

At a score of 6 it is teetering on the brink of unplayability but some enjoyment could still be found. These games would be bargain bin treasures. Certainly not worth full value but they will quickly decrease in cost and might be worth $15 or so.

That said, all of this should merely be used to augment the real meat of the review, which is the words.

 

[SEALBoy]

Well, now I know why some people freaked when I gave Assassin's Creed a 5. I thought 5, being in the middle of 1-10, was the literal definition of average. But according to everyone else, a 5 is god awful/inhumanly bad.

I understand your point now, SealBoy, but I don't agree with keeping the system the same way. I think it's better to use the entire spectrum of numbers at your disposal in a 1-10 scale rather than limit to just half that amount. I think you can get a bit more granular and detailed with your opinions, too, rather than being stuck with 9=great, 8=good, 7=average, 6=sucks.

I'm completely against grade inflation, believe me, it sucks that the number of 10's GameStop handed out this year is equal to the TOTAL amount they had handed out in the past decade. But I don't think the scale should be changed. One problem I think you guys are facing is that you review only a few select, high-profile games that would probably land between 7-10 on ANYONE'S scale. When you start reviewing every game under the sun, you will use the whole scale and you will find out the usefulness of the 1-6 scale in differentiating absolute WTF garbage and a playable, albeit mediocre, title. Just use 0.5 divisions if you need more space on the 7-10 scale.

 

[tmeacham]

I actually prefer smaller scales. For instance the way rotten tomatoes does it. I'm not talking about the tomato meter I'm talking about how the reviewers that submit their reviews pick either good or bad and that's it. So simple. Same goes for Ebert and Roeper. It's thumbs or thumbs down.

But I do see how that becomes somewhat impractical for games so beyond a point scale I prefer the "pass it," "rent it," "buy it" scale. After all, isn't that the information you want from a review. "Should I buy this game?"

And no... we won't be adding a "pirate it" as a tier in the scale.

 

[SnareSpectre]

I have to agree almost 100% with SEALBoy on this one. I think the scale of 1-10 works quite well (granted reviewers are unbiased, and that's hard to do). I read something that I really liked from (I think) Greg Kasavin of Gamespot at the time; he said that their average score is a 7 because, in general, games are good. A 7 (or 70%) in school is a C, which is considered "average." I like this scale because I completely agree. Heck, a game has to get at least a few points just for being able to run on someone's computer or console.

And on the issue of Gamespot or whoever being owned by Cnet - I actually prefer Gamespot to most other rating sites. I couldn't care less how they operate, because I compare the scores I would give to games to the scores Gamespot gives them, and I generally agree. There are definitely anomalies, like Assassin's Creed (holy crap, a freaking 9.0???? That game kind of sucked!), Halo 3 (I would give it an 8.0 or 8.3), GTA 4 (self-explanatory), or some underrated games like Titan Quest. I'd give Titan Quest atleast an 8.5, and I even sacreligiously like it better than Diablo, but sites docked it craploads of points for lack of originality. However, these outliers set aside, I generally agree with their scores. Therefore I go to look at the scores they put out and usually know whether or not I'll like a game based on their score AND review.

 

[SEALBoy]

Yep, you have to realize the difference between the scale and the scores. The scale has been the same since forever. You go back 10-15 years and "good" games still earned scores of 7-10. The problem lies with artificially inflated grades, but the scale does not cause that. The vast majority of game grades are still uninflated. Only in overhyped, extreme high-profile games does inflation really come into play because of the fear of backlash from fanboys as well as conflicts of interest in advertising. Personal bias also plays a role here (ie the reviewer desperately wants the game to be good).

 

[amdfangirl]

^ Um, haven't you heard of someone telling you there Grade Point Average? It's presented in a number.

I think that he is talking about percentages though, and it has quite a ring of truth to it, at least until college where they use curves much more often. Anyhow, it is common in schools for grades to break down like so:
100-90 = A
89-80 = B
79-70 = C
69-60 = D
Everything under is fail, and even a D is pretty much a fail in some instances, like in regards to your major.

In Australia:

Outstanding (A)
High (B)
Satisfactory (C)
Basic (D)
Limited (E)

All part of the rating changing thing...

I'm flabbergasted. I'm totally serious, I never knew about this. And I'm not making light of it. It's just that it still doesn't make sense to me. If game review sites/mags want to make 6 a D or or an F...then why not just use a 1-5 scale? I mean, you're only using a few numbers with the 7-10 scale, so just ditch the rest of the spectrum.

I see what you mean... most give 7-10s. I remeber a site somewhere that promises to use the full scale... somewhere. If you wanna use the full scale (which seems more logical) have text after it like average, commendable and etc.

 

[SnareSpectre]

Gamespot has given a game a 1.0 before (their lowest score), and several others have received between 1.0 and 2.0. They definitely use their entire scale...I just believe that they care more to rate games that people will actually play instead of wasting time on value games. People who buy value games aren't the same gamers who go to sites like these to check out game reviews. Someone earlier mentioned that Tom's only reviews high-profile games, and I totally understand - limited reviewers, so why review movie-based games when they're not the ones people are going to go play?

I agree with amdfangirl, but it should also be pointed out that several sites do already. I read reviews from Gamespot and IGN all the time, because I agree with their scores the most, and then obviously I read Tom's too. Both Gamespot and IGN have word scores at the ends of their reviews, and they both pretty much use a 6-10 scale. Maybe Tom's could adopt this too..I would definitely not be opposed to it. I was disappointed in Assassin's Creed, but I sure as heck thought it was better than a 5, lol.

 

[tmeacham]

Snare, that's really the best way to go about it. When it comes down to what the reader gets out of a review, no one reviewer has any more say than another so you have to sort through them, find someone you agree with most of the time and stick with them. It's all a crapshoot really.

At the same time reviewers do push their scores up to avoid fanboy backlash on the larger titles. And that I don't like. This goes back to the whole what-is-a-10-game argument. Not everyone does it like this but Rob and I have a tendency to score a game harder, wait for the backlash to start and then leap feet first into it to make our points heard.

 

[bokaj]

I also think the 1 - 10 scale is working ok.

It is only natural that most games scores 7-10 because only relative good games or games with high focus are reviewed. (and this applies especially to Toms Hardware!!!!)

Lots of games are released (at least for the PC) every month that no reviewer is looking at .. simple because they sucks and are low budget non-hyped titles.

"Tank Universal: Challenger Eight
Release Date: Aug 21, 2008
Take part in a 3D, "Tron-like" world as you make your way through 20 levels of tank battles. "

You think this would get a score above 7? We will never know because it will not be reviewed.

Actually Gamespot have given 7 out of the last 20 games a score of 5 or below (PC). It is because very few good games are released in the summer and there are time to review the bad ones too.

But all in all .. the system is fine.

And like one person said. It doesn't matter if a game gets 1 or 4 .. it suuucks.. but 7 or 10 .. thats another story.

 

[laharle_koa]

honestly the numbers or objects used in a scale don't matter. you can take a 7-10 scale and say 6 and lower means "pass it", 8 and lower means "rent it", and 9 and higher means "buy it". all the scales are the same as long as you explain them. What really needs to go away is the censoring.

Going back to gamespot and their kane and lynch review. The fact that the person who originally reviewed the game and gave it a bad score was conveniently fired soon thereafter and his review replaced by a much more favorable one (since every single add on the page for the month or two prior was advertising that game) means to me that any review from them possibly is a lie and has no bearing on whether or not the game is a good one.

I, for one, am a fan of both x-play (even though they have been dropping a little recently) and zero punctuation. x-play uses the 0-5 system and isn't afraid to use the 0-1 area. Also, as part of their rating, they will flat out tell you things like "If you are a fan of the series/genre, go buy this game. if you aren't then rent it". this covers the games like "hello kitty island adventure" where sure it might be a great game and have absolutely zero bugs and get a score of 10, but theres no way in hell im going to buy, rent, or pirate the game.

Yahtzee (zero punctuation) on the other hand doesnt even use a rating system. he says "this is what sucks about the game. if that bugs you then dont play it" along with the regular how he liked it. He also seems less biased than any other reviewer i know of.

So to sum everything up, ratings are lazy and don't really mean too much to begin with. Not everyone likes the same games. Reviews need to give information about the game so readers can make an educated decision.

 

[purplerat]

I've always been a fan of the multi-facited scoring system where several different areas are scored along with an overall score. I remember Nintendo Power magazine using this type of system way back in the day when deciding which game I would ask for Christmas was a big decision. Always seemed to work pretty well.

 

[infornography42]

I also like the multifaceted scoring.
Game informer has a pretty good system if a bit biased reviewers. They have been better lately than they were a few years ago though.

For instance I really value good character development and plot and bad gameplay can ruin any game. However I don't care quite so much about the graphics and sound.

 

[radnor]

In Australia:

Outstanding (A)
High (B)
Satisfactory (C)
Basic (D)
Limited (E)

In Portugal

1 to 20. 20 being "Perfect" or "Flawless Victory" and 1 being "Why the sack did you even come" ?

Rating games from 7-10 is silly imho. The scale is 1 to 10 in the different areas. You can do a average of the different points being evaluated, but lets be honest, if a game is a 5 it is a 5. If you have your method of evaluation pretty well explained, the rest is a matter of opinions. And opinions, well, mostly aren't so founded or backed by methods. Just moronic opinions sometimes.

I would never consider BattleToads a good game. Maybe a 2. But there people that liked it.