Second Take: Fixing Game Reviews

[bennyblanx]

Scores or no scores? Grade inflation. Comparing ratings. In light of recent controversies, Ben and Rob discuss how best to do game reviews.

Watch Video: http://www.tomsgames.com/us/site/flash_videos/second_take_fixing_game_reviews.html

We take forum feedback seriously and thought we should do an episode based on some of the discussion threads from the Assassin's Creed, Kane and Lynch reviews. and last week's Gamespot-gate episode.

-Ben
ps. What do you guys think of the new intro?

 

[_Morphine_]

Track Record.

Some critics you listen to and some you ignore. Its takes time to get to know a critic. His job is to give his opinion. I dont want a number. I want an impression of the game from a person I trust. (or dont trust, and therefore have an idea of what I will think)

Rob has proven to me that:

A) We enjoy similar aspects of games.
B) He is as conscious as I am about balance. (graphics vs gameplay, teamplay, stratgey)
C) His value of money and time is similar to mine.

(EDITED) I read/watch his reviews because it isnt a waste of my time. (unless Ben is talking) I guess what Im saying is you dont have to agree, just understand where they are coming from. A number cannot reflect an impression or feeling. I cant read a number between the lines. Just continue to be yourselves... Even you Ben =)

To change how you do things is catering to people with no taste. They will play and love anything, screw them.

PS. Screw the intro too.

 

[TenBeers]

OK, maybe I was a little critical of your criticalness (is that a word? it is now!) about Assassin's Creed.

You are right, it's a painting that you didn't like, but I liked it quite a bit, and most of the time we like the same things. And there's nothing wrong with that. Looking at the other posts, it's obvious that people either love it or are disappointed (not quite hate-it). But your review was very clear about what you thought was good, what you didn't like, and the rating you gave it was reasonable for your experience.

I see your dilemma. If you don't like something but think other people might, then you'll end up giving all your scores in the 7 to 10 range. And that's not right.

My take: Keep doing what you are doing. Continue being honest and explaining your position. You have a good understanding of the problems with ratings and have a good system, don't go changing things. Differing viewpoints are what it is all about, and it helps the gaming community.

TF2 Rocks!

 

[GullyStateWarrior]

5 Point scale is the compromise. As it is, on a 1-10 scale or a % scale, 60-100 or 6-10 are the only values really being used anyways. Personally I would just go ebert and roeper and go thumbs up/ thumbs down

 

[_Morphine_]

You cant walk into a museum and start pointing at paintings, spouting a number scale that deems it worthy of your praise or not. Like Rob said, maybe the game (or painting) just doesnt speak to you. (i.e. Salvador Dali for me XD)

Dont let the immaturity of the review reader sway your conclusions. You cant dummy down the truth to a score that works for the target demographic.

Teach them what is good by sticking to your convictions.

 

[name101]

Hey,
I've been lurking around for a little while but i just had to comment on this video.

The 1 to 10 scale is great. because of the exact reasons that you suggested.
but "TenBeers" made a great point you cannot walk into a museum and point at paintings and call them trash but someone else would pay alot of money for that painting.
All reviewers are paid to make an opinion of a game.. all reviews should be taken with a grain of salt. a classic example is a website gave bioshock a "10/10 master peice" but my friend hated it. disliked the story didnt like the "Adam and Eve". simply this is all based on opinion.

I like the numbering system because if im reading a few reviews from one person I have an idea what this person is going to say. most of the time I am lazy and skip to the end look at the number and read the conlusion. if its a low number I may backtrack and look into why...Also a Comment section is great because I also read comments [if i have time] because this is another bunch of critics. for or against the game.
maybe a user rating system. for example
http://www.tomsgames.com/us/2007/12/10/umbrella_chronicles_review/page3.html
"Ryan Lord" gave this title a "6.5/10"
Maybe a User rating system.
lets say 300 people rate this game a "6/10" this does give the game a solid review in my opinion and the author has done a great job.

I personally love SupremeCommander:ForgedAlliance. but did not like the single player campain but love the multiplayer aspect of the game.
I was reading a review on SC:FA it was a low review based on the fact SP is boring...... But the reviewer did not take into account The multiplayer Aspect of a game. This is a Critical Flaw of the Review and the reviewer!

The suggestion is..
When creating a Review Take into consideration of Multiplayer within the game. some games are based on multiplayer and are recieving low reviews because the SP is boring.
A True Classic game has a great story line and great MP. In my opinion

but what about games that are designed for SP only? I suppose that has its own area. Bioshock which I liked [watching] had epic story line but I do not know of MP game play.

as all reviews this should be taken witha grain of salt. every one is entitled to there opinions..

 

[randomizer]

The grade inflation is everywhere. It renders the score useless because if, as Rob said, it is not 7 or higher, the game (or the review) is written off. I think that the reviewer should give their honest opinion at all times as long as it is not "I don't really like FPS games, so FPS_game was boring." kind of bias. Keep the review reasonably objective where necessary but state your opinion where you can. I feel like I'm contradicting myself here I know what I want but it's hard to get out in words without sounding jumbled up. Keep the score if you want, but due to the fact that nearly all reviews give 7 or more anyway, it should really just be 1-3 :lol:

What about that lawsuit you mentioned Ben? Or is that another vid?

Collective score... bleep that out of the video, don't even consider that idea. The review would be total trash if you tried to combine multiple decent opinions into 3 pages.

With regards to the intro, I liked it the first two times (thumbs up to the person who made it, was pretty funny) but it does take up 15 seconds that adds to the other 15 seconds of ads at the start before the video really starts.

Overall I think this was something that definitely needed to be addressed, and I thank Ben, Rob and the producers for getting the ball rolling.

 

[V8VENOM]

Scores are important, for quick reads I just wanna know the "average" and then I'll decide the other aspects. I read Maximum PC and PC Gamer for the most part, and I've found that any game that is 90 or higher is usually going to be good. 80-90 range is roll the dice. 80 less is forget it. That has worked for me over time.

However, what IS NOT addressed in reviews are the serious issues around a game -- as in how much time are you gonna need surfing the net to find solutions (if any) to problems at the hardware level. It's ridiculous just how much time the typical PC gamer has to spend to getting a game to work -- you rarely hear the hardware issues in any game review.

For me, it's important to see this VERY critical aspect of a game, how "compatibile" is it with the end gamers hardware. We can't just keep ignoring the fact that Console games are out selling PC games by a factor of 15:1 primarily because console games just work out of the box. Gamers don't want to spend endless hours researching how to get a game to work with their hardware, they just want to be entertained. The industry (nVidia, ATI, Microsoft) seem to just ignore this serious problem (even worse on Vista).

Why don't reviewers talk about this in detail? It's so important to the survival of PC Gaming. Do you folks even want PC Gaming to survive? Maybe you don't.

 

[kilroy0097]

Come up with a collective list of 100 specific features and points on a game that can be answered with Yes or No. Do a check down the entire list and then tally up the yes count. If you have for example 73/100 yes then you have a 7.3 score. Then in addition to the score is the analysis of the reviewer in the article.

I personally like:
http://www.metacritic.com/games/

As it takes the average of all the scores out there which is a better feeling I think of the general gaming community.

So make a list. Find 100 points about games in general that are important. Work on that list over a long period of time and get input from a lot of people. Tweak it as you need to. Then test it on games you know scores on. See if it holds up. If it does then use it.

The other option is to use a very simple scoring such as "Play It" or "Trash It". Perhaps another could be "Own It", "Rent It", "Borrow It", "Avoid It". Each meaning in order, it's good enough to own it and keep it so get it, it's good enough to play it but not own it, it's ok but nothing good and it's certainly not worth any money (borrowing is free), and finally it's sucks hardcore and you shouldn't even touch it.

So that's my 2 cents.

 

[caamsa]

I think people need to be informed consumers. Read more than one review. Post in the forums about the games, ask questions about the game. Play the demo if you can. It is up to the individual to take personal responsibility.

Guys in reference to your videos......ahhh they are much too long. If you are going to talk that long then you need to entertain us. Show us a chart or a graph. Be a little more creative in your presentations. I give you a score of 70.

 

[robwright]

I think people need to be informed consumers. Read more than one review. Post in the forums about the games, ask questions about the game. Play the demo if you can. It is up to the individual to take personal responsibility.

Guys in reference to your videos......ahhh they are much too long. If you are going to talk that long then you need to entertain us. Show us a chart or a graph. Be a little more creative in your presentations. I give you a score of 70.

That's totally my fault -- I rambled way too much lately, which has led to longer videos. We'll make sure that we have plenty of game footage and other elements for future vidoes, like next week's Games of the Year Edition.

Now about that 70 score...is that a 70 on a 1 to 100 scale, or a 70 to 100 scale?

 

[caamsa]

That's totally my fault -- I rambled way too much lately, which has led to longer videos. We'll make sure that we have plenty of game footage and other elements for future vidoes, like next week's Games of the Year Edition.

Now about that 70 score...is that a 70 on a 1 to 100 scale, or a 70 to 100 scale?

:lol: Now that made me laugh.

Yea 9+ minutes was kinda long. The 70 is based on a 1 to 100 scale. Of course that is a C which is not too shabby.

 

[_Morphine_]

Guys in reference to your videos......ahhh they are much too long. If you are going to talk that long then you need to entertain us.

I dunno. I just tab it open and listen while I surf. If there was something to watch and require my attention I might not be able to digest it all at once. I like the fact I dont need to watch.

However, It did take me some time to come to that conclusion.

 

[randomizer]

Guys in reference to your videos......ahhh they are much too long. If you are going to talk that long then you need to entertain us.

I dunno. I just tab it open and listen while I surf. If there was something to watch and require my attention I might not be able to digest it all at once. I like the fact I dont need to watch.

However, It did take me some time to come to that conclusion.

I do the same thing, but my inability to multi-task well means I can miss stuff easily.

 

[jimbobhickville]

I think the problem is that in school, anything below a 60% is an F. And an F means you really suck badly. D means you're dumb, but not a total loser, C means you're just kinda dumb, B is ok, A is smart. But, I do agree that review scores are silly when there's no way to get anything below a 7. What's worse is when you read a review, and the reviewer sounds like they hated the game, then the score is an 8.5 or something. If you hated the game, there's no way it should get above a 5.

 

[killersnowman]

if the goal of the review is conveying if the game is "worth" playing then why not do just that? i emphasize "worth" because, a game, like every other thing in life costs a set amount of money. why not tell us how much the game is worth. if the game is amazing then you say i would buy this game if it was $80. if its terrible then you say that i would buy this game if it was $2.

 

[randomizer]

That's an interesting concept. Though I say no game is worth more than $30, and that's for the best of the best.

 

[Chazwuzzer]

While I appreciate the lament, I still see trouble. The ad bucks are gonna follow the better (highly-scored) reviews and as a result those publications are going to be well financed, slick and ranked highly on search engines.

Rob's reviews are solid. Some minor disagreements here, but overall he appears to be giving the titles an honest, thoughtful review.

One thing that I've always been curious about is what difficulty level games are reviewed on. I always automatically crank the difficulty up to its highest and disable any and all wuss-out options. Sometimes when I'm reading a review, I get the vibe that the reviewer was playing on normal (i.e. dumbass) difficulty and as a result they come away with a very different perspective on the game.

(BTW, I've only been unable to finish a game on the max difficulty once. It was Mechwarrior Mercenaries. There's a mission toward the end of the game where you are leading your squad of (semi-retarded) mechwarriors on raid and you get jumped by a full company of enemies. My idiotic squad of Keystone Cops would just run right into the thick of the elite enemy assault mechs and would be cut down in a comedically short period of time, leaving me to deal with the whole mess myself.)

 

[Grub]

Tom's has always had a good editorial staff, including some of the lean years right after Tom stepped away. Omar was one of my favorites...he let people have it with both barrels. He was especially 'frank' with trolls and juvenile posters. I see Rob as incredibly patient with the readers here at Tom's. On one level, its admirable, readers are what pays the bills (at least indirectly), right? On another level, its an exercise in futility...you'll never please them all. Here's my input, Second take has evolved into one of my favorite aspects of Tom's. The production values seem pretty low when compared with the relative popularity of Tom's. Maybe, I just have unrealistic expectations and impressions about how much revenue Tom's generates...you all get six figure salaries, right? In any case, a little bit better set, maybe cut to some different scenes, oh...and most important: ROB NEEDS THEME MUSIC!!! Something Mario-esque... I would really like to see some off campus videos of you guys at trade shows, lans, and the like. You don't know how bad I wanted to see you guys Rock the 'Rock Band' game at the show you were at. Anywho, keep up the good work, and keep your integrity...