Thoughts on game reviews

[Graham Mitchell]

So, we've definitely debated to death on these forums about how basically useless number-based reviews are. Yet every time I get an issue of Game Informer and I read the review section I want to go punch the wall.

Now, I haven't even played the game yet, but the review that triggered me into this rage today was Game Informer's review of Shin Megami Tensei: Catherine. They gave it a 7 and talked about how boring and frustrating it gets. They recently gave the new Alice a 6.25 or something awful as well, but it's a perfectly fine game, and does what it's supposed to do admirably IMO.

I was pondering all of this and trying to figure out why I disagree with these guys so often (they take a shit on nearly every game that sounds interesting to me) and it suddenly hit me. I think that in order to write the review, the writers are expected to play the game compulsively and obsessively until they beat it and squeeze as much life out of it as they can. But that's not how I play. I change games a lot. I'll get into one game intensively one week, then shelve it and come back to it in a couple months. Know why? Because ALL games get boring after too much repitition.

Even the Zelda and Metroid games, as much as I love them, can get old. After 2-3 hours, sometimes I've just fallen down that hole in the dungeon one too many friggin' times, and I really don't feel like going through the whole level again. So I stop playing. When I pick it up the next day or whenever, it's fresher to me, and my interest is easily re-captured.

Anybody who's worked in game testing will tell you that compulsively playing a game for 40 hours+ a week will make you hate it. I have friends that play tested for Microsoft and Nintendo and they ended up just not playing video games at all for a year after they quit. If that's how reviewers are expected to play the games, then their interpretations are basically meaningless to a phenotypically normal person with no impulse control disorders who plays the game in a healthy fashion.

I also think this is why Heavy Rain got a 9+ from many reviewers, despite the fact that the gameplay is basically Dragon's Lair with more buttons and minimal replay value. The story has lots of sex and violence in it, and is pretty captivating at times. I think this lends to the game being more palatable in extended plays, despite it being not much of a game.

Thoughts? You guys think I'm nuts or am I on to something here?

[Icarus Moonsight]

I am certain getting high quality, objective with reasonable analysis and comparisons, review prior to or on release is entirely possible. As it stands, the people charged to doing it are failing at it. But then, print is dead, so who cares?

[Aussie2B]

I don't think professional game reviewers are playing games anything like game testers. I don't think they're playing games obsessively at all, and I definitely don't think their employers expect that of them either. The employers just want the piece done and written, as fast as possible, so they have something to go along with all of the ad space they're trying to sell. Because of that, it's true that the reviewers are sometimes forced to marathon games because they're given so little time to complete a review. They do aim to beat a game at least (but I can almost guarantee you that some games are written about without the credits ever being seen), which is is no sweat for a game under 10 hours, but it can be a nightmare for something like a long Japanese RPG. Because of that, I'd say the problem is the exact opposite. They're not playing the games enough. They're rushing like mad to get to the end, so they don't have the opportunity to dig deep and really learn and appreciate (or loathe) what it has to offer. Bottom line, reviewers need more time to more casually and extensively play a game, and their employers need to appreciate them more, pay them better, and encourage excellence in the quality of their work. There are indeed bad game journalists, no doubt about that, but the main problem is the structure of the industry created by the higher ups.

[Graham Mitchell]

I don't think professional game reviewers are playing games anything like game testers. I don't think they're playing games obsessively at all, and I definitely don't think their employers expect that of them either. The employers just want the piece done and written, as fast as possible, so they have something to go along with all of the ad space they're trying to sell. Because of that, it's true that the reviewers are sometimes forced to marathon games because they're given so little time to complete a review. They do aim to beat a game at least (but I can almost guarantee you that some games are written about without the credits ever being seen), which is is no sweat for a game under 10 hours, but it can be a nightmare for something like a long Japanese RPG. Because of that, I'd say the problem is the exact opposite. They're not playing the games enough. They're rushing like mad to get to the end, so they don't have the opportunity to dig deep and really learn and appreciate (or loathe) what it has to offer. Bottom line, reviewers need more time to more casually and extensively play a game, and their employers need to appreciate them more, pay them better, and encourage excellence in the quality of their work. There are indeed bad game journalists, no doubt about that, but the main problem is the structure of the industry created by the higher ups.

Well, if that's true, then it's no wonder all the JRPG's get dumped on all the time.

It's not so much the lack of objectivity that bothers me about these reviews. It's more the fact that so many of these reviewers (especially in the print world) seem to expect their games to be more of a movie than a game. If the game has any kind of story, they expect gameplay and cinematics to be seamlessly integrated like a Valve game or Mass Effect. I'm constantly reading how "out of place" the actual gameplay seems in the contrast to the cinematics. They comment about this in the Catherine review, in fact. I also remember Velvet Assasin getting just trashed on basically because it's level-based, and because each segment of the game is a little vignette, and none of them seem interrelated. They completely ignore the game's stellar visuals, intelligent writing, and all around FUN stealth gameplay. Yeah, when you die, you restart from the checkpoint. So what? That's a theme that's been around since Super Mario Bros. Why is it suddenly so uncool for a game to be challenging or demand a certain level of precision?

But, maybe you're right, Icarus. Maybe it all just because of a lack of objectivity. But I think if these people went back and played stuf from the pre-PS1 era they might understand that a game can be enjoyable despite not having scripted sequences and being slightly hard.

Just my 2 mesetas.

[Icarus Moonsight]

It's genuinely hard to write about experiences while taking yourself out of the picture. Reviewing works well also requires a broad experience base in the area you are writing about. Seems to contradict, but that's how it is. The only way I could write a review for a shmup is for other shmup fans without really pulling hard on bias to get it down in a fair light. I couldn't write a Star Trek game review for a similar reason, but rather for people that generally dislike it. And when I say could/couldn't I mean, with my integrity intact. If you are just doing the review for yourself and anyone that cares to look and read, then the standards relax quite a bit.

Perhaps they are serving their positions completely well to these points and expectations, we're just on the outside of it. I'm also fine with that.

[Aussie2B]

Objectivity is impossible, seeing as reviews are critiques, which are completely subjective by definition. I think what you're getting at is bias. There are too many reviewers out there that will love any garbage just because it's in a particular genre or features some kind of content that they worship, or, conversely, will rag on something just because it's in an unfavored genre or such.

[dgdgagdae]

They comment about this in the Catherine review, in fact.

You mention the review of Catherine twice. Is that really what's upsetting you? And have you played the demo? That was enough to tell me that the gameplay had nothing whatsoever to do with the story, and that I wasn't going to play slideyblocks to watch some amime film.

To summarize: The Japanese are weird.

[Graham Mitchell]

You mention the review of Catherine twice. Is that really what's upsetting you? And have you played the demo? That was enough to tell me that the gameplay had nothing whatsoever to do with the story, and that I wasn't going to play slideyblocks to watch some amime film.

To summarize: The Japanese are weird.

No, this has been bugging me for years. But the Catherine article was what prompted me to write this post. It's also just an example that's fresh in my mind because I read it yesterday. I haven't played the demo yet because I preordered the game and I prefer to be surprised when it shows up.

I do agree with some of their reviews. I just saw that they gave Portal 2 a 9.5, and I think it deserved it.

Definitely, this stuff is not worth freaking out about. But I am/was curious to understand why my they consistently knock just about anything that's not a 2-D fighting game from Capcom or an FPS.

The bottom line is that you have to just digest these reviews as entertainment, I guess. Trying to understand why I disagree with them so often is basically pointless. I don't think I'll ever have an answer. And that's fine.

[Gamereviewgod]

Because of that, it's true that the reviewers are sometimes forced to marathon games because they're given so little time to complete a review. They do aim to beat a game at least (but I can almost guarantee you that some games are written about without the credits ever being seen), which is is no sweat for a game under 10 hours, but it can be a nightmare for something like a long Japanese RPG. Because of that, I'd say the problem is the exact opposite. They're not playing the games enough.

The thing is, with games, they don't really need to develop.

Today, I sat down with Call of Juraez: The Cartel. The second I fired the gun it was trouble. It was sloppy, collision was suspect, and the guns were weak. It was 30 seconds. Four hours later, I'm taking a break for a bit, and I can assure you, for another four hours today I'll be using those same mechanics, doing the same things. I don't really need those other 10 or so hours to know it's pretty much a mess. The story is only the hole in the process now.

Reviewing games is vastly different than other media. Movies need to be a complete narrative, so do books. Games are really mechanical, selling themselves on their feel. You don't always have to play a game "enough" to know where you stand. You do because it's a job and it's required of you, but the opinion isn't going to change whether or not you beat it. I'm not going to hate Juarez anymore than I do now simply because I have a deadline.

People love to pick apart reviews, like the Catherine thing. I talk to the reviewer all the time. He was prepared for a backlash. Gamers are simply rabid when it comes to this stuff. They see a "7" and freakout. Alice? Yeah, Alice was rough. Levels dragged on for hours with no end in sight. One sitting, two sittings, doesn't matter. Maybe that didn't bother you. I was invested in the world but there was too much of it. A 6.5 is more than fair in my eyes, maybe not in yours. 'Tis the nature of reviews.

Keep in mind that in a great game, something truly special, you're never tired of it. The design is such that it takes an inherently repetitive medium and makes it something spectacular. Most games today are a lot of filler to fill some requisite length, and they're going to be called on it. The great ones stand out and you don't want that marathon session to end.

[Aussie2B]

The thing is, with games, they don't really need to develop.

Today, I sat down with Call of Juraez: The Cartel. The second I fired the gun it was trouble. It was sloppy, collision was suspect, and the guns were weak. It was 30 seconds. Four hours later, I'm taking a break for a bit, and I can assure you, for another four hours today I'll be using those same mechanics, doing the same things. I don't really need those other 10 or so hours to know it's pretty much a mess. The story is only the hole in the process now.

Reviewing games is vastly different than other media. Movies need to be a complete narrative, so do books. Games are really mechanical, selling themselves on their feel. You don't always have to play a game "enough" to know where you stand. You do because it's a job and it's required of you, but the opinion isn't going to change whether or not you beat it. I'm not going to hate Juarez anymore than I do now simply because I have a deadline.

People love to pick apart reviews, like the Catherine thing. I talk to the reviewer all the time. He was prepared for a backlash. Gamers are simply rabid when it comes to this stuff. They see a "7" and freakout. Alice? Yeah, Alice was rough. Levels dragged on for hours with no end in sight. One sitting, two sittings, doesn't matter. Maybe that didn't bother you. I was invested in the world but there was too much of it. A 6.5 is more than fair in my eyes, maybe not in yours. 'Tis the nature of reviews.

Keep in mind that in a great game, something truly special, you're never tired of it. The design is such that it takes an inherently repetitive medium and makes it something spectacular. Most games today are a lot of filler to fill some requisite length, and they're going to be called on it. The great ones stand out and you don't want that marathon session to end.

That works for bad games and for games that are awesome all throughout, but it's not like there isn't anything in between. I've played many games in which I found them reasonably enjoyable until something soured my experience near the end. Be it a game that's fun for awhile but goes on for too long and gets far too repetitive and boring or a game that just has something completely unenjoyable near the end, like a final area that's really annoying and also huge so you're stuck dealing with it for a considerable amount of time. There are definitely games I'd be more favorable to without the final 5, 10, however many hours. It's always valuable to beat a game before reviewing if at all possible, unless the game is so wretched from the get-go that even a complete 180 (as unlikely as it would be) wouldn't make up for the unbearable portions at the beginning.

[calthaer]

I believe that game reviewers are at a distinct disadvantage vis a vis journalists (or "journalists", depending on one's view of the game-news business) who review movies, books, and so forth. Games typically take much longer to complete; it's almost impossible to do so in a reasonable amount of time. It's often a year or more before some games are revealed to be the true gems that they are; sometimes, it takes several years, if the game is niche and it takes a while for people to slot the game into their personal play-schedules.

I'm going to float an idea here - and I expect it to be a bit controversial even as I formulate this idea - but anyways...

It is in the best interests of everyone involved, I think, if game creators were to either create a "special version" of their game or to inherently enable "cheat codes" into all versions of the game to enable people to automatically skip around to any and all places in the game at any time. I'm sure that the creators must have something like this internally to enable them to test the game before sending it out; it's not too much of a stretch for them to create some kind of "test" version for game reviewers.

I realize this is kind of a dangerous concept. It means that there might arise a whole "bootleg" market for these "test versions" of the games, even if the companies required the game (which could certainly be copied) to be returned to them after the reviewer is done playing. I suppose each "test" version could have hard-coded info on the person to whom it went, but there are probably ways around that, too. That's why I'm suggesting the possibility that game developers would incorporate a "skip to the end" or "god mode" feature in each and every copy of the game they sell - possibly unlockable via an inputted code, or something.

The ramifications for that would be enormous. Imagine having every long JRPG having a god mode, that would allow one to skip at any point in the story (possibly with cookie-cutter party setups), at any time, and / or to give one's self any item at any time. It has the potential to completely "ruin" the game.

But - you know what? Who cares? With the possible exception of something like Pokemon, where players are trading with each other, and where they'd obviously want people not to be able to cheat like this, the "cheater" here is "hurting" nobody but themselves. Even with Pokemon, people already give themselves these "god-like" powers with just a little bit of effort and a flash-cartridge / save editor. Game Genie or Gameshark or Action Replay or whatever its current incarnation is these days would become obsolete - but their existence proves that there is a demand for this kind of feature.

It's the kind of feature that any and all board games have always had. It's totally possible to create a chess game that is halfway, or three-quarters, or whatever amount near its conclusion - a lot of good chess players probably do this to put themselves into various scenarios. One could do the same with any board game - it's just that people typically don't. They do have the freedom to do so, though.

I think video game designers / creators need to start recognizing the need for this sort of feature. This would allow any reviewer - great or small - to play a more comprehensive part of the game, and to give it a much fairer and balanced review than they'd otherwise be able to do. This might, in turn, help the good games to shine, and the bad games to be revealed, a lot more easily. At least...that's my theory. Maybe there are holes in it.

[kupomogli]

People love to pick apart reviews, like the Catherine thing. I talk to the reviewer all the time. He was prepared for a backlash. Gamers are simply rabid when it comes to this stuff. They see a "7" and freakout. Alice? Yeah, Alice was rough. Levels dragged on for hours with no end in sight. One sitting, two sittings, doesn't matter. Maybe that didn't bother you. I was invested in the world but there was too much of it. A 6.5 is more than fair in my eyes, maybe not in yours. 'Tis the nature of reviews.

Even a 6.5 is way too high for a game with as many problems as I've heard Alice 2 has. All the reviews I've read would translate to the usual 2/5 I'd rate for this gen, or on the 10 scale 4 or 5/10. 6.5/10 is 3/5, basically saying the game is worth playing, and most games this gen just aren't worth playing.

The problem is that reviewers rate everything too high. It's a combination of both the fanboys that flip out with anything lower than a certain number as well as the developers who will pull advertising unless they get a certain rating. Regardless. You're defending the reviewers when they're just as bad as the fanboys who rant about the ratings. Professional review sites are the reason why review scores are inflated.

[G-Boobie]

On the subject of numbered reviews being worthless, its interesting to note the difference in perceived value between a literary, movie, or music critic and a game reviewer(the fact that the word 'critic' is seldom applied to game reviewers is pretty telling, too).

A critic of film, novels, or music is held in high regard, and his opinion is often weighed when formulating an opinion on something when you're deeply into whatever their chosen subject is. A professional game reviewer is an asshole and pretty much everyone hates him. There are certain people whose opinion I trust in the gaming world, like Jeremy Parish or the Rock Paper Shotgun guys, but normally I just consult the forums.

[Rob2600]

So, we've definitely debated to death on these forums about how basically useless number-based reviews are.

To me, 10-point and 100-point rating scales are useless and juvenile. What's the difference between a 7.1 and a 7.2?

All we really need is a 3-point scale:

3 - Totally worth playing.
2 - Maybe worth playing. Rent it and see.
1 - Not worth playing at all.

[kafa111]

To me, 10-point and 100-point rating scales are useless and juvenile. What's the difference between a 7.1 and a 7.2?

All we really need is a 3-point scale:

3 - Totally worth playing.
2 - Maybe worth playing. Rent it and see.
1 - Not worth playing at all.

i would write 2 everytime. impossible to mess up

[kafa111]

wah double post

[Berserker]

I look at professional game reviews the same way I look at Ron Popeil infomercials: as chunks of paid programming. Does anyone actually base their purchases on what comes out of this small handful of institutions?

It seems to me that, by this point, we've almost completely cast aside any pretense that these serve to inform us in any real capacity. Instead, we view them now only as public measurements of games we've already purchased and played, having been informed by sources we find trustworthy, such as peer recommendation or our own intuition.

Whatever discussion occurs now focuses only on the public measurement as an object, and how close or far it comes to our own personal measurements; but the question of whether or not they are at all genuine or uncompromised is beyond discussion.

In a nutshell, we've twisted our views of these things to synchronize with the advertising industry that created them, because that's the only way their existence makes sense to us.

So my thought isn't that professional game journalism is bad - it's that it hasn't been invented yet. My hope is that at some point, the number of professional writers who are disillusioned and tired of trying to conform to this medium's boney grip reaches some sort of critical mass which might lead to its true birth.

[kupomogli]

I read reviews to find out what a game is like rather than whether I should purchase it or not. I do look and see how people have enjoyed it, but it's really if I find a value in it for myself before I purchase it or not.

As for the 10 point scale, it's not that it's bad, it's just that no one uses it properly. A 6/10 is supposed to be an decent game, yet the way professional review sites make it look is that 6/10 is the crap shovelware. It's rare that a game is 6/10 at all if it's at any of the professional revew sites. I think most of the games this gen should be below 6/10 or 3/5.

I think everything should really go by the five scale. It's more simple. When you have people looking at it and Infamous comes at a 3/5, it's giving the game a 6/10, where on the 10 scale people would flip out about it but on the five scale they'll be more like "oh, okay."

With the five scale. Five is near perfect or perfect. Four is very good, three is good, two needs some work, and one is crap. I don't really like the "three is average" as most games would fall under three then, and rather than looking like the game should be skipped, it looks like it should be played instead. Three being average is nice when over half the games in that specific gen don't suck.

[Icarus Moonsight]

To me, 10-point and 100-point rating scales are useless and juvenile. What's the difference between a 7.1 and a 7.2?

All we really need is a 3-point scale:

3 - Totally worth playing.
2 - Maybe worth playing. Rent it and see.
1 - Not worth playing at all.

Traffic light gaming reviews. That could serve well for a fast at-a-glance reference, at least when it comes to plow-in or full-stop. The middle ground is always the murkiest waters to navigate, and if you look, that's where a majority of the reviews range already. That's when the article itself is critical for informing a potential customer. The Metacritic aggregate review process just makes the middle-ground wider, but also accentuates the top and bottom. I've been using this myself already, so it does have some pragmatic quality.

Berserker is spot on. There is a better way, not yet realized. For most of us, at our level of involvement, we can disseminate information extremely well on our own. So the need to invest in this new way is somewhat diminished. But, as more folks come over to the dark-side that incentive will increase, and the usual sources will become that much less relevant. It will emerge when it's needed, like the Hero of Time.

[Gameguy]

I usually use Metacritic to average out reviews from many sources, rather than just use one source. Idealy it's best to find a reviewer who has a similar taste to yours rather than someone who's just "objective". Also, plenty of reviewers don't like certain genres and when they have to review a game from a genre they don't like the review is obviously more negative than it should be.