5 ms·

Game reviews on Metacritic: why we avoid inclusion

35 points by barredo 16y ago

ghshephard 16y ago
"Our practice of giving games either a "Buy, Rent or Skip" verdict is too vague for Metacritic to arbitrarily take our score and tell its readers what it thinks we meant."
- nopassrecover 16y ago
  Instinct suggests 85/65/30. If you felt inclined you could probably have some fun with parsing the reviews for influential words too.
  - mortenjorck 16y ago
    They actually have a policy for scoreless reviews: However, this does pose a problem for our METASCORE computations, which are based on numbers, not qualitative concepts like art and emotions. (If only all of life were like that!) Thus, our staff must assign a numeric score, from 0-100, to each review that is not already scored by the critic. Naturally, there is some discretion involved here, and there will be times when you disagree with the score we assigned. However, our staffers have read a lot of reviews--and we mean a lot--and thus through experience are able to maintain consistency both from film to film and from reviewer to reviewer. When you read over 200 reviews from Manohla Dargis, you begin to develop a decent idea about when she's indicating a 90 and when she's indicating an 80. http://www.metacritic.com/about/scoring.shtml http://www.metacritic.com/about/scoring.shtml
smackfu 16y ago
Not surprised. This is the same Ars where they used to rate other things (like iPods) with a numeric scale and ditched it because people were fixating on the rating and ignoring the article.
electromagnetic 16y ago
Having worked as a game reviewer, they're correct. There's a ton of hidden pressure behind the scenes to positively review some games, for threat of losing the publishers review materials. Some publishers were great, and some were complete asses putting up embargoes on review release dates to hand them off to the bigger magazines first. Overall it just made the experience of reviewing them rather negative, which has tainted my view of games as a whole. I used to be an avid gamer, now I'd barely classify myself as a casual gamer. There are some great and truly innovative games in all sorts of terms from gameplay to story, however mostly I just see Arbitrary Action Element 12 followed by Unsurprising Plot Development 8, even in some reputedly great games.
- jacquesm 16y ago
  The golden days of game development imo were the late 80's, just enough computer power to make things interesting and not enough big money in it to make it hard to get in to for creative small companies. It's been very much downhill since then, with the exception of a few interesting developments in the console market.
  - _delirium 16y ago
    I think it was reasonably agile up to the mid 90s, at least in some areas. A lot of the top PC games of the early/mid 90s had smallish teams and smallish budgets: Quake (1996), for example, had 9 employees on it. These days a 9-person game company has little chance of breaking out of the "indie" category (though there's still some interesting indie stuff happening).
    - jacquesm 16y ago
      Ok, I'll settle for 'doom' as the cut-off :) 1993. (I was doubting about 'wolfenstein', but that only makes it one year earlier). After that it really got out of hand. 9 people is already a fair sized team. Compare that to Jeff Minter or Braben & Bell.
      - 16y ago
sliverstorm 16y ago
They kind of paint Metacritic in a poor light. It's not Metacritic's fault THQ has the policies they do.
- jjs 16y ago
  They kind of paint Metacritic in a poor light. It's not Metacritic's fault THQ has the policies they do. Nah, they just gave Metacritic a B-minus. It's not Ars's fault that Metacritic interprets that as a 67 out of 100.
- shadowsun7 16y ago
  I think what surprised me the most is how much Metacritic matters to the gaming industry. You don't get the same issues with music, say, or movie releases. It's true - Metacritic isn't at fault for THQ's policies, but then that's what you get for coming in and changing the mechanics of the game review world.
  - henrikschroder 16y ago
    I think it's because a lot of gamers treat game reviews as some kind of absolute truth, instead of just the opinion of one guy. People seem to have no problem treating movie reviews like someone's personal opinion, and find a reviewer they like and trust, and ignore the rest. But when a game gets 8/10, it somehow either is an 8/10 game or the game reviewer is wrong. Somewhere along the line people forgot that reviews are subjective, not objective.
    - arohner 16y ago
      Yes, but the stakes are higher. A movie takes $10 and 2 hours of your time. (New) Games cost $60, and occupy a variable amount of time. Since I'm bootstrapping my startup i.e. poor, I only buy games with a high expected value. I typically look for $1 / hour of entertainment, or better. This means I buy games that I can expect to play for 50+ hours before getting bored. Modern Warfare 1, for example, I played for hundreds of hours, making it a great bargain. The game review scale is also highly compressed. The difference between 8/10 and 9/10 is often the difference between "good game" and "unplayable".

35 points by barredo 16y ago

"Our practice of giving games either a "Buy, Rent or Skip" verdict is too vague for Metacritic to arbitrarily take our score and tell its readers what it thinks we meant."

nopassrecover 16y ago
Instinct suggests 85/65/30. If you felt inclined you could probably have some fun with parsing the reviews for influential words too.
- mortenjorck 16y ago
  They actually have a policy for scoreless reviews: However, this does pose a problem for our METASCORE computations, which are based on numbers, not qualitative concepts like art and emotions. (If only all of life were like that!) Thus, our staff must assign a numeric score, from 0-100, to each review that is not already scored by the critic. Naturally, there is some discretion involved here, and there will be times when you disagree with the score we assigned. However, our staffers have read a lot of reviews--and we mean a lot--and thus through experience are able to maintain consistency both from film to film and from reviewer to reviewer. When you read over 200 reviews from Manohla Dargis, you begin to develop a decent idea about when she's indicating a 90 and when she's indicating an 80. http://www.metacritic.com/about/scoring.shtml http://www.metacritic.com/about/scoring.shtml

smackfu 16y ago

Not surprised. This is the same Ars where they used to rate other things (like iPods) with a numeric scale and ditched it because people were fixating on the rating and ignoring the article.

electromagnetic 16y ago

Having worked as a game reviewer, they're correct. There's a ton of hidden pressure behind the scenes to positively review some games, for threat of losing the publishers review materials. Some publishers were great, and some were complete asses putting up embargoes on review release dates to hand them off to the bigger magazines first. Overall it just made the experience of reviewing them rather negative, which has tainted my view of games as a whole. I used to be an avid gamer, now I'd barely classify myself as a casual gamer. There are some great and truly innovative games in all sorts of terms from gameplay to story, however mostly I just see Arbitrary Action Element 12 followed by Unsurprising Plot Development 8, even in some reputedly great games.

jacquesm 16y ago
The golden days of game development imo were the late 80's, just enough computer power to make things interesting and not enough big money in it to make it hard to get in to for creative small companies. It's been very much downhill since then, with the exception of a few interesting developments in the console market.
- _delirium 16y ago
  I think it was reasonably agile up to the mid 90s, at least in some areas. A lot of the top PC games of the early/mid 90s had smallish teams and smallish budgets: Quake (1996), for example, had 9 employees on it. These days a 9-person game company has little chance of breaking out of the "indie" category (though there's still some interesting indie stuff happening).
  - jacquesm 16y ago
    Ok, I'll settle for 'doom' as the cut-off :) 1993. (I was doubting about 'wolfenstein', but that only makes it one year earlier). After that it really got out of hand. 9 people is already a fair sized team. Compare that to Jeff Minter or Braben & Bell.
    - 16y ago

sliverstorm 16y ago

They kind of paint Metacritic in a poor light. It's not Metacritic's fault THQ has the policies they do.

jjs 16y ago
They kind of paint Metacritic in a poor light. It's not Metacritic's fault THQ has the policies they do. Nah, they just gave Metacritic a B-minus. It's not Ars's fault that Metacritic interprets that as a 67 out of 100.
shadowsun7 16y ago
I think what surprised me the most is how much Metacritic matters to the gaming industry. You don't get the same issues with music, say, or movie releases. It's true - Metacritic isn't at fault for THQ's policies, but then that's what you get for coming in and changing the mechanics of the game review world.
- henrikschroder 16y ago
  I think it's because a lot of gamers treat game reviews as some kind of absolute truth, instead of just the opinion of one guy. People seem to have no problem treating movie reviews like someone's personal opinion, and find a reviewer they like and trust, and ignore the rest. But when a game gets 8/10, it somehow either is an 8/10 game or the game reviewer is wrong. Somewhere along the line people forgot that reviews are subjective, not objective.
  - arohner 16y ago
    Yes, but the stakes are higher. A movie takes $10 and 2 hours of your time. (New) Games cost $60, and occupy a variable amount of time. Since I'm bootstrapping my startup i.e. poor, I only buy games with a high expected value. I typically look for $1 / hour of entertainment, or better. This means I buy games that I can expect to play for 50+ hours before getting bored. Modern Warfare 1, for example, I played for hundreds of hours, making it a great bargain. The game review scale is also highly compressed. The difference between 8/10 and 9/10 is often the difference between "good game" and "unplayable".