Rating consistency: How I rate games

I first published this article for the 29th ludumdare. This is a slightly revised version that better suits my views after 10 participation to the event, while keeping the new “opt-out” option in mind.

 

Hello community!

Life being life, I had a lot of time on my hands during the rating period of the ludumdare 29. While this allowed my game to be featured in the “most coolness” section of the rating page, I was also confronted with a problem I never faced before: rating consistency.
Like most of you, when I find a game that I REALLY like I give a lot of stars. Even in categories where the game doesn’t deserve much stars. And yes, the opposite is also true: I will give fewer stars than you deserve if there is something I really don’t like about your game.
Those bias are often unconscious, but after a few sessions I found myself questioning my past ratings. Things like “Did that gorgeous game really deserved 5/5 overall?” or “Wait… I gave 4 stars to that other game, and this one is clearly better. But not 5/5 better”
With the amount of game I had already rated, and considering the amount of games I still intended to rate, I felt that I needed a scoring system if I wanted to be fair with everyone.

I am NOT telling you how I think you should rate games. I post this for those who feel like their rating is not consistent. Or for those who find themselves giving either 1 or 5 stars in some categories. Or even as a tool to analyse your ratings if you like.

 

General rule

[N/A] I used to leave some categories empty when I felt there was no intention from the author to create a mood/humor AND when I felt nothing special. With the new opt-out option I always rate a category when I encounter it.
[1] You tried to do something but it was more an annoyance than anything else OR I don’t understand why you didn’t opted-out of that category. (To me, giving 1 star is a punitive action. It’s my way of saying “don’t do that again”)
[2] You tried to do something… but really REALLY poorly. This is often the rating I give when it feels scamped, pushed-in-there…
[3] I can see you made some effort with it (even if I don’t think you accomplished what you wanted)
[4] Pretty good, but not perfect
[5] I hardly see how it could be better

more specifically:

MOOD

[1] Expressed intention from the author to create a mood (either in the description, in a text early in the game, or by not opting-out of the category), but nothing in the game to support that.
[2] Crap sound/visual effects, “telling me” what I am supposed to feel (“you arrive in a spooky town”), dark visuals that hurt the gameplay instead of serving it …
[3] Meh… Yes maybe
[4] Something feels “off”. (don’t have a better way to describe it, depends on the game)
[5] I was “in the flow” of the game

HUMOR

[1] Not funny at all. No really : NOT. FUNNY.
[2] Something in the game made me genuinely smile, but I don’t think it was intentional
[3/4] I smiled/laugh in multiple occasions
[5] I kept playing because I knew it would make me laugh

AUDIO

[1] Either a lack of sound that harms the game, or if I muted the game
[2] Sound effects that are not unpleasant (basically any game that use exclusively bfxr sounds)
[3] Some music and/or sound effects of good quality or well-integrated (ie. proper bfxr sounds with a well-balanced volume).
[4/5] Depends on the game, but the game must have REALLY good music and sounds. Otherwise, it’s a 3

GRAPHICS

If it is a text-based game I rate based on the clarity of the interface (with bonus points if you have some effects)
[1] I don’t understand what is happening on the screen. Seriously, some games do just fine with colored squares!
[2] I understand what is going on
[3] You took the time to create some assets and/or have some effects here and there
[4] Good assets OR a nice aesthetic (style, colors…) OR some notably cool effects
[5] Everything listed in point 4, or REALLY beautiful art.

THEME

[1] You did nothing with the theme (Come on… LudumDare is a theme-based jam) OR the link to the theme was explained in the description but I felt it was just “Here is my take on the theme : [absurd explanation]. Now play the game I wanted to do anyway”.
[2] A take on the theme that is not original, or the theme is only mentioned at the beginning/end of the game
[3] A not-so-common take on the theme, but used only to set the context of the game.
[4] The theme is used in someway in your game
[5] A fundamental mechanic of the game is built from the theme, or the mechanics of the game all revolve around the theme

FUN

[1] I was forcing myself to play the game (Frustrating game: no clear objective, bad controls, bugs…)
[2-3] I didn’t feel the need to keep playing after a game over or after a few levels
[4] I kept playing for a while
[5] I want to come back to this game later

INNOVATION

[1] Nothing unique OR a copy of another game
[2] Classic mechanics with a minor twist (often poorly integrated)
[3] Any “abstract” game, or classic mechanics with a nice twist
[4] Multiple cool features, or an unusual mix of mechanics, or a clever mechanic (often associated with the theme)
[5] Unique gameplay

OVERALL

[1] Neither a game nor an interactive experience. I also give 1 when there is just a mechanic with nothing built around it (a guy running around)
[2-3] A game I won’t remember
[4] Great game overall (often have multiple 5 in other categories) but don’t feel like a “complete” game
[5] A game that feels complete OR with an excellent rating in every other categories

 

Now, a few random thoughts:

  • Read the description. Sometime you don’t want to read a wall of text (especially if it is a big thesaurus of all the entities in the game), but the description is the only medium for the author to communicate with you.
  • Post a comment. It’s not always easy to post something nice, but for some people comments ARE more important than the actual rating. I suspect I am not the only one to check my page at least once a day.
  • Be constructive. I have seen multiple games with comments that were not far from “nice art, shitty game”…
  • Don’t get your expectations too high : The goal of LudumDare is to make a game and to get feedbacks on it, not to win it

 

Leave a Reply