Farewell 'Completely Stable', Hello Medal Rating System

September 3, 2018

Greetings Linux gamers!

Starting today, SPCR will change the way we're asking reporters to assess their experience with the games they are reporting using Steam Play. Previously we bucketed these against a spectrum of a game's 'status'. Those options were: Won't Start, Crashed, Unplayable, Unstable, Stable, and Completely Stable. This metric usefully described how it felt to play a game, and even captured attention for the utility and understatement of the Steam Play system on launch.

We quickly discovered a few kinks in this way of measuring, namely:

  • Objective clarity was lacking and significant variance existed between reporters' assessments. What separated Completely Stable from Stable, and from Stable to Unstable, was unclear.
  • The options were not mutually exclusive. For example, a game could crash upon exit but otherwise operate normally.
  • Gathering reports into a broader metric, say an average rating, was tricky, due to the above

To mitigate this, we've decided to rally around a new system, derived very closely from one that's been employed for several years by WineHQ's Application Database. It pivots from being a description of status to a medal rating system. Here are the options:

  • Platinum (runs perfectly out of the box)
  • Gold (runs perfectly after tweaks)
  • Silver (runs with minor issues, but generally is playable)
  • Bronze (runs, but often crashes or has issues preventing from playing comfortably)
  • Borked (game either won't start or is crucially unplayable)

These options are now available on the Google Form used for contributing, and will soon be available in a new form system we're building directly into the site.

We hope that this will provide a clear and measurable framework for reports present and future. To leverage the reports we already have, we've combed through them all and migrated status to rating as we felt appropriate. It's possible we've made mistakes, so if you notice any glaring ones, please let us know on Discord.

Finally, I'd like to give a big thanks to everyone in the community for helping bring this site and the Steam Play initiative forward with your feedback and over 7,000 reports. We're starting to get some attention with all of this. In particular, I'd like to thank AlexH and dreamer_ for the careful dedication to migrate our existing reports to the new system. Cheers!

migelius (@buck)

Following Post

SPCR relaunches as ProtonDB!(October 27, 2018)
,