Welcome to the home of Noah Brier. I'm the co-founder of Variance and general internet tinkerer. Most of my writing these days is happening over at Why is this interesting?, a daily email full of interesting stuff. This site has been around since 2004. Feel free to get in touch. Good places to get started are my Framework of the Day posts or my favorite books and podcasts. Get in touch.

You can subscribe to this site via RSS (the humanity!) or .

Rating Systems and Personal Rules

Rating systems are something I find myself discussing fairly frequently. Partly because the idea shows up in a lot of projects and partly because I usually end up relating it to the rules people set for themselves within different social networks.

Let me explain. First off, this is all inspired by something Tyler Cowen wrote recently about rating systems:

The old rating system granted up to five stars but now the maximum number of stars is ten. This signals that they wish to start exaggerating the quality of the product. When there are only five stars you know that they are laying their reputation on the line when they grant five stars to a new CD. (Michelin of course won’t give a restaurant more than three stars. They don’t calculate out to the fourth decimal place along a scale of one thousand.) If the music isn’t good you can decide to stop trusting them. But say they give a new release eight, nine, or who knows maybe eight and a half stars? What exactly are they trying to say? Yes they are putting their reputation on the line when they give ten stars, but this will happen so infrequently that it will be harder to judge their overall trustworthiness.

His point is a good one, but one that is made much easier within a single publication. There is no reason that as part of the change Spin couldn’t provide a key to go along with the system, one where they lay out what a 10 really means in terms of past album reviews.

With that said, I don’t really want to talk about Spin’s rating system, I want to talk about mine. When I rate things within iTunes there are specific rules in my head. If something gets one star, it’s pretty damn awesome. That’s because only songs that I really like get a star. Therefore, a five star song is my very favorite ever (I don’t think I currently have it in rotation, but November Rain might get it). Now I’m sure you have some other way of rating things. Maybe 0 stars is the worst song you’ve ever heard and you work from there. Whatever it is, the point is, it’s probably different than mine. This isn’t a big deal when we’re the ones consuming our music, after all it’s pretty easy to keep track of our own criteria, however, it becomes a lot more confusing when more and more folks are exposed to it.

But exposing folks isn’t half the problem, the real trouble starts when we start to build systems that aggregate ratings. At that point, everyone’s criteria is thrown into a big bucket, stirred around and then shown off to the world in a way that ultimately means very little. What’s a five star video on YouTube? Who the hell knows. For me, the only time I ever rate anything I give it five stars. (For whatever that’s worth.)

I’m not sure there’s a real point to all this, but I’m going to try and get there. First off, it speaks to implicit rating systems. Knowing the stuff people are actually paying attention to (a la last.fm) is a whole lot more effective than asking people their favorite things. People are generally not that good at telling you what they really like or pay attention to. (I don’t have anything to back that up, but it’s probably true.)

Also, and this is where things get really tangential, I think this relates to the criteria people have for “friending” on different social networks. Who do you connect with on Facebook? How about LinkedIn? How about Twitter? You probably have thought these things through and answered them for yourself. Equally likely is that my answer is different. We all have different criteria for the decisions we make in these places.

Now none of this really matters and we could all go along happily, except people forget that and get upset when they’re not friended, followed or connected with. I get a kick when I read things like this from David Pogue on his adoption of Twitter: “One guy took me to task for asking ‘dopey questions.’ Others criticized me for various infractions, like not following enough other people, writing too much about nontech topics or sending too many or too few messages.” That’s insane. (Something Evan Williams acknowledges in the article.) The truth with all this stuff is there are no rules. Or, more accurately, there are an infinite number of them.

So yeah, there’s that.

February 25, 2009