Todd Speaks: 1/2 Stars

Dear blogfans: Todd is apparently more busy than I am and consequently this was the only way I could drag him on stage here. Be nice or he won’t come back. Todd? I’ve got a gajillion people here who would really like to know why we don’t have ½ stars. You’ve read their comments; you saw the poll we ran. Is this going to happen or not, and if not, why not?

Todd: Hello Everyone in blogland. First of all, thanks for your debate on the topic. Your comment chain sounded a lot like the voices in my head for the past 6 months. As a film fan, like you, I had an instinct that ½ stars would be preferable to the 5 whole stars we currently use. But as you have pointed out so well, there are good cases that can be made on both sides of the argument. If I might re-iterate for those just tuning in: on one hand, 5 stars is simple and pretty good for its level of simplicity. On the other hand, it doesn’t create the kind of fine tuning and accuracy that even we use in our predictions. Since someone can always rate on the even star increments, it would appear that adding ½ star options is a plus with no down side. Or on the outside chance there IS a downside, a preference setting seems like it would solve that. This is a compelling argument.

Now from the Netflix vantage point.
As I said, your hunches are almost identical to mine. So here’s what I learned from months of testing this across the country: when we make the ½ star options possible, we get fewer ratings. Significantly fewer ratings. We have argued these results internally for some time, and our best guess is that the complexity of doubling the number of choices from 5 to 10 deters many people from rating, so they just give up. (“3 stars? No, 3 ½ stars.. no… 3 stars… no… oh forget it…”) At Netflix we want people to rate so we can give them a better site experience (better suggestions, better predictions, better use of the pages to showcase movies they’re less likely to have seen).

I like the improved accuracy, but I’d rather have more people rate than fewer. The preference setting is a marginal solution because, as Michael has discussed in this blog before, very few people ever find and change their preferences (even if we simplified the preference page, which we should do anyway!) – so the standard behavior has to be pretty ideal. For those of you who are concerned that there is no good middle (no opinion vote), please consider using 3 stars for that purpose because that is the way we use it in our recommendations system. 4-5 star ratings tell us to boost up movies like it when predicting for you and 1-2 star ratings tell us to punish movies like it when we predict for you, but we treat 3 stars as a neutral signal.

I have not given up on this issue, because you and I generally agree it seems like it would be a good thing. But it isn’t a slam dunk and I have more work to do to make sure this helps everyone and is simple. Consequently, the vote is still out, but don’t hold your breath. I’ll let you know if something changes for sure one way or the other. Sorry it’s not what you want to hear. But I hope it will give you some context.