Monday, August 6, 2007

Todd Speaks: 1/2 Stars

Dear blogfans: Todd is apparently more busy than I am and consequently this was the only way I could drag him on stage here. Be nice or he won’t come back. Todd? I’ve got a gajillion people here who would really like to know why we don’t have ½ stars. You’ve read their comments; you saw the poll we ran. Is this going to happen or not, and if not, why not?

Todd: Hello Everyone in blogland. First of all, thanks for your debate on the topic. Your comment chain sounded a lot like the voices in my head for the past 6 months. As a film fan, like you, I had an instinct that ½ stars would be preferable to the 5 whole stars we currently use. But as you have pointed out so well, there are good cases that can be made on both sides of the argument. If I might re-iterate for those just tuning in: on one hand, 5 stars is simple and pretty good for its level of simplicity. On the other hand, it doesn’t create the kind of fine tuning and accuracy that even we use in our predictions. Since someone can always rate on the even star increments, it would appear that adding ½ star options is a plus with no down side. Or on the outside chance there IS a downside, a preference setting seems like it would solve that. This is a compelling argument.

Now from the Netflix vantage point.
As I said, your hunches are almost identical to mine. So here’s what I learned from months of testing this across the country: when we make the ½ star options possible, we get fewer ratings. Significantly fewer ratings. We have argued these results internally for some time, and our best guess is that the complexity of doubling the number of choices from 5 to 10 deters many people from rating, so they just give up. (“3 stars? No, 3 ½ stars.. no… 3 stars… no… oh forget it…”) At Netflix we want people to rate so we can give them a better site experience (better suggestions, better predictions, better use of the pages to showcase movies they’re less likely to have seen).

I like the improved accuracy, but I’d rather have more people rate than fewer. The preference setting is a marginal solution because, as Michael has discussed in this blog before, very few people ever find and change their preferences (even if we simplified the preference page, which we should do anyway!) – so the standard behavior has to be pretty ideal. For those of you who are concerned that there is no good middle (no opinion vote), please consider using 3 stars for that purpose because that is the way we use it in our recommendations system. 4-5 star ratings tell us to boost up movies like it when predicting for you and 1-2 star ratings tell us to punish movies like it when we predict for you, but we treat 3 stars as a neutral signal.

I have not given up on this issue, because you and I generally agree it seems like it would be a good thing. But it isn’t a slam dunk and I have more work to do to make sure this helps everyone and is simple. Consequently, the vote is still out, but don’t hold your breath. I’ll let you know if something changes for sure one way or the other. Sorry it’s not what you want to hear. But I hope it will give you some context.

67 comments:

  1. Why not just set the default as 5 stars with the option of turning on the 1/2 stars under preferences?

    Reversing the process seems to remove the problem.

    ReplyDelete
  2. Just because the majority of people vote on here to have 5 stars - doesn't make it a real majority.

    Todd, how about having a way, like sending out an email to all members, to get more members to vote on the blog poll, besides those who vote directly on the blog? This way, if more people vote, the percentage of those interested in 1/2 stars may actually decrease.

    ReplyDelete
  3. I agree with nuuuu. The segment of Netflix users visiting this blog is very small (No offense), and the ones who are visiting have strong opinions about the stars issue. And... not much else ;)

    I am pretty sure in my Netflix preferences somewhere, I agreed to accept email surveys (As well as all kinds of newsletters that you guys never even send) from you. So, go ahead and send one out to all Netflix users, ask important questions you are wondering about and be done with it.

    I know I am so tired of hearing about it, when your time could be spent on so many other things. That's still a thumbs down from me, in case you are wondering.

    ReplyDelete
  4. Quote:

    "For those of you who are concerned that there is no good middle (no opinion vote), please consider using 3 stars for that purpose because that is the way we use it in our recommendations system. 4-5 star ratings tell us to boost up movies like it when predicting for you and 1-2 star ratings tell us to punish movies like it when we predict for you, but we treat 3 stars as a neutral signal."

    Thank you for letting us know here. Now, I think it is a good idea to put it somewhere on the main site, so that all members who rate movies know how it's helping them to rate. I personally, jumped from just over 500 recommendations, to a little over 700. For me that's a huge jump and now I'm getting the recommendations that I've been wanting to get all the time. I changed a lot of the 3's to 4's and many of the 4's to 5's.

    ReplyDelete
  5. Thanks for letting us know where you are on this, Todd - pretty much where we are, it seems : torn! I agree that a comprehensive questionnaire to all Netflix members might well resolve a lot of these issues that we relative few keep going around in circles about. I'd just ask, if The Team decides to do that, that you include the possibility of opting in or out of half stars (something you don't mention), since that would indeed seem to be the perfect solution.

    ReplyDelete
  6. Apologies, you do mention it. But was it ever optional in your tests? Because surely anyone motivated enough to switch ON 1/2 stars in their preferences will be equally motivated to vote in 1/2 stars, thus improving the overall accuracy of the site?

    ReplyDelete
  7. Was wondering how your software processes Greasemonkey's 1/2 stars. As 2.5, or just 2? If the latter, that might energize a few more people who think they have a fix to vote for 1/2 stars. Thanx.

    ReplyDelete
  8. "For those of you who are concerned that there is no good middle (no opinion vote), please consider using 3 stars for that purpose because that is the way we use it in our recommendations system. 4-5 star ratings tell us to boost up movies like it when predicting for you and 1-2 star ratings tell us to punish movies like it when we predict for you, but we treat 3 stars as a neutral signal."

    This is exactly why I think that the labels for each star rating should be removed. I know I don't use them and a lot of others here don't either - and in this case, they're somewhat misleading, because 3 stars is labelled "liked it", but for your purposes, it is considered neutral.

    Everyone I know attaches their own labels to the star ratings, and in some cases, they are wildly different (for example, one friend of mine uses the differentiation between 3 and 4 stars as whether or not he would want to watch it a second time). But it has never had a detrimental effect on our abilities to understand each other's opinions of a movie.

    ReplyDelete
  9. Todd, in case you missed it, THANK YOU for showing up here.

    ReplyDelete
  10. Thanks for the insights Todd, I suppose that makes sense to a certain degree. How about this scenario:

    You sign up for Netflix and rent some movies. You start rating them based on the 5 star system. After x amount of ratings (10?) A dialog comes up that says "We see you like rating your movies... blah blah blah, would like to find tune your results by enabling 1/2 stars?" They can say yes or no to such a dialog and be alerted that they can always switch back in their preferences if they want to. This solves a lot of the problems in my opinion, you make the option available as a preference, you let people know the option is available who would otherwise not check their preferences page, and you offer a very simple way to accept or deny it.

    ReplyDelete
  11. I've already said my piece on 1/2 stars in the previous thread. My only reiteration would be to support the idea of having whole stars as the default with the option to enable 1/2 stars.

    On a separate note, I am VERY interested in knowing how the Netflix recommender processes 1/2 star ratings made using the Greasemonkey script: If I rate something 2.5 stars... does that get processed as 2 stars (negative) or 3 stars (neutral)?

    I sincerely hope you're rounding up. It would be nice to know that I can keep my 2.5 star ratings without it effecting my recommendations for similar films... and also to know that I can use the 3.5 star rating to boost a movie up to "positive" in the recommendation engine, while still keeping it in the "liked it" range of ratings.

    ReplyDelete
  12. Todd, many thanks for your time! Id like to disagree with your priorities. It doesn't bother me at all that fewer people will rate movies if you give them more stars. I don't need more people rating "The Simpsons". I need more discerning people rating movies.

    ReplyDelete
  13. Thanks for the insight on how ratings affect recommendations. It brings up another question in my mind though:

    How (if at all) does marking a movie "not interested" affect things? Does it get factored into the recommendation algorithm or just flag it so it never shows up as a recommendation?

    ReplyDelete
  14. @anonymous (the last one) - I'd say that Todd's priorities are right where they should be (and also where his boss says they should be). Having more people rate more movies is exactly what Netflix wants, as it has a positive effect on customer loyalty. I have no reason to want to leave Netflix for a competitor, but even if I did, the thought of having to transfer my 1200+ ratings would probably be enough of a deterrent to keep me around.

    Having more people rating more movies also enhances YOUR Netflix experience, no matter how good or bad the tastes of those people may be. The Netflix recommendation system relies on comparing your ratings with the ratings of other people. So the more people there are to compare with yourself, the better your recommendations are going to be.

    ReplyDelete
  15. @Amanda - Unless they personally use it, I doubt the Netflix guys know how the 1/2 star greasemonkey script works. The conversion from half to whole stars almost assuredly is done by the script itself rather than by any code written by Netflix employees. Your best bet for an answer is to look at the script code yourself or ask the person who wrote it.

    ReplyDelete
  16. Todd, I'd like to echo Shaun's comment earlier, that the labels on the stars should be changed. If 3 stars is effectively ignored by the ratings system, it shouldn't be labeled as "liked it." If I like a movie, I expect that a recommendations system would take that into account.

    ReplyDelete
  17. @Michael, Todd, Steve, Meghan, Vikram, et al, we all very much appreciate your great efforts on our behalf, and know you're all crazy busy, but would it be possible, say at the end of the week, to take a few minutes to answer a few of the very specific questions that were posted during that week (like "How (if at all) does marking a movie "not interested" affect things?) - as Michael often pops up and does, bless him - because otherwise they just get buried deeper and deeper in the posts and never get answered. Oh, and customer service rarely seems to know these things, even with (but occasionally without) all the best will in the world. Many thanks.

    ReplyDelete
  18. @eric - Actually, there is no conversion of 1/2 star ratings on the Greasemonkey script. They are fed into the Netflix system and stored in Netflix's database as the 1/2 star ratings you give... they are displayed on the Netflix website as rated 1/2 stars... and even all of your Netflix friends can see when you've rated a movie 1/2 stars using the script.

    I just want to know how the recommender engine treats a 1/2 star rating for the purpose of giving recommendations -- does it round up, round down, or are 1/2 star ratings just ignored?

    ReplyDelete
  19. Accurate ratings are more important than making it easier for the masses.

    If those people are too stupid to decide between 3 and 3-1/2 stars, screw them.

    ReplyDelete
  20. But Eric, Shaun also wants everyone to follow their bliss when assigning a value to stars, so one person's 'liked it' will be another person 'blah'. Let's all get on the same page so these stars start communicating the same thing to everyone equally (most importantly Netflix), not just to our iconoclastic friends; otherwise we'll have to start adding footnotes.

    ReplyDelete
  21. Oh dear, Anonymous @11.32 am. We were trying SO hard not to frighten Todd away, as instructed....

    ReplyDelete
  22. is there someway one could enable half star ratings on their account if they choose to have them, perhaps?

    ReplyDelete
  23. In my limited experience over at 'that other website', the problem of assigning 1/2 stars is never with movies you've just seen, but with ones you haven't seen in a long time, especially when you're trying to plough through pages and pages of them to bump up the number of recs. But if I ever felt a brain freeze coming on I'd just revert to whole stars rather than poop out completely. Maybe that instruction could be given when the next 1/2 star test area is chosen, emphasizing that if people prefer whole stars they should stick to those?

    ReplyDelete
  24. Thanks for speaking, Todd. It really helps to know how you guys on the inside are thinking. I appreciate hearing that 3 stars are neutral, that is often my intention, although not always. It's also interesting to know about the significant drop-off rate with half-stars. While not being able to relate personally, like most hard-core raters, at least I can understand it. Wasn't there some discussion awhile back about 6 stars instead of 5?

    ReplyDelete
  25. Yes, was zero stars ever tested? That might be a good compromise to separate 2, 3 and 4 stars a bit more if 1/2 stars end up getting nixed (sigh).

    ReplyDelete
  26. Hi Todd. Thannks for entering fray.

    Okay, so now after rating 1600+ movies I find out that 3 stars is considered neutral by the Netflix system? Would it be possible to consider changing the labels on the stars to accurately reflect how they are going to be "viewed" by Netflix?

    I reallly don't care for giving something I felt neutral about a 3 star rating. That will give my friends the idea, per your star lables, that I actually liked something that I may actually feel ambivilant about.

    ReplyDelete
  27. I don't understand why Netflix would cater to people who don't take full advantage of the Preferences menu.

    If people do not wish to take advantage of the features offered to them by Netflix, that's their prerogative. However, I think it's kind of ridiculous to limit those who do wish to take full advantage of Netflix features.

    If there really is concern for people who cannot be bothered to figure out how to work the Preferences section, perhaps an e-mail should be drafted giving step-by-step directions on how to change the ratings system.

    ReplyDelete
  28. I may be wrong, but didn't 1 and 2 stars mean "Didn't like it" & "It was OK" at one point in time?

    ReplyDelete
  29. I have a couple of friends who roll their eyes whenever I try to bring up any of this stuff, giving me a "just send me the movies and leave me alone" kinda look - and maybe Netflix has gotten a lot of that in their research outside of this particularly savvy blogosphere. But if the preferences were all to be defaulted to stripped-down basic, and centralized on one easy to find and easy to understand page, I would have though both camps could be kept happy?

    ReplyDelete
  30. @Dustin. Yes, you're correct. That is what it used to be. The labels changed to their current state 2-3 years ago, I believe.

    ReplyDelete
  31. I think Todd's proposed solution of 10 half stars is the reason why users were scared away.

    Simply, why not provide 5 stars, and code it in JavaScript to allow users to click on the left side of the star to make it a half, or the right side of the star to make it a full star?

    This will solve the problem for all:

    1. Users that want to more accurately define their movie tastes can do so
    2. People will not be turned away with more choices to make, since functionally it will appear and act pretty much the same

    ReplyDelete
  32. @Eric - The options were always 10 whole stars - or - 10 half stars using the existing 5 star format (left and right side clickable as you describe) - or - 5 whole stars. All with or without an additional zero. Not sure what your point is?

    ReplyDelete
  33. even restaurant critics have moved away from 1/2 stars. netflix is fabulous as is, don't bog them down with nitpicking ideas. keep up the good work netflix. i have been a member for like four years now and have not one complaint. thank you

    ReplyDelete
  34. I agree with mysteryshopper. Let's not bog down with such minutiae as half star ratings. Everyone, just make a decision on your ratings rather than riding the fence with a 2 1/2. Let Netflix work on the more pressing issues.

    ReplyDelete
  35. I've got to agree with the people who are saying if 3 stars is supposed to be neutral, it needs to be labeled that way. And it's a little late to be telling us now. Not to mention the people reading this blog are probably less than 1 percent of netflix users, so what good does it do. It would be even better to make them something other than stars, to make it blatantly obvious. Thumbs up, down and in the middle for example. Though I'm sure that will never happen. Unfortunately, Shaun is right, and everyone probably puts their own interpretation on what X number of stars represents. But I think that seriously undermines the whole recommendation system.

    Here's some suggestions for new labels:

    5) would watch again
    4) liked it
    3) who cares
    2) sucks
    1) the makers of this movie must be punished

    It'd be really nice to give a zero, and the we could shift everything down one and make 5) great!

    ReplyDelete
  36. I was going to say the same thing preston said. The default should be the same as it has been (5 stars), while in the Preferences you should be able to turn on the option of using 1/2 stars.

    ReplyDelete
  37. riding the fence with a 2 1/2

    Enough with your tired phrase. Why don't you go ride something until you come up with a new motto?

    ReplyDelete
  38. And it's not even indecisiveness, as implied, it's trying to be more precise to better guide the reader. As it is, 3 stars stretches all the way from 'I couldn't care less' to 'I liked it'. It may not be 'riding the fence', but the fact is that 3 stars helps me NOT AT ALL decide whether to rent the movie or not. But 3 1/2 stars (ranging from "slightly more than indifferent" to "I really liked it, but with reservations") WOULD tell me something.

    ReplyDelete
  39. Tood, from NetflixAugust 8, 2007 at 8:44 AM

    How (if at all) does marking a movie "not interested" affect things? Does it get factored into the recommendation algorithm or just flag it so it never shows up as a recommendation?

    Dustin: good question. The NOT INTERESTED button tells us not to display the movie again, AND not to present movies like it--it acts as a weak negative rating. Please remember that we never keep any of our 80k+ titles away from any of our users. You can always search for everything on the site, regardless of our rating system.

    ReplyDelete
  40. I would vote for 5 stars, with 1/2 stars available by a preference, off by default. The people who really care can turn it on.

    I think it's best not to have labels and let people rate as they feel it makes sense. But, I think it's really important to add the average rating to everyone -- then someone who's looking can sort of calibrate that person's ratings based on their average.

    Finally, I really wish Not Interested worked as it does but without getting counted as a Rating. I don't consider a movie that I mark as Not Interested as rated. Who knows -- I may end up seeing it and rating it later and it gives a false impression of how many movies I've actually rated.

    ReplyDelete
  41. "I think it's best not to have labels and let people rate as they feel it makes sense".

    Is that you Shaun? If so, please buy a new drum. If not, Oh Lordie, TWO of 'em wanting the star system to be totally useless to the other 6 million of us!

    ReplyDelete
  42. I rated many movies I liked 3 stars, because that's what the explanation says: "liked it". I did not realize that this would be considered "neutral". I'm all for the idea of a "neutral" category, but Netflix' explanation of their star system would have to reflect this. And I'll have to re-rate many of my movies ...

    ReplyDelete
  43. I don't particularly appreciate getting called out for something I didn't say, but I do find it humorous that an anonymous poster accused me of posting anonymously. But I digress...

    I don't see how removing labels really changes anything drastically. The fact is, many folks already ignore them - of my 8 Netflix friends, I don't know a single one that uses them. I would venture to say a healthy percentage (not a majority, perhaps, but a decent chunk) of the userbase does their own thing. Everyone has an implicit understanding of what "1 star" or "5 stars" means, and though they might slightly differ (for me 1 star might be "I hated it", for someone else it might be "I fell asleep during it"), the fact is that they are about the same.

    To prove my point, consider IMDB. Whether or not you like their 10 star system, the fact is that they are unlabeled. Do you really think that IMDB's rating system is useless? Considering it's probably the most used and respected movie site in the world, I would say they're doing a pretty good job...

    ReplyDelete
  44. This comment has been removed by the author.

    ReplyDelete
  45. Todd (or is that Tood?): Thanks for the confirmation of the meaning of "not interested".

    Anonymous (August 8, 2007 11:14 AM): If you don't want "not interested" to count as a rating and you may wind up seeing it later, don't click "not interested". The only ones I mark that way are ones that I know I never ever want to see.

    ReplyDelete
  46. @Shaun - You're right to annoyed about a masked attack, although who knows if even our registered names mean anything? Plus I often try to send something using my 'real' name and it won't recognize my password. I'm mainly teasing you, having read your argument many times on different posts, and it never made much sense to me. I certainly see that one person's 'liked it' is the same as another person's 'good movie', as long as the enthusiasm and recommendation quotients are more or less the same. I just think that the current dual system of 3 stars meaning 'neutral' to some people (and to NF, it turns out), but 'liked it' (as per the banner) to others is nuts; and your argument seemed to be advocating that. But, you're right, IMDB works just fine - so, truce?

    ReplyDelete
  47. I didn't bother to read all the other comments, sorry.

    in my eyes the solution is obvious- if having half stars be the default lowers the amount of ratings that happen, just make whole stars the default and half stars an available option in preferences.seawol

    ReplyDelete
  48. These thoughts have been brewing for a couple days now, so forgive me if I stumble through it a bit.

    I think that Todd perhaps phrased the "3 stars = neutral" thing in an unclear manner. He didn't actually say that 3 stars means you feel neutral about a movie. What he said was that the movie recommender does not take 3-star ratings into account.

    This actually makes sense, because a recommender can only reliably draw conclusions about movies you feel strongly about. While I do, in fact, like all the movies I rate 3 stars, they are not movies I like strongly enough that films similar to them should be recommended to me just by virtue of the fact that they share characteristics with some other movie I liked to some unspecified degree. I must have rated over 1000 movies 3 stars, but I'd be surprised if I have more than 300 4 or 5-star ratings combined. The fact that I rate those select few higher is an indication to the recommendation system that they should be paid attention to.

    The problem is that, while it's fine for the recommender to only take our more extreme ratings into account, that doesn't help us with our desire to communicate our feelings about movies through our ratings clearly to our friends and the Netflix Community at large. So Todd's suggestion that we use the 3-star rating to "fill the void" of the lacking "neutral" rating does not work. 3-stars isn't neutral, it's most certainly positive... just not positive enough for the Netflix recommender to care.

    I think that there are two separate ratings issues going on here:
    1) How our ratings lead to better recommendations from Netflix.
    2) How we use our ratings to communicate to other users.

    For #1, a more simplified system is better:
    -"Give me more like this"
    -"Don't give me anything like this"
    -"Ignore this."
    All we need to achieve this is thumbs up, down, and middle. The goal here is to make recommendations better. In order to achieve this goal, the more ratings a user makes, the better. Having refined ratings doesn't matter at all.

    Unfortunately, "good" "bad" and "meh" doesn't even come close to doing the trick for #2. The goal, in this case, is building a stronger community. The goal is achieved by adding rating levels this time -- we need more variations to hint at our more complex feelings about each film. Some people think 5 stars is enough. I feel 1/2 stars are necessary to express myself... but that's just my opinion (as I've so clearly documented already).

    I just wanted point out that I think the two issues are confusing the discussion because they are getting mixed together.

    ReplyDelete
  49. Agreed, A++ Amanda. But how to satisfy the demands of both #1 and #2 in a whole star system?

    It would seem to me that 1/2 stars then become all the more important, since Netflix' 'recommender' could - in Todd's terminology - start 'rewarding' movies at 3 1/2 stars (while the wider world would know that we liked it but be warned not to rush out and buy it) climbing to an 'essential viewing' of 5 stars. 3 and 2 1/2 could then both be 'neutral' (while allowing for a little shading for our friends) and anything 2 and below could be increasingly 'punished' by Netflix down to either 1/2 or zero?

    But maybe this is precisely the kind of tortured thinking that caused the rating volumes to go down in the test areas, so 5 whole stars should certainly be the primary engine, with a 1/2 star option for the more -------- (I'll leave the word to you) amongst us.

    ReplyDelete
  50. To those who are saying: survey more people! That might be valuable but Todd's post isn't about the desire for 1/2 stars. It's about what happens when 1/2 stars are available. I'm delighted to hear that Netflix has done some testing on this. If not already done, maybe N. should test the suggestion posted above -- what happens when users can opt-in to 1/2 stars? It's quite possible that even these users will end up rating less frequently, in line with the observations Todd's already made. Would be good data to have.

    Of course there is a possible side effect to consider: how would it affect the recommendation engine if people use two different rating systems? Netflix's recommendation system is the most significant reason I stay with the service. It's led me to so many great movies that I'd never heard of! Probably half the movies in my 400+ queue are based on N's predicted ratings. I've come to trust them so much that I upgraded my Netflix subscription. In 2 years, I've had only one disappointing recommendation. Even my favorite movie critics haven't guided me so well!

    Personally, I don't much care whether there are 1/2 stars. But I do care whether the effort to be all things to all people creates excess complexity. That might result in problems for other parts of the service.

    Anyway, great post, Todd. Really appreciate the insight!

    ReplyDelete
  51. I think an easy solution would be to use a scaled click able image. That is you have an image of 5 stars to appease the masses, however the image would be mapped so 1/2 star ratings would be recorded based on where on the image you clicked.

    The scale would be (.5)(1)(1.5)(2)(2.5)(3)(3.5)(4)(4.5)(5) This way if I clicked to the left of center by the first star, the rating would be .5, if between 1 & 2, the recorded rating would be 1.5.

    You would accomplish the 1/2 star rating system to more fine tune the process, without making the sheeple think too hard.

    ReplyDelete
  52. i don't need half stars, would really love zero stars and also a star rating that is labeled OK/neutral or something to that effect. if 3 stars is supposed to mean neutral than let's make sure everyone knows that's what 3 stars means.

    ReplyDelete
  53. the reason so many on here are "nitpicking" about ratings is b/c we use the ratings and enjoy rating our movies. we want it to accurately depict our opinions and be enjoyable.

    ReplyDelete
  54. Hmm. 3 Stars treated as neutral!! Well, I agree with others on this who wish we had been made aware of this before. I'll probably have to move most of my 3 stars to 4, and 4 to 5.

    ReplyDelete
  55. Instead of adding 1.5, 2.5, etc why can you not add just half a star at the beginning or end. You can set it apart and make it stagnant meaning it doesn't highlight when you hover. That way if someone is giving a novie 2.5 stars, they can click the 2 stars and the half star. It's virtually idiot proof.. virtually!

    ReplyDelete
  56. Slightly off topic, but love the new very clear 'rate your genre' layout! Not sure we're giving you guys enough credit for all the minor tweaks going on amidst all the major upheavals.

    Just a thought on the punish/reward component of your recommendation engine - how about showing on the Stars Legend (and wherever else possible) that 1 & 2 stars will reduce recommendations for that kind on movie, while 4 and 5 stars will boost them. I think that would have a major impact on how people vote. I personally think that the boost should kick in at 3 stars, based on your 'liked it' banner, but then again I don't have access to 10 years of voting data. Thanks.

    ReplyDelete
  57. Personally, I would prefer three "stars" - bad, neutral, and good. Then, if you really want precision, Netflix could output the mean, median, and standard deviation of the ratings of each movie to 3 decimal places. Those of you who need more and more stars could then interpret that number as a two-thousand star rating! Never in my wildest dreams though would I have imagined there was such an intense debate over this subject.

    ReplyDelete
  58. So, Ben, when it comes to other things in your life, do you also either love them, feel indifferent, or hate them? If so, I'd maybe see someone about that.

    ReplyDelete
  59. Logically there's no difference between 3 1/2 stars out of 5 and 7 stars out of 10 — except, inevitably, someone would want to give a rating of 7 1/2 (or maybe 6 1/2 or 7 1/4...ad nauseum). I like a 5-point scale because it's simple to understand and has enough breadth to assign some shading of meaning (making it better than 3-points). As I recall Siskel and Ebert used a 2-point scale: thumbs up or thumbs down. (But then both were voting so maybe it was really a 4-point scale...hey! I could do this all day. ;-))

    ReplyDelete
  60. I Dont really care about half star ratings but I sure would like something in between disliked and liked. What about all the so-so, just ok, movies out there. I dont dislike them but I wouldnt watch them again. I feel forced to rate everything as 'liked' which becomes meaningless, or I'm forced to rate them as disliked which isnt true. Can we just get another star on the lower level?

    ReplyDelete
  61. I want half stars, I need half stars, seriously give them to us.

    ReplyDelete
  62. Stars, stars, stars, stars, stars, stars, stars…

    Thank you, thank you for all the hard work and openness.

    Please… Keep the easy and popular 5 star as default (with more accurate labeling that reflects how they are actually applied by Netflix), and always make sure it’s the power users who have to take the extra step to fine-tune their usage in preferences. They are the one’s who are motivated to do it after all.

    If it is really possible to have both… Please… when you have a spare moment… as if… go ahead and implement the half star option as an opt-in for those people who want it so we can all point them to their preferences page and be done with it.

    Personally I think it is a waste of time and effort, and needlessly complicates things, but this is really getting tedious, even for us power users.

    The longer a decision stretches out the worse it will get.

    With 5 stars, you have a natural neutral position. (3) I just don’t get why anyone thinks adding 0 would help. The problem isn’t in the rating system, it’s in your head. That’s not meant to be offensive I just mean, ‘it’s all in how you think about it.’ Get your head right and it works. If 3 is neutral, “take it or leave it,” and the highest position, in this case ‘5,’ is “Great,” then ‘4,’ being half way between, is going to be “Good,” or whatever corresponding descriptive is half way between “neutral” & “great” as far as you are concerned. That’s how it will always naturally, or logically, work, whether they have 5 stars, or 10 sour pickles. (in which case correspondingly 5.5 sour pickles would be neutral)

    It’s clear with all the different methods being applied to the rating system we have now, that they will never be an “accurate” way to communicate our individual opinion. Because even if there is a clear ratings policy, it’s obvious that a huge number of people will interpret ratings in an entirely different manner no matter how much thought we put into it. I used to go through the same agonies about ratings but the ratings will always be a fuzzy indicator for that very reason, stars or half stars, so I hardly think there is ever going to be any way to “increase their accuracy.” Just go with your gut, rate the thing, then relax… let it go. Because no one but you is going to care that much. Unless you are just plane enjoying yourself agonizing away like that, you’re really wasting your energy. If you really want to tell someone what you think and have a real chance that they will actually understand what you think, you have to write a review.

    Does anyone remember the name of that freaky phenomena where if you ask enough people how many jellybeans are in a jar their average will actually predict how many jellybeans are in the jar? No matter how off each individual is, the collective ‘mind’ somehow seems to know the answer?

    Finally: “Not Interested” is being used to get those suggested titles out of the way because they are constantly loading at the top of the pages we are trying to use to find new movies. I wish movie suggestions changed on their own more often so I don’t have to scroll through the same titles again and again to get to the one’s I haven’t seen yet. The thing should just know that, ‘hey, she keeps ignoring these movies and not adding them to her queue, maybe I should show her some of the other hundreds of titles she might like instead.’ (tax tax tax) But it doesn’t, so I get annoyed and throw them off the page with ‘not interested’ to make room for something new to show up. So no. I don’t want them to count as a rating, and I don’t want them to mean I won’t get similar movies suggested just because that particular one didn’t look good enough to squeeze into my queue that hundredth time I’d seen it.

    If good is thumbs up, and bad is thumbs down. Exactly what finger, in what position, is in the middle?

    ReplyDelete
  63. i fully agree, just the fact that majority of people vote on here to have 5 stars - doesn't make it a real majority.

    ReplyDelete
  64. I am going to echo "anonymous" of August 18, 2007 who stated we only need one more star. Neutral or "just ok". I can't believe the long paragraphs about 1/2 stars (too complicated) but we definitely need something between "liked" and "disliked".

    ReplyDelete
  65. I don't think the current labeling of the stars is appropriate. Why are the 5 options loved it, really liked it, liked it, didn't like it, hated it? I would consider switching "didn't like it" to "it was ok" and switch "hated it" to "didn't like it". So the rating system would be loved it, really liked it, liked it, it was ok, didn't like it. I often want to rate a movie, ok, but there is no appropriate label for that. I also don't think you have to go to the point of saying you hated a movie. I believe "I didn't like it" is sufficient. The option for "not interested" works fine for never want to see this movie now or ever again. It would be interesing to see how many people rate movies "hated it". I know if there's a movie I see that I really dislike, I just mark it "not interested"... why even give it a star?

    ReplyDelete