Comments on The Official Netflix Blog: Todd Speaks: 1/2 Stars

I don't think the current labeling of the stars is...

2008-07-08T20:14:00.000-07:00

I don't think the current labeling of the stars is appropriate. Why are the 5 options loved it, really liked it, liked it, didn't like it, hated it? I would consider switching "didn't like it" to "it was ok" and switch "hated it" to "didn't like it". So the rating system would be loved it, really liked it, liked it, it was ok, didn't like it. I often want to rate a movie, ok, but there is no appropriate label for that. I also don't think you have to go to the point of saying you hated a movie. I believe "I didn't like it" is sufficient. The option for "not interested" works fine for never want to see this movie now or ever again. It would be interesing to see how many people rate movies "hated it". I know if there's a movie I see that I really dislike, I just mark it "not interested"... why even give it a star?

I am going to echo "anonymous" of August 18, 2007 ...

2008-02-27T15:40:00.000-08:00

I am going to echo "anonymous" of August 18, 2007 who stated we only need one more star. Neutral or "just ok". I can't believe the long paragraphs about 1/2 stars (too complicated) but we definitely need something between "liked" and "disliked".

i fully agree, just the fact that majority of peop...

2007-11-09T09:32:00.000-08:00

i fully agree, just the fact that majority of people vote on here to have 5 stars - doesn't make it a real majority.

Stars, stars, stars, stars, stars, stars, stars…Th...

2007-09-09T00:31:00.000-07:00

Stars, stars, stars, stars, stars, stars, stars…

Thank you, thank you for all the hard work and openness.

Please… Keep the easy and popular 5 star as default (with more accurate labeling that reflects how they are actually applied by Netflix), and always make sure it’s the power users who have to take the extra step to fine-tune their usage in preferences. They are the one’s who are motivated to do it after all.

If it is really possible to have both… Please… when you have a spare moment… as if… go ahead and implement the half star option as an opt-in for those people who want it so we can all point them to their preferences page and be done with it.

Personally I think it is a waste of time and effort, and needlessly complicates things, but this is really getting tedious, even for us power users.

The longer a decision stretches out the worse it will get.

With 5 stars, you have a natural neutral position. (3) I just don’t get why anyone thinks adding 0 would help. The problem isn’t in the rating system, it’s in your head. That’s not meant to be offensive I just mean, ‘it’s all in how you think about it.’ Get your head right and it works. If 3 is neutral, “take it or leave it,” and the highest position, in this case ‘5,’ is “Great,” then ‘4,’ being half way between, is going to be “Good,” or whatever corresponding descriptive is half way between “neutral” & “great” as far as you are concerned. That’s how it will always naturally, or logically, work, whether they have 5 stars, or 10 sour pickles. (in which case correspondingly 5.5 sour pickles would be neutral)

It’s clear with all the different methods being applied to the rating system we have now, that they will never be an “accurate” way to communicate our individual opinion. Because even if there is a clear ratings policy, it’s obvious that a huge number of people will interpret ratings in an entirely different manner no matter how much thought we put into it. I used to go through the same agonies about ratings but the ratings will always be a fuzzy indicator for that very reason, stars or half stars, so I hardly think there is ever going to be any way to “increase their accuracy.” Just go with your gut, rate the thing, then relax… let it go. Because no one but you is going to care that much. Unless you are just plane enjoying yourself agonizing away like that, you’re really wasting your energy. If you really want to tell someone what you think and have a real chance that they will actually understand what you think, you have to write a review.

Does anyone remember the name of that freaky phenomena where if you ask enough people how many jellybeans are in a jar their average will actually predict how many jellybeans are in the jar? No matter how off each individual is, the collective ‘mind’ somehow seems to know the answer?

Finally: “Not Interested” is being used to get those suggested titles out of the way because they are constantly loading at the top of the pages we are trying to use to find new movies. I wish movie suggestions changed on their own more often so I don’t have to scroll through the same titles again and again to get to the one’s I haven’t seen yet. The thing should just know that, ‘hey, she keeps ignoring these movies and not adding them to her queue, maybe I should show her some of the other hundreds of titles she might like instead.’ (tax tax tax) But it doesn’t, so I get annoyed and throw them off the page with ‘not interested’ to make room for something new to show up. So no. I don’t want them to count as a rating, and I don’t want them to mean I won’t get similar movies suggested just because that particular one didn’t look good enough to squeeze into my queue that hundredth time I’d seen it.

If good is thumbs up, and bad is thumbs down. Exactly what finger, in what position, is in the middle?

I want half stars, I need half stars, seriously gi...

2007-08-19T19:48:00.000-07:00

I want half stars, I need half stars, seriously give them to us.

I Dont really care about half star ratings but I s...

2007-08-18T08:55:00.000-07:00

I Dont really care about half star ratings but I sure would like something in between disliked and liked. What about all the so-so, just ok, movies out there. I dont dislike them but I wouldnt watch them again. I feel forced to rate everything as 'liked' which becomes meaningless, or I'm forced to rate them as disliked which isnt true. Can we just get another star on the lower level?

Logically there's no difference between 3 1/2 star...

2007-08-16T15:15:00.000-07:00

Logically there's no difference between 3 1/2 stars out of 5 and 7 stars out of 10 — except, inevitably, someone would want to give a rating of 7 1/2 (or maybe 6 1/2 or 7 1/4...ad nauseum). I like a 5-point scale because it's simple to understand and has enough breadth to assign some shading of meaning (making it better than 3-points). As I recall Siskel and Ebert used a 2-point scale: thumbs up or thumbs down. (But then both were voting so maybe it was really a 4-point scale...hey! I could do this all day. ;-))

So, Ben, when it comes to other things in your lif...

2007-08-14T01:21:00.000-07:00

So, Ben, when it comes to other things in your life, do you also either love them, feel indifferent, or hate them? If so, I'd maybe see someone about that.

Personally, I would prefer three "stars" - bad, ne...

2007-08-13T17:56:00.000-07:00

Personally, I would prefer three "stars" - bad, neutral, and good. Then, if you really want precision, Netflix could output the mean, median, and standard deviation of the ratings of each movie to 3 decimal places. Those of you who need more and more stars could then interpret that number as a two-thousand star rating! Never in my wildest dreams though would I have imagined there was such an intense debate over this subject.

Slightly off topic, but love the new very clear 'r...

2007-08-10T07:02:00.000-07:00

Slightly off topic, but love the new very clear 'rate your genre' layout! Not sure we're giving you guys enough credit for all the minor tweaks going on amidst all the major upheavals.

Just a thought on the punish/reward component of your recommendation engine - how about showing on the Stars Legend (and wherever else possible) that 1 & 2 stars will reduce recommendations for that kind on movie, while 4 and 5 stars will boost them. I think that would have a major impact on how people vote. I personally think that the boost should kick in at 3 stars, based on your 'liked it' banner, but then again I don't have access to 10 years of voting data. Thanks.

Instead of adding 1.5, 2.5, etc why can you not ad...

2007-08-09T19:56:00.000-07:00

Instead of adding 1.5, 2.5, etc why can you not add just half a star at the beginning or end. You can set it apart and make it stagnant meaning it doesn't highlight when you hover. That way if someone is giving a novie 2.5 stars, they can click the 2 stars and the half star. It's virtually idiot proof.. virtually!

Hmm. 3 Stars treated as neutral!! Well, I agree wi...

2007-08-09T17:35:00.000-07:00

Hmm. 3 Stars treated as neutral!! Well, I agree with others on this who wish we had been made aware of this before. I'll probably have to move most of my 3 stars to 4, and 4 to 5.

the reason so many on here are "nitpicking" about ...

2007-08-09T16:05:00.000-07:00

the reason so many on here are "nitpicking" about ratings is b/c we use the ratings and enjoy rating our movies. we want it to accurately depict our opinions and be enjoyable.

i don't need half stars, would really love zero st...

2007-08-09T16:03:00.000-07:00

i don't need half stars, would really love zero stars and also a star rating that is labeled OK/neutral or something to that effect. if 3 stars is supposed to mean neutral than let's make sure everyone knows that's what 3 stars means.

I think an easy solution would be to use a scaled ...

2007-08-09T12:20:00.000-07:00

I think an easy solution would be to use a scaled click able image. That is you have an image of 5 stars to appease the masses, however the image would be mapped so 1/2 star ratings would be recorded based on where on the image you clicked.

The scale would be (.5)(1)(1.5)(2)(2.5)(3)(3.5)(4)(4.5)(5) This way if I clicked to the left of center by the first star, the rating would be .5, if between 1 & 2, the recorded rating would be 1.5.

You would accomplish the 1/2 star rating system to more fine tune the process, without making the sheeple think too hard.

To those who are saying: survey more people! That ...

2007-08-09T07:22:00.000-07:00

To those who are saying: survey more people! That might be valuable but Todd's post isn't about the desire for 1/2 stars. It's about what happens when 1/2 stars are available. I'm delighted to hear that Netflix has done some testing on this. If not already done, maybe N. should test the suggestion posted above -- what happens when users can opt-in to 1/2 stars? It's quite possible that even these users will end up rating less frequently, in line with the observations Todd's already made. Would be good data to have.

Of course there is a possible side effect to consider: how would it affect the recommendation engine if people use two different rating systems? Netflix's recommendation system is the most significant reason I stay with the service. It's led me to so many great movies that I'd never heard of! Probably half the movies in my 400+ queue are based on N's predicted ratings. I've come to trust them so much that I upgraded my Netflix subscription. In 2 years, I've had only one disappointing recommendation. Even my favorite movie critics haven't guided me so well!

Personally, I don't much care whether there are 1/2 stars. But I do care whether the effort to be all things to all people creates excess complexity. That might result in problems for other parts of the service.

Anyway, great post, Todd. Really appreciate the insight!

Agreed, A++ Amanda. But how to satisfy the demands...

2007-08-09T07:16:00.000-07:00

Agreed, A++ Amanda. But how to satisfy the demands of both #1 and #2 in a whole star system?

It would seem to me that 1/2 stars then become all the more important, since Netflix' 'recommender' could - in Todd's terminology - start 'rewarding' movies at 3 1/2 stars (while the wider world would know that we liked it but be warned not to rush out and buy it) climbing to an 'essential viewing' of 5 stars. 3 and 2 1/2 could then both be 'neutral' (while allowing for a little shading for our friends) and anything 2 and below could be increasingly 'punished' by Netflix down to either 1/2 or zero?

But maybe this is precisely the kind of tortured thinking that caused the rating volumes to go down in the test areas, so 5 whole stars should certainly be the primary engine, with a 1/2 star option for the more -------- (I'll leave the word to you) amongst us.

Very well said amanda...

2007-08-09T05:06:00.000-07:00

Very well said amanda...

These thoughts have been brewing for a couple days...

2007-08-09T02:00:00.000-07:00

These thoughts have been brewing for a couple days now, so forgive me if I stumble through it a bit.

I think that Todd perhaps phrased the "3 stars = neutral" thing in an unclear manner. He didn't actually say that 3 stars means you feel neutral about a movie. What he said was that the movie recommender does not take 3-star ratings into account.

This actually makes sense, because a recommender can only reliably draw conclusions about movies you feel strongly about. While I do, in fact, like all the movies I rate 3 stars, they are not movies I like strongly enough that films similar to them should be recommended to me just by virtue of the fact that they share characteristics with some other movie I liked to some unspecified degree. I must have rated over 1000 movies 3 stars, but I'd be surprised if I have more than 300 4 or 5-star ratings combined. The fact that I rate those select few higher is an indication to the recommendation system that they should be paid attention to.

The problem is that, while it's fine for the recommender to only take our more extreme ratings into account, that doesn't help us with our desire to communicate our feelings about movies through our ratings clearly to our friends and the Netflix Community at large. So Todd's suggestion that we use the 3-star rating to "fill the void" of the lacking "neutral" rating does not work. 3-stars isn't neutral, it's most certainly positive... just not positive enough for the Netflix recommender to care.

I think that there are two separate ratings issues going on here:
1) How our ratings lead to better recommendations from Netflix.
2) How we use our ratings to communicate to other users.

For #1, a more simplified system is better:
-"Give me more like this"
-"Don't give me anything like this"
-"Ignore this."
All we need to achieve this is thumbs up, down, and middle. The goal here is to make recommendations better. In order to achieve this goal, the more ratings a user makes, the better. Having refined ratings doesn't matter at all.

Unfortunately, "good" "bad" and "meh" doesn't even come close to doing the trick for #2. The goal, in this case, is building a stronger community. The goal is achieved by adding rating levels this time -- we need more variations to hint at our more complex feelings about each film. Some people think 5 stars is enough. I feel 1/2 stars are necessary to express myself... but that's just my opinion (as I've so clearly documented already).

I just wanted point out that I think the two issues are confusing the discussion because they are getting mixed together.

I didn't bother to read all the other comments, so...

2007-08-08T21:40:00.000-07:00

I didn't bother to read all the other comments, sorry.

in my eyes the solution is obvious- if having half stars be the default lowers the amount of ratings that happen, just make whole stars the default and half stars an available option in preferences.seawol

@Shaun - You're right to annoyed about a masked at...

2007-08-08T15:32:00.000-07:00

@Shaun - You're right to annoyed about a masked attack, although who knows if even our registered names mean anything? Plus I often try to send something using my 'real' name and it won't recognize my password. I'm mainly teasing you, having read your argument many times on different posts, and it never made much sense to me. I certainly see that one person's 'liked it' is the same as another person's 'good movie', as long as the enthusiasm and recommendation quotients are more or less the same. I just think that the current dual system of 3 stars meaning 'neutral' to some people (and to NF, it turns out), but 'liked it' (as per the banner) to others is nuts; and your argument seemed to be advocating that. But, you're right, IMDB works just fine - so, truce?

Todd (or is that Tood?): Thanks for the confirmat...

2007-08-08T15:14:00.000-07:00

Todd (or is that Tood?): Thanks for the confirmation of the meaning of "not interested".

Anonymous (August 8, 2007 11:14 AM): If you don't want "not interested" to count as a rating and you may wind up seeing it later, don't click "not interested". The only ones I mark that way are ones that I know I never ever want to see.

I don't particularly appreciate getting called out...

2007-08-08T14:04:00.000-07:00

I don't particularly appreciate getting called out for something I didn't say, but I do find it humorous that an anonymous poster accused me of posting anonymously. But I digress...

I don't see how removing labels really changes anything drastically. The fact is, many folks already ignore them - of my 8 Netflix friends, I don't know a single one that uses them. I would venture to say a healthy percentage (not a majority, perhaps, but a decent chunk) of the userbase does their own thing. Everyone has an implicit understanding of what "1 star" or "5 stars" means, and though they might slightly differ (for me 1 star might be "I hated it", for someone else it might be "I fell asleep during it"), the fact is that they are about the same.

To prove my point, consider IMDB. Whether or not you like their 10 star system, the fact is that they are unlabeled. Do you really think that IMDB's rating system is useless? Considering it's probably the most used and respected movie site in the world, I would say they're doing a pretty good job...

I rated many movies I liked 3 stars, because that'...

2007-08-08T14:03:00.000-07:00

I rated many movies I liked 3 stars, because that's what the explanation says: "liked it". I did not realize that this would be considered "neutral". I'm all for the idea of a "neutral" category, but Netflix' explanation of their star system would have to reflect this. And I'll have to re-rate many of my movies ...

"I think it's best not to have labels and let peop...

2007-08-08T11:40:00.000-07:00

"I think it's best not to have labels and let people rate as they feel it makes sense".

Is that you Shaun? If so, please buy a new drum. If not, Oh Lordie, TWO of 'em wanting the star system to be totally useless to the other 6 million of us!