Thursday, July 19, 2007

Fixing Up Reviews

Be assured my friends, getting these reviews to be more useful for ya'll is a high priority for our team. I'll lay out some details shortly. Lots to do, folks. Lot's to do. Thanks for your insights.

Oh, by the way -- i'm implementing some polls over in the right column. It's a start. (Is there anyone better than polldaddy? Seems pretty cool.


  1. Comment on the poll:
    While "Community" gives the clearest meaning among the choices, I voted for Flixville, since it carries part of the Netflix brand name with it AND also conveys a sense of community.

  2. Three things nobody seems to mention when they get all up in arms about 1/2 star ratings:

    1) The important thing really isn't YOUR individual rating; it's the 'meta-rating' that comes out of everyone's collective input. If a million people have all rated "The Godfather" and it's averaging a 4.5, it's probably a very good movie, regardless of whether you personally gave it a four or a five. Look at Rotten Tomatoes - it boils every review down to just 'good' or 'bad', yet may have the best system on the web, because of the number of reviews it draws from.

    2) We all know what your ratings mean. Four or Five were very good, Three were nothing special, and One or Two sucked. I don't need the 1.5 or 2.5 stars to tell me the precise level of suck; I'm not interested. I don't need 3.5 to tell me it was almost special, and 4.5 to tell me it was almost perfect. They serve no additional purpose. You are not writing a review a million people are planning their weekend fare around. It's for your friends to read, and to round out Netflix's meta-data.

    3) I've rated 1,800+ movies, and I'm anal-retentive. That means that should 1/2 stars be implemented, I'm going to get bored one night and ask myself 1,800+ times, "Now is that really a X stars movie, or is it X+1/2 or X-1/2?" And then I'll be annoyed.

    Just because a vocal minority wants an improvement doesn't mean it will better serve the overall Netflix population. Only hard-core users want 1/2 stars, just as I'm sure only hard-core users are reading this blog. Of my 10 friends, only 6 regularly rate the movies they've seen. If that many people don't rate the movies anything, how many of the remaining people really care whether they can rate at 1/2 star intervals?

    1/2 stars will not help Netflix. For every three people who would have rated a movie 4, one will stay at 4, one will go 4.5, and one will change to 3.5. What does that improve?

  3. But why not tell us what you really think, Billy? Nobutseriously, your trifurcated point is a good one, but even if we do all live in pitiful ego bubbles that barely register on any but our nearest and dearest, what's the harm in indulging our delusional sense of self-empowerment as long as we all end up with a reasonably good sense of whether or not a movie is worth renting?

  4. I think Billy's point is that if the only real benefit of the feature is "indulging our delusional sense of self-empowerment", then it's not worth Netflix's time, especially if it comes at the cost of other features.

    That being said, I am used to rating on IMDB's scale of 10 so I would find 1/2 stars handy. Necessary? No, because I basically say that IMDB 1/10 and 2/10 = Netflix 1 star, and so on. But that's just my philosophy...

  5. Hey, with regards to the poll, glad you added one to the blog! That makes me happy.

    I picked Flixville as well, because it has personality. Community does mean Community, but it sounds sterile and overused. Flixville could be very nice :)

    So, we're going to have a different poll every day? That'd be pretty great, and a reason for me to come check the blog daily, if there aren't new posts.

    This site keeps getting better and better.

  6. Great points Billy, and very precisely stated. Half points won't really improve the accuracy of the ratings. Much more important is disallowing the possibility of multiple voting on both movies and reviews. Maybe 3 stars should be redefined to "just ok", "fair", or "average".

  7. Oh, and you asked if "Anyone was better than Polldaddy".

    Honestly, I've built some websites in the past, and it is hard enough to find a good poll creation site that works, so if you get one that works, you should stick with them.

    Only thing, why didn't you choose a pretty color for the poll box rather than gray? ;) You could pick any color in the rainbow.

    (Just checked their website, it says you have to use CSS to change the poll color... I am not well versed in that.)

  8. OK, so maybe tongue-in-cheek flipness doesn't translate well in writing (no levity allowed?), but if the half-stars make the site more satisfying and enjoyable for so many people and isn't too technically difficult or byte consuming what's the downside, is all I was asking?

  9. I'm with Billy 100%. I think that five stars are plenty. The New York Times, Detroit Free Press and many others use five star ratings. I don't understand how ½-stars will be helpful. If we need greater variety, why don't we go 0-100? I've been on's website for years, like Shaun. I don't think that the 10-star system works any better for me than the 5-star system, especially with the definitions loved it, really liked it, liked it, didn't like it (which many seem to confuse with dislike-- I don't like Indiana, but it isn't worth disliking.) and hated (a word used too loosely) it.

    In any case, too many people have asked for half-stars. Michael will convey this sentiment to the Netflix gods, as he should. And some night, when all the DVDs are in the mail, like Billy, I'll sit at my computer for 3 hours fiddling with ratings with no satisfaction.

  10. " Billy, I'll sit at my computer for 3 hours fiddling with ratings with no satisfaction." But WHY, Mort? I used to be with that other rental company beginning with 'B' that has 1/2 stars, and, like you two, just found them confusing and annoying so stuck to 1 thru 5. But why send up prayers to the Netflix gods to deny other people the option if they want it??

  11. I don't necessarily need 1/2 stars either. I just need a rating between dislike & like. Something like ok or neutral or average.

  12. michael, from netflixJuly 19, 2007 at 6:48 PM

    What is your glitch, people? Why does every posting become a debate about 1/2 stars? I know this is a big issue for you, and I promise that when he's ready, Todd will talk about it. But you're spamming each and every posting with your debate. I cherish your debate but show a little restraint where you argue. It really disheartens me and makes me want to stop talking with you when people keep sidetracking the discussions at hand. I'm counting on the "regulars" out there to keep everyone else on track.

  13. anonymous, I agree with you: all things being equal, adding 1/2 stars makes a lot of people happy, and those who don't like them don't have to use them. Unfortunately, the real X-factor is: how much effort is involved here? Not sure that's a question we'll get the answer to, so all we can do is express our opinions and wait, I guess...

  14. "What is your glitch, people? Why does every posting become a debate about 1/2 stars?"

    I love that - what is your glitch :) I'm going to have to use that one.

    I agree with you Michael, this 1/2 star debate is really getting insane. There are more important things that the Netflix people can be working on than adding 1/2 a star to every movie. I hope you guys never add this 1/2 star rating everyone is whining about.

    I want Netflix to collectively work on actual NEW features, rather than try and please the few goons out there who beg and beg for something so pointless.

    We need to stay on topic here, and not push a point until they close this Blog down. That is what it will come to people, if we keep going on about something so unrelated to what the blog posts are saying.

    Give Netflix a break for a change.

  15. Sowwy, Daddy. But HE started it.....

  16. Oh, but heavens, lets wish for anything that the almighty Billy doesn't find necessary! I would hate for him to spend any more time crafting such overwrought posts.

    Cripes! Why get your knickers in such a twist over it? If we end up with half stars then those who want to use them can, and those who don't want to don't have to.

  17. Please don't get disheartened, Michael. For all our microvisioned 'spamming' you must at least appreciate that we care (if too much) about the issues, and honor your determination to make this the best Netflix it can be.

  18. Although 'the regulars' sounds alarmingly oligarchic for something as messy as a 'democracy', that being "...the worst form of government, except for all the rest", as Sir Winston Churchill once put it. Over and out.

  19. We don't need 1/2 stars. The person rating the movie just needs to nut up and make a decision. You either hated it, didn't like it, liked it, Really liked it, or loved it. Needing something between liked it and didn't like it just means you can't make a decision. Stop riding the fence.

  20. Michael, as your sit there on the beach with your head in your hands thinking 'why, why, why did we open Pandora's Community Box?'.......

  21. I'm sorry for stirring up a hornets' nest, Michael. I'm new to the blog, surfed around and read most of the posts and replies. Everywhere I looked, it was pro-1/2 star replies. I just thought I'd chime in on behalf of the opposition that it's not a very useful feature, and I wish Netflix would focus on more important issues. I apologize if you've already heard enough on the subject to make your ears bleed, but when I read "Fixing Up Reviews" I thought it concerned 1/2 stars.

  22. FlixBerg! FlixBerg!

    Does that have a nice ring or WHAT? Flixville is just too beige. Not enough snap. Comparitively speaking.

    As for the half-star ridiculousness...I say get a life. Or, alternatively, moderate comments and don't post the Half-Star Drones.

    Or? Banish them all to Flixville. Meanwhile, we'll all be in FlixBerg having a very large time.

  23. Don't worry Billy. This thread was far more entertaining than most of the movies I've seen this week. But let's give Michael, et al, a break and only address issues pertinent to 'original post', as Ms Mcwhammer suggests.

  24. Michael,

    What's your glitch? We're paying. You're getting paid. I don't want to dump on you, but that's the way it is. You're the host. If you want to aim the conversation, aim it. If it goes somewhere else, ignore it, do with it what you said you were here to do (listen), or create a special place for people to go to discuss topics you think are finished. Don't make some puerile comment like 'What's your glitch?' and expect everyone to take it with a smile. What demographic do you think you're dealing with? I hope you aren't there to tell us to shut the hell up, but that's what you just did. I resent it.

    It's been a nice ride. It had made me think more of Netflix. Now perhaps I should stop participating before it makes me think less of Netflix.

  25. Star-Bellied SneetchJuly 19, 2007 at 11:18 PM

    If the good people at Netflix ever implemented a half star option, why couldn't the Half-Nots just ignore the fact that the star rating options are any different than they ever were, and the Halves could use them to their hearts' content?

    Why fight over it? Can't we all just get along?

  26. Michael,

    Since you count on the "regulars" to keep everyone else on track, would you mind cluing us all in on just who these regulars are and the extent of their responsibilities in policing the boards?

    If some of us who appear to be joe or jane average poster are more equal than the rest of us, it would be nice to know.

    Thank you.


  27. Gotta agree with mort and clover on this one, Michael. Threads do get hijacked on occasion, unfortunately, and trying to force them back on course often just makes the problem worse.

    Also, just because I comment more than most, I don't consider my opinion on what should be discussed to be more valid or pertinent than anyone else's.

    I think a polite request to return on-topic probably would have gone over better with the folks...

  28. michael, from netflixJuly 20, 2007 at 5:20 AM

    Clover - no one is more important than anyone else here. My suggestion that "regulars" have additional responsibility is only to the degree that some folks who are new don't realize there have been postings on an entire range of topics - and those postings are good places to continue the conversations... whether it be on suggestions for new features like RSS feeds, how we name/talk about things, or 1/2 stars.

    These "newbies" tend only to read the top few postings perhaps and not realize there is a proper place to carry on discussions. But "regulars" tend to know this. And would therefore be fine folks to help steer others to the right place. It's not a "job" for anyone, but rather, typical community behavior. Everyone can abdicate social responsibility and hope the "police" will come keep everyone in line, but one of the great things about communities (like here or netflix) is the more liberal attitude, and trust that everyone contributes to the conversation as well as the order, to some degree. When conversations naturally drift, I tend to think that's fine and it has happened all over this blog. But sometimes, like now, I believe the topic is getting hijacked. The 1/2 star topic has been nicely begun in other threads and to bring it up here as if that somehow puts it on the netflix radar in a stronger way actually hurts the use of this blog. But no one is more equal than anyone else. Everyone can speak. I edit no postings (at least not so far).

    My "real" job is to make the website as good as i can. I choose to speak directly to you, our more interested and active members, to hear your opinions. But if this becomes something not useful in the process, it won't be worth my time. It's not some marketing vehicle to make y'all feel good about netflix. Its one tool among many to get feedback and hear individuals. It has nothing to do with me and I don't feel dumped on when you don't like something about the site -- i think that's good dialog. I'm not now nor ever telling anyone to "shut up." I'm reminding everyone to be respectful to the others in the community. I do think that is part of my job. And frankly, even a discourse like this one helps me understand the benefits and limits of online communities -- like ours at netflix -- and how they function. Thanks.
    (and Sam: maybe...)
    (and Shaun: you may be correct - perhaps a simple request to keep on track may have been more effective than my snipe. But i don't agree that a few people should have the freedom to take every posting in a direction they want without the community in general having a role in the focus, no? (As a poster here, I think you are just as good as anyone to help steer-- it doesn't make you "special" so much as "active." Anyway, I'm still learning and feeling out the ways these things work. And like I said, it's all informing. Thanks.

  29. My apologies for whatever disruption I might have contributed to last night's food fight, Michael. I'm not just an irregular newbie to this site, but to any site, having never blogged before, so don't really know the ground rules or etiquette. All I can say is that compared to various blog threads I've seen, if not contributed to, my fellow flixers are like a consortium of astro-physicists! I'll try to comport myself accordingly in the future, and stick to both the subject and the tone at hand.

  30. Since this thread got completely messed-up how about starting a new 'Fixing up reviews' page? Lots of important issues to address and no-where to do it now.

  31. I'd like to see a new categoty of reviews to highlight technical issues. For instance, often the available languages/subtitles will be wrong. (Especially true for many asian films; often Korean films are copied and redubbed in Hong Kong and the Korean language track is replaced with Cantonese.)

  32. Michael,

    I'm well aware of how message boards work, having joined my first one approximately eight years ago. As a result, I am also well aware of newbie posting patterns. What prompted my question about who is in charge here is that every message board I have belonged to has had an admin and generally a few moderators as well, and it was known up front who those people were. Your statement about the "regulars" and what you are counting on them to do made me wonder what the structure was here.


  33. Back to 'Reviews' for a second, I'm finding that 2 times out of 3 that I try to post a review I get a 'Netflix couldn't process your request' notice; and if I try again 2 copies appear on the movie page (but 24 hrs later on my review page so I can't erase one of them right away - the shame!). Is this my own personal - and very minor - hell, or are others having the same problem?

  34. A brief comment on reviews...
    I noticed a homophobic comment (a comment, I might add, inciting violence) and promptly flagged it. For those of you who care, please join me in alerting netflix to any and all comments of such nature (be it homophobic, sexist, racist, etc.). For the record, the message has been flagged and I spoke with customer service. They were responsive and are actively concerned with this issue.
    In peace.

  35. Will do. But rest assured that those kind of idiotic comments in this kind of forum are much more likely to invalidate the review and the reviewer than do any harm to the target group. And, unfortunately, anyone that nods in agreement needs no written encouragement to keep their mind and heart closed.

  36. I have just visited the page for "Bridge to Terabithia" and in the first six pages of customer reviews I found - and flagged - 9 (count them, nine) "reviewers" who chose to spoil a massively critical plot surprise without even the courtesy of posting a spoiler alert. Plus at least two or three others in those same six pages who gave away the same surprise, but with slightly more finesse.

    I want to thank Netflix for having added the option to flag anothers' review as a "Spoiler" (I assume Netflix verifies and then somehow marks those submissions).

    But - geesh. What an appalling development by guests of the service, evidently deliberately ruining a film for others.

    This smells of an organized campaign - for several of those reviews used some very similar phrasing and - yes, I checked - most of these spoiler-reviewers have only "reviewed" one film, which of course was: "Bridge to Terabithia."

    I haven't seen the film myself yet - and now I probably won't. I described this problem to a friend who was very moved by the film and he lamented "Those jerks!- Yes they absolutely ruined it for you."

    Qustion: Netflix does not instantly publish customers' reviews - Netflix vets them before publishing. So I ask: why was Netflix staff obviously sitting on their hands in this case, allowing those reviews to go through? Maybe that's my bigger disappointment.

    I love Netflix and have been a faithful subscriber for the better part of a decade. Please keep up your wonderful standards - and please don't let some a##hole customers ruin your content by giving away critical plotpoints en masse.

  37. @Jerry - I'm glad that you flagged the reviews that contain spoilers. How long has it been, and has Netflix taken any action?

    The reason I ask is that I have been trying to get something done about libelous comments posted under the guise of a review, and Netflix just does not seem to care. The comments posted have nothing at all to do with the film, but rather are statements of actions that the "reviewer" claims to have witnessed. Nevermind that no such charges have been filed against the person being discussed at any time over a very lengthy career in public service, nor has anything ever been reported in teh press about such behavior. I've flagged. I've e-mailed. I've called. Yet the comments, which again contain NO reference to the movie at all, remain on the review page which I thought was a place for honest opinions -both positive and negative- about movies.

    I guess this kind of garbage is just peachy keen as far as Netflix is concerned. What follows in the "review" in it's entirety, although I have removed the name of the subject. Now don't you think if this sort of thing was going on, and the person acting out in the rampant manner described was one of national prominence, information about it would have leaked out somehow? Even if it was a known issue, how is this a a film review?

    I worked with ***** this past summer, and let me tell you - the man's lost his marbles. He sexually harrassed every female he came across, even bringing some to tears. His has come a long way from the proud ******* he once was. Do not support his legacy, he has trashed it himself. The once shining star is nothing but a smoldering old creep.

    If someone at Netflix does review this stuff before it is posted, then someone was sure on autopilot when this was approved.

  38. michael, from netflixJuly 29, 2007 at 11:02 PM

    For TK and Jerry, and everyone else: thanks for flagging the problem reviews, and i'm sorry it appears nothing has been done. With the introduction of those "flags" on reviews last month, we were giving the community (that is, you) the ability to help police the site- just as you have done. But the system is automated. We do not individually check over the reviews as they come in. We have millions of reviews and more come in all the time. Each flag has a different threshhold that triggers an event. Sometimes the reviewers are notified their review was flagged and they are given an opportunity to adjust it; if lots of flags come in from different people, other actions happen. We are still tuning the number of flags required for a bad review to be removed, but alerting us here is a fine path if you feel nothing else was working. The flagging system is new and we continue to improve it with your help. That review you cited really should be removed. I'll try to get that handled expeditiously.

  39. Michael,
    Thanks very much for the reply. I knew that if I could get the attention of a real, live person then the garbage post I quoted would be handled. I've been told by a couple of friends who run very heavily trafficked message boards that there can be liability involved when something of that nature is posted.

    You know, I wasn't even aware that the flagging option was relatively new. I had never felt the need to alert anyone about a review prior to that one, so I had never looked for a way to do so. While some reviews baffle me and others are in direct opposition to my own opinions, they don't bother me as long as the comments are relevant to the content of the DVD.

    Lastly, if you ever decide to hire someone to screen reviews and such, pick me! Pick Me! I live within commute distance of San Jose, and would love to work for Netflix! Uh, sorry. Got carried away there .

    Thanks again for your help.

  40. When people click 'see ----'s other reviews' they also get to see a list of our 5 star movies. I'm puzzled why 2 of these are headlined with pics, seemingly at random, somehow suggesting that those are our most favorite of all - which in my case happens to be so much not the case that I plan to demote both to 4 stars. Also, it would be great to have a link to people's 4, 3, 2 and 1 star movies as well (no pics needed with these either!), which is often just as interesting and instructive as knowing their favorites. Thanks.

  41. DUH! OK, scrub the 'random' part, since they're obviously alphabetical, but still distracting and unnecessary.

  42. michael, from netflixJuly 30, 2007 at 9:25 AM

    thanks for feedback -- on those two movies that sit atop the pile of your favorite movies (or really, any "Top 10" -aka Custom List), they are currently just the first two in the list; but in a forthcoming bit of software they will be selected to be the two movies from the list of favorites that have the highest liklihood of being two the viewer would be interested in. But for now, yes, it's kinda goofy just showing the top two (which always seem to be something like Apollo 13, Amadeus, or Annie Hall...)

  43. Other Film Festival Awards, pleaseAugust 5, 2007 at 9:01 PM

    I'd like to be able to find reviews/awards from other film festivals (aside from Sundance) included in the Netflix Awards section when I browse. For example, Toronto film festival is not listed, but is a big deal, especially their people's choice award.

  44. hello, i've been submitting reviews within the netflix required guidelines for over 2 weeks and they never show up on the site? what's going on? thank you.

  45. One thing I might add to the review flag drop-down, besides "not a review" for people who say the movie was unplayable is "unqualified to make a judgment" for people who say "loved the book, hope the movie is as good" or "terrible, only watched the first 30 seconds".

  46. I know a 'not a review' addition to the 'flag' is being considered for scratch (etc) complaints, but, given the number of non-reviews we're seeing flash before our eyes in the movie waterfall, maybe you could also post something under the 'write a review' box and just above the review stream - something along the lines on "Please use the link in the 'at home' section of your queue to report DVD problems. No action can be taken if they are posted in a review". And maybe a mass e-mailing as well, because it's not only annoying for us, but must piss them off when they get no response. I'm also posting this in 'New Features', so forgive the repetition.

  47. I occasionally need a 3rd box marked 'very entertaining' for reviews that i don't think give good advice about the movie but which are totally hilarious, like " of the machines grabs Dakota Fanning pulling her to certain death. Tom Cruise shouts, No! No!". I shouted "Yes! Yes!".

  48. Can you add the ability to preview your review? And allow some html like the basic tags?