Wednesday, September 5, 2007

Flagging Reviews

At the same time as we release Favorites, we release the first major improvements to the reviews flagging since it was introduced. This should fix many if not most of the issues you've had with policing content. Most noteworthy, i suppose, is the addition of the "Not A Review" flag -- your suggestion -- to deep six those spammy, customer servicey, non-reviewery elements. Remember, if you abuse the flagging to try gaming the reviews system, we know who you are. And this would be bad.

A few other notes. If the threshold is reached for the Objectionable Content flag on a certain movie review, it is only deleted from public view -- from the movie display page and the community home page and those types of presentations; it will still remain on your Reviews & Lists page. (Presumably, if you really want to read someones nasty, stupid objectionable reviews, you still can -- but they're not foisted on the rest of us).

If a reviewer is flagged as having objectionable nicknames or avatars, their name and image is automatically reset. If this happens to you, and you are CERTAIN there is nothing objectionable going on, go ahead and put the same photo and nickname back. But if it gets flagged AGAIN, you could lose your account and have to deal with customer service to deal with the situation. Let's not let it get to this, okay?

Spoiler Alerts. We're still improving this, but when a review gets tagged as a spoiler, a warning will appear at the start of the review. This is still getting fixed so it won't be working as soon as the rest of these go into effect. Reviewers will also have the opportunity to tag their own reviews as having spoilers, which amounts to the same thing. I'll let you know when this is all working. (Of course, it may be obvious too).

And as you may have assumed, getting your reviews flagged in various ways really can reduce your influence score, and thus your rank as a reviewer, if that matters to you. LOOK FOR THESE FEATURES MID-MONTH. I'll also post it here when these are live.

58 comments:

  1. Sounds excellent! Just one question - when you say "If a reviewer is flagged as having objectionable nicknames or avatars, their name and image is automatically reset" I'm assuming that you mean "if the number of flags from various different accounts reaches a pre-set threshhold, their name.....reset"? Otherwise we're all very vulnerable to pranksters and people who happen to object strongly to our viewpoint.

    ReplyDelete
  2. Michael: This is all very exciting and good news for the community of avid reviewers. I am particularly excited about "Influence Ratings", "Faves" and "Fans". However, I am still having problems. None of my latest reviews (approx 12) have posted to the site over the past 2 weeks. Customer Service was at a loss but said they would "look into it". Any thoughts on why this would be occuring?

    JayMikey in Baltimore

    ReplyDelete
  3. If a reviewer is flagged as having objectionable nicknames or avatars, their name and image is automatically reset. If this happens to you, and you are CERTAIN there is nothing objectionable going on, go ahead and put the same photo and nickname back. But if it gets flagged AGAIN, you could lose your account and have to deal with customer service to deal with the situation.

    Um... this concerns me. I don't plan to have any objectional nicknames or avatars, but this just seems like it could be problematic.

    ReplyDelete
  4. joniw,

    Even if your name/avatar is flagged a second time, your account will not be closed unless it is truly objectionable.

    ReplyDelete
  5. I'm sure this is an obvious conclusion, I am simply confirming.

    When the kiddies (or adults) go in and write a review for a movie like "High School Musical 2" and their review is along the line of…

    (*****) I haven't seen HSM2 yet but just know it will be better then HSM which ROCKED!

    That would fall under the definition of "This is not a Review".

    Thanks for overlooking the obvious

    ☺☺☺☺

    ReplyDelete
  6. Sample of Not a Review could be:

    rated 1 star - "I couldn't watch it because the disk was broken and I haven't gotten the replacement yet."

    Those, I hate. Especially on what I consider a 5 star movie.

    ReplyDelete
  7. anonymous 1-i'm sure that there won't be very much of this going on-remember, we are all Paying for this service. Unlike other places that are free to access, like myspace and yahoo. So i wouldn't stress about this happening.

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete
  9. Hmmm....does one flag this non review ?

    "I could not rate it because [it] did not have English Captions for the deaf...." (The reading Room).

    If I was deaf I'd want to know!

    ReplyDelete
  10. If a reviewer is flagged as having objectionable nicknames or avatars, their name and image is automatically reset. If this happens to you, and you are CERTAIN there is nothing objectionable going on, go ahead and put the same photo and nickname back. But if it gets flagged AGAIN, you could lose your account and have to deal with customer service to deal with the situation.

    Is the person who flags someone's nickname or avatar tracked in some way so that if this DOES happen as a prank, the flagger can be dealt with?

    Hypethetical situation (or a plot for a really, really bad movie!)

    What if I was the tenth highest ranked reviewer on the site? I want to be numero uno, so I go and flag each person ahead of me as "objectionable". They each lose their nickname and avatar and have to reset it. All I have to do is flag them again and bam! They lose their account privileges. Like taking candy from a baby. Muhahaha!

    PS: Everything else sounds really cool and very helpful, though.

    ReplyDelete
  11. michael, from netflixSeptember 5, 2007 at 8:23 PM

    @ang - you should be a screenwriter... no seriously, that is a plausible case, but as was pointed out, everyone here is an account holder and we can see who is causing problems...

    The other thing a number of you have brought up is how you keep a fringe minority from gaming otherwise good (but perhaps controversial) reviews. this is a good question. we cannot pre-screen reviews as they come in -- we get thousands every day -- but i think we can count of the wisdom of the crowd to find truly bad ones, and that we have tuned our system well-enough to separate the real signal from the noise. It's never going to be a perfect system (and we won't be deleting reviews, only demoting them more strongly), but in our months of tuning the threshholds (which is why some obviously bad ones didn't get removed... yet) you'd be surprised how clear the signals are. We hand checked all the material getting flagged and think we can do a good job cleaning up the site, and keeping good stuff from getting gamed. But we'll keep working on it if it's not quite right.
    I'll let you know when it releases.

    ReplyDelete
  12. Thanks, Michael. Well done.

    ReplyDelete
  13. "...(and we won't be deleting reviews, only demoting them more strongly)..."


    Everything sounds very nicely thought through in terms of keeping mischief-makers under control, but I'm slightly puzzled as to how you're going to "do a good job cleaning up the site" if you're only going to be "demoting reviews more strongly", as opposed to "removing them". I'm sure most people won't care that their 'influence rating' is reduced, so won't the site still be cluttered with multiply flagged reviews? Or perhaps I'm misunderstanding?

    ReplyDelete
  14. I'm sure most people won't care that their 'influence rating' is reduced, so won't the site still be cluttered with multiply flagged reviews? Or perhaps I'm misunderstanding?

    Flagged reviews will not show up on the movie pages or on the scroll reviews on the community page. They will still show on the reviewer's personal page though.

    ReplyDelete
  15. I like the new "Flag" button under everyones reviews. It is easier to let Netflix know of a possable issue instead of searching for who to send an email to.

    @Jeff - What I think Michael is saying is that it will temporarily demote a review until someone can take a look at it and make sure there is something that can be found objections. It would be unfaire if someone disagreed with a review and flagged it and the person who worked hard on writing the review found it deleted without anyone checking the content. A computer can only do what we tell it to do. What Netflix might want to do is make it more difficult for someone to flag a review by seperating the "Find objectionable content and/or language...". That can mean anything from i just don't like the review to actually having some bad language. If they create a more difinitive reasons to find something objectionable then it allows the people on Netflix to look for specific things in a review.

    ReplyDelete
  16. Thanks for the clarification, guys. So this really is beginning to be a 'community'! Nice!

    ReplyDelete
  17. I'd like to chime in that I agree with everyone regarding malicious use of the flagging system. One area I can see this being a real problem is with political movies or movies like An Inconvenient Truth where you have a major schism between the Democrat/Independent point of view (i.e. Right) and the Republican point of view (i.e. Wrong).

    People are going to misuse this and make people have to deal with customer service as a way to silence dissent. Just wait.

    ReplyDelete
  18. Why not pre-screen reviews for language with word filters? Screen the usual subjects like F*ck, sh*t, c*ck, c*nt, d*ck and so on. You'd have to also search for variants of it like "fvck" and so on, though. People are crafty and know how to get around word filters.

    Just a thought, why not have an "adult" version of Netflix and "kiddie" version?

    How you would do it would be like so:

    1. When the user signs up, they could have an addendum to the TOS and agree that they would like to see only the "sanitized" reviews OR they would like to see everything, unedited, uncensored.

    2. When the person is making a review, there would be a large area explaining to them that they can post their review/comment to EITHER the "sanitized" area, where it must be devoid of racist, sexist, lewd language and suitable for children OR they can post their review to the "uncensored" area where the free flow of language goes unmodified.

    3. If and when people erroneously post unacceptable reviews to the "sanitized" area the aforementioned method you guys came up with apply, but the "uncensored" area would be much harder for people to complain about since they signed the Terms Of Service agreeing not to be offended by adult language, ideas or epithets.

    4. Anyone would at any time be able to change their ability to see only "sanitized" or "uncensored" reviews at any time in the Options area of their account.

    ReplyDelete
  19. I pray for your sake that you're joking about the Adult and Kiddie Netflix versions! As to the potential for flagging abuse, I think the extremists on either side of the aisle/sanity divide will be content to carry on abusing 'unhelpful', since they know that excessive and unsubstantiated flagging will endanger their account more than that of the person they're trying to silence. But yes, I'll wait and see, as instructed.

    ReplyDelete
  20. Wasn't kidding at all. Personally I think it would be a nice way for Netflix to start carrying porn as well. I would rarely make use of this feature but if there were interesting porn out there (say, extreme mud wrestling with midgets) I might be inclined to rent it.

    Michael, why hasn't Netflix carried porn yet? With the correct controls in place (i.e. marrying the account with a credit card to ensure age) there is little chance of children getting porn.

    ReplyDelete
  21. Try Intelliflix or a quadrillion internet sites, Buster. I'm no prude, but this is a 'movie/film' rental company, and porn ain't cinema by any stretch of the imagination.

    ReplyDelete
  22. Michael, why hasn't Netflix carried porn yet?

    There is a company out there that does porn, along with regular movies and until recently they also did video games. They are called Intelliflix (and also DVDBarn), but I HIGHLY recommend AGAINST using their service. I tried out their 1 year subscription, and it was unbelievably awful. They sent movies randomly from the queue, even after they claimed to make big improvements on it. They would often wait days (and sometimes weeks) to send out my next movie. Complaining got nothing done, just a "we are processing your next movie". They sent far more movies from 3000 miles away than they did from my own state, so almost everything took 3 days both directions. So, don't try them out unless you are a glutton for punishment.

    ReplyDelete
  23. I believe it's a DVD rental place. If you want pure "cinema" then I suggest going to a local independent theatre. Nowhere on Netflix's page is "cinema" mentioned, only DVD.

    As an aside, I really wish Netflix would start renting video games ala Gamerang.com, Gamefly.com, Gameznflix.com. I'm probably going to sign up with Gamefly, but if Netflix offered games then that'd be extra money I wouldn't be sending to someone else.

    ReplyDelete
  24. It always seemed to me that games would be more profitable, since people would tend to keep them longer than movies.

    Though it would be harder to predict how popular a game will be in order to purchase the right number. Also, I suppose everyone would want the game on day 1, and few would be renting it 6 months later. Games age much faster than movies.

    ReplyDelete
  25. "I believe it's a DVD rental place".

    Excuse me, but it is called 'Net-Flicks', and last time I checked flick = movie. But yes, if you're being totally literal, you're right, since they have TV, exercise videos, etc.... But every company has its 'mission profile' and I think you're being cretinously tin-eared about Netlix'. Why don't you ask Reed Hastings on his blog how he feels about porn and video games?

    ReplyDelete
  26. I personally have no problem either way with whether or not Netflix decides to rent porn or video games.

    I will say, however, that if Netflix offered video game rentals... I would definitely pay more for that feature.

    ReplyDelete
  27. Michael, why hasn't Netflix carried porn yet?
    Actually Netflix used to carry adult (soft porn) titles. I've been a member since early 2000 and these titles were available back in the day.

    ReplyDelete
  28. The problem with porn titles is extremely clear, and I for one applaud Netflix for choosing NOT to offer the titles.

    1.) I'm not a prude, I have many titles at home.

    2.) But where to draw the line on selection?

    Straight, gay, bondage, lesbian, gang…

    Then you deal with the real prudes who will boycott not because they can see the titles, but because they are even being offered.

    3.) The real problem stems from the fact that most parents do not monitor their kids on-line activity, and they expect websites to babysit THEIR children for them.

    So here's the scenario if Netflix offered adult titles.

    Junior logs on to mommies Netflix account, sees a xxx movie featuring big boobs and rents it. Mommy doesn't know until she "in horror" finds it in the mail box.

    Mommy will be more outraged at Netflix for sending it then at Junior… Why? Because Junior is a child and Netflix is supposed to protect them and babysit them.

    Then all mommy has to do is decide to start a grass root drive to boycott Netflix, gaining other "I don't have time to babysit my own kids on-line" mommies and Netflix goes out of business.

    ReplyDelete
  29. Here's another scenario if Netflix offered porn: shipping mistakes. I suppose it rarely happens, but if you add up all the times that the wrong disc is put in a sleeve, or you ordered DVD 1 and you got DVD 2, or any other very rare but entirely possible mistake -- imagine the consequences if the disc that arrived was porn.

    Mommy puts the Barney DVD in for her kids, not paying attention, and some smut shows up! Horror. Does she send it back to Netflix? Not before calling the New York Times. If it even happened once the costs would be large.

    It doesn't surprise me in the least that Netflix no longer has porn.

    ReplyDelete
  30. I agree. VERY slippery slope. And Lord knows we already hear enough hysterical trumpeting about the occasional 'naughty bits' in regular movies!

    ReplyDelete
  31. Is a "NR" movie that is the uncut version of a "Rated R", not considered semi-porn, if there's sex in there? There's plenty of "semi porn" on netflix.

    I think that it's a parents fault if they are not careful, and since mistakes do happen, they should always check out DVDs that they receive before having their kids watch it. It's negligence on the Parents if they simply trust that netflix can always be 100% accurate, which can't ever be the case.

    ReplyDelete
  32. Mr. Tangent said...
    I'd like to chime in that I agree with everyone regarding malicious use of the flagging system. One area I can see this being a real problem is with political movies or movies like An Inconvenient Truth where you have a major schism between the Democrat/Independent point of view (i.e. Right) and the Republican point of view (i.e. Wrong).

    People are going to misuse this and make people have to deal with customer service as a way to silence dissent. Just wait.
    September 6, 2007 12:25 PM


    This, of course, would be flagged as a non review (I hope).

    ReplyDelete
  33. Michael, do I have to go through all 500+ reviews and type "SPOILER" on most of them? This a movie review place, and of course there are going to be spoilers in practically every review. Unless you want to turn NF into the mess that adult swim is.

    ReplyDelete
  34. Michael, do I have to go through all 500+ reviews and type "SPOILER" on most of them?

    No. Any of your reviews that have spoilers in them will get flagged by readers. But in the future you should flag your new reviews as spoilers (if they are) as you write them.

    FYI: Good reviews contain little or no spoilers. Reviews should be critique/praise for the overall movie, not a rundown of the plot, or scene-by-scene analysis. (I'm not claiming to be a great reviewer myself, just giving some general advice.)

    ReplyDelete
  35. The joke is that the blurb often gives away far more of the story than it need do. That and the many errors in the movie descriptions should be reported through the 'Contact Us' link. Not that I've EVER seen a mistake corrected in 4+ years!!

    ReplyDelete
  36. Just wondering how you all feel about "reviews" that are really just information about the disks. For example, letting other renters know that the disks are out of order so they should start with a different disk (see Farscape season 1 or the first disk in the classic Doctor Who series). It's not a review, but in my opinion it's important to know. Would you leave those alone or flag them?

    ReplyDelete
  37. If you ask me, Ang, if the "review" is useful and informative, I can't imagine needing to flag it as not a review. Something that is not a review would need to be spam or a complaint or something. But not a bit of helpful advice. imho.

    ReplyDelete
  38. or the first disk in the classic Doctor Who series

    That tidbit was actually helpful to me when I put them in my queue, so I certainly wouldn't flag it.

    ReplyDelete
  39. I agree that info about the order of a disk and even the sound quality not being tops are good info to have in the review section. As long as the reviewer rates the film on their impression of the content not the problem.

    Before I received Eddie Izzard's Unrepeatable I saw reviews saying how terrible the sound quality was (from the recording - not a DVD problem) so I was prepared for it and didn't let it bother me.

    "Reviews" I'd want to flag are the ones that rate a film 1 star because they are angry that the DVD is the wrong one or broken and unwatchable. I've also seen a couple of reviews that are complete spoilers because the person hated the movie and appears to want to spoil it for others. I've written reviews on movies I've hated but didn't need to spoil the horrible film for others who might like the terrible thing.

    Please get this feature quickly. I ran into a review yesterday in Masters of Horror: Jenifer that told everything just to stop people from watching it. I flagged it but I've never seen anything done on these flags. This reviewer didn't like the show so trys to stop others from seeing it. I've seen that movie 2 times and don't love it but appreciate it for what it is and would like that review either marked or demoted from the first page.

    I hope when the new flagging system is implemented it will help.

    ReplyDelete
  40. Umm-- what does "gaming the reviews system" mean?

    ReplyDelete
  41. Umm-- what does "gaming the reviews system" mean?

    An example: you and 10 friends rate each other's reviews as helpful and everyone else's as unhelpful. Netflix would want to ban these people or at least tell the system to ignore all their input about other people's reviews.

    Basically it means taking advantage of the inherent flaws in the system. Its a bad thing.

    ReplyDelete
  42. Yes, like the poll, each account should only be able to vote once. A NF friend recently voted my reviews 'helpful' twice -- currently possible once every 24 hrs -- "just to be nice" (I asked her to cease and desist, I swear!), so who knows what goes on when people actually make a game-plan to boost each other?

    ReplyDelete
  43. Yes, the multiple voting (every 24 hrs by all 4 allowed accounts per member), and now the "Flag this" will only lead to more Trolls and more Flaming (there's too much of this now).

    ReplyDelete
  44. The 'helpful' button should be disabled on your own reviews, as it is on a lot of sites. Although how desperate and creepy is that, to vote for yourself?

    ReplyDelete
  45. michael, from netflixSeptember 9, 2007 at 11:54 AM

    The ability to give multiple "helpful" votes is definitely a bug, and it's a loophole we expect to close shortly. The flags are engineered from the get-go so that you can only vote once. I suppose everything is "gameable" if you try hard enough, but i think we'll be okay here.

    ReplyDelete
  46. A Monthly Netflix Review Prize? Video Reviews?

    I love the Netflix Member reviews (they really do help me find great films), and I also love the Netflix Prize concept. Why not bring the two together and offer a $1000 per month to the reviewer that receives the most member votes for a particular review? This would seem to fit nicely with your concept of "Friends and Faves" as well.

    Also, have you thought of letting members upload video reviews of the films they watch? This could lead to some pretty clever reviews (satire, costumes, etc.) that would generate greater loyalty. Most laptops now have webcams so it's pretty easy -- Facebook's new Video messaging application makes this look easy.

    ReplyDelete
  47. Well, it's pretty simple then....Netflix members like myself will ignore the reviews. It seems that there's no simple way to trust a fellow member's opinion anyway. If one must enter a review under the guise of being 'Top' or 'Favorite' and risk being 'Flagged'.....because of shallow grade-school like behavior; it's just too much of a hassle. Netflix is making this too much of an issue with too many hurdles to jump. I just want to watch movies and send them back and watch more. Simple idea, don't you think?

    Forget the reviews.........

    ReplyDelete
  48. They have other services that do exactly the same thing but for porn, like http://www.hugeadultdvd.com and that way they everyone can be happy and Netflix can focus on non-porn.

    ReplyDelete
  49. Can we have an Other category? For example, there's a "review" on Midnight Cowboy which is actually for Urban Cowboy. Some "reviews" say simply "My DVD was scratched and unplayable" .

    ReplyDelete
  50. Art, just flag those with "this is not a review".

    ReplyDelete
  51. Art, just flag those with "this is not a review".

    Just don't hold your breath waiting for anything to happen. Unless several hundred others flag it also, Netflix won't even bother to look at it.

    ReplyDelete
  52. My review of "Black Book" disappeared from my Review & Lists every day for three days and another review "Offside" or "Bob Le Flambeur" had the same problem. The customer rep said they may have been flagged, but they were straight forward reviews and did not violate NetFlix's policy. Is this a glitch at the website?
    Also, I commented in my "Bob Le Flambeur" review that the NetFlix synopsis was wrong. I could not get this review approved for posting until I removed the word "NetFlix" from the text. What's up with that?

    ReplyDelete
  53. You can find a porn program like netflix at http://www.netflixforporn.com

    ReplyDelete
  54. Honestly, I prefer to have Netflix stay the way it is. when I want a movie, I'll go to Netflix. I don't want to go there and have porno suggestions appear on my Netflix account while I'm trying to rent a Disney movie with a nephew or something. Then when I want my porn DVDs, I can wait til the kids are asleep and log into http://www.sugaradultdvd.com and get all the porn I want. I just prefer them to be seperate sites altogether.

    ReplyDelete
  55. http://www.WantedList.com has been called the netflix of porn. You can take a look there.

    ReplyDelete
  56. They don't rent porn because they don't want smudges on their DVDs.

    ReplyDelete
  57. I have a suggestion to lessen the number of non-reviews (i.e. This movie didn't work for me, or Disc 7 is special features...).

    These comments are legitimate movie topic discussion, but simply not a review.

    Why not make a forum/discussion board for each movie/title where members can discuss the films and not review them?

    ReplyDelete