Wednesday, July 25, 2007

Why I'm Not Writing About The Website Outage:

First of all, i'm not a company spokesman. Second, I write about what i'm working on here, and what I know personally. The operation of the network, or the databases, isn't in my field of expertise. I can tell you it was down a long time, and that it was the most significant outage in the last 9 years, and lots of people appeared to be working very hard to get things fixed. And I can tell you things seem to be fixed now. But the thing I wanted to say about it was this: as bad as yesterday was in so many dimensions, it was also incredible to witness and experience. We came together internally, there was a sort of steely determination and focus that really seemed to bring out the best in everyone. There is an odd clarity that comes from pain like that. There was no finger pointing, just lots of teamwork. And more than anything in recent memory, it actually made me feel great to be working at netflix, taking on these challenges, and excited for what we are doing. I know it was lousy for every one of you out there. It was pretty lousy in here, too. Sorry for the inconvenience.

Anyway, I'm glad it's a new day. Back to work.

18 comments:

  1. It was good for me too. It not only gave me a chance to discover that they'd made a book from the movie 'Dr Zhivago', but also to buy up all the shares that the meltdowners were selling off. Sweet!

    ReplyDelete
  2. I had a couple of films I wanted to move around on my queue, but it wasn't really all that inconvenient I had to wait until last night.

    People who were really upset need to get a life. Expecting 24x7, "five nines" uptime is ridiculous.

    ReplyDelete
  3. My issue with the situation wasn't that netflix was down... that happens. I work in IT, I certainly know that downtime happens.

    The issue as I see it (and I commented in the suggestions thread) is that it seems that netflix could be more upfront about problems, whether that is downtime or Doctor Who Season 2 Disks or whatever.

    I love netflix and I recommend it to everybody I know (and I give them reasons to choose it over BB:TA, too). The occasional bad day will not change my mind.

    But when there are problems, I'd like more info about the situation and when DVDs are involved, maybe more info about what is being done to correct the issue.

    Perhaps the little notification we all got when trying to access netflix could have mentioned something about DVDs being shipped & processed so we didn't have to call customer service or look at other websites to find out whether or not that was happening. That would have been nice.

    I think fewer people would have freaked out yesterday if there'd been a bit more information up about what was wrong and some assurances that dvds were being processed and folks were working hard to correct the problem as quickly as possible.

    ReplyDelete
  4. You guys are doing a good job. Keep it up!

    ReplyDelete
  5. Seconded. 364 good days out of 365 ain't bad. In fact if my cell phone company manages to do better than 50/50, good days vs bad days, I'm mightily relieved. Netflix rox.

    ReplyDelete
  6. I agree that you guys are Great!! The site being down was a small annoyance for a few hours and, yes, more info that the movies were being shipped would have been helpful and instilled confidence that the entire company wasn't shut down. But overall - I wish every vendor I have to deal with offered the service you guys do!

    Thanks for the great service - you guys - designers, programmers, film selectors (whatever they are called), engineers, shippers all deserve a lot of praise!!

    ReplyDelete
  7. "when there are problems, I'd like more info about the situation and when DVDs are involved, maybe more info about what is being done to correct the issue."

    "more info that the movies were being shipped would have been helpful and instilled confidence"

    Curious as to why some people thought that they weren't going to get their freakin' DVDs. The problem wasn't with the USPS that ships the disks for crying out loud. Nowhere did Netflix say people weren't going to get their disks, yet that's what people automatically assumed.

    ReplyDelete
  8. Well, there are certainly worse things to be addicted to than movies, but yes, the withdrawal symptoms that became manifest all over the web within 8 hrs were indeed somewhat disconcerting, given that those reaching out for help and reassurance were clearly some of our best and brightest.

    ReplyDelete
  9. Curious as to why some people thought that they weren't going to get their freakin' DVDs. The problem wasn't with the USPS that ships the disks for crying out loud.

    No need to throw attitude at the questions. It is not unreasonable for some to have wondered if the website outage meant that other internal systems were impacted as well. No one here questioned the USPS end of things, however it does appear that there was concern that Netflix staff may not have had access to the info needed to process orders. It was not unreasonable for those who wondered to ask about it.

    As for me, I have to say that of all the websites I use, Netflix has by far been the most reliable. Nice work!

    ReplyDelete
  10. I dislike blockbuster because of the service that i've received in the stores and from the 2 week free trial that i once gave a shot ( dvds arrived later than Netflix and 1 out of 4 movies sent was the wrong movie). I hope that i'm not forced to go to blockbuster because netflix can't hold its position and good service. Competition is good though , yes ??

    ReplyDelete
  11. First off, I'm enjoying Netflix immensely, and the "Instant Watch" is way cool.

    Is it possible to let us know via a post to the blog, what is being updated/fixed/added during each maintenance time? Also, is it possible to let us know in advance that the website won't be available during the hours of _____.

    Another question I have, what is the best day of the week to return a DVD to ensure fastest receiving of next one in queue? And is netflix planning on having a 6-day week instead of 5 for sending out DVDs?

    Netflix you rock!

    ReplyDelete
  12. Uh-oh, 'unavailable' again! I think it's the word 'scheduled' on your advisory that's confusing. Leaving aside yesterday's unforeseeable problems, if the date and time of a regular maintenance is known ahead of time, why not just post it on the blog so people know not to try to log-on during those hours? Because you must know by now what a hair-trigger startle-reflex we cineasts have!

    ReplyDelete
  13. "Because you must know by now what a hair-trigger startle-reflex we cineasts have!"

    So you think it's better to just be notified when you go to the site? It would be easier to know in advance for scheduled maintenance. Or am I the only one?

    ReplyDelete
  14. That's exactly was I was attempting to say, that it would be good to have some advance notice of shutdowns so we can plan accordingly, whether it's finding an alternative to the 'Watch Now' movie we'd intended to see, shifting a movie to the top of the queue before the current #1 gets into the shipping process, ordering up movies for the family before heading off on a business trip to Bhutan, etc...

    ReplyDelete
  15. Hi Michael - thanks for the comment. I too work for a large internet site, and I can not imagine what sort of madhouse would have ensued if this had happened in my workplace.

    As your customer, I did go through a few withdrawals, but comforted myself by remembering why I keep such a well-stocked queue.

    I wasn't worried.

    Clinton, MA

    ReplyDelete
  16. People who were really upset need to get a life. Expecting 24x7, "five nines" uptime is ridiculous.

    ReplyDelete
  17. Looks like you guys have a Watch Instantly Service outage again at the moment.

    ReplyDelete
  18. I got my first DVD with "infinifilm". It says, "Press 'Enter/Select' To:". I have a Macintosh G5 and there is neither an "Enter" nor a "Select" key. How do I access infinifilm features?

    ReplyDelete