Showing posts with label User Experience Rants. Show all posts
Showing posts with label User Experience Rants. Show all posts

Thursday, January 17, 2008

Nouveau Netflix Lexicon

I have to alert y'all about a very cool blog posting I just saw called "Terminology for the Netflix Fanatic" - where Ryan Sims (and his readers) present some terms for Netflixers, for instance:

- the act of Netflixing a movie, watching 15 minutes of it, deciding you don't like it and thus, promptly return it.

Notflixing - the act of Netflixing a movie which sits on your coffee table for at least a week (if not much, much longer) and you send it back without watching it.

Netflux – the period between movies, for those of us on the 1 DVD at a time plan.

- Recieve movie, start watching movie, becomes a bit boring, or not what you expected but you still want to know what happens in the end so you watch the movie in fast forward, occationally watching portions that look halfway intresting.

Read his entire set and the comments for the full entertaining flavor. (thanks to Andrea for sending this in)

Sunday, August 19, 2007

Moving the Needle

When I add something to the site — like the Latest Reviews block or the Members Top 10 Lists, I am looking to see how many of our subscribers use it, and come back to use it again. For whom does this feature add value? My goal is to add lots of value for as many people as i can. A new feature doesn’t have to appeal to everyone, of course. We have a lot of subscribers, after all, and you all have a very wide range of needs and preferences. Some things we do might be useful to everyone (say, improving the way the Queue works) and some only impact a subset of users (for instance, changing the way you see your Friends). We do both kinds of features, and are always looking to balance the efforts creating each of them.

When I look at features on the Netflix site (and in the Community area in particular) I have two buckets that everything falls into: moving the needle and polishing the apple.

What I want is to move the needle: I want to make a change or add a feature that materially improves our site. Maybe it makes it so much easier to manage your queue or find exceptionally good movies that it totally changes the way you use and enjoy Netflix. That would be good. That would be “moving the needle.”

Often times I’m just polishing the apple: it’s good to make this thing better, easier, clearer, whatever – but it really isn’t going to change how many people use it or enjoy it – at least not in a meaningful way.

The drag and drop queue – that was a needle mover. It makes the queue much easier to work for every one of our subscribers. Good feature. Adding the Friends Activity headlines to the Community Home page? Well, it was a good thing to do, but it was only for Friends users, and quite possibly was only a bit of apple polish. Removing the dates from the Top 10 Lists block? (releasing next week) Definitely apple polish. I wouldn’t have done it at all except it only took about 10 seconds and it was driving a few of us crazy. We call this “opportunistic” improvement. You’re fixing something anyway and it is easy to do. So make the thing better even if it doesn’t move any needles.

There is a general sense that you have to do a little bit of polish or in time the site just isn’t that good, it isn’t easy to use or it slowly irritates y’all to the point of distraction. Search is an example. It works. But it could be so much better. Most of us are sure this is a needle mover. I think the entire Community effort – comprised of many many little interlocking features – could be a needle mover. I am trying to discover if that is the case.

The problem often comes when me and my teams’ natural desire is to polish apples – make the things we have much smoother or better (think the sliding of the Latest Reviews or the adding of movies to Top 10 Lists), but then realize that the small bits of work to improve these things adds up and the big new features that are certainly more likely to move some needles are getting postponed.

Anyway, it’s a balance. Like a stock portfolio. Some high risk stuff, some slow growth but low risk stuff, a few wildcards… but all adding up to a strong healthy portfolio that weathers short fluctuations and delivers good results in the long haul.

The features I want, and that you guys often suggest, are almost always good things to do and would results in genuine improvement of the site. The question I ask – and you can ask yourself if you want – is will it make a difference, and if so – how much? For whom? I could change the font on all the tabs and it would certainly make the site nicer looking and perhaps even easier to read – but would it change our business? (Maybe that’s too extreme an example). Perhaps better: let’s make it easy to sort the movie reviews on all the movie pages – by helpful ratings, by similarity percentage to you, by recency, etc. Seems like a no-brainer. It certainly would be great. It’s already on my list of things to do. But let’s step back for a moment. Will it help more people find better movies? If you don’t read movie reviews (and many people don’t, actually), will it get you to try it? Most people ask for these options, but rarely use them. If we provide the ability to sort in a dozen ways, how many people will use the feature? Simplicity would say: discover the most useful of all the ways to sort these things, and make that the default – because that’s what most people will see. Then add the sorting later, opportunistically. And what are the thresholds? If I told you only 1% of our users would sort, would you make this a top priority? How about 10%? What would it take – 20%? 50% (Complicate this with the effort: what if it took a day to build, or a month? How would that affect your decision?) The question is rhetorical, but the issue is very real. It’s not a bad feature. In fact it is a brilliant and obvious feature. But should it be the first thing I build? Or is there something else that is also brilliant but that will be “game changing” – a needle mover! Because if you were given a choice, you’d probably tell me to do that.

I’m reminded of a bit of business advice I got when I was raising money to start a company back in 1993. It went something like this: “Just because something is a truly great idea does not necessarily mean it will make a good business. And just because something would be a terrific business does not necessarily mean it is a good investment.” You guys offer us ideas – often great ideas. Our job is to decide if they are worthy investments. There is often no question they are good ideas.

That’s why I really absorb all of your suggestions, and that’s also why you don’t see all of them getting implemented right away. It is without question the hardest part of this job.

Wednesday, July 25, 2007

The Product Tax

Your feature ideas are great, and your obvious passion is a tribute, I think, to what we've done so far. So I'm going to have another one of those "out on a limb" conversations here about product design. (It's possibly too much for a blog like this, but...) there was something in a few of your postings that comes up often, and it includes the expression that the solution to a problem is "obvious." This has triggered something in me. A number of new community features will roll out next week and it seemed like a good time to bring you deeper into my world.

There is this idea in product design that can be called a "product tax." It has a few components. The taxes are in addition to the more obvious cost of a new feature -- in terms of the work of designers and engineers. This is what most people think about when prioritizing 1/2 stars or movie privacy. I think inexperienced product developers are too highly focused on this as they prioritize what to build. The first small tax on this is the unanticipated amount of work it really takes to build and support the feature. Take our Movie Privacy feature.

As I said back when it rolled out, the creation was opportunistic: we had built it a year ago and it hadn't been released, it was not particularly "designed" - but it did solve a need and I felt it would be important to Friends users. And heck, Blockbuster even had it. Alright, so it wouldn't take too much time to finish it up and get it on the site, as long as I didn't spend a lot of time making it pretty and efficient. It just "is."

But once it was connected up to other pages, unexpected questions arose: if a movie is hidden, what happens if you review it? Does that "unhide it" or does the review show anyway. What about Watch Now movies? Are those hidden too? Someone discovered that hidden movies DO show up in the RSS Feeds. Should we hide the movies there? Each place where movies need to be hidden requires a little more work to implement. And in some cases, say RSS Feeds, there are engineering reasons why it's harder to hide something than in your Friends' sliders. Now I'm going to make up some numbers to illustrate this, but they are just made up: let's say 25% of our members use Friends. And 10% use Watch Now. It's possible that only 1% of Watch Now users are also Friends users. And not everyone in that group wants or needs privacy-- maybe only 10% of them. So to add this simple Movie Privacy page, and make it work "right" we have to think about and engineer this hiding for Watch Now even though it's highly unnecessary: 25% of our Friends users need Movie Privacy, but only a fraction of a % need Privacy applied in the Watch Now area. (And for the record, Movie Privacy DOES work on Watch Now titles). The point is that it was easy to create the page, but finding all the strings leading away is harder and creates a tax on releasing the feature. We tend to make something work only when a reasonably large percentage of people want and need the feature, or if common sense dictates we do it anyway, expecting more people to be using it later on. But as obvious as the feature is, one needs to be careful following all these strings and considering that additional work in the feature. But as i said, this is only the "small tax" here. I think product designers are well aware of this tax.

The larger tax is the one frequently missed. Once the new feature is created, it adds a lot of software to an already large pile of software that is our website. And now whenever someone working on any other part of the site has the idea for some cool new feature -- one that might be VERY important to lots of people, and VERY cool, the building of it must take into consideration all the other things it touches. As Steve goes and builds Watch Now features, he is belabored with the strings of Movie Privacy. If I want to add something about notes or reviews, I can't just build it quickly and get it out, i have to consider all the permutations that may involve privacy and make sure each is logical. And the more of these things you add, the more difficult and slow it is to do other features.

This is the hidden tax. It's starts small, but expands rapidly. Products (and websites) often get slower and slower to release new features because they spend an increasingly large percentage of their time just dealing with the baggage of the old features. If those old features are great, that's just business. But if those old features don't add much value, or only add value to the 1/2% of people who write reviews, use RSS feeds AND use Friends, then one must consider if the cost is worth it.

The discipline, therefore, is to generate new features, make them good for a lot of people, and keep an eye on how much value they really add. And after some period of time (it could be half a year, or a few years) if they aren't being used by many people, kill them. Otherwise they stack up in an endless pile that must be weeded through by customers and engineers alike.

And there is one more factor to consider: in many cases, adding more features diminishes the usability of the existing features. People only have so much attention (particularly for a website). If I have one button, you can click on the button. But if I add a second button, we have seen over and over again, that often people don't click on either button. There is confusion. There is choice where there was none. Overall user experience can get worse when you add features, and the exact opposite of what you want to happen occurs. We think we've added something people want, but fewer people see the features in total. Weird, but true. Less is more.

You are sincere when you suggest a new obvious feature, and it may even be clear to us the feature is cool. We consider the costs and benefits and then build them. But we also think of the tax, and need to think of the bigger picture, of a site 3 years from now that does all kinds of great things, things we KNOW many people (most people) want. If this cool little thing now is going to impact that supercool big thing then, we have to decide whether this thing is worth it. These are HARD decisions, there is no right answer in many cases.

As I mentioned, my brother is a Hollywood screenwriter, and I was a movie and TV editor in a former life. Writers have this notion of "killing your young" -- a rather crass way to describe the painful process of writing and editing out stuff that you like. For me as an editor, the way I learned it was like this: editing a movie is not about taking out the bad stuff. editing is about taking out the good stuff to make the remaining stuff better. You don't usually see all the stuff removed to make your favorite movies so good. But trust me, some of it is great, and you'd think they were idiots for cutting those scenes or lines. But the result is the thing.

Please keep your suggestions and debate flowing. We all read this stuff and it's good to hear. Many of your ideas are being implemented while you read this. So thank you.

Anyway, welcome to the inner conversation.

Saturday, July 14, 2007

Product Development Insights

Once again, i'm going to go out on a limb here and talk pretty candidly with y'all. Hang on. We who make this website, we are called "product developers" because we think of the website as a product, and we take the development of this product pretty seriously. We seriously want to make it better and not worse. That seems simple enough.

Here are a couple little insights into our jobs here:
1) I'm not the only person working on the website. This shouldn't come as a surprise. I already introduced you to Meghan and Vikram and the rest; we're all working on this stuff; and thus:
2) Because you get feature A doesn't mean you don't get feature B. These things are not mutually exclusive. Just because I see a quick opportunity to roll out Movie Privacy doesn't mean Todd isn't going to work on 1/2 stars, or whatever. For the community-type features, my stuff, I am a little more cavalier with than how we are with, say, the Queue. Parts of the site are different, and we generally don't compete with each other for resources. Works goes on in parallel. Follow this?

But it begs the question: If we KNOW something is a feature you want, or a feature we want, why isn't it on the site already -- or why is it taking so long to release? These are good and logical questions.

So here's the answer: sometimes it is because a simple feature is more complicated than it at first appeared, and sometimes its because much more important features are on the top of our engineers' priorities list, but one of the most significant reasons is that we don't want to make the website more complicated than it needs to be. The site not only has to work for powerusers but also for my mom. Or your mom. We have to consider this kind of "feature creep" all the time and it keeps us from just dumping a ton of odd functions into the website.

Do you think we don't want to have the screenwriter on movie pages? (For cryin'outloud--some of my best friends are screenwriters. Heck, my brother is a screenwriter. Believe me, I think it would be great to get that info on movie pages.) So once you and we all agree something like that would be great, we start a serious exploration of the pros and cons, with lots of designing, with lots of testing, and make sure that it really is a good feature and not just another cool doohickie. In most cases, these features do make it to the website. But we are exceptionally careful at this. And the website is pretty good precisely because we are careful at this.

I'm reminded of a great talk by David Pogue, the technology columnist for the New York Times, that he gave at the distinguished TED Conference last year. Pogue is both entertaining and incredibly insightful, and his advice on interface design is very applicable to websites, and a cautionary tale for website builders like us (and his musical interludes are amusing). This runs 20 minutes--but is worthy:

We are movie lovers here at Netflix--the gang of us who build this site--and we have a lot in common with people who read blogs about Netflix. But when we release features, we work hard to get our egos out of the process, and tease out what truly makes the site better in all ways. We value your input here. We combine it with our own instincts and other research, and add new features. I hope it doesn't seem like we are reacting slowly. We're just methodical. Thank you so much for your suggestions. We hope that every month you're with us, you see the service and website improve.