Moving the Needle

When I add something to the site — like the Latest Reviews block or the Members Top 10 Lists, I am looking to see how many of our subscribers use it, and come back to use it again. For whom does this feature add value? My goal is to add lots of value for as many people as i can. A new feature doesn’t have to appeal to everyone, of course. We have a lot of subscribers, after all, and you all have a very wide range of needs and preferences. Some things we do might be useful to everyone (say, improving the way the Queue works) and some only impact a subset of users (for instance, changing the way you see your Friends). We do both kinds of features, and are always looking to balance the efforts creating each of them.

When I look at features on the Netflix site (and in the Community area in particular) I have two buckets that everything falls into: moving the needle and polishing the apple.

What I want is to move the needle: I want to make a change or add a feature that materially improves our site. Maybe it makes it so much easier to manage your queue or find exceptionally good movies that it totally changes the way you use and enjoy Netflix. That would be good. That would be “moving the needle.”

Often times I’m just polishing the apple: it’s good to make this thing better, easier, clearer, whatever – but it really isn’t going to change how many people use it or enjoy it – at least not in a meaningful way.

The drag and drop queue – that was a needle mover. It makes the queue much easier to work for every one of our subscribers. Good feature. Adding the Friends Activity headlines to the Community Home page? Well, it was a good thing to do, but it was only for Friends users, and quite possibly was only a bit of apple polish. Removing the dates from the Top 10 Lists block? (releasing next week) Definitely apple polish. I wouldn’t have done it at all except it only took about 10 seconds and it was driving a few of us crazy. We call this “opportunistic” improvement. You’re fixing something anyway and it is easy to do. So make the thing better even if it doesn’t move any needles.

There is a general sense that you have to do a little bit of polish or in time the site just isn’t that good, it isn’t easy to use or it slowly irritates y’all to the point of distraction. Search is an example. It works. But it could be so much better. Most of us are sure this is a needle mover. I think the entire Community effort – comprised of many many little interlocking features – could be a needle mover. I am trying to discover if that is the case.

The problem often comes when me and my teams’ natural desire is to polish apples – make the things we have much smoother or better (think the sliding of the Latest Reviews or the adding of movies to Top 10 Lists), but then realize that the small bits of work to improve these things adds up and the big new features that are certainly more likely to move some needles are getting postponed.

Anyway, it’s a balance. Like a stock portfolio. Some high risk stuff, some slow growth but low risk stuff, a few wildcards… but all adding up to a strong healthy portfolio that weathers short fluctuations and delivers good results in the long haul.

The features I want, and that you guys often suggest, are almost always good things to do and would results in genuine improvement of the site. The question I ask – and you can ask yourself if you want – is will it make a difference, and if so – how much? For whom? I could change the font on all the tabs and it would certainly make the site nicer looking and perhaps even easier to read – but would it change our business? (Maybe that’s too extreme an example). Perhaps better: let’s make it easy to sort the movie reviews on all the movie pages – by helpful ratings, by similarity percentage to you, by recency, etc. Seems like a no-brainer. It certainly would be great. It’s already on my list of things to do. But let’s step back for a moment. Will it help more people find better movies? If you don’t read movie reviews (and many people don’t, actually), will it get you to try it? Most people ask for these options, but rarely use them. If we provide the ability to sort in a dozen ways, how many people will use the feature? Simplicity would say: discover the most useful of all the ways to sort these things, and make that the default – because that’s what most people will see. Then add the sorting later, opportunistically. And what are the thresholds? If I told you only 1% of our users would sort, would you make this a top priority? How about 10%? What would it take – 20%? 50% (Complicate this with the effort: what if it took a day to build, or a month? How would that affect your decision?) The question is rhetorical, but the issue is very real. It’s not a bad feature. In fact it is a brilliant and obvious feature. But should it be the first thing I build? Or is there something else that is also brilliant but that will be “game changing” – a needle mover! Because if you were given a choice, you’d probably tell me to do that.

I’m reminded of a bit of business advice I got when I was raising money to start a company back in 1993. It went something like this: “Just because something is a truly great idea does not necessarily mean it will make a good business. And just because something would be a terrific business does not necessarily mean it is a good investment.” You guys offer us ideas – often great ideas. Our job is to decide if they are worthy investments. There is often no question they are good ideas.

That’s why I really absorb all of your suggestions, and that’s also why you don’t see all of them getting implemented right away. It is without question the hardest part of this job.