4 ms·

It's often impossible to get metrics on what new features you should build since your users have not yet experienced them, and those that give you feedback are

by wooyi 12y ago

It's often impossible to get metrics on what new features you should build since your users have not yet experienced them, and those that give you feedback are not necessarily the ones you should listen to. My experience is that you push out the feature and test them. If there us usage, you keep them, if not, you kill them. Testing works a lot better than trying to measure future usage.

palakchokshi 12y ago
I think you bring up a good point. These metrics, formulae, etc. can only be applied when thinking about new features. When building the core product we must rely on our knowledge of the market/users to decide what features end up in v1.0. It is also prudent to bake in anonymized usage tracking code so you know how users are using your v1.0. This will provide a guideline on what area of the product to concentrate on when thinking about new features. This way we don't end up spending time pushing out features that few will use when our data had showed we should be concentrating on a different area of the product.
- timboslice 12y ago
  Any reason you suggest anonymized tracking? I like to pass user-id / segment information to my tracking tools so we can get a picture of WHO is using feature X in addition to general adoption rates
  - palakchokshi 12y ago
    We track user specific actions too but that is more for providing a personalized experience for the user (this is specific to our use case). We notify users of why we are collecting that data. To get an idea of what feature is popular we don't need to know WHO is using it rather HOW MANY are using it and for HOW long? Another reason to do anonymized tracking would be to address any privacy concerns users might have. If someone raises such a concern we have data we can show that says we did not do any user specific tracking. This is specific to our use case and in other cases it might be important to track who is using what feature.
    - timboslice 12y ago
      Fair enough, thanks for the response
tarr11 12y ago
"d" is not usually a known quantity beforehand and therein lies the problem OP's approach. The "average user" is also not necessarily the most important one to be designing for. I've seen this kind of analysis "(b*d) / c" lead to a lot of useless SEO tweaks. Ie, imagine you've got some section of your site that is driving a lot of organic SEO traffic. If you are able to increase conversion on that part of the site by a small percent, it would have a huge impact on the business and users would experience more meaningful parts of your product. Of course, it is very difficult to increase conversion on those parts of your site and most of those initiatives fail. Better to spend more time focusing on making the experience stellar for a smaller, but strategically important section of users.
- jrochkind1 12y ago
  > Better to spend more time focusing on making the experience stellar for a smaller, but strategically important section of users. OP agrees for the 'early days' of your product: > There is a bit more to say about the breadth and depth of your product. In a company’s earliest days, it is important to place more priority on d in order to ensure that you have product/market fit. It is useless to try to grow or scale before finding 100 users who really love your product. Only then should you refocus on b and grow your user base... But at some point, you do need to grow your userbase, right?
jrochkind1 12y ago
> My experience is that you push out the feature and test them. If there us usage, you keep them There are virtually an infinite number of features you could be developing. But you have finite resources to develop them. And some features will take a lot more of your finite resources to develop than others. You need some way to prioritize what to actually develop. If your product doesn't actually do too much but is essentially just clickbait, then, sure, your features are very cheap to implement, and you just throw everything you can think of out there and A/B test.

by wooyi 12y ago

palakchokshi 12y ago

I think you bring up a good point. These metrics, formulae, etc. can only be applied when thinking about new features. When building the core product we must rely on our knowledge of the market/users to decide what features end up in v1.0. It is also prudent to bake in anonymized usage tracking code so you know how users are using your v1.0. This will provide a guideline on what area of the product to concentrate on when thinking about new features. This way we don't end up spending time pushing out features that few will use when our data had showed we should be concentrating on a different area of the product.

timboslice 12y ago
Any reason you suggest anonymized tracking? I like to pass user-id / segment information to my tracking tools so we can get a picture of WHO is using feature X in addition to general adoption rates
- palakchokshi 12y ago
  We track user specific actions too but that is more for providing a personalized experience for the user (this is specific to our use case). We notify users of why we are collecting that data. To get an idea of what feature is popular we don't need to know WHO is using it rather HOW MANY are using it and for HOW long? Another reason to do anonymized tracking would be to address any privacy concerns users might have. If someone raises such a concern we have data we can show that says we did not do any user specific tracking. This is specific to our use case and in other cases it might be important to track who is using what feature.
  - timboslice 12y ago
    Fair enough, thanks for the response

tarr11 12y ago

"d" is not usually a known quantity beforehand and therein lies the problem OP's approach. The "average user" is also not necessarily the most important one to be designing for. I've seen this kind of analysis "(b*d) / c" lead to a lot of useless SEO tweaks. Ie, imagine you've got some section of your site that is driving a lot of organic SEO traffic. If you are able to increase conversion on that part of the site by a small percent, it would have a huge impact on the business and users would experience more meaningful parts of your product. Of course, it is very difficult to increase conversion on those parts of your site and most of those initiatives fail. Better to spend more time focusing on making the experience stellar for a smaller, but strategically important section of users.

jrochkind1 12y ago
> Better to spend more time focusing on making the experience stellar for a smaller, but strategically important section of users. OP agrees for the 'early days' of your product: > There is a bit more to say about the breadth and depth of your product. In a company’s earliest days, it is important to place more priority on d in order to ensure that you have product/market fit. It is useless to try to grow or scale before finding 100 users who really love your product. Only then should you refocus on b and grow your user base... But at some point, you do need to grow your userbase, right?

jrochkind1 12y ago

> My experience is that you push out the feature and test them. If there us usage, you keep them There are virtually an infinite number of features you could be developing. But you have finite resources to develop them. And some features will take a lot more of your finite resources to develop than others. You need some way to prioritize what to actually develop. If your product doesn't actually do too much but is essentially just clickbait, then, sure, your features are very cheap to implement, and you just throw everything you can think of out there and A/B test.