4 ms·

I Only semi-agree. To truly understand if your product has hit the mark, you need to have the metrics/analytics to be able to assess those factors, look for ti

by hoi 13y ago

I Only semi-agree. To truly understand if your product has hit the mark, you need to have the metrics/analytics to be able to assess those factors, look for tipping points, know what to remove etc. Sure, don't do premature micro optimization.. but massive optimization to get product market fit is still a form of optimization that needs to be done, for example, A/B testing different landing pages to discover what people are actually looking for, each page with different subsets of 'features'.

jonathanjaeger 13y ago
It takes a lot of data to get good A/B test results and you can't run all these tests at the same time and in parallel. A lot of newer startups simply don't have the traffic or data available to optimize like crazy. I think the key is just to focus on your core metrics, like engagement for a consumer product or whatever your metric is for a B2B company (contracts, revenue, etc.).
- usujason 13y ago
  I've worked with many startups to help them build a culture of data-driven decision making and I'll say from my experience that you don't need massive data to run highly valuable A/B tests. Also, with proper segmentation techniques, you can run many tests in parallel.
  - jonathanjaeger 13y ago
    Fair enough. My experience is more on A/B testing high trafficked paid lead gen pages, and I understand you can still make conclusions with less data (doesn't need to be massive). But I do agree with the author of the original post that it can be easy to worry about all these different A/B tests that aren't that illuminating in the grand scheme of things when you haven't even got a solid product.
    - usujason 13y ago
      Yep, we agree there, the advice of the author is spot on. I think too many startups, and large companies for that matter, think optimization is going to fix all the problems that are inherent in their product/business by just blindly slaping optimization vendor A's tags on the site and money just falls from the sky. Just doesn't happen like that.

by hoi 13y ago

jonathanjaeger 13y ago

It takes a lot of data to get good A/B test results and you can't run all these tests at the same time and in parallel. A lot of newer startups simply don't have the traffic or data available to optimize like crazy. I think the key is just to focus on your core metrics, like engagement for a consumer product or whatever your metric is for a B2B company (contracts, revenue, etc.).

usujason 13y ago
I've worked with many startups to help them build a culture of data-driven decision making and I'll say from my experience that you don't need massive data to run highly valuable A/B tests. Also, with proper segmentation techniques, you can run many tests in parallel.
- jonathanjaeger 13y ago
  Fair enough. My experience is more on A/B testing high trafficked paid lead gen pages, and I understand you can still make conclusions with less data (doesn't need to be massive). But I do agree with the author of the original post that it can be easy to worry about all these different A/B tests that aren't that illuminating in the grand scheme of things when you haven't even got a solid product.
  - usujason 13y ago
    Yep, we agree there, the advice of the author is spot on. I think too many startups, and large companies for that matter, think optimization is going to fix all the problems that are inherent in their product/business by just blindly slaping optimization vendor A's tags on the site and money just falls from the sky. Just doesn't happen like that.