3 ms·
I worked for a startup that employed this strategy. It didn't work - not even close. That's not to say it can't. But I became skeptical. Among the problems I ob
by gruseom 18y ago
I worked for a startup that employed this strategy. It didn't work - not even close. That's not to say it can't. But I became skeptical. Among the problems I observed were:
1. What's a successful test? We collected lots of data, but in the end it was still a gut decision which product(s) to go with, with all the usual emotional factors in play.
2. Just because people say they want a product, it does not follow that if you build what you thought they meant, they will buy it.
3. Tests can get a lot more expensive than you expect, and you may need to run a lot of them. This is related to #1, since you have no objective criterion to know when you're done testing.
4. This is a mechanistic model for making products, which assumes that you can turn a product "on" or "off" depending on data. But many successful products don't come into the world that way. They are driven by vision and total commitment on the part of the makers. (Think of that guy at Apple whose project was canceled and who snuck into the building for a year to keep working on it.) This is psychologically incompatible with the idea of testing a bunch of ideas, none of which you care about that much, and picking the best one based on some test results. That approach that tends to be favored by managerial types who don't much care what it is they're managing - not the classical entrepreneur who's consumed by a passion. Who do you think is more likely to have the kind of persistence needed to carry a product forward?
I came out of this with two conclusions. One was that there already is a mechanism for testing which ideas will have market success: the market itself. So the strategy here is really to beat the open market, which gives some idea of how hard it is to execute.
The second was that this way of working is not for me. I want to work on something that I am passionately commited to because it is a creative expression of my being - as well as something that people will pay for. Sometimes the one has to precede the other.
- DenisM 18y agoWho do you think is more likely to have the kind of persistence needed to carry a product forward? Someone who is confident that he is building something people want. You have to make up your mind on whether you're building something people want, or something you want. Either is fine, but you have to walk into this with open eyes.
- deleted 18y ago[deleted]
- gruseom 18y agoEveryone is confident they're building something people want. The question is, how do you propose to tell if it's true? Ultimately, the only way to know for sure is to bring it to market. Measuring sign-ups on landing pages from Adwords did not (in the cases I observed) live up to its promise as a short-cut in this process, for the reasons cited. Edit: I certainly believe in adapting what one's doing in response to market evidence - e.g. listening to one's customers - rather than stubbornly assuming that one knows what people want. That, however, is not the point of the OP.
- mechanical_fish 18y agoI think you're expecting too much from this method. This is a means of checking the pulse of the market. When you approach an injured person to give them first aid, checking their pulse is one of the first things you do. If they don't have a pulse you take appropriate action. But if they do have a pulse, you don't continue checking the pulse again and again, using ever-more-sophisticated pulse-testing technologies. The quality of a paramedic's care is not proportional to how precisely and accurately she can measure your pulse. This test might tell you whether your marketing is going to be very difficult. If you can't get anyone to click on a teaser for your product you've obviously got a problem, and you might want to refine your pitch a bit before doing anything else. Or at least make a plan that involves a relatively long, slow, profitless period during which you shop your minimalist prototype to people and try to build word of mouth. You're right that the real size of your market is hard to estimate with this technique. Tim Ferriss goes so far as to recommend that you take potential customers' emails [1] before putting up the screen that clearly informs them, with great sorrow, that you're "out of stock" and offering to email them when you've got product to ship. I think he's right that this is the only way to be sure that the rubberneckers on your website are actual potential customers, but I'm not sure I'd go so far as to do this. (Even though it looks, to my surprise, like the FTC would condone it if you used the right fine print...) And I'm pretty sure it wouldn't work for software. People don't pay for software unless someone has reviewed it. They're not fools. As for the fact that this model is "mechanistic"... That's why it's a useful spiritual exercise. The tendency to fall in love with your first idea is strong, and the tendency to build the thing that most inspires you -- as opposed to the thing that will cause customers to give you money -- is strong. But it's often easier to make a profitable thing lovable than the other way around. [1] I originally stated, in this post, that Ferriss recommended taking down credit card numbers as well. After looking closely at this part of his book, this turns out not to be true. The fine print matters a lot in cases like this, so I'm sorry for the mistake.
- gruseom 18y agoTim Ferris goes so far as to recommend that you take potential customers' credit cards and emails before [telling them] that you're "out of stock" Tim Ferris recommends lying to your customers? That is all I need to know about Tim Ferris. (I don't have any objections to your quite reasonable comments - you may be right that such tests are useful if you don't expect too much from them.)
- johnrob 18y agoI think getting hands on experience with the marketing of your unborn product is a very healthy thing to do. It will probably influence what you eventually build. And if it doesn't, at least the work is stuff you will need to do down the road regardless.