4 ms·

I thought this was going to be a post about customer development, where the issue is not generally the formatting of the questions, but asking the wrong questio

I thought this was going to be a post about customer development, where the issue is not generally the formatting of the questions, but asking the wrong questions in the first place, or misinterpreting the answers. Surveys can be a useful tool, but nothing replaces a real conversation. Even asking more open ended questions in an email or survey form can give you so much more information than radio button answers (and unless you're a huge company, it's not too hard to read a few hundred open-ended responses). Along those lines, I'll share a snippet from patio11's Microconf presentation last year (posted on HN yesterday), regarding interpreting the responses you get to a more open-ended conversation: "If you’re solving a problem people actually have, they will say at this point, 'Shut up and take my money.' If someone says, 'That’s kind of interesting, tell me when that exists,' you have not successfully identified a problem that people actually have." You don't get that kind of insight from radio buttons.

lobotryas 13y ago
I also think the article title is misleading or off-base. As for the contents: it's a PR promotional post for the startup disguised as a list of tips. The article also misses the point: "talking" involves dialogue, which is a two-way street. Conversely, a survey is a formulaic, one-way exchange of information that pales in comparison to a real conversation.
wpietri 13y ago
Agreed. One of the biggest mistakes I see startups making when talking to users is using surveys at all. The theory seems to be "Oh, we'll get more information that way." Or, from the more honest ones, "This was less scary than trying to talk with people one on one." But surveys are mainly useful for confirming or denying particular generalizations from real conversations. Surveys throw out all sorts of data. Every raised eyebrow. Every excited look. Every long pause where they try to figure out what they hell you are talking about. Every gesture, every tone of voice, every tangent. You really need that data when you're first starting out. Early on, the problem isn't the things you know you don't know. It's the things that you're pretty sure about that are false.
- usaar333 13y ago
  I'd say it is more of a "this is more tractable". Given that the barrier to doing a survey is so much lower for your users, the guys who accept "real conversations" will probably be significantly more optimistic about your product than those who don't..
  - Evbn 13y ago
    People who are optimistic at the only people that matter. A company with only 1% of the world population as customers is one of the greatest companies in history.
    - usaar333 13y ago
      If there are enough of them. Most B2Bs do not have anywhere close to 1% of the world's population as customers.
  - adrianhoward 13y ago
    I'd say it is more of a "this is more tractable". Given that the barrier to doing a survey is so much lower for your users, the guys who accept "real conversations" will probably be significantly more optimistic about your product than those who don't.. Not true in my experience. In fact I see exactly the opposite. The folk doing surveys seem far more likely to fool themselves that there onto something. They're usually some way to cut up the quant data so it looks like "success" by some category or another. Surveys are a great tool in the right sort of context - but early customer development isn't one of 'em. I do a little workshop on survey design. The first slide is: The First Rule of Surveys: Don't do them. The Second Rule of Surveys (for experts only!): Don't do them yet. (yes - it is a deliberate paraphase of Michael Jackson's rules of optimisation ;-)

lobotryas 13y ago

I also think the article title is misleading or off-base. As for the contents: it's a PR promotional post for the startup disguised as a list of tips. The article also misses the point: "talking" involves dialogue, which is a two-way street. Conversely, a survey is a formulaic, one-way exchange of information that pales in comparison to a real conversation.

wpietri 13y ago

Agreed. One of the biggest mistakes I see startups making when talking to users is using surveys at all. The theory seems to be "Oh, we'll get more information that way." Or, from the more honest ones, "This was less scary than trying to talk with people one on one." But surveys are mainly useful for confirming or denying particular generalizations from real conversations. Surveys throw out all sorts of data. Every raised eyebrow. Every excited look. Every long pause where they try to figure out what they hell you are talking about. Every gesture, every tone of voice, every tangent. You really need that data when you're first starting out. Early on, the problem isn't the things you know you don't know. It's the things that you're pretty sure about that are false.

usaar333 13y ago
I'd say it is more of a "this is more tractable". Given that the barrier to doing a survey is so much lower for your users, the guys who accept "real conversations" will probably be significantly more optimistic about your product than those who don't..
- Evbn 13y ago
  People who are optimistic at the only people that matter. A company with only 1% of the world population as customers is one of the greatest companies in history.
  - usaar333 13y ago
    If there are enough of them. Most B2Bs do not have anywhere close to 1% of the world's population as customers.
- adrianhoward 13y ago
  I'd say it is more of a "this is more tractable". Given that the barrier to doing a survey is so much lower for your users, the guys who accept "real conversations" will probably be significantly more optimistic about your product than those who don't.. Not true in my experience. In fact I see exactly the opposite. The folk doing surveys seem far more likely to fool themselves that there onto something. They're usually some way to cut up the quant data so it looks like "success" by some category or another. Surveys are a great tool in the right sort of context - but early customer development isn't one of 'em. I do a little workshop on survey design. The first slide is: The First Rule of Surveys: Don't do them. The Second Rule of Surveys (for experts only!): Don't do them yet. (yes - it is a deliberate paraphase of Michael Jackson's rules of optimisation ;-)