3 ms·

This is great in some cases, but can also be really dangerous. It's structuring the entire company around the selection bias of people who complain about your

by avar 8y ago

This is great in some cases, but can also be really dangerous. It's structuring the entire company around the selection bias of people who complain about your product, and not focusing on the hypothetical majority of users who never have issues with it, who may outnumber the group of complainers by 10x or 100x. At that point the right thing to do is to pay way less attention to things that represent a support load, and iterate on already working features used by the 10x or 100x. Sometimes it's the best thing to do, sometimes it's a horrible idea. It's not some no-brainer that software development in general should be structured like this.

ddebernardy 8y ago
Agreed in principle, but not in practice. I've yet to join a company that had a marketing department on top of things enough to pinpoint what the hypothetical majority of users who never had issues actually want. (I'm sure there are some out there; but I've yet to find one that will hire me. And for clarity, I usually join companies as part of the marketing team. After a few weeks of getting a feel of what end-users actually want, fixing the existing product niggles turns out for one reason or another - maybe it's just the type of companies I work with - to be the or one of the lowest hanging fruits from an ROI standpoint.)
- avar 8y ago
  You wouldn't give this to marketing, but make the decisions data-driven where you try as hard as you can to get a sample of what your average user is doing, and whether they're being hindered by bugs or some features. This is easiest with hosted SAAS. It's why companies like Google care so little about user feedback. They can e.g. do an A/B test on their UI and find from their data that moving some button makes it easier to find (fewer false clicks), no matter what the minority of users who contact them and swear up & down that the new UI sucks say. With self-hosted or desktop software this is harder. You might have some phone-home feature that sends aggregate statistics, or simply pro-actively contact a random segment of your userbase. "Hey, you use our product. Does it mostly work for you, or are you being hindered by issues? If so which?". That's still a huge selection bias (people who care enough to respond to a survey), but still beats the even bigger bias of people filing bug reports. And I'm saying this as exactly the sort of person who'd be annoyed enough to file a very detailed bug report against some piece of software I use.
  - DenisM 8y ago
    Moving the button around the screen is the kind of thing that will drive you to a local optimum and dump you there, with no sense of purpose or direction to a better place. Direct user feedback shakes you out of that dead-end, and towards a newer, much better dead-end on the path to success. Maybe the button is solving the wrong problem, but users latch onto it as the first thing that comes to mind. You can't find that out unless you interrogate your customers, and most engineers wouldn't do it unless their back is against the wall. Happened to me last week - spent an hour on the phone+hangouts with a customer, and it turned out they needed something other than what they asked (and we didn't have). Happens all the time. [BIG-CO] employees are just talking to each other, and then A/B testing their way to success along the nearest promotion-worthy metric. The courage, or the necessity, to overcome the aversion of the users is a competitive advantage for a small company. EDIT: want to add that I am enjoying our conversation here, thanks for joining.
  - nicoburns 8y ago
    Not sure that google is a good advert for this method. Their products tend to be increibly frustrating to use...
toast0 8y ago
The expectation here is not for engineers to see requests that can be handled by the frontline support team. And when you have a sizable number of people asking for features you don't want to build (for a good or bad reason), you can pretty easily turn that into something that the frontline team (or their automation) can come back with: "Thank you for your suggestion. This is a common request" and then "but we're not planning to build it for [some reason]" or "we're considering it for the future". Depending on if you want articles about how dumb your reasons are, or articles about how you promised you were working on it three years ago, and it never happened.
otakucode 8y ago
Your comment makes me think of one of the saddest tragedies of software development that bit me in the past. Ages ago I used a 'shareware' (thats what it was called back then) FTP client called BulletProof FTP. It was magnificent. It performed its task marvelously well and only had one bug I was aware of that ever caused me any trouble. Then they kept working on it. With every version, they bolted on some extraneous garbage that made the rest of the application worse and worse. MP3s were popular, so they added searching FTP sites for MP3s... something that I find hard to imagine any human being ever did or wanted to do at any point. I eventually just had to stop using it, they made it far too aggravating. And they never even fixed the bug (after a disconnect, it would use the random port previously used for a data connection to try to establish a command connection and if the reconnect had been quick enough, the server would still be spewing binary data into what the client expected to be a plaintext connection, resulting in random garbage).
wpietri 8y ago
I am suspicious of your division: "complainers" and "majority who never has issues". I agree some people are born complainers, and you should not implement things mentioned by just one user or one especially noisy user. I also agree that support requests shouldn't be the only way one understands users. But I suspect there are very few improvements valuable to people who don't contact support that would never be mentioned by someone who does. To me the people who contact support with suggestions aren't just the most irritating fraction of your user base. They're also the most dedicated users. The ones who love your product the most. The ones who are doing something you didn't expect because they are an early adopter in a different market. The new users that you'd like to have more of but won't until your product is easier to learn.
blihp 8y ago
Presumably, that's why you still have at least level 1 support: make sure they are able to filter out the noise and get 'real' issues sent to level 2. (aggregation can be part of the filter for complaints/feature requests that aren't bugs) There are other ways to accomplish it, but the basic idea is to never let product development get too far removed from the actual customers. Large companies can get away with not listening to the customer (for a while at least), smaller ones not so much.
badestrand 8y ago
I think improvements for products can be devided into positives (e.g. brings value, new features) and solving frustrations (bugs, slow speed, bad UX, etc). By listening to all user complaints you can almost complete eradicate frustrations - which makes your product fun to use and can massively reduce churn.

by avar 8y ago

ddebernardy 8y ago

Agreed in principle, but not in practice. I've yet to join a company that had a marketing department on top of things enough to pinpoint what the hypothetical majority of users who never had issues actually want. (I'm sure there are some out there; but I've yet to find one that will hire me. And for clarity, I usually join companies as part of the marketing team. After a few weeks of getting a feel of what end-users actually want, fixing the existing product niggles turns out for one reason or another - maybe it's just the type of companies I work with - to be the or one of the lowest hanging fruits from an ROI standpoint.)

avar 8y ago
You wouldn't give this to marketing, but make the decisions data-driven where you try as hard as you can to get a sample of what your average user is doing, and whether they're being hindered by bugs or some features. This is easiest with hosted SAAS. It's why companies like Google care so little about user feedback. They can e.g. do an A/B test on their UI and find from their data that moving some button makes it easier to find (fewer false clicks), no matter what the minority of users who contact them and swear up & down that the new UI sucks say. With self-hosted or desktop software this is harder. You might have some phone-home feature that sends aggregate statistics, or simply pro-actively contact a random segment of your userbase. "Hey, you use our product. Does it mostly work for you, or are you being hindered by issues? If so which?". That's still a huge selection bias (people who care enough to respond to a survey), but still beats the even bigger bias of people filing bug reports. And I'm saying this as exactly the sort of person who'd be annoyed enough to file a very detailed bug report against some piece of software I use.
- DenisM 8y ago
  Moving the button around the screen is the kind of thing that will drive you to a local optimum and dump you there, with no sense of purpose or direction to a better place. Direct user feedback shakes you out of that dead-end, and towards a newer, much better dead-end on the path to success. Maybe the button is solving the wrong problem, but users latch onto it as the first thing that comes to mind. You can't find that out unless you interrogate your customers, and most engineers wouldn't do it unless their back is against the wall. Happened to me last week - spent an hour on the phone+hangouts with a customer, and it turned out they needed something other than what they asked (and we didn't have). Happens all the time. [BIG-CO] employees are just talking to each other, and then A/B testing their way to success along the nearest promotion-worthy metric. The courage, or the necessity, to overcome the aversion of the users is a competitive advantage for a small company. EDIT: want to add that I am enjoying our conversation here, thanks for joining.
- nicoburns 8y ago
  Not sure that google is a good advert for this method. Their products tend to be increibly frustrating to use...

toast0 8y ago

The expectation here is not for engineers to see requests that can be handled by the frontline support team. And when you have a sizable number of people asking for features you don't want to build (for a good or bad reason), you can pretty easily turn that into something that the frontline team (or their automation) can come back with: "Thank you for your suggestion. This is a common request" and then "but we're not planning to build it for [some reason]" or "we're considering it for the future". Depending on if you want articles about how dumb your reasons are, or articles about how you promised you were working on it three years ago, and it never happened.

otakucode 8y ago

Your comment makes me think of one of the saddest tragedies of software development that bit me in the past. Ages ago I used a 'shareware' (thats what it was called back then) FTP client called BulletProof FTP. It was magnificent. It performed its task marvelously well and only had one bug I was aware of that ever caused me any trouble. Then they kept working on it. With every version, they bolted on some extraneous garbage that made the rest of the application worse and worse. MP3s were popular, so they added searching FTP sites for MP3s... something that I find hard to imagine any human being ever did or wanted to do at any point. I eventually just had to stop using it, they made it far too aggravating. And they never even fixed the bug (after a disconnect, it would use the random port previously used for a data connection to try to establish a command connection and if the reconnect had been quick enough, the server would still be spewing binary data into what the client expected to be a plaintext connection, resulting in random garbage).

wpietri 8y ago

I am suspicious of your division: "complainers" and "majority who never has issues". I agree some people are born complainers, and you should not implement things mentioned by just one user or one especially noisy user. I also agree that support requests shouldn't be the only way one understands users. But I suspect there are very few improvements valuable to people who don't contact support that would never be mentioned by someone who does. To me the people who contact support with suggestions aren't just the most irritating fraction of your user base. They're also the most dedicated users. The ones who love your product the most. The ones who are doing something you didn't expect because they are an early adopter in a different market. The new users that you'd like to have more of but won't until your product is easier to learn.

blihp 8y ago

Presumably, that's why you still have at least level 1 support: make sure they are able to filter out the noise and get 'real' issues sent to level 2. (aggregation can be part of the filter for complaints/feature requests that aren't bugs) There are other ways to accomplish it, but the basic idea is to never let product development get too far removed from the actual customers. Large companies can get away with not listening to the customer (for a while at least), smaller ones not so much.

badestrand 8y ago

I think improvements for products can be devided into positives (e.g. brings value, new features) and solving frustrations (bugs, slow speed, bad UX, etc). By listening to all user complaints you can almost complete eradicate frustrations - which makes your product fun to use and can massively reduce churn.