4 ms·

Encouraged users to send emails for support, and answer every single one, no matter how unpleasant (one exception). At first I got angry with "stupid" questions

by DenisM 8y ago

Encouraged users to send emails for support, and answer every single one, no matter how unpleasant (one exception). At first I got angry with "stupid" questions, but then I realized that angry is not going to stop new emails, so I rolled up my sleeves and started fixing UX problems as they came in. At the peak I had 40k active users, and the deluge of communication was like drinking from the firehose, but with continuous improvements the stream of support emails withered and then dried up completely. Eventually I got to a point where had only one or two per day, and they were more of a leisurely conversation than support requests.

ddebernardy 8y ago
Similar story here. In terms of scalability, my takeaway has been this: in every company I've worked since, I've fought tooth and nail to make sure that the engineers are the ones handling the level 2 support and up. Support is invariably happy because they only need to triage and process the really dumb questions. They offload the time consuming stuff and end up cutting their budget needs. The engineers, on the other hand, invariably get extremely annoyed by the deluge of UX and/or recurring questions that swamp them. Oftentimes it's to the point where they spend so much time on them that they've no time to build new features anymore. And then, magically, they aggressively prioritize and fix the more time consuming/support intensive issues to get them off of their hands. Everyone is better off. Clients are happier; support is happier; marketing and sales are happier; finance is happier; and so are the engineers. The latter are less happy initially, but they rapidly get a better feel of (and end up appreciating) how their work impacts end-users.
- avar 8y ago
  This is great in some cases, but can also be really dangerous. It's structuring the entire company around the selection bias of people who complain about your product, and not focusing on the hypothetical majority of users who never have issues with it, who may outnumber the group of complainers by 10x or 100x. At that point the right thing to do is to pay way less attention to things that represent a support load, and iterate on already working features used by the 10x or 100x. Sometimes it's the best thing to do, sometimes it's a horrible idea. It's not some no-brainer that software development in general should be structured like this.
  - ddebernardy 8y ago
    Agreed in principle, but not in practice. I've yet to join a company that had a marketing department on top of things enough to pinpoint what the hypothetical majority of users who never had issues actually want. (I'm sure there are some out there; but I've yet to find one that will hire me. And for clarity, I usually join companies as part of the marketing team. After a few weeks of getting a feel of what end-users actually want, fixing the existing product niggles turns out for one reason or another - maybe it's just the type of companies I work with - to be the or one of the lowest hanging fruits from an ROI standpoint.)
    - avar 8y ago
      You wouldn't give this to marketing, but make the decisions data-driven where you try as hard as you can to get a sample of what your average user is doing, and whether they're being hindered by bugs or some features. This is easiest with hosted SAAS. It's why companies like Google care so little about user feedback. They can e.g. do an A/B test on their UI and find from their data that moving some button makes it easier to find (fewer false clicks), no matter what the minority of users who contact them and swear up & down that the new UI sucks say. With self-hosted or desktop software this is harder. You might have some phone-home feature that sends aggregate statistics, or simply pro-actively contact a random segment of your userbase. "Hey, you use our product. Does it mostly work for you, or are you being hindered by issues? If so which?". That's still a huge selection bias (people who care enough to respond to a survey), but still beats the even bigger bias of people filing bug reports. And I'm saying this as exactly the sort of person who'd be annoyed enough to file a very detailed bug report against some piece of software I use.
      - DenisM 8y ago
        Moving the button around the screen is the kind of thing that will drive you to a local optimum and dump you there, with no sense of purpose or direction to a better place. Direct user feedback shakes you out of that dead-end, and towards a newer, much better dead-end on the path to success. Maybe the button is solving the wrong problem, but users latch onto it as the first thing that comes to mind. You can't find that out unless you interrogate your customers, and most engineers wouldn't do it unless their back is against the wall. Happened to me last week - spent an hour on the phone+hangouts with a customer, and it turned out they needed something other than what they asked (and we didn't have). Happens all the time. [BIG-CO] employees are just talking to each other, and then A/B testing their way to success along the nearest promotion-worthy metric. The courage, or the necessity, to overcome the aversion of the users is a competitive advantage for a small company. EDIT: want to add that I am enjoying our conversation here, thanks for joining.
- tomjen3 8y ago
  That assumes the engineers can prioritize the tasks and can't just get the list of frequent issues from support. Also it assumes that the engineers don't find a better way to deal with the support questions, such as canned "answers".
  - ddebernardy 8y ago
    > That assumes the engineers can prioritize the tasks It doesn't. If the product manager is prioritizing things, he or she invariably gets backlash from the engineers when they put forward that they're swamped and can't deliver unless they deal with X, Y, Z first. > and can't just get the list of frequent issues from support Which are then, all too often in my experience, ignored or downplayed by whoever prioritizes tasks over at engineering. When the engineers have skin in the support game, they pay extra attention to what may or may not lighten their "not very fun" workload.
    - afarrell 8y ago
      No, this strategy really does depend on the engineers having a good way to prioritize things. If you have engineers doing level-2 support and you have product managers saying "Do this other thing", then the engineer now repeatedly has to resolve a conflict among the possibilities of: A) Check email and slack to see if something new has come in. B) Check your error reporting/monitoring to see if Sentry has reported any new exceptions/errors from existing services. C) Issue #1 from a customer D) Issue #2 from a different sort of customer who has more revenue impact E) Issue #3 which was actually bubbled up from someone in sales. F) Non-urgent but important Project X from the product manager which you are actually 30% of the way finished with and consequently, it is the thing that is sitting in your brain. G) Check hackernews as a salve to the stress of a bunch of ambiguously-prioritized tasks. Accomplishing engineering tasks requires allocating your brain's working memory and directing your focus on ONE thing at a time. The harder that is, the harder it is to do anything but option G. If you put engineers in this situation, but you don't give engineers the training and structure to arbitrate decisions about prioritization, then you're setting them up for needless conflict and failure. If you just designate all of your engineers as being responsible for level-2 support without any additional structured then each one needs to tell the product manager, "I'm not going to work on this project which will bring the company $X Monthly Recurring Revenue for another week, but I'm going to deal with these support requests. I am in charge of prioritizing what gets worked on. Your job as a product manager is not to decide priorities." Which is a silly thing to tell a product manager. There should be an existing consensus around how this sort of thing is handled and prioritized. If your product managers aren't allocating time for fixing frequently-encountered bugs or UX problems, then the solution isn't to set them up for a conflict with engineers. The solution is to have someone exercise leadership of the product managers so that they is alignment on the general importance of getting bugs fixed quarterly goals and so that when someone (PM or engineer, whoever is triaging level-2 support queries) exercises judgement about the priority of a bug relative to other work, others broadly agree on that decision. You need to have some structure accomplishes this.
- gwbas1c 8y ago
  That requires a good support team that follows process. It falls apart when support doesn't train newcomers in what engineering needs, or when support just "throws things over the wall" to engineering. I once worked with a very bad "director" of support who couldn't figure out that a customer was sharing something to the wrong email address. She did absolutely no diagnostics. When I called her on it, and explained that she couldn't throw everything over the wall to me, she quit the next day. Her replacement was much, much better.
  - ddebernardy 8y ago
    > That requires a good support team that follows process. > It falls apart when support doesn't train newcomers in what engineering needs It requires as low skill level from support as you can get. Some rudimental diagnostics skills and understanding of the product itself are needed; from there, if it's not in the FAQ, let engineers handle it in full. One of three things then happens - in my experience, anyways: * The engineers prioritize fixing the stuff they don't want to repeatedly deal with. * The engineers populate a FAQ for the stuff when it's low impact enough that they're OK with support copy/pasting the FAQ item until they address it - if ever. Which can lead to further questions that get fixed or expand the FAQ, but that's fine too. * (In rarer cases) some of the engineers quit because they think support is beneath their pay grade.
    - gwbas1c 8y ago
      > It requires as low skill level from support as you can get. Some rudimental diagnostics skills and understanding of the product itself are needed; from there, if it's not in the FAQ, let engineers handle it in full. Management has to enforce that tickets aren't thrown over the wall. I can't count the number of times that your approach is misinterpreted as, "I don't need to know much about the product or try to understand the customer's problem and communicate it correctly." Anyway, without support that takes pride in, well, support, it leaves customers very unsatisfied.
- hyperpallium 8y ago
  I've always found it easy to see the whole system, and how it looks to others. (Ironically,) I can't understand how most engineers don't, and get annoyed instead. I once spent a couple of weeks working out how to give helpful error messages in the yacc parser of an open source project, for issues that arose repetitively, annoying the maintainers... but it was ignored. They are still getting annoyed by those issues to this day. And I'm still annoyed about it.
  - nicoburns 8y ago
    Same here! I figure there must be places that apprecuate this kind of thinking. So now I'm focused on only working in those kind of placea.
anitil 8y ago
Wow that takes some serious gumption!

by DenisM 8y ago

ddebernardy 8y ago

Similar story here. In terms of scalability, my takeaway has been this: in every company I've worked since, I've fought tooth and nail to make sure that the engineers are the ones handling the level 2 support and up. Support is invariably happy because they only need to triage and process the really dumb questions. They offload the time consuming stuff and end up cutting their budget needs. The engineers, on the other hand, invariably get extremely annoyed by the deluge of UX and/or recurring questions that swamp them. Oftentimes it's to the point where they spend so much time on them that they've no time to build new features anymore. And then, magically, they aggressively prioritize and fix the more time consuming/support intensive issues to get them off of their hands. Everyone is better off. Clients are happier; support is happier; marketing and sales are happier; finance is happier; and so are the engineers. The latter are less happy initially, but they rapidly get a better feel of (and end up appreciating) how their work impacts end-users.

avar 8y ago
This is great in some cases, but can also be really dangerous. It's structuring the entire company around the selection bias of people who complain about your product, and not focusing on the hypothetical majority of users who never have issues with it, who may outnumber the group of complainers by 10x or 100x. At that point the right thing to do is to pay way less attention to things that represent a support load, and iterate on already working features used by the 10x or 100x. Sometimes it's the best thing to do, sometimes it's a horrible idea. It's not some no-brainer that software development in general should be structured like this.
- ddebernardy 8y ago
  Agreed in principle, but not in practice. I've yet to join a company that had a marketing department on top of things enough to pinpoint what the hypothetical majority of users who never had issues actually want. (I'm sure there are some out there; but I've yet to find one that will hire me. And for clarity, I usually join companies as part of the marketing team. After a few weeks of getting a feel of what end-users actually want, fixing the existing product niggles turns out for one reason or another - maybe it's just the type of companies I work with - to be the or one of the lowest hanging fruits from an ROI standpoint.)
  - avar 8y ago
    You wouldn't give this to marketing, but make the decisions data-driven where you try as hard as you can to get a sample of what your average user is doing, and whether they're being hindered by bugs or some features. This is easiest with hosted SAAS. It's why companies like Google care so little about user feedback. They can e.g. do an A/B test on their UI and find from their data that moving some button makes it easier to find (fewer false clicks), no matter what the minority of users who contact them and swear up & down that the new UI sucks say. With self-hosted or desktop software this is harder. You might have some phone-home feature that sends aggregate statistics, or simply pro-actively contact a random segment of your userbase. "Hey, you use our product. Does it mostly work for you, or are you being hindered by issues? If so which?". That's still a huge selection bias (people who care enough to respond to a survey), but still beats the even bigger bias of people filing bug reports. And I'm saying this as exactly the sort of person who'd be annoyed enough to file a very detailed bug report against some piece of software I use.
    - DenisM 8y ago
      Moving the button around the screen is the kind of thing that will drive you to a local optimum and dump you there, with no sense of purpose or direction to a better place. Direct user feedback shakes you out of that dead-end, and towards a newer, much better dead-end on the path to success. Maybe the button is solving the wrong problem, but users latch onto it as the first thing that comes to mind. You can't find that out unless you interrogate your customers, and most engineers wouldn't do it unless their back is against the wall. Happened to me last week - spent an hour on the phone+hangouts with a customer, and it turned out they needed something other than what they asked (and we didn't have). Happens all the time. [BIG-CO] employees are just talking to each other, and then A/B testing their way to success along the nearest promotion-worthy metric. The courage, or the necessity, to overcome the aversion of the users is a competitive advantage for a small company. EDIT: want to add that I am enjoying our conversation here, thanks for joining.
tomjen3 8y ago
That assumes the engineers can prioritize the tasks and can't just get the list of frequent issues from support. Also it assumes that the engineers don't find a better way to deal with the support questions, such as canned "answers".
- ddebernardy 8y ago
  > That assumes the engineers can prioritize the tasks It doesn't. If the product manager is prioritizing things, he or she invariably gets backlash from the engineers when they put forward that they're swamped and can't deliver unless they deal with X, Y, Z first. > and can't just get the list of frequent issues from support Which are then, all too often in my experience, ignored or downplayed by whoever prioritizes tasks over at engineering. When the engineers have skin in the support game, they pay extra attention to what may or may not lighten their "not very fun" workload.
  - afarrell 8y ago
    No, this strategy really does depend on the engineers having a good way to prioritize things. If you have engineers doing level-2 support and you have product managers saying "Do this other thing", then the engineer now repeatedly has to resolve a conflict among the possibilities of: A) Check email and slack to see if something new has come in. B) Check your error reporting/monitoring to see if Sentry has reported any new exceptions/errors from existing services. C) Issue #1 from a customer D) Issue #2 from a different sort of customer who has more revenue impact E) Issue #3 which was actually bubbled up from someone in sales. F) Non-urgent but important Project X from the product manager which you are actually 30% of the way finished with and consequently, it is the thing that is sitting in your brain. G) Check hackernews as a salve to the stress of a bunch of ambiguously-prioritized tasks. Accomplishing engineering tasks requires allocating your brain's working memory and directing your focus on ONE thing at a time. The harder that is, the harder it is to do anything but option G. If you put engineers in this situation, but you don't give engineers the training and structure to arbitrate decisions about prioritization, then you're setting them up for needless conflict and failure. If you just designate all of your engineers as being responsible for level-2 support without any additional structured then each one needs to tell the product manager, "I'm not going to work on this project which will bring the company $X Monthly Recurring Revenue for another week, but I'm going to deal with these support requests. I am in charge of prioritizing what gets worked on. Your job as a product manager is not to decide priorities." Which is a silly thing to tell a product manager. There should be an existing consensus around how this sort of thing is handled and prioritized. If your product managers aren't allocating time for fixing frequently-encountered bugs or UX problems, then the solution isn't to set them up for a conflict with engineers. The solution is to have someone exercise leadership of the product managers so that they is alignment on the general importance of getting bugs fixed quarterly goals and so that when someone (PM or engineer, whoever is triaging level-2 support queries) exercises judgement about the priority of a bug relative to other work, others broadly agree on that decision. You need to have some structure accomplishes this.
gwbas1c 8y ago
That requires a good support team that follows process. It falls apart when support doesn't train newcomers in what engineering needs, or when support just "throws things over the wall" to engineering. I once worked with a very bad "director" of support who couldn't figure out that a customer was sharing something to the wrong email address. She did absolutely no diagnostics. When I called her on it, and explained that she couldn't throw everything over the wall to me, she quit the next day. Her replacement was much, much better.
- ddebernardy 8y ago
  > That requires a good support team that follows process. > It falls apart when support doesn't train newcomers in what engineering needs It requires as low skill level from support as you can get. Some rudimental diagnostics skills and understanding of the product itself are needed; from there, if it's not in the FAQ, let engineers handle it in full. One of three things then happens - in my experience, anyways: * The engineers prioritize fixing the stuff they don't want to repeatedly deal with. * The engineers populate a FAQ for the stuff when it's low impact enough that they're OK with support copy/pasting the FAQ item until they address it - if ever. Which can lead to further questions that get fixed or expand the FAQ, but that's fine too. * (In rarer cases) some of the engineers quit because they think support is beneath their pay grade.
  - gwbas1c 8y ago
    > It requires as low skill level from support as you can get. Some rudimental diagnostics skills and understanding of the product itself are needed; from there, if it's not in the FAQ, let engineers handle it in full. Management has to enforce that tickets aren't thrown over the wall. I can't count the number of times that your approach is misinterpreted as, "I don't need to know much about the product or try to understand the customer's problem and communicate it correctly." Anyway, without support that takes pride in, well, support, it leaves customers very unsatisfied.
hyperpallium 8y ago
I've always found it easy to see the whole system, and how it looks to others. (Ironically,) I can't understand how most engineers don't, and get annoyed instead. I once spent a couple of weeks working out how to give helpful error messages in the yacc parser of an open source project, for issues that arose repetitively, annoying the maintainers... but it was ignored. They are still getting annoyed by those issues to this day. And I'm still annoyed about it.
- nicoburns 8y ago
  Same here! I figure there must be places that apprecuate this kind of thinking. So now I'm focused on only working in those kind of placea.

anitil 8y ago

Wow that takes some serious gumption!