8 ms·

I have a slightly different take on this. I think every company should dedicate a certain percentage of their revenue to handling bug reports from users. Yes, m

by jwr 7y ago

I have a slightly different take on this. I think every company should dedicate a certain percentage of their revenue to handling bug reports from users. Yes, many bug reports will be bogus, but many will be pure gold (I speak from experience). Yes, it involves a lot of manual work. But it is so incredibly worth it — you get to catch the odd bugs that you will never catch with automated or in-house testing. In contrast, big companies like Apple try very hard to avoid and ignore user bug reports. There is no way to easily E-mail them, "filing a radar" is hard, and even if you do they do their best to ignore your report. When I wrote "percentage of revenue", I meant it. In my software business (https://partsbox.io/ https://partsbox.io/), I dedicate about 10% of my time to reading, analyzing, and responding to user bug reports. Carefully thinking about every report let me catch bugs and problems that I never would have found. I also learned that every user bug report carries some information: even if something isn't actually a bug, getting multiple reports about it might point to a usability or a documentation problem. If Apple also spent 10% of its revenue on processing bug reports and fixing bugs, they would not have a quality problem like they do now.

mtgx 7y ago
It's basically why Windows 10 is also so much buggier than previous versions: Nadella cut a good portion of Microsoft's Q&A team.
- SCdF 7y ago
  If this your anecdote, or are their studies around this? I ask because anecdotally everyone I know finds Windows 10 to be the most stable Windows they've ever used.
  - SmellyGeekBoy 7y ago
    Did they not use 2000? Or XP? Or even 7 for that matter?
    - SCdF 7y ago
      I've used Windows on and off since 3.11, and have used 98, xp, 7 and 10 a lot. As much fun as it is to hate on Windows, if I'm honest with myself 10 has been the most stable experience I've had with Windows, with the possible exception of 3.11 (though that was a long time ago).
      - acollins1331 7y ago
        This seems pretty far removed from most other people's experiences.
        SCdF 7y ago
        Which is why I was asking if there was any kind of quality study / survey / anything better than opinions around this, because as you say the anecdotes in my anecdote arsenal seem to be quite a bit different from other folk (or at least other highly verbal folk) on the internet.
        IggleSniggle 7y ago
        I agree with you SCdF. Win 10 has been the most reliable Windows for me since 3.11, except maybe late-stage XP when all the worst bugs had worked out. I think it’s always the case that people that have bad experiences are going to be vocal, because you don’t go around talking about “oh hey, everything worked normally today.” The deletion bug WAS pretty bad, but it also affected a VERY tiny group within the subset of users that had opted in to experimental/advanced-preview updates.
        thijsvandien 7y ago
        All previous versions of Windows stabilized because they were more or less feature complete. The problem with W10 is that it’s continuously in motion. 1809 becomes stable, but before you know it you’re on 1903 and the trouble starts all over again.
        7y ago
  - dooglius 7y ago
    There was a big data deletion bug from an update that happened around a year ago, I don't think there was anything that bad in Windows 7
  - hootbootscoot 7y ago
    Dear Flag-happy HN reader: flag this. It's an opinion. If you can't handle a subjective opinion from a subjective being then don't expect anyone to handle your subjective opinion, subjective being. <rant> Every windows OS since XP SP3 has been terrible, IMO. The UI gets more and more unusable and I am no longer able to even assist an old person do something simple. Microsofts entire product line is BROKEN, IMO. Outlook.com? a lousy SPA that has been broken for months (reloads the entire app every 4 seconds. the "official solution" is to delete "ALL BROWSER COOKIES" (so they say. I advise you delete the Outlook.com cookies instead, but YMMV) at which point this same problem will occur again in a few days.) No "basic html" mode for Outlook (thanks Gmail!) just a foisted-upon-us SPA that is total garbage. (Email is arguably the last place I want an SPA. I don't mind reloads when going from a list of messages to an individual one, but who cares what users want, right?) Office365? terrible. They are pulling an Oracle and seeing how long the helpless victims of their 30 year MS Office monopoly will put up with being boiled slowly. Seriously, compare a proper in-office DC, roaming profiles (on the LAN!) etc with this SAAS debacle and tell me tales of progress. MS seems to be on a mission to lose all of their revenue streams at once. I enjoy their sudden support for FOSS and Linux, I just wonder if it's not simply a matter of escaping a sinking ship... </rant> studies, around an obvious state of affairs that one only needs to open ones eyes to see? I suppose that could be arranged, corrupted (paid off), or contested in endless petty arguments in which freedom is slavery and war is peace etc, but I do believe that the emperor wears no clothes. Oh, so Crapple is also going downhill now? Surprise surprise, and you won't be allowed to use the lifeboats, as you are lower-deck paeon, so drown like a good little fanboi, indeed. Someone, anyone: STEAL THEIR LUNCH. it's ripe for the taking. at this point even the smallest OEM selling pre-loaded Linux laptops has a leg-up. It's going to end up the de-facto standard at the point that Windows heiroglyphics no longer reveal any means of doing anything with clicks grunts nudges etc... we are almost there...
    - hootbootscoot 7y ago
      oh no. i lost "points"
Mindwipe 7y ago
If Apple spent more than $11 dollars on documentation they also might not be in the situation they're currently in. So much of what happened in Catalina is just flat out not written down in the Developer Documentation. Apple's documentation has been waning for a while, but it's fallen off a cliff in the last eighteen months. It's impossible to believe that's not having an impact on external software quality, and it's difficult to believe it isn't having an impact on Apple's own applications too.
- p_l 7y ago
  I remember doing my first hackintosh, due to an attempt at a startup where we wanted to have a iOS and OSX client. The documentation blew me away, nicely packaged with XCode, and made it really look like a platform that would care about developers, especially given how niche it was. That was 10.6 time. Currently, I have a Mac forced on me by company. I look at docs... and they are the same docs I remember, with bits missing, a "deprecated banner" from 5 years ago. Deprecation notices give no information what APIs one should use in place, or where new documentation is.
  - bloopernova 7y ago
    This boggles my mind. My flabber is ghasted. Documentation is of paramount importance! How else will people build an ecosystem around your software if they can't figure out how to use it in a variety of ways? I really hope this isn't some trend of just abandoning the commons, taking without adding back in.
    - kkarakk 7y ago
      I think the point is to get the average beginner dev frustrated and not make macos apps. They've been pruning the platform slowly but surely for quite some time now ipad seems to be their focus for "pro" stuff now. Now that iPad apps can run on macos i think that's a clear sign that if you want to make a macos app - don't bother, make an ipad app
      - 7y ago
2rsf 7y ago
You are writing this based on gut feeling, big companies simply can't go over every bug report- it's not scalable. I used to work on a product at a big company, we took users input very seriously, not only from bug reports but also by analyzing telemetrics data and simple users "star" feedback. At the end of the day it's simply impossible to both fix everything and still go forward when the product is big and complex. To this day the product has terrible reputation as buggy and unfriendly. Even the simple task of finding what the problem is, fixing it and then closing related problem reports is very complex since many bugs manifest themselves in different ways.
- crummy 7y ago
  Does the Chrome team go over every bug report? It seems like it to me, though I've only filed a few.
  - dredmorbius 7y ago
    They also ignore bugs for a decade or more.
- jwr 7y ago
  > You are writing this based on gut feeling, big companies simply can't go over every bug report- it's not scalable. I often hear this repeated, and I keep wondering: why? Why is it not scalable? Apple had 265 billion dollars of revenue in 2018. I'm spending 10% of my budget on QA. So, in the case of Apple, if you had a 26 billion dollar budget (give or take a few billion) for your QA department, why couldn't you go over every bug report? It's definitely possible, it's just that companies are making a different tradeoff, assuming that bugs will not influence buying decisions that much. And to a certain extent they are right.
  - thrower123 7y ago
    Because software is just not something you can just throw more money and people at and expect it to get better. In fact, that is the surest way to make a piece of software worse. Software, despite all the attempts to come up with ways to commoditize its production, does not scale.
dimitar 7y ago
Definitely put a number on it! And track how close you actually are. I think 10% of revenue will be deemed too much for most organizations. Usually Support is seen as a distraction to new product development or whatever the strategy is, but often I see management overestimate how much time and money is spent on it. So, just tracking that cost/time will likely increase it and improve customer satisfaction. Unfortunately, improving Customer Support is not rewarded in organizations as much as cutting costs, creating new products or expanding revenue, so you'll probably need the CEO to care a lot about it to improve matters. And CEOs can't really put a number on improving customer support, so it is optimized to cost less over the near term, not deliver a better service. I also see the worrying tendency to keep only Customer Service or equivalent responsible, as if they can fix the bugs and improve the design.
00deadbeef 7y ago
Yeah I have a bug where the disk storage calculations show Messages as using a negative amount of disk space. Nothing I could have done as a user could have caused that, but they've ingored the bug report I sent.
hobs 7y ago
Just a not that there is a way to get a bug report filed - call support, get the issue escalated because its not fixed, if they cant fix it it will go to engineering. I have personally witnessed hundreds of support cases go through this various process, in some cases taking months of work (for communication mostly from eng to customer) of T2 support reps. Now - is their current spend sufficient? Probably not, but Apple consistently ranks among the highest customer service ratings - so they probably think spending more here is a waste of time.
- alphakappa 7y ago
  While this is true, it is a solution for people dedicated to getting a problem fixed. Calling up a support line is a huge barrier, and companies lose out when people cannot file bugs or express their frustration with little issues with ease.
  - hobs 7y ago
    Absolutely - and out of the first year of purchase you gotta pay - people LOVE paying! But that last part matters - if a company already wins the customer service metrics - how do you convince management the money is worth it?
mschaef 7y ago
> If Apple also spent 10% of its revenue on processing bug reports and fixing bugs Just to be clear, you're suggesting they dedicate $25B/year to bug fixing? Seems high, given their total R&D budget for 2018 was only ~56% of that. https://www.macrumors.com/2018/11/05/apple-2018-form-10-k-highlights/ https://www.macrumors.com/2018/11/05/apple-2018-form-10-k-hi...
- Strom 7y ago
  Sure, why not? It's not like they're doing anything else with the money. It doesn't have to be only software fixes either. There's plenty of hardware issues like the keyboard. The spending doesn't have to be instant either. Just start increasing the spending and measure user satisfaction and bug report counts. Once you've arrived deep into diminishing returns you can stop increasing the spending.
  - mschaef 7y ago
    > Once you've arrived deep into diminishing returns you can stop increasing the spending. How do you know they're not there now? It's hard to believe they're not thinking very carefully about these sorts of things, given how successful they've been establishing a place in the market.
    - Strom 7y ago
      > How do you know they're not there now? I don't know for a fact but I have more respect towards Apple than to think that the current state is the best they could do.
      - mschaef 7y ago
        > I have more respect towards Apple than to think that the current state is the best they could do. $265B in Revenue and and $59B in profit seems to indicate they're already doing pretty well. Maybe your definition of 'best' isn't in perfect alignment with the rest of the marketplace? I get that there are things about their products you can point at and wish were done better, or at least differently. I use Apple products myself, so I have a list of these things myself. But, the thing is, this isn't unique to Apple. Have you ever been completely satisfied with a product or service? I can't really say that I have. About the best that I've ever been able to achieve is something that's a close enough approximation to my 'requirements' that I can adapt what I'm doing to make it a solution. Point is - you're never going to find something perfect. There will always be a gap and hopefully effort on both sides to close that gap. Those financial numbers aren't a perfect indication by any means, but they do signal that Apple's doing a reasonably good job of closing that gap for a reasonably large number of people willing to part with their earnings to buy Apple's products.

by jwr 7y ago

mtgx 7y ago

It's basically why Windows 10 is also so much buggier than previous versions: Nadella cut a good portion of Microsoft's Q&A team.

SCdF 7y ago
If this your anecdote, or are their studies around this? I ask because anecdotally everyone I know finds Windows 10 to be the most stable Windows they've ever used.
- SmellyGeekBoy 7y ago
  Did they not use 2000? Or XP? Or even 7 for that matter?
  - SCdF 7y ago
    I've used Windows on and off since 3.11, and have used 98, xp, 7 and 10 a lot. As much fun as it is to hate on Windows, if I'm honest with myself 10 has been the most stable experience I've had with Windows, with the possible exception of 3.11 (though that was a long time ago).
    - acollins1331 7y ago
      This seems pretty far removed from most other people's experiences.
      - SCdF 7y ago
        Which is why I was asking if there was any kind of quality study / survey / anything better than opinions around this, because as you say the anecdotes in my anecdote arsenal seem to be quite a bit different from other folk (or at least other highly verbal folk) on the internet.
        IggleSniggle 7y ago
        I agree with you SCdF. Win 10 has been the most reliable Windows for me since 3.11, except maybe late-stage XP when all the worst bugs had worked out. I think it’s always the case that people that have bad experiences are going to be vocal, because you don’t go around talking about “oh hey, everything worked normally today.” The deletion bug WAS pretty bad, but it also affected a VERY tiny group within the subset of users that had opted in to experimental/advanced-preview updates.
        thijsvandien 7y ago
        All previous versions of Windows stabilized because they were more or less feature complete. The problem with W10 is that it’s continuously in motion. 1809 becomes stable, but before you know it you’re on 1903 and the trouble starts all over again.
      - 7y ago
- dooglius 7y ago
  There was a big data deletion bug from an update that happened around a year ago, I don't think there was anything that bad in Windows 7
- hootbootscoot 7y ago
  Dear Flag-happy HN reader: flag this. It's an opinion. If you can't handle a subjective opinion from a subjective being then don't expect anyone to handle your subjective opinion, subjective being. <rant> Every windows OS since XP SP3 has been terrible, IMO. The UI gets more and more unusable and I am no longer able to even assist an old person do something simple. Microsofts entire product line is BROKEN, IMO. Outlook.com? a lousy SPA that has been broken for months (reloads the entire app every 4 seconds. the "official solution" is to delete "ALL BROWSER COOKIES" (so they say. I advise you delete the Outlook.com cookies instead, but YMMV) at which point this same problem will occur again in a few days.) No "basic html" mode for Outlook (thanks Gmail!) just a foisted-upon-us SPA that is total garbage. (Email is arguably the last place I want an SPA. I don't mind reloads when going from a list of messages to an individual one, but who cares what users want, right?) Office365? terrible. They are pulling an Oracle and seeing how long the helpless victims of their 30 year MS Office monopoly will put up with being boiled slowly. Seriously, compare a proper in-office DC, roaming profiles (on the LAN!) etc with this SAAS debacle and tell me tales of progress. MS seems to be on a mission to lose all of their revenue streams at once. I enjoy their sudden support for FOSS and Linux, I just wonder if it's not simply a matter of escaping a sinking ship... </rant> studies, around an obvious state of affairs that one only needs to open ones eyes to see? I suppose that could be arranged, corrupted (paid off), or contested in endless petty arguments in which freedom is slavery and war is peace etc, but I do believe that the emperor wears no clothes. Oh, so Crapple is also going downhill now? Surprise surprise, and you won't be allowed to use the lifeboats, as you are lower-deck paeon, so drown like a good little fanboi, indeed. Someone, anyone: STEAL THEIR LUNCH. it's ripe for the taking. at this point even the smallest OEM selling pre-loaded Linux laptops has a leg-up. It's going to end up the de-facto standard at the point that Windows heiroglyphics no longer reveal any means of doing anything with clicks grunts nudges etc... we are almost there...
  - hootbootscoot 7y ago
    oh no. i lost "points"

Mindwipe 7y ago

If Apple spent more than $11 dollars on documentation they also might not be in the situation they're currently in. So much of what happened in Catalina is just flat out not written down in the Developer Documentation. Apple's documentation has been waning for a while, but it's fallen off a cliff in the last eighteen months. It's impossible to believe that's not having an impact on external software quality, and it's difficult to believe it isn't having an impact on Apple's own applications too.

p_l 7y ago
I remember doing my first hackintosh, due to an attempt at a startup where we wanted to have a iOS and OSX client. The documentation blew me away, nicely packaged with XCode, and made it really look like a platform that would care about developers, especially given how niche it was. That was 10.6 time. Currently, I have a Mac forced on me by company. I look at docs... and they are the same docs I remember, with bits missing, a "deprecated banner" from 5 years ago. Deprecation notices give no information what APIs one should use in place, or where new documentation is.
- bloopernova 7y ago
  This boggles my mind. My flabber is ghasted. Documentation is of paramount importance! How else will people build an ecosystem around your software if they can't figure out how to use it in a variety of ways? I really hope this isn't some trend of just abandoning the commons, taking without adding back in.
  - kkarakk 7y ago
    I think the point is to get the average beginner dev frustrated and not make macos apps. They've been pruning the platform slowly but surely for quite some time now ipad seems to be their focus for "pro" stuff now. Now that iPad apps can run on macos i think that's a clear sign that if you want to make a macos app - don't bother, make an ipad app
    - 7y ago

2rsf 7y ago

You are writing this based on gut feeling, big companies simply can't go over every bug report- it's not scalable. I used to work on a product at a big company, we took users input very seriously, not only from bug reports but also by analyzing telemetrics data and simple users "star" feedback. At the end of the day it's simply impossible to both fix everything and still go forward when the product is big and complex. To this day the product has terrible reputation as buggy and unfriendly. Even the simple task of finding what the problem is, fixing it and then closing related problem reports is very complex since many bugs manifest themselves in different ways.

crummy 7y ago
Does the Chrome team go over every bug report? It seems like it to me, though I've only filed a few.
- dredmorbius 7y ago
  They also ignore bugs for a decade or more.
jwr 7y ago
> You are writing this based on gut feeling, big companies simply can't go over every bug report- it's not scalable. I often hear this repeated, and I keep wondering: why? Why is it not scalable? Apple had 265 billion dollars of revenue in 2018. I'm spending 10% of my budget on QA. So, in the case of Apple, if you had a 26 billion dollar budget (give or take a few billion) for your QA department, why couldn't you go over every bug report? It's definitely possible, it's just that companies are making a different tradeoff, assuming that bugs will not influence buying decisions that much. And to a certain extent they are right.
- thrower123 7y ago
  Because software is just not something you can just throw more money and people at and expect it to get better. In fact, that is the surest way to make a piece of software worse. Software, despite all the attempts to come up with ways to commoditize its production, does not scale.

dimitar 7y ago

Definitely put a number on it! And track how close you actually are. I think 10% of revenue will be deemed too much for most organizations. Usually Support is seen as a distraction to new product development or whatever the strategy is, but often I see management overestimate how much time and money is spent on it. So, just tracking that cost/time will likely increase it and improve customer satisfaction. Unfortunately, improving Customer Support is not rewarded in organizations as much as cutting costs, creating new products or expanding revenue, so you'll probably need the CEO to care a lot about it to improve matters. And CEOs can't really put a number on improving customer support, so it is optimized to cost less over the near term, not deliver a better service. I also see the worrying tendency to keep only Customer Service or equivalent responsible, as if they can fix the bugs and improve the design.

00deadbeef 7y ago

Yeah I have a bug where the disk storage calculations show Messages as using a negative amount of disk space. Nothing I could have done as a user could have caused that, but they've ingored the bug report I sent.

hobs 7y ago

Just a not that there is a way to get a bug report filed - call support, get the issue escalated because its not fixed, if they cant fix it it will go to engineering. I have personally witnessed hundreds of support cases go through this various process, in some cases taking months of work (for communication mostly from eng to customer) of T2 support reps. Now - is their current spend sufficient? Probably not, but Apple consistently ranks among the highest customer service ratings - so they probably think spending more here is a waste of time.

alphakappa 7y ago
While this is true, it is a solution for people dedicated to getting a problem fixed. Calling up a support line is a huge barrier, and companies lose out when people cannot file bugs or express their frustration with little issues with ease.
- hobs 7y ago
  Absolutely - and out of the first year of purchase you gotta pay - people LOVE paying! But that last part matters - if a company already wins the customer service metrics - how do you convince management the money is worth it?

mschaef 7y ago

> If Apple also spent 10% of its revenue on processing bug reports and fixing bugs Just to be clear, you're suggesting they dedicate $25B/year to bug fixing? Seems high, given their total R&D budget for 2018 was only ~56% of that. https://www.macrumors.com/2018/11/05/apple-2018-form-10-k-highlights/ https://www.macrumors.com/2018/11/05/apple-2018-form-10-k-hi...

Strom 7y ago
Sure, why not? It's not like they're doing anything else with the money. It doesn't have to be only software fixes either. There's plenty of hardware issues like the keyboard. The spending doesn't have to be instant either. Just start increasing the spending and measure user satisfaction and bug report counts. Once you've arrived deep into diminishing returns you can stop increasing the spending.
- mschaef 7y ago
  > Once you've arrived deep into diminishing returns you can stop increasing the spending. How do you know they're not there now? It's hard to believe they're not thinking very carefully about these sorts of things, given how successful they've been establishing a place in the market.
  - Strom 7y ago
    > How do you know they're not there now? I don't know for a fact but I have more respect towards Apple than to think that the current state is the best they could do.
    - mschaef 7y ago
      > I have more respect towards Apple than to think that the current state is the best they could do. $265B in Revenue and and $59B in profit seems to indicate they're already doing pretty well. Maybe your definition of 'best' isn't in perfect alignment with the rest of the marketplace? I get that there are things about their products you can point at and wish were done better, or at least differently. I use Apple products myself, so I have a list of these things myself. But, the thing is, this isn't unique to Apple. Have you ever been completely satisfied with a product or service? I can't really say that I have. About the best that I've ever been able to achieve is something that's a close enough approximation to my 'requirements' that I can adapt what I'm doing to make it a solution. Point is - you're never going to find something perfect. There will always be a gap and hopefully effort on both sides to close that gap. Those financial numbers aren't a perfect indication by any means, but they do signal that Apple's doing a reasonably good job of closing that gap for a reasonably large number of people willing to part with their earnings to buy Apple's products.