4 ms·

Its not just open source; a lot of people won't tell you about bugs in commercial projects or internal systems either. You need really good telemetry/monitoring

Its not just open source; a lot of people won't tell you about bugs in commercial projects or internal systems either. You need really good telemetry/monitoring infrastructure and proactively find the bugs yourself. But it isn't really practical to apply that strategy in open source projects.

bluedino 7y ago
Out of all the remote offices, 90% of the calls and complaints came from one guy, we'll call him Larry. Larry was one of those users who knew 'just enough to be dangerous', he was a power user who often bragged about going to 'computer school'. All of the help desk techs thought of him as a big thorn in their side, to them it seemed like he called about all the most insignificant issues, and it was almost like he intentionally tried to break things. This crept over to the developers and management, who would ridicule the guy. But what they didn't know was that he was probably their best tester! All of the other remote offices didn't care that things didn't work right, and assumed the bugs were already known and were never going to be fixed. I truly felt bad for the guy, he was just trying to get his job done, but ended up being the victim of both bad software, and bad technical support.
- ycombonator 7y ago
  Every company has a over enthusiastic “Larry” who went to computer school.
  - 1996 7y ago
    I wish I had had one at last company.
- pixl97 7y ago
  Yep, done IT for decades and one thing you find out is in office groups one person becomes the spokesperson for software problems. Other people report their problems to that one person who contacts technical support. At my moms job she was the leiasion for printer problems. You'd think she broke the printer a hundred times, but instead shes the only person that would call support when they happened.
xrd 7y ago
I would go further. It's worse in commercial projects. Most of the time, no one will be fired if you report a bug in an open source project. But, and this is the important part: depending on how the company is structured, reporting a bug in a closed source project will result in unplanned work. If the company takes a "everyone is responsible for all bugs, we don't assign blame" then it is likely someone will report it. If the company, whether explicitly or implicitly, is communicating that if you report the bug, you fix it, then there is a good chance you will not report it, if you own it. If you own something, you can be fired if you break it. So there is a perverse incentive to not report your own bugs, especially if you inherit a mess.
- mwfunk 7y ago
  Bugs are inevitable in software, the best anyone can do is try to minimize them. I don’t know any companies that would fire someone for introducing a regression unless it was from malice, or a level of carelessness indistinguishable from malice. Maybe this is different in other countries or industries? If you ever find yourself working for a company that would fire someone for that, definitely find a new employer ASAP, because that company is far more interested in meaningless ass-covering by management than they are in great engineering or shipping great products.
  - mmsimanga 7y ago
    Unfortunately some of live in regions where getting a new job isn't an option. The parent post raises a very valid point which I have observed in companies whose core business isn't IT but have an IT department. You end up having a fair number of non-technical managers who don't understand the intricacies of software development They don't understand that bugs do not mean you didn't test enough or you are a bad developer. Bugs delay projects or require resources and reporting them is likely to make someone look bad so there tends to be push back from IT. It isn't right but it is what it is. Its valuable to be aware of this prepare accordingly.
    - mwfunk 7y ago
      That was really what I was wondering about, it sounded strange to me but that's the bubble I'm in. Different industries or different countries might have very different norms surrounding this kind of thing. In my bubble, you have to be very careful about hiring because it can be very difficult to fire anyone at a large company. Or it just ends up being such a long, awkward, multi-stage process, that even objectively terrible engineers will get transferred elsewhere within a large company rather than get fired, because it's that much easier for the manager who wants to get rid of them. Some of it might be a cultural affinity for giving people second chances, but I think a lot of it is just the hoops you have to jump through to avoid the possibility of a future wrongful termination lawsuit if you fire somebody. But I've heard that it can be very different elsewhere, in different industries or countries. I bet it has a lot to do with the legal system too, like whether or not there's a framework of worker protection laws in place.
Geeflow 7y ago
I once discovered and fixed a bug in a commercial ordering system which prevented one of the addon-products to be ordered. I checked version control - the bug must have been there for months. Nobody noticed that the product could not be ordered. It probably wasn't the most popular product to begin with, but I am still amazed that nobody even noticed the drop in sales.
meristem 7y ago
Internal systems have their own, complex, reporting failure modes. Generally that is mixture of political concerns and internal deprioritization of tools (a cost center) vis-a-vis projects that directly produce revenue.
lawn 7y ago
Hah. We've been working on an internal project for a couple of years, with several internal projects using our API. Yet the feedback has been almost non-existent. We just have to assume it all works as intended, but we know that's too good to be true. When we grab people they say "oh yeah, I was going to report a bug but now I can't remember" or we'll see that people just stop using it because of some game-breaking bug, which they never report. Some people even say "don't use it, it's broken" and just assume we magically know about the bugs. The worst part is many of these projects are explicitly made to help test the API, and help improve it. Yet they don't report anything.
nitwit005 7y ago
They often have no way to do so. Your CTO, who signed the deal, will get emailed some support information. They'll promptly archive or delete the email. Worse still is putting your manuals behind a login. I know some companies believe they're valuable secrets for some reason, but most of your customers will simply never be able to log in to get them.
- arbie 7y ago
  There are many industries where the workflow espoused in the manual IS the secret sauce, and, once replicated, can be replaced by a much cheaper product. In these cases, you will never recoup your NRE.
  - nitwit005 7y ago
    Except, it's easy for competitors to get a copy if they care to. They can ask around for a copy, or just buy your product and get a login. Oracle has sued competitors for handing out copies of the manual, suggesting requiring a login isn't much of a barrier: https://www.bizjournals.com/sanjose/news/2013/06/18/oracle-wins-lawsuit-over-tiny.html https://www.bizjournals.com/sanjose/news/2013/06/18/oracle-w...
hinkley 7y ago
I have snuck in user studies at every place I’ve worked, just by watching coworkers fat finger things or hit the wrong button or twist in the wind. You have to learn to really watch people when they’re using your team’s code. It’s so much clearer than listening to them bitch about the three things that triggered them.
_trampeltier 7y ago
After a crash, the software often like to send a crash report. So you get just these errors. But until now, I never saw a "Report a bug" button somewhere in a menu.
D-Coder 7y ago
If it's possible, I think it's really useful to watch the users using the software for a while.
- arbie 7y ago
  I agree completely. Just observing users stumbling through what you had thought was a blindingly-obvious, low-friction workflow is an eye-opener. Sometimes, you have used your own code way too much to judge effectiveness. Having a no-agenda meeting with users is also a great idea, but it is important to set scope on feature requests before it snowballs into a request for the moon.

bluedino 7y ago

Out of all the remote offices, 90% of the calls and complaints came from one guy, we'll call him Larry. Larry was one of those users who knew 'just enough to be dangerous', he was a power user who often bragged about going to 'computer school'. All of the help desk techs thought of him as a big thorn in their side, to them it seemed like he called about all the most insignificant issues, and it was almost like he intentionally tried to break things. This crept over to the developers and management, who would ridicule the guy. But what they didn't know was that he was probably their best tester! All of the other remote offices didn't care that things didn't work right, and assumed the bugs were already known and were never going to be fixed. I truly felt bad for the guy, he was just trying to get his job done, but ended up being the victim of both bad software, and bad technical support.

ycombonator 7y ago
Every company has a over enthusiastic “Larry” who went to computer school.
- 1996 7y ago
  I wish I had had one at last company.
pixl97 7y ago
Yep, done IT for decades and one thing you find out is in office groups one person becomes the spokesperson for software problems. Other people report their problems to that one person who contacts technical support. At my moms job she was the leiasion for printer problems. You'd think she broke the printer a hundred times, but instead shes the only person that would call support when they happened.

xrd 7y ago

I would go further. It's worse in commercial projects. Most of the time, no one will be fired if you report a bug in an open source project. But, and this is the important part: depending on how the company is structured, reporting a bug in a closed source project will result in unplanned work. If the company takes a "everyone is responsible for all bugs, we don't assign blame" then it is likely someone will report it. If the company, whether explicitly or implicitly, is communicating that if you report the bug, you fix it, then there is a good chance you will not report it, if you own it. If you own something, you can be fired if you break it. So there is a perverse incentive to not report your own bugs, especially if you inherit a mess.

mwfunk 7y ago
Bugs are inevitable in software, the best anyone can do is try to minimize them. I don’t know any companies that would fire someone for introducing a regression unless it was from malice, or a level of carelessness indistinguishable from malice. Maybe this is different in other countries or industries? If you ever find yourself working for a company that would fire someone for that, definitely find a new employer ASAP, because that company is far more interested in meaningless ass-covering by management than they are in great engineering or shipping great products.
- mmsimanga 7y ago
  Unfortunately some of live in regions where getting a new job isn't an option. The parent post raises a very valid point which I have observed in companies whose core business isn't IT but have an IT department. You end up having a fair number of non-technical managers who don't understand the intricacies of software development They don't understand that bugs do not mean you didn't test enough or you are a bad developer. Bugs delay projects or require resources and reporting them is likely to make someone look bad so there tends to be push back from IT. It isn't right but it is what it is. Its valuable to be aware of this prepare accordingly.
  - mwfunk 7y ago
    That was really what I was wondering about, it sounded strange to me but that's the bubble I'm in. Different industries or different countries might have very different norms surrounding this kind of thing. In my bubble, you have to be very careful about hiring because it can be very difficult to fire anyone at a large company. Or it just ends up being such a long, awkward, multi-stage process, that even objectively terrible engineers will get transferred elsewhere within a large company rather than get fired, because it's that much easier for the manager who wants to get rid of them. Some of it might be a cultural affinity for giving people second chances, but I think a lot of it is just the hoops you have to jump through to avoid the possibility of a future wrongful termination lawsuit if you fire somebody. But I've heard that it can be very different elsewhere, in different industries or countries. I bet it has a lot to do with the legal system too, like whether or not there's a framework of worker protection laws in place.

Geeflow 7y ago

I once discovered and fixed a bug in a commercial ordering system which prevented one of the addon-products to be ordered. I checked version control - the bug must have been there for months. Nobody noticed that the product could not be ordered. It probably wasn't the most popular product to begin with, but I am still amazed that nobody even noticed the drop in sales.

meristem 7y ago

Internal systems have their own, complex, reporting failure modes. Generally that is mixture of political concerns and internal deprioritization of tools (a cost center) vis-a-vis projects that directly produce revenue.

lawn 7y ago

Hah. We've been working on an internal project for a couple of years, with several internal projects using our API. Yet the feedback has been almost non-existent. We just have to assume it all works as intended, but we know that's too good to be true. When we grab people they say "oh yeah, I was going to report a bug but now I can't remember" or we'll see that people just stop using it because of some game-breaking bug, which they never report. Some people even say "don't use it, it's broken" and just assume we magically know about the bugs. The worst part is many of these projects are explicitly made to help test the API, and help improve it. Yet they don't report anything.

nitwit005 7y ago

They often have no way to do so. Your CTO, who signed the deal, will get emailed some support information. They'll promptly archive or delete the email. Worse still is putting your manuals behind a login. I know some companies believe they're valuable secrets for some reason, but most of your customers will simply never be able to log in to get them.

arbie 7y ago
There are many industries where the workflow espoused in the manual IS the secret sauce, and, once replicated, can be replaced by a much cheaper product. In these cases, you will never recoup your NRE.
- nitwit005 7y ago
  Except, it's easy for competitors to get a copy if they care to. They can ask around for a copy, or just buy your product and get a login. Oracle has sued competitors for handing out copies of the manual, suggesting requiring a login isn't much of a barrier: https://www.bizjournals.com/sanjose/news/2013/06/18/oracle-wins-lawsuit-over-tiny.html https://www.bizjournals.com/sanjose/news/2013/06/18/oracle-w...

hinkley 7y ago

I have snuck in user studies at every place I’ve worked, just by watching coworkers fat finger things or hit the wrong button or twist in the wind. You have to learn to really watch people when they’re using your team’s code. It’s so much clearer than listening to them bitch about the three things that triggered them.

_trampeltier 7y ago

After a crash, the software often like to send a crash report. So you get just these errors. But until now, I never saw a "Report a bug" button somewhere in a menu.

D-Coder 7y ago

If it's possible, I think it's really useful to watch the users using the software for a while.

arbie 7y ago
I agree completely. Just observing users stumbling through what you had thought was a blindingly-obvious, low-friction workflow is an eye-opener. Sometimes, you have used your own code way too much to judge effectiveness. Having a no-agenda meeting with users is also a great idea, but it is important to set scope on feature requests before it snowballs into a request for the moon.