4 ms·

Please, no! I worked with a 3rd party library that had this mentality. "A bug is a bug so the assert fails and thus the code is now in an unknown state thus Th

by j1elo 2y ago

Please, no! I worked with a 3rd party library that had this mentality. "A bug is a bug so the assert fails and thus the code is now in an unknown state thus The right thing to do would be to ABORT the process execution immediately". Oh my. Just do "if (pointer)" and when that fails, error out from the smallest context possible that applies to that pointer, and nothing more than that. I.e. the real BEST thing to do is to abort the current connection. To skip the current file with an error. To fail writing that piece of memory. Whatever. But never abort (unless maybe in debug builds). The end result of this library was that we had a WebRTC server handling 100s of simultaneous video calls, and then when a single new user tripped up during connection and went through a bogus code path, the library would decide "oh something is not as I expected so I'll abort, of course!" and the whole production server was brought down with it. That kind of behavior does not help achieving high production quality and providing robust and reliable services. We ended up removing the library's runtime assertions, which meant that connections that would bug the library code would just end up failing with an error somewhere else, that could be used to just discard the attempt and try again. All in all, numbers showed it was a huge positive in stability for the service.

ratmeadow 2y ago
I think the original comment was more directed to scenarios where the precondition fail is let slide, NOTHING happens, and that's not the desired behaviour, so you have a bug. Such code exists too often and it's just poor quality. If there's anything I've learnt about error handling, it's that it must be approached with very careful consideration of what you want the app to do (conceptually) when the error occurs. Sometimes that's crash the program, sometimes it's throw an error, sometimes it's log and move on. The issue comes when devs don't want to think about this, whereby the simplest solution is absorb any error and forget about it.
- mikeschinkel 2y ago
  Agree with your assumption, but then I also know that developers often read recommendations and convert them into dogma, ignoring the specifics for when those recommendations should be applied. Early in my career I taught programmers and I was horrified to come back to one client and find they followed my advice, but not where it applied meaning that what they did was actively harmful. Ever since that, I realized when a developer makes a recommendation they need to be very explicit about when it applies otherwise they are doing more harm than good and hopefully someone will publicly challenge their recommendation to bring their unstated qualifications to light.
tsss 2y ago
Any sane person will throw an exception, which, when unhandled, will crash the program.
samiv 2y ago
So the library was full of bugs. (IMHO) The right thing to do would be to - Fix the actual bugs - Provide super visor and isolation as a protection mechanism and resiliciency. What you've now done is to essentially just hammer it quiet while sweeping all the issues under the rug and pretend everything is great when the library is actually in ill defined state. How's that possibly any better? You're probably experiencing hard to detect bugs, occasional runtime corruptions and all the fun stuff now that you'll likely never be able to fix :) "That kind of behavior does not help achieving high production quality and providing robust and reliable services." It absolutely does. When bugs are obvious and caught early they're easier to fix and this leads to higher quality more robust and reliable service. Pretending that everything is good is never the solution.
- j1elo 2y ago
  I agree with you. But with a product trying to gain a good name and its first loyal customers, a crash that brings down the complete service is a no-go, no matter the excellence in software development that one seeks theoretically. Bugs won't be fixed either if the early customers fly away. In practice, ignoring the bugs meant better numbers. Only the user connections affected by those bugs would fail, which then a retry system transparently solved. This is how the library should have worked to begin with. Bugs are always going to exist, it's the first law of software, that there is no such thing as a software without bugs. So no, I don't believe on aborting a whole application is ever an acceptable behavior for a library. Do the if(pointer) else return error, not the assert(pointer) else abort.
  - samiv 2y ago
    I'm sure you made the best judgement call given your circumstances. Of course from business perspective software quality is irrelevant. It's best achieved by letting the PR/marketing team take care of it. However, the problem is that when you start with this route that you allow buggy/incorrect code to continue running you cannot reason about your program anymore. You cannot make any smart decisions when the program is allowed to continue after hitting a bug. If I call function "foobar()" and I assume it's buggy but it continues and leaves my program in a bad state what should I do? How can I determine that the result that foobar() produced is garbage? So maybe foobar() returns a bool/success/flag value that indicates that it bugged out. But then what? Maybe I want to log this as an error but what if the logging function also has bugs in it? So maybe the logging function didn't work as intended because I called made a bug when I called it when trying to deal with the bug that happened inside foobar(). How do I propagate this error correctly without introducing more bugs to my callers who then must all do something. The fact is most programmers can barely get the "happy path" right. Even normal logical error conditions ("file not found") cause plenty of software to fail because it cannot do proper error handling and propagation. So if you let incorrect code to keep on executing nothing good will ever come out of it and there's 0 chance anyone will be able to write correct code on top of incorrect code. The point is once you start writing code to "deal with" incorrect logic its like trying to do math after a division of zero. None of the rules apply, none of the logic applies. Your program state is random garbage. All these problems disappear when you make the simple rule. You don't write code for bugs. Simply just abort. Mind you from the library perspective what is your bug might not be a bug from my perspective. In other words if I provide a library for random 3rd parties I can assume they will use it wrong. Therefore their buggy code is what I must expect and return some error value etc. But if I'm calling code that I wrote from my own code I don't write a single line of code for bugs. I simply assert and abort. And to your last point of bugs always existing. Yes I agree, and the best we can do is to squash them as soon as possible and make them loud and clear and as easy to debug as possible (i.e. direct callstack / core dump). Not doing this does not fix them but smply makes them harder to fix.
Mawr 2y ago
Oh my indeed. > That kind of behavior does not help achieving high production quality and providing robust and reliable services. Right, so since it's the messenger (assertion) that brought news of defeat in battle, you kill the messenger instead of trying to win the next battle. Your problem is that the entire business seems to hinge on a single point of failure - if the server binary dies, you lose customers. This has nothing to do with what some library does or does not do, it's just a horribly designed system. That crash should have made you all stop and think real hard about what you're doing because clearly it's not working: 1. Don't write bugs in the first place. No I'm not kidding. Your tests clearly suck so fix them, make sure you do proper PR reviews, make sure to leverage your type system to the fullest, use all available tooling, etc. 2. Design your system be resilient to the server binary dying because I assure you, there's nothing you can do to prevent it from happening for one reason or another.

by j1elo 2y ago

ratmeadow 2y ago

I think the original comment was more directed to scenarios where the precondition fail is let slide, NOTHING happens, and that's not the desired behaviour, so you have a bug. Such code exists too often and it's just poor quality. If there's anything I've learnt about error handling, it's that it must be approached with very careful consideration of what you want the app to do (conceptually) when the error occurs. Sometimes that's crash the program, sometimes it's throw an error, sometimes it's log and move on. The issue comes when devs don't want to think about this, whereby the simplest solution is absorb any error and forget about it.

mikeschinkel 2y ago
Agree with your assumption, but then I also know that developers often read recommendations and convert them into dogma, ignoring the specifics for when those recommendations should be applied. Early in my career I taught programmers and I was horrified to come back to one client and find they followed my advice, but not where it applied meaning that what they did was actively harmful. Ever since that, I realized when a developer makes a recommendation they need to be very explicit about when it applies otherwise they are doing more harm than good and hopefully someone will publicly challenge their recommendation to bring their unstated qualifications to light.

tsss 2y ago

Any sane person will throw an exception, which, when unhandled, will crash the program.

samiv 2y ago

So the library was full of bugs. (IMHO) The right thing to do would be to - Fix the actual bugs - Provide super visor and isolation as a protection mechanism and resiliciency. What you've now done is to essentially just hammer it quiet while sweeping all the issues under the rug and pretend everything is great when the library is actually in ill defined state. How's that possibly any better? You're probably experiencing hard to detect bugs, occasional runtime corruptions and all the fun stuff now that you'll likely never be able to fix :) "That kind of behavior does not help achieving high production quality and providing robust and reliable services." It absolutely does. When bugs are obvious and caught early they're easier to fix and this leads to higher quality more robust and reliable service. Pretending that everything is good is never the solution.

j1elo 2y ago
I agree with you. But with a product trying to gain a good name and its first loyal customers, a crash that brings down the complete service is a no-go, no matter the excellence in software development that one seeks theoretically. Bugs won't be fixed either if the early customers fly away. In practice, ignoring the bugs meant better numbers. Only the user connections affected by those bugs would fail, which then a retry system transparently solved. This is how the library should have worked to begin with. Bugs are always going to exist, it's the first law of software, that there is no such thing as a software without bugs. So no, I don't believe on aborting a whole application is ever an acceptable behavior for a library. Do the if(pointer) else return error, not the assert(pointer) else abort.
- samiv 2y ago
  I'm sure you made the best judgement call given your circumstances. Of course from business perspective software quality is irrelevant. It's best achieved by letting the PR/marketing team take care of it. However, the problem is that when you start with this route that you allow buggy/incorrect code to continue running you cannot reason about your program anymore. You cannot make any smart decisions when the program is allowed to continue after hitting a bug. If I call function "foobar()" and I assume it's buggy but it continues and leaves my program in a bad state what should I do? How can I determine that the result that foobar() produced is garbage? So maybe foobar() returns a bool/success/flag value that indicates that it bugged out. But then what? Maybe I want to log this as an error but what if the logging function also has bugs in it? So maybe the logging function didn't work as intended because I called made a bug when I called it when trying to deal with the bug that happened inside foobar(). How do I propagate this error correctly without introducing more bugs to my callers who then must all do something. The fact is most programmers can barely get the "happy path" right. Even normal logical error conditions ("file not found") cause plenty of software to fail because it cannot do proper error handling and propagation. So if you let incorrect code to keep on executing nothing good will ever come out of it and there's 0 chance anyone will be able to write correct code on top of incorrect code. The point is once you start writing code to "deal with" incorrect logic its like trying to do math after a division of zero. None of the rules apply, none of the logic applies. Your program state is random garbage. All these problems disappear when you make the simple rule. You don't write code for bugs. Simply just abort. Mind you from the library perspective what is your bug might not be a bug from my perspective. In other words if I provide a library for random 3rd parties I can assume they will use it wrong. Therefore their buggy code is what I must expect and return some error value etc. But if I'm calling code that I wrote from my own code I don't write a single line of code for bugs. I simply assert and abort. And to your last point of bugs always existing. Yes I agree, and the best we can do is to squash them as soon as possible and make them loud and clear and as easy to debug as possible (i.e. direct callstack / core dump). Not doing this does not fix them but smply makes them harder to fix.

Mawr 2y ago

Oh my indeed. > That kind of behavior does not help achieving high production quality and providing robust and reliable services. Right, so since it's the messenger (assertion) that brought news of defeat in battle, you kill the messenger instead of trying to win the next battle. Your problem is that the entire business seems to hinge on a single point of failure - if the server binary dies, you lose customers. This has nothing to do with what some library does or does not do, it's just a horribly designed system. That crash should have made you all stop and think real hard about what you're doing because clearly it's not working: 1. Don't write bugs in the first place. No I'm not kidding. Your tests clearly suck so fix them, make sure you do proper PR reviews, make sure to leverage your type system to the fullest, use all available tooling, etc. 2. Design your system be resilient to the server binary dying because I assure you, there's nothing you can do to prevent it from happening for one reason or another.