8 ms·

Defensive Programming

106 points by tyhoff 6y ago

andi999 6y ago
Well... in standard linux malloc never fails... not sure if it is a good thing though..
- mcguire 6y ago
  Not really, no. My first experience with that behavior was with AIX in the early '90s: allocation always succeeds (unless you've exhausted your address space) because it is backed by virtual memory. If you do run low on physical memory, every process gets a SIGDANGER and is expected to free up memory. If that fails, the kernel starts sending SIGKILLs. Unfortunately, exactly none of the normal utilities understood the protocol and upon running low, the kernel just started shooting processes in the head until the situation improved. For some reason, one of the first processes chosen was usually inetd, meaning you could no longer remotely log in to see what was going on. When I complained, I was told that the machines were frequently used for scientific computing, where they would allocate giant arrays which were then used sparsely, so it was the intended behavior. And that the X server at the time made use of the behavior. So, ... yech.
- q3k 6y ago
  It does though? $ uname -a Linux mimeomia 5.9.11 #1-NixOS SMP Tue Nov 24 12:39:15 UTC 2020 x86_64 GNU/Linux $ cat main.c #include <stdio.h> #include <stdlib.h> int main() { printf("malloc: %lx\n", malloc(128LL << 30)); // 128GiB perror("malloc err"); } $ gcc main.c -o main && ./main malloc: 0 malloc err: Cannot allocate memory
  - dragontamer 6y ago
    The original poster is almost certainly talking about Linux's overcommit policy: http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommit.html http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommi... You can't adopt a defensive / offensive stance with regards to out-of-memory issues, because on Linux... malloc doesn't fail when you run out of physical memory. Your "error" occurs on a pointer-dereference, which could be anywhere in your program. Once more memory is dereferenced than the system has capacity for, the OOM-killer is activated. There's no defensive stance possible, aside from setting overcommit to "Type2" (disabling the overcommit behavior of the Kernel). But that's somewhat non-standard, and possibly non-helpful, since some applications depend on overcommit.
    - theamk 6y ago
      Disabling overcommit is totally feasible on some embedded boxes. And if you don’t want this for some reason, you can also mlock/madvice/manually touch all the memory to get errors right away as well. Yes, overcommit makes things tricker, but it is hardly “impossible”
      - dragontamer 6y ago
        > you can also mlock/madvice/manually touch all the memory to get errors right away as well. All this does is run the OOM killer, which doesn't necessarily kill your process. Maybe its a process you didn't care about, maybe its a process you actually are communicating with and suddenly you're now in a weird deadlock state. Etc. etc. Even if YOU touch all your memory like you're supposed to, there's no guarantee that OTHER processes have touched their memory. So you touch memory unnecessarily: which means you're now officially reserving RAM (wasting it). Then, some OTHER process touches their memory and runs the OOM-killer. That wouldn't have happened if you just didn't touch your RAM. ----------- If Linux Overcommit is on default, you live with it and die with it. No point trying to fight OS-policy. Change the policy (aka: Overcommit to 2), and then carefully select applications to run with that follow the same overcommit policy as you.
        andi999 6y ago
        Exactly. The misunderstanding a lot of ppl have about the OOM killer is, is that the process touching memory which cannot be fulfilled will be killed. This is not the case (for all practial reasonings one shd assume it kills random processes)
        theamk 6y ago
        If you have an idea where do OOMs come from, you can (and should!) use "oom_score_adj"/"oom_adj" so that OOM kills processes are not random. There are many interesting strategies here: - Set your watchdog and logging to "never kill". Set watchdog to reboot on OOM, after uploading error report. This way, if the system OOMs, you can log the problems to your logging server, and then watchdog will reboot the system to recover. - If you have a "UI" process which can also eat all the memory, set it to "kill me first". Yes, the UI will crash, but the rest of system will restart it, and your updater would still work. - If you have separate resource-intensive process, you can set it to "kill me first" -- then the rest of the system can stay up and maybe even show a descriptive error message. - The latter approach works pretty nicely for general systems. Is your OOM killer activating because your build system started too many gcc instances? run it with "kill me first" setting -- it inherits to children processes, so you only build processes would be killed, the rest of the system (UI, browser, etc..) would all stay alive.
curtis3389 6y ago
In Erlang, instead of calling it Offensive Programming they call it "Let it Crash". A very valuable concept either way: if a function encounters an error and can't somehow resolve it and still do its work successfully, then it should fail and not do something else.
- macintux 6y ago
  Agreed. Among the other fantastic things about Erlang, the fact that = is effectively an assertion means that unlike other languages where you turn off assertions in production, Erlang retains them in every environment.
- pjbk 6y ago
  Based on my experience, many, many, many headaches would be saved if this advice was followed more closely. IMHO defensive techniques including recovery should be placed in runtime monitors and supervisors, not in underpinning code which should have a minimum set of behaviors. It should also be kept as simple as possible to avoid code bloat and help reviewers and tools like static analyzers do their job. Old computing hardware was designed like that. Then software attempted to replicate hardware "exceptions", however not realizing that the model was intrinsically different and completely exploding in intricacies dealing with interrupts, stacks, memory management and threads. In a similar way, supervisors should be as decoupled as possible from the target code to provide reliability and avoid overall system complexity, with their own set of narrowed down requirements. Sadly our industry is plagued with historical, ad-hoc programming techniques that are not relevant anymore or were plain empirical bad advice in first place. For example, blindly relying on stateful paradigms, or side effects by design or as an acceptable consequence of certain programming idioms.
- alpaca128 6y ago
  And it's valuable in other languages as well. Not many things are as frustrating as silent failures in some library. And it's still one of my biggest gripes with JavaScript that when it encounters an error there's a good chance it just keeps on running in some undefined state, hiding the root cause of the problem.
Geminidog 6y ago
I was a python programmer before I became a c++ developer and believe it or not despite lack of type checking, big python apps have much less bugs then a comparable c++ app due in no part to offensive programming features built into python. In other words you don’t need to offensively program in python, it’s a given. Firmware is just a pain in the ass to deal with. Wish there was a better way.
- ben-schaaf 6y ago
  > believe it or not despite lack of type checking, big python apps have much less bugs then a comparable c++ app Despite the lack of static typing, python has memory safety and a GIL; eliminating a much larger class of bugs, so it's not that hard to believe.
  - Geminidog 6y ago
    Additionally clear error messages similar to offensive programming. Any runtime error in python reads like a compile time error with the interpreter giving you the exact location where the error occured and what error occured. You'd be surprised how many c++ programmers think python is a terrible language just simply due to the lack of type checking and indentations. A lot of them have no idea how much easier and how much more productive life is outside of the c++ world. Take a look at the other replier. He's thinking "perhaps" it's true, he has no idea.
- rutthenut 6y ago
  Firmware requires serious design, code and test regimes to deliver safe, and reliable components. That same pain-in-the-arse requirement is true of other software, albeit higher-level platforms can perhaps save your arse when bugs are encountered
  - Geminidog 6y ago
    No. This shows a very narrow regime of experience. Testing is not the only way to verify software additionally the error surface area of certain languages is simply much much smaller in most languages outside of c++. This had nothing to do with a language saving your arse. This has everything to do with the simple non-existence of a type of bug in other languages, for example memory safety in rust and python. Additionally c++ often simply does not tell you where an error occurs and you have to use a coredump to figure everything out. It's not an offensive programming strategy by default. Python on the other hand makes every single type of error read like a c++ syntax error giving you a line number and an error type by default. Another thing to consider is raw speed and lower memory requirements. These are the only advantages of c++. If your hardware isn't strict on the above requirements consider micropython. I'm finding c++ programmers have this very narrow and cocky view of the world. They turn their nose up at languages like python without truly experiencing how much life is easier and more productive in those worlds.
greatgib 6y ago
The arguments of this article are ridiculously bad in my opinion. Defensive programming does not mean that you ignore errors. It means that the dev should always expect the worse and decide right away how to deal with it early. Imagine you have a car that suddenly block and explode each time an unexpected error is encountered. So that you are sure to catch it. So, you are driving on the highway, then a there a window fuse that blow and because of that your car decide to brake and explode... Especially in hardware and embedded, you have to expect the unexpected to happen. Imagine the car will suddenly stop when a lightbulb break or there is a shortcut in windows control. Sure you will know, but you will have 300 000 cars all around the world bricked!
- uxp100 6y ago
  I believe "offensive" programming is heavily used in automotive. If there is an issue with the connection to the camera, it is better to show nothing than corrupted and potentially misleading images. Better not to try to guess about uncertain throttle position. Defensive programming can sometimes look like silently ignoring errors. A haphazard test of defensively written code (that, for example falls back to a different encryption scheme or io device) could pass incorrectly. But there is a place for defensive programming too. I think most important is being aware of which is which.
- tyhoff 6y ago
  Author here! I don't think we are in disagreement, just mostly in different interpretations. You are talking about a car, with likely 200-300 embedded systems backed into it. If the light sensor or the braking module goes into a completely unexpected state where it can no longer be communicated with, it has corruption, etc, you need to do something about it. The solution isn't to reboot the car, it's to reboot the individual module and quickly get it back into working order. Rebooting the braking or lighting module should take on the order of milliseconds (if done correctly), and that is much, much better than having a piece of software in a corrupted state for seconds/minutes/hours. In a car, there is a mothership board that is orchestrating everything, pinging every module, and ensuring everything is in working order. Just like a kubernetes setup, if any one piece fails to ping or looks off, just reboot it and get it working ASAP.
- 6y ago
uxp100 6y ago
As a side note, this article suggests using assert. Fine, but make sure you know what your assert does, and when it does that, if you have a debug and release build (which I think you probably shouldn't.) I've spent some time with "offensive" programming in the context of both security (like key leakage) and safety (like firmware integrity). It was just kinda funny thinking about meetings spent determining what we could do if i2c or something wasn't responding, and different approaches to try and fix it, and then a year later going back and saying, ok, if something isn't working, we've got an easy answer. Lockdown and halt boot. Anything seems weird, just stop trying.
- iso8859-1 6y ago
  Runtime assertions are easily inserted, but why not change the types, make that case impossible? Dependent types can make that happen. Static assertions are enabled by dependent typing. But unlike dependent typing, static assertions are only assertions, for example they do not enable you to write a function that returns different types based on its arguments. Static assertions do not allow you to write a compile-time vector type.
  - AnimalMuppet 6y ago
    "Making that case impossible" is hard for i2c not responding. You can represent "not responding" in a way that isn't confused with real data (even if zero-length), and you can refuse to go down the path that expects data. But no dependent type can make it respond. And you're still stuck with the questions: "What do I do then? How do I handle it? Can my program continue at all? How do I tell the user what's wrong?" As far as I can see, dependent types don't help with the larger problem at all.
    - lmm 6y ago
      Types (you don't even need full dependent types) let you check your reasoning about why something is true, which is IME the core of the larger problem. Most of the time the bug doesn't happen because everyone forgot that it's possible for i2c to not respond, it's because component a thought that it should pass the i2c response straight through to component b and component b would handle the i2c not responding, whereas component b thought that component a had already handled the case where the i2c didn't respond (or rather, component b is a pass-through middleware that calls component c that wraps component d and component d thought that i2c not responding had already been handled). Types absolutely help with that.
mauvehaus 6y ago
I've worked in both embedded and in the Windows file system stack. What's unsaid in this article is that your ability to break noisily varies a lot based on what the end user's expectations are and how much of the device you own. If you're building a single-purpose embedded device, sure, you can probably halt and catch fire if its use-case allows doing so safely. This might involve making sure the outside world is safely brought to a halt, but that might be feasible if you're building a device that works on a production line. On the other hand, you almost certainly don't want to bugcheck some random big corporate file server that might be doing goodness knows what else to get a crash dump. At least not without the blessing of the client. Like so many other things in engineering, how to handle errors depends greatly on the use case, the users, and the maintainers (which might or might not overlap with the users). You can't baldly claim that one approach or another is best without understanding those.
tyhoff 6y ago
Author here. Firmware, for better and worse, is stuck in C/C++ land, so many of the topics here are essential to keep a complex, multi-MCU, multi-board setup in working order. I'm actually really curious what you all do in C/C++ to prevent bad operations from ever being performed in the first place. For example, should we just change our internal malloc / free to use double pointers instead? bool malloc(size_t n, void **ptr) { *ptr = <memory> } void free(void **ptr) { ... <free> *ptr = NULL } This way, if anyone actually tries to use that pointer, it will crash the system (hardfault), instead of potentially using corrupted memory, which IMO is much worse.
- ffffjfjjfj 6y ago
  This gives a false sense of security, since the most common use-after-free bugs are if you kept a pointer around. Your method doesn't stop things like this: void *p1, *p2; malloc(100, &p1); p2 = p1; ... free(&p1); *p2; // oops! Obviously its trivial in an example like this, but when p2 leaves the lexical scope then there's a problem.
  - deleted 6y ago
    [deleted]
  - btrask 6y ago
    If you are transferring ownership, you would do p2 = p1; p1 = NULL; However if you are intentionally doing multiple ownership, then yes you can still have problems.
- dragontamer 6y ago
  > I'm actually really curious what you all do in C/C++ to prevent bad operations from ever being performed in the first place. 1. Go as far as you can with static memory. Don't call the allocator: its slow, sometimes requires inter-thread communications. 2. Dynamic uses often can use the stack: almost always in L1 of your local core. Stack allocations are self-cleaning, just don't pass those pointers to anyone else. 3. unique_ptr<blah> covers most of the true dynamic issues. 4. shared_ptr<blah> covers the rest of them. --------- The above advice is just general-purpose / low-performance code. If you're entering high performance coding (aka: you're actually worried about fragmentation, memory sizes, and such), then things get more complicated. Embedded is often RAM-restricted, so you end up in this complicated place where you need to think of highly-efficient techniques. That's almost always a complexity / resource tradeoff. Efficient techniques are just innately more complex than general purpose techniques. Try not to reach for efficient coding unless you know its something you need to do
anordal 6y ago
Excellent writeup and intro to the underdiscussed topic of offensive programming, and how DefP and OffP are not contradictory (any more than graceful fallback and rejection of bugs has to be), but rather applies to different categories of errors, with that crucial line of defense inbetween. http://en.wikipedia.org/wiki/Offensive%20programming http://en.wikipedia.org/wiki/Offensive%20programming

106 points by tyhoff 6y ago

andi999 6y ago

Well... in standard linux malloc never fails... not sure if it is a good thing though..

mcguire 6y ago
Not really, no. My first experience with that behavior was with AIX in the early '90s: allocation always succeeds (unless you've exhausted your address space) because it is backed by virtual memory. If you do run low on physical memory, every process gets a SIGDANGER and is expected to free up memory. If that fails, the kernel starts sending SIGKILLs. Unfortunately, exactly none of the normal utilities understood the protocol and upon running low, the kernel just started shooting processes in the head until the situation improved. For some reason, one of the first processes chosen was usually inetd, meaning you could no longer remotely log in to see what was going on. When I complained, I was told that the machines were frequently used for scientific computing, where they would allocate giant arrays which were then used sparsely, so it was the intended behavior. And that the X server at the time made use of the behavior. So, ... yech.
q3k 6y ago
It does though? $ uname -a Linux mimeomia 5.9.11 #1-NixOS SMP Tue Nov 24 12:39:15 UTC 2020 x86_64 GNU/Linux $ cat main.c #include <stdio.h> #include <stdlib.h> int main() { printf("malloc: %lx\n", malloc(128LL << 30)); // 128GiB perror("malloc err"); } $ gcc main.c -o main && ./main malloc: 0 malloc err: Cannot allocate memory
- dragontamer 6y ago
  The original poster is almost certainly talking about Linux's overcommit policy: http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommit.html http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommi... You can't adopt a defensive / offensive stance with regards to out-of-memory issues, because on Linux... malloc doesn't fail when you run out of physical memory. Your "error" occurs on a pointer-dereference, which could be anywhere in your program. Once more memory is dereferenced than the system has capacity for, the OOM-killer is activated. There's no defensive stance possible, aside from setting overcommit to "Type2" (disabling the overcommit behavior of the Kernel). But that's somewhat non-standard, and possibly non-helpful, since some applications depend on overcommit.
  - theamk 6y ago
    Disabling overcommit is totally feasible on some embedded boxes. And if you don’t want this for some reason, you can also mlock/madvice/manually touch all the memory to get errors right away as well. Yes, overcommit makes things tricker, but it is hardly “impossible”
    - dragontamer 6y ago
      > you can also mlock/madvice/manually touch all the memory to get errors right away as well. All this does is run the OOM killer, which doesn't necessarily kill your process. Maybe its a process you didn't care about, maybe its a process you actually are communicating with and suddenly you're now in a weird deadlock state. Etc. etc. Even if YOU touch all your memory like you're supposed to, there's no guarantee that OTHER processes have touched their memory. So you touch memory unnecessarily: which means you're now officially reserving RAM (wasting it). Then, some OTHER process touches their memory and runs the OOM-killer. That wouldn't have happened if you just didn't touch your RAM. ----------- If Linux Overcommit is on default, you live with it and die with it. No point trying to fight OS-policy. Change the policy (aka: Overcommit to 2), and then carefully select applications to run with that follow the same overcommit policy as you.
      - andi999 6y ago
        Exactly. The misunderstanding a lot of ppl have about the OOM killer is, is that the process touching memory which cannot be fulfilled will be killed. This is not the case (for all practial reasonings one shd assume it kills random processes)
        theamk 6y ago
        If you have an idea where do OOMs come from, you can (and should!) use "oom_score_adj"/"oom_adj" so that OOM kills processes are not random. There are many interesting strategies here: - Set your watchdog and logging to "never kill". Set watchdog to reboot on OOM, after uploading error report. This way, if the system OOMs, you can log the problems to your logging server, and then watchdog will reboot the system to recover. - If you have a "UI" process which can also eat all the memory, set it to "kill me first". Yes, the UI will crash, but the rest of system will restart it, and your updater would still work. - If you have separate resource-intensive process, you can set it to "kill me first" -- then the rest of the system can stay up and maybe even show a descriptive error message. - The latter approach works pretty nicely for general systems. Is your OOM killer activating because your build system started too many gcc instances? run it with "kill me first" setting -- it inherits to children processes, so you only build processes would be killed, the rest of the system (UI, browser, etc..) would all stay alive.

curtis3389 6y ago

In Erlang, instead of calling it Offensive Programming they call it "Let it Crash". A very valuable concept either way: if a function encounters an error and can't somehow resolve it and still do its work successfully, then it should fail and not do something else.

macintux 6y ago
Agreed. Among the other fantastic things about Erlang, the fact that = is effectively an assertion means that unlike other languages where you turn off assertions in production, Erlang retains them in every environment.
pjbk 6y ago
Based on my experience, many, many, many headaches would be saved if this advice was followed more closely. IMHO defensive techniques including recovery should be placed in runtime monitors and supervisors, not in underpinning code which should have a minimum set of behaviors. It should also be kept as simple as possible to avoid code bloat and help reviewers and tools like static analyzers do their job. Old computing hardware was designed like that. Then software attempted to replicate hardware "exceptions", however not realizing that the model was intrinsically different and completely exploding in intricacies dealing with interrupts, stacks, memory management and threads. In a similar way, supervisors should be as decoupled as possible from the target code to provide reliability and avoid overall system complexity, with their own set of narrowed down requirements. Sadly our industry is plagued with historical, ad-hoc programming techniques that are not relevant anymore or were plain empirical bad advice in first place. For example, blindly relying on stateful paradigms, or side effects by design or as an acceptable consequence of certain programming idioms.
alpaca128 6y ago
And it's valuable in other languages as well. Not many things are as frustrating as silent failures in some library. And it's still one of my biggest gripes with JavaScript that when it encounters an error there's a good chance it just keeps on running in some undefined state, hiding the root cause of the problem.

Geminidog 6y ago

I was a python programmer before I became a c++ developer and believe it or not despite lack of type checking, big python apps have much less bugs then a comparable c++ app due in no part to offensive programming features built into python. In other words you don’t need to offensively program in python, it’s a given. Firmware is just a pain in the ass to deal with. Wish there was a better way.

ben-schaaf 6y ago
> believe it or not despite lack of type checking, big python apps have much less bugs then a comparable c++ app Despite the lack of static typing, python has memory safety and a GIL; eliminating a much larger class of bugs, so it's not that hard to believe.
- Geminidog 6y ago
  Additionally clear error messages similar to offensive programming. Any runtime error in python reads like a compile time error with the interpreter giving you the exact location where the error occured and what error occured. You'd be surprised how many c++ programmers think python is a terrible language just simply due to the lack of type checking and indentations. A lot of them have no idea how much easier and how much more productive life is outside of the c++ world. Take a look at the other replier. He's thinking "perhaps" it's true, he has no idea.
rutthenut 6y ago
Firmware requires serious design, code and test regimes to deliver safe, and reliable components. That same pain-in-the-arse requirement is true of other software, albeit higher-level platforms can perhaps save your arse when bugs are encountered
- Geminidog 6y ago
  No. This shows a very narrow regime of experience. Testing is not the only way to verify software additionally the error surface area of certain languages is simply much much smaller in most languages outside of c++. This had nothing to do with a language saving your arse. This has everything to do with the simple non-existence of a type of bug in other languages, for example memory safety in rust and python. Additionally c++ often simply does not tell you where an error occurs and you have to use a coredump to figure everything out. It's not an offensive programming strategy by default. Python on the other hand makes every single type of error read like a c++ syntax error giving you a line number and an error type by default. Another thing to consider is raw speed and lower memory requirements. These are the only advantages of c++. If your hardware isn't strict on the above requirements consider micropython. I'm finding c++ programmers have this very narrow and cocky view of the world. They turn their nose up at languages like python without truly experiencing how much life is easier and more productive in those worlds.

greatgib 6y ago

The arguments of this article are ridiculously bad in my opinion. Defensive programming does not mean that you ignore errors. It means that the dev should always expect the worse and decide right away how to deal with it early. Imagine you have a car that suddenly block and explode each time an unexpected error is encountered. So that you are sure to catch it. So, you are driving on the highway, then a there a window fuse that blow and because of that your car decide to brake and explode... Especially in hardware and embedded, you have to expect the unexpected to happen. Imagine the car will suddenly stop when a lightbulb break or there is a shortcut in windows control. Sure you will know, but you will have 300 000 cars all around the world bricked!

uxp100 6y ago
I believe "offensive" programming is heavily used in automotive. If there is an issue with the connection to the camera, it is better to show nothing than corrupted and potentially misleading images. Better not to try to guess about uncertain throttle position. Defensive programming can sometimes look like silently ignoring errors. A haphazard test of defensively written code (that, for example falls back to a different encryption scheme or io device) could pass incorrectly. But there is a place for defensive programming too. I think most important is being aware of which is which.
tyhoff 6y ago
Author here! I don't think we are in disagreement, just mostly in different interpretations. You are talking about a car, with likely 200-300 embedded systems backed into it. If the light sensor or the braking module goes into a completely unexpected state where it can no longer be communicated with, it has corruption, etc, you need to do something about it. The solution isn't to reboot the car, it's to reboot the individual module and quickly get it back into working order. Rebooting the braking or lighting module should take on the order of milliseconds (if done correctly), and that is much, much better than having a piece of software in a corrupted state for seconds/minutes/hours. In a car, there is a mothership board that is orchestrating everything, pinging every module, and ensuring everything is in working order. Just like a kubernetes setup, if any one piece fails to ping or looks off, just reboot it and get it working ASAP.
6y ago

uxp100 6y ago

As a side note, this article suggests using assert. Fine, but make sure you know what your assert does, and when it does that, if you have a debug and release build (which I think you probably shouldn't.) I've spent some time with "offensive" programming in the context of both security (like key leakage) and safety (like firmware integrity). It was just kinda funny thinking about meetings spent determining what we could do if i2c or something wasn't responding, and different approaches to try and fix it, and then a year later going back and saying, ok, if something isn't working, we've got an easy answer. Lockdown and halt boot. Anything seems weird, just stop trying.

iso8859-1 6y ago
Runtime assertions are easily inserted, but why not change the types, make that case impossible? Dependent types can make that happen. Static assertions are enabled by dependent typing. But unlike dependent typing, static assertions are only assertions, for example they do not enable you to write a function that returns different types based on its arguments. Static assertions do not allow you to write a compile-time vector type.
- AnimalMuppet 6y ago
  "Making that case impossible" is hard for i2c not responding. You can represent "not responding" in a way that isn't confused with real data (even if zero-length), and you can refuse to go down the path that expects data. But no dependent type can make it respond. And you're still stuck with the questions: "What do I do then? How do I handle it? Can my program continue at all? How do I tell the user what's wrong?" As far as I can see, dependent types don't help with the larger problem at all.
  - lmm 6y ago
    Types (you don't even need full dependent types) let you check your reasoning about why something is true, which is IME the core of the larger problem. Most of the time the bug doesn't happen because everyone forgot that it's possible for i2c to not respond, it's because component a thought that it should pass the i2c response straight through to component b and component b would handle the i2c not responding, whereas component b thought that component a had already handled the case where the i2c didn't respond (or rather, component b is a pass-through middleware that calls component c that wraps component d and component d thought that i2c not responding had already been handled). Types absolutely help with that.

mauvehaus 6y ago

I've worked in both embedded and in the Windows file system stack. What's unsaid in this article is that your ability to break noisily varies a lot based on what the end user's expectations are and how much of the device you own. If you're building a single-purpose embedded device, sure, you can probably halt and catch fire if its use-case allows doing so safely. This might involve making sure the outside world is safely brought to a halt, but that might be feasible if you're building a device that works on a production line. On the other hand, you almost certainly don't want to bugcheck some random big corporate file server that might be doing goodness knows what else to get a crash dump. At least not without the blessing of the client. Like so many other things in engineering, how to handle errors depends greatly on the use case, the users, and the maintainers (which might or might not overlap with the users). You can't baldly claim that one approach or another is best without understanding those.

tyhoff 6y ago

Author here. Firmware, for better and worse, is stuck in C/C++ land, so many of the topics here are essential to keep a complex, multi-MCU, multi-board setup in working order. I'm actually really curious what you all do in C/C++ to prevent bad operations from ever being performed in the first place. For example, should we just change our internal malloc / free to use double pointers instead? bool malloc(size_t n, void **ptr) { *ptr = <memory> } void free(void **ptr) { ... <free> *ptr = NULL } This way, if anyone actually tries to use that pointer, it will crash the system (hardfault), instead of potentially using corrupted memory, which IMO is much worse.

ffffjfjjfj 6y ago
This gives a false sense of security, since the most common use-after-free bugs are if you kept a pointer around. Your method doesn't stop things like this: void *p1, *p2; malloc(100, &p1); p2 = p1; ... free(&p1); *p2; // oops! Obviously its trivial in an example like this, but when p2 leaves the lexical scope then there's a problem.
- deleted 6y ago
  [deleted]
- btrask 6y ago
  If you are transferring ownership, you would do p2 = p1; p1 = NULL; However if you are intentionally doing multiple ownership, then yes you can still have problems.
dragontamer 6y ago
> I'm actually really curious what you all do in C/C++ to prevent bad operations from ever being performed in the first place. 1. Go as far as you can with static memory. Don't call the allocator: its slow, sometimes requires inter-thread communications. 2. Dynamic uses often can use the stack: almost always in L1 of your local core. Stack allocations are self-cleaning, just don't pass those pointers to anyone else. 3. unique_ptr<blah> covers most of the true dynamic issues. 4. shared_ptr<blah> covers the rest of them. --------- The above advice is just general-purpose / low-performance code. If you're entering high performance coding (aka: you're actually worried about fragmentation, memory sizes, and such), then things get more complicated. Embedded is often RAM-restricted, so you end up in this complicated place where you need to think of highly-efficient techniques. That's almost always a complexity / resource tradeoff. Efficient techniques are just innately more complex than general purpose techniques. Try not to reach for efficient coding unless you know its something you need to do

anordal 6y ago

Excellent writeup and intro to the underdiscussed topic of offensive programming, and how DefP and OffP are not contradictory (any more than graceful fallback and rejection of bugs has to be), but rather applies to different categories of errors, with that crucial line of defense inbetween. http://en.wikipedia.org/wiki/Offensive%20programming http://en.wikipedia.org/wiki/Offensive%20programming