5 ms·

To respond to some of the comments. It is not that there is anything intrinsically wrong with these functions. You can technically use all of them and I have b

To respond to some of the comments. It is not that there is anything intrinsically wrong with these functions. You can technically use all of them and I have been using all of them, safely, for decades. The issue is they are huge traps to the point that in a larger piece of software one can say "well, it's just not worth it". You can go much, much, much further than that. In couple embedded projects I worked some of the rules were: * dynamic allocation after application has started is banned -- any heap buffers and data structures must be allocated at the start of the application and after that any allocation is a compile time error, * any constructs that would prevent statically calculating stack usage were banned (for example any form of recursion except when exact recursion depth is ensured statically), * any locks were banned, * absolutely every data structure must have size ensured, in a simple way, beyond any reasonable doubt, etc.

xondono 6y ago
Anything enforcing MISRA has essentially (almost) no way of allocating memory at runtime.
- fsociety 6y ago
  It’s funny, I worked exclusively with MISRA at the start of my career. Eventually I started a job at a FAANG and received quizzical comments on why I implemented a memory arena. The argument was to allocate memory freely and let it pool memory as necessary. Fair enough, it was simpler and fit the standard expectation of development. The issue is that if you talk with the allocator team they complain of not being able to fix performance issues fast enough due to allocations firing off left and right in the middle of a request. I never realized that my view of C programming is heavily influenced by MISRA until your comment. I know game engine programming follows a similar, perhaps unspoken, convention.
  - munchbunny 6y ago
    The lack of runtime allocations in game engine programming comes from a different motivation: allocations are expensive, garbage collections are expensive, cache coherency matters, and you're chucking around a lot of very similar looking objects, so... object pools!
    - orwin 6y ago
      Yeah, the first time we coded a scroller shooting game with my friend (at school), we were baffled that our terminal-based scroller lagged more than the raycaster we did two weeks prior. Was it a C vs C++ thing? Turns out, creating then destroying every single missile/enemy was extremely costly
  - hctaw 6y ago
    Custom allocators are quite common, it's not an arcane convention. I think the rule of thumb is preallocate until it gets questionable in complexity, then write your custom allocator - and really it's only applicable to code with a real-time deadline (hard or soft). Otherwise the system allocator is going to be a lot smarter than yours once it leaves microbenchmarks.
- watersb 6y ago
  MISRA: "Motor Industry Software Reliability Association" https://en.m.wikipedia.org/wiki/MISRA_C https://en.m.wikipedia.org/wiki/MISRA_C
closeparen 6y ago
How often does the dynamic allocation rule lead to an ad-hoc allocator appearing inside the program? Also doesn’t the OS lie? I thought the memory wasn’t really physically assigned until first use.
- syncsynchalt 6y ago
  In my experience dynamic allocation is banned in either (a) small embedded environments or (b) high scrutiny environments (soft realtime, safety critical, etc). In both cases the project size is small enough, or the scrutiny is high enough that the ad-hoc allocator doesn't develop. The environment is also simple enough that the memory cheats you're thinking of don't exist (or you can squash them by touching all allocated memory up front).
- rightbyte 6y ago
  > How often does the dynamic allocation rule lead to an ad-hoc allocator appearing inside the program? You could maybe call filescope buffers with an size counter a dynamic memory allocation. I.e. for storing RS232 or CAN messages. Since they shrink and grow. The important thing is that you want to know that flooding one buffer wont flood another, which malloc could result in if it was used for unrelated buffers.
- xxpor 6y ago
  It's an embedded system, it's very likely there's no OS in the first place.
  - lmilcin 6y ago
    Actually it is irrelevant whether you use an operating system or not. One project I worked with these rules was on Verix OS. The rules are more intended on reducing application complexity and unpredictability which is typically helping reliability regardless of the setting.
    - xxpor 6y ago
      It was mostly on if the OS would actually allocate physical memory, or if it'd do a linux style overcommit. Without an OS, the latter is very unlikely.
zwieback 6y ago
The stack thing was always the big worry for me. Without a comprehensive static code analysis tool that's hard to do. And runtime stack checking adds quite a bit of overhead, especially if you also have to worry about running on the interrupt stack and possibly switching.
whatisthiseven 6y ago
It is interesting to read the rules you came up with to limit memory usage, and then to think of the criticisms one gets in Java for limiting memory usage. In Java we try to limit new as much as possible to prevent the GC from pausing too much, or inconveniently, or for too long. And basically all the rules you say are what we also use in Java. Except when you have these rules in Java, the ironic counter-point is "if you are doing this much memory control yourself, you should just use C or C++ or something". I'll keep your comment in mind next time I see that rebuttal. Thank you.
- lmilcin 6y ago
  Having almost 20 years of experience with Java... but are not following recent garbage collector developments. There is a bunch of misconceptions about Java. Java is actually very performant and memory allocation is generally cheaper than in C (except for inability to have good use of stack in Java). What's slow about Java is all the shit that has been implemented on top of it, but that's another story for another time. For example, allocation in Java is basically incrementing the pointer. And deallocation for most objects is basically forgetting the object exists. No, you don't want to "limit the use of new", that's wrong approach. What you want is to have objects that are either permanent or last very short amount of time. The worst types of objects are ones that have kind of intermediate lifetime ie if they are allowed to mature from eden. These cost a lot to collect. The objects that have very short lifetime are extremely cheap to collect. So if your function takes arguments, creates couple of intermediate objects and then never returns them (for example they were just necessary for inner working of the function) and your function does not call a lot of other heavy stuff, then it is very likely the cost of those temporary objects will be very low. Also, they tend to be allocated very close to each other and so pretty well cached.
  - binwiederhier 6y ago
    This was very insightful. Thank you.
  - scns 6y ago
    Well that seems to align pretty well with Entities/Aggregates and ValueObjects in DDD.
unnouinceput 6y ago
Actually I experienced worse restrictions when I was at Siemens writing embedded. Expanding on your list, here are some extras: - ternary operator("?") was strictly forbidden. One had to use full "if () {..}else {..}" syntax with comments inside each branch even if the branch was empty - a dynamic array written in an abstract way, when used and implemented specifically for current project had to become a constant static one, with values precalculated and copy/pasted to current project source. This was a fun one to do maintenance work years later. - magic numbers inside code was forbidden. All numbers had to be defined in a specific header, with explanation why is that number said value. - no variable parameters. All functions to have fixed parameters - use of macros as minimum as possible. Code review was wasted sometime on 50% time over use of macros that were not already "classic" from the project point of view - operator overload strictly forbidden. Also overloading functions was forbidden too.
- gpanders 6y ago
  All of these except for the first two seem like good rules in general. Was the ban on the ternary operator just a style/readability thing?
  - unnouinceput 6y ago
    Maintenance reasons. Future juniors to not have problems diving into projects directly. Chaining several ternary operators in same line is great to showoff your C prowess but a PITA to decipher what was doing, hence the rules.
signa11 6y ago
> dynamic allocation after application has started is banned -- any heap buffers and data structures must be allocated at the start of the application and after that any allocation is a compile time error, how do ensure that?
- 4gotunameagain 6y ago
  One way I can think of is to include the banned.h after you have performed all init processes (It would have to be in the .c files, not the headers, might not be so clean)

xondono 6y ago

Anything enforcing MISRA has essentially (almost) no way of allocating memory at runtime.

fsociety 6y ago
It’s funny, I worked exclusively with MISRA at the start of my career. Eventually I started a job at a FAANG and received quizzical comments on why I implemented a memory arena. The argument was to allocate memory freely and let it pool memory as necessary. Fair enough, it was simpler and fit the standard expectation of development. The issue is that if you talk with the allocator team they complain of not being able to fix performance issues fast enough due to allocations firing off left and right in the middle of a request. I never realized that my view of C programming is heavily influenced by MISRA until your comment. I know game engine programming follows a similar, perhaps unspoken, convention.
- munchbunny 6y ago
  The lack of runtime allocations in game engine programming comes from a different motivation: allocations are expensive, garbage collections are expensive, cache coherency matters, and you're chucking around a lot of very similar looking objects, so... object pools!
  - orwin 6y ago
    Yeah, the first time we coded a scroller shooting game with my friend (at school), we were baffled that our terminal-based scroller lagged more than the raycaster we did two weeks prior. Was it a C vs C++ thing? Turns out, creating then destroying every single missile/enemy was extremely costly
- hctaw 6y ago
  Custom allocators are quite common, it's not an arcane convention. I think the rule of thumb is preallocate until it gets questionable in complexity, then write your custom allocator - and really it's only applicable to code with a real-time deadline (hard or soft). Otherwise the system allocator is going to be a lot smarter than yours once it leaves microbenchmarks.
watersb 6y ago
MISRA: "Motor Industry Software Reliability Association" https://en.m.wikipedia.org/wiki/MISRA_C https://en.m.wikipedia.org/wiki/MISRA_C

closeparen 6y ago

How often does the dynamic allocation rule lead to an ad-hoc allocator appearing inside the program? Also doesn’t the OS lie? I thought the memory wasn’t really physically assigned until first use.

syncsynchalt 6y ago
In my experience dynamic allocation is banned in either (a) small embedded environments or (b) high scrutiny environments (soft realtime, safety critical, etc). In both cases the project size is small enough, or the scrutiny is high enough that the ad-hoc allocator doesn't develop. The environment is also simple enough that the memory cheats you're thinking of don't exist (or you can squash them by touching all allocated memory up front).
rightbyte 6y ago
> How often does the dynamic allocation rule lead to an ad-hoc allocator appearing inside the program? You could maybe call filescope buffers with an size counter a dynamic memory allocation. I.e. for storing RS232 or CAN messages. Since they shrink and grow. The important thing is that you want to know that flooding one buffer wont flood another, which malloc could result in if it was used for unrelated buffers.
xxpor 6y ago
It's an embedded system, it's very likely there's no OS in the first place.
- lmilcin 6y ago
  Actually it is irrelevant whether you use an operating system or not. One project I worked with these rules was on Verix OS. The rules are more intended on reducing application complexity and unpredictability which is typically helping reliability regardless of the setting.
  - xxpor 6y ago
    It was mostly on if the OS would actually allocate physical memory, or if it'd do a linux style overcommit. Without an OS, the latter is very unlikely.

zwieback 6y ago

The stack thing was always the big worry for me. Without a comprehensive static code analysis tool that's hard to do. And runtime stack checking adds quite a bit of overhead, especially if you also have to worry about running on the interrupt stack and possibly switching.

whatisthiseven 6y ago

It is interesting to read the rules you came up with to limit memory usage, and then to think of the criticisms one gets in Java for limiting memory usage. In Java we try to limit new as much as possible to prevent the GC from pausing too much, or inconveniently, or for too long. And basically all the rules you say are what we also use in Java. Except when you have these rules in Java, the ironic counter-point is "if you are doing this much memory control yourself, you should just use C or C++ or something". I'll keep your comment in mind next time I see that rebuttal. Thank you.

lmilcin 6y ago
Having almost 20 years of experience with Java... but are not following recent garbage collector developments. There is a bunch of misconceptions about Java. Java is actually very performant and memory allocation is generally cheaper than in C (except for inability to have good use of stack in Java). What's slow about Java is all the shit that has been implemented on top of it, but that's another story for another time. For example, allocation in Java is basically incrementing the pointer. And deallocation for most objects is basically forgetting the object exists. No, you don't want to "limit the use of new", that's wrong approach. What you want is to have objects that are either permanent or last very short amount of time. The worst types of objects are ones that have kind of intermediate lifetime ie if they are allowed to mature from eden. These cost a lot to collect. The objects that have very short lifetime are extremely cheap to collect. So if your function takes arguments, creates couple of intermediate objects and then never returns them (for example they were just necessary for inner working of the function) and your function does not call a lot of other heavy stuff, then it is very likely the cost of those temporary objects will be very low. Also, they tend to be allocated very close to each other and so pretty well cached.
- binwiederhier 6y ago
  This was very insightful. Thank you.
- scns 6y ago
  Well that seems to align pretty well with Entities/Aggregates and ValueObjects in DDD.

unnouinceput 6y ago

Actually I experienced worse restrictions when I was at Siemens writing embedded. Expanding on your list, here are some extras: - ternary operator("?") was strictly forbidden. One had to use full "if () {..}else {..}" syntax with comments inside each branch even if the branch was empty - a dynamic array written in an abstract way, when used and implemented specifically for current project had to become a constant static one, with values precalculated and copy/pasted to current project source. This was a fun one to do maintenance work years later. - magic numbers inside code was forbidden. All numbers had to be defined in a specific header, with explanation why is that number said value. - no variable parameters. All functions to have fixed parameters - use of macros as minimum as possible. Code review was wasted sometime on 50% time over use of macros that were not already "classic" from the project point of view - operator overload strictly forbidden. Also overloading functions was forbidden too.

gpanders 6y ago
All of these except for the first two seem like good rules in general. Was the ban on the ternary operator just a style/readability thing?
- unnouinceput 6y ago
  Maintenance reasons. Future juniors to not have problems diving into projects directly. Chaining several ternary operators in same line is great to showoff your C prowess but a PITA to decipher what was doing, hence the rules.

signa11 6y ago

> dynamic allocation after application has started is banned -- any heap buffers and data structures must be allocated at the start of the application and after that any allocation is a compile time error, how do ensure that?

4gotunameagain 6y ago
One way I can think of is to include the banned.h after you have performed all init processes (It would have to be in the .c files, not the headers, might not be so clean)