3 ms·

I've poked around with -fsanitize=kernel-address in embedded environments--and I think there's real potential there. However, in this case, I wonder how well -

I've poked around with -fsanitize=kernel-address in embedded environments--and I think there's real potential there. However, in this case, I wonder how well -fsanitize=leak might work. Override free() to show when it is called & print a stack trace, and you get a good look at your system's actual allocation pattern. As an aside, there's nothing wrong with malloc in these embedded environments, it's the freeing that causes memory fragmentation. If you can also avoid running out of memory, you're good to go, since you essentially have a fancy form of static allocation!

a1369209993 4y ago
> it's [just] the freeing that causes memory fragmentation. Nope. Consider for examle a microcontroller with 64K of address space, consisting of 32K of low RAM, 8K of ROM/MMIO, and 24K of high RAM. Allocate buffers of size 23K, 15K, and 15K. With a straightforward first-fit allocator[0], you'll put the 23K in low RAM, leaving 9K, put the first 15K in high RAM, leaving 9K there, and fail to allocate the second 15K, because even though you have 18K free, it's fragmented into two 9K regions. If you'd allocated that memory statically, you could have put the 23K in high RAM, and the two 15Ks in low RAM, with 2K low and 1K high left over. It's debatable whether the fragmentation is 'caused' by malloc itself or by the ROM region in the middle of the address space, but you never called free, so it's not caused by free. 0: Less naive allocation strategies take more work to 'trap' like this, but I can work out a example for best-fit or whatever if you're sceptical that this isn't just a problem specific to first-fit.
- flaviut 4y ago
  That makes a lot of sense, I hadn't considered that! Thank you for pointing out a gap in my knowledge.
  - a1369209993 4y ago
    The underlying problem is that malloc-style dynamic memory requires the allocations to stay in place from malloc until free, so if malloc makes a bad decision early on, it can't fix it once it finds out it made a mistake. And any nontrivial decision malloc makes can be made retroactively bad with the right (wrong?) set of subsequent requests.
- girvo 4y ago
  This is part of why ESP-IDF has the heap_caps_* (malloc, calloc, free, etc.) interfaces, but it’s some decent mental overhead to have to use a lot of it. Has meant a lot of planning and white boarding for the few places we haven’t been able to avoid it.
hra5th 4y ago
In addition to the other comment, the primary problem with malloc() in embedded environments is that you push the memory exhaustion error to runtime, rather than compile time (not memory fragmentation). A call to malloc() in a rarely-called function might not exhaust available RAM until a system has been deployed for days or weeks, which is much worse than finding out your program requires too much memory at compile time, or at boot time. While stack overflows are always possible, many embedded projects at least have static analysis tools that can (with varying degrees of soundness) calculate worst-case stack use.

a1369209993 4y ago

> it's [just] the freeing that causes memory fragmentation. Nope. Consider for examle a microcontroller with 64K of address space, consisting of 32K of low RAM, 8K of ROM/MMIO, and 24K of high RAM. Allocate buffers of size 23K, 15K, and 15K. With a straightforward first-fit allocator[0], you'll put the 23K in low RAM, leaving 9K, put the first 15K in high RAM, leaving 9K there, and fail to allocate the second 15K, because even though you have 18K free, it's fragmented into two 9K regions. If you'd allocated that memory statically, you could have put the 23K in high RAM, and the two 15Ks in low RAM, with 2K low and 1K high left over. It's debatable whether the fragmentation is 'caused' by malloc itself or by the ROM region in the middle of the address space, but you never called free, so it's not caused by free. 0: Less naive allocation strategies take more work to 'trap' like this, but I can work out a example for best-fit or whatever if you're sceptical that this isn't just a problem specific to first-fit.

flaviut 4y ago
That makes a lot of sense, I hadn't considered that! Thank you for pointing out a gap in my knowledge.
- a1369209993 4y ago
  The underlying problem is that malloc-style dynamic memory requires the allocations to stay in place from malloc until free, so if malloc makes a bad decision early on, it can't fix it once it finds out it made a mistake. And any nontrivial decision malloc makes can be made retroactively bad with the right (wrong?) set of subsequent requests.
girvo 4y ago
This is part of why ESP-IDF has the heap_caps_* (malloc, calloc, free, etc.) interfaces, but it’s some decent mental overhead to have to use a lot of it. Has meant a lot of planning and white boarding for the few places we haven’t been able to avoid it.

hra5th 4y ago

In addition to the other comment, the primary problem with malloc() in embedded environments is that you push the memory exhaustion error to runtime, rather than compile time (not memory fragmentation). A call to malloc() in a rarely-called function might not exhaust available RAM until a system has been deployed for days or weeks, which is much worse than finding out your program requires too much memory at compile time, or at boot time. While stack overflows are always possible, many embedded projects at least have static analysis tools that can (with varying degrees of soundness) calculate worst-case stack use.