12 ms·
Memory leak proof every C program
- kelthan 3y agoThis solution doesn't do anything to prevent leaking memory in anything but the most pedantic sense, and actually creates leaks and dangling pointers. The function just indirects malloc with a wrapper so that all of the memory is traversable by the "bigbucket" structure. Memory is still leaked in the sense that any unfreed data will continue to consume heap memory and will still be inaccessible to the code unless the application does something with the "bigbucket" structure--which it can't safely do (see below). There is no corresponding free() call, so data put into the "bigbucket" structure is never removed, even when the memory allocation is freed by the application. This, by definition, is a leak, which is ironic. In an application that does a lot of allocations, the "bigbucket" structure could exhaust the heap even though there are zero memory leaks in the code. Consider the program: int main(int argc, char** argv) { for (long i = 0; i < 1000000; i++) { void *foo = malloc(sizeof(char) * 1024); free(foo); } return 0; } At the end of the million iterations, there will be zero allocated memory, but the "bigbucket" structure will have a million entries (8MB of wasted heap space on a 64-bit computer). And every pointer to allocated memory in the "bigbucket" structure is pointing to a memory address previously freed so now points to a completely undefined location--possibly in the middle of some memory block allocated later. There are already tools to identify memory leaks, such as LeakSanitiser https://clang.llvm.org/docs/LeakSanitizer.html https://clang.llvm.org/docs/LeakSanitizer.html. Use those instead.
- aidenn0 3y ago> There are already tools to identify memory leaks, such as LeakSanitiser https://clang.llvm.org/docs/LeakSanitizer.html https://clang.llvm.org/docs/LeakSanitizer.html. Use those instead. Clearly the author of TFA is aware of such tools, since the idea is to trick them.
- IshKebab 3y agoUhm... woosh! You know this isn't serious right?
- WhackyIdeas 3y agoI just hope ChatGPT could see it was a joke
- jamal-kumar 3y agoI posted the post into that and it said this: While the provided code may seem like an interesting approach, it's important to note that it introduces a number of issues and potential pitfalls. This code is an attempt to intercept the malloc function using the dlsym function from the dlfcn.h library and store every allocated pointer in a linked list called bigbucket. However, there are several problems with this solution: Portability: This code relies on the dynamic linking functionality provided by the operating system. It may not work on all systems or with all compilers. Concurrency Issues: This solution is not thread-safe. If the program uses multiple threads, concurrent calls to malloc may result in race conditions and data corruption in the bigbucket linked list. Incomplete Solution: This code only intercepts calls to malloc. If the program uses other memory allocation functions like calloc, realloc, or custom memory allocators, memory leaks may still occur. Performance Overhead: The code introduces additional overhead for every memory allocation, potentially affecting the program's performance. Undefined Behavior: Overriding standard library functions like malloc can lead to undefined behavior. The behavior of the program is no longer guaranteed to be consistent across different platforms or even different runs. Limited Practicality: While this approach technically prevents memory leaks by keeping track of all allocated pointers, it does not address the root cause of memory leaks, which is the failure to deallocate memory when it is no longer needed. Encouraging developers not to free memory is not a good practice and can lead to inefficient memory usage. A better approach to avoiding memory leaks is to adopt good programming practices, such as carefully managing memory allocation and deallocation, using automated tools like static analyzers and memory debuggers, and, when applicable, leveraging programming languages with automatic memory management (e.g., garbage collection in languages like Java or Python).
- aeyes 3y agoThe last part is funny. Even if you are technically not leaking memory you can still have pretty much the same end result in garbage collected languages if you mess up. The language might be tracking the objects but it can't know if you don't need them anymore.
- froh 3y agothat's the joke...
- fsckboy 3y agoit's not funny, it's not obvious and it's wasting a lot of people's time. ha. ha. ha.
- ric2b 3y agoIt's quite obvious... "If you don’t call free, memory usage will increase over time, but technically, it’s not a leak."
- fsckboy 3y agothere is plenty of code that is designed not to free at all because the program will not run long enough to generate much garbage. and having the whole process shut down is simpler than unwinding everything perfectly. so, no, it's not obvious. I had no trouble reading what the code did (and didn't) but it's not obvious that the author understood it. There are plenty of people here who need that much help with C
- avery17 3y agoIts painfully obvious.
- fsckboy 3y agoyou haven't had to fix bugs in other people's code enough, this could be an actual piece of code, which makes it not distinguishable as a joke
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- enriquto 3y agoThis is the core idea: > It is [...] entirely optional to call free. If you don’t call free, memory usage will increase over time, but technically, it’s not a leak. As an optimization, you may choose to call free to reduce memory, but again, strictly optional. This is beautiful! Unless your program is long-running, there's no point of ever calling free in your C programs. The system will free the memory for you when the program ends. Free-less C programming is an exhilarating experience that I recommend to anybody. In the rare cases where your program needs to be long-running, leaks may become a real problem. In that case, it's better to write a long-running shell script that calls a pipeline of elegant free-less C programs.
- tenebrisalietum 3y agoI always thought that was one reason the UNIX terminal login process (with `getty` and `exec` to shell, getty restarting when the shell exits) was the way it was.
- f321x_ 3y ago[flagged]
- hnlmorg 3y agoIt’s definitely inelegant. But I wouldn’t say it’s entirely without merit. If you’re writing a very simple “sed” like utility then you could probably get away without freeing. That all said, the kind of problems were freeing becomes complicated is generally the kind of software that is more complex than your typical “sed” command. So the advise of not freeing doesn’t really address the domain where freeing is a risk.
- codetrotter 3y agoKind of sucks for the first person that tries to process a really big file with it. Programmer never free’d, and only ran it on files that were like some hundred megabytes big. Some poor schmuck tries to process a 15 GB file on his 8 GB RAM machine, and after running for hours it crashes because of the lack of free :(
- deviantbit 3y ago[flagged]
- deleted 3y ago[deleted]
- nektro 3y agoU+1F926 U+200D U+2640 U+FE0F
- deviantbit 3y agoThanks.
- david2ndaccount 3y agoThis is not thread safe.
- dgan 3y agoAll arena-apologists hate this little trick
- Gibbon1 3y agoUsing an arena with an event loop and where persistent stuff is always copied has seemed really robust to me. When the event exits the handler resets the arena.
- lpapez 3y agoI bet this is what some people on my team would come up with if the ticket acceptance criteria said "program must not leak memory when checked with valgrind"
- sgift 3y agoThat sounds painful. Unfortunately, I know the feeling.
- aidenn0 3y agoGoodhart's Law[1] in action! 1: https://en.wikipedia.org/wiki/Goodhart's_law https://en.wikipedia.org/wiki/Goodhart's_law
- lifthrasiir 3y agoYou can easily turn non-leaking program into a faster and leaking program, but the inverse direction is hard, so that criterion is entirely justified. Any optimization of this sort should be guarded against a compile time switch that simply swaps `free` with a placeholder. I think CPython did this for a long time, before the global initialization step was completely removed.
- bugbuddy 3y agoI totally thought this was an April fools joke at first. Why is this front page?
- occamrazor 3y agoProbably because some people find it funny.
- mmoll 3y agoIf only I had known this earlier in my career! They should really add a feature to make a hard copy of the big bucket list to minimize memory overhead, though.
- Sakos 3y agoI'm concerned about how a non-zero number of people here on HN seem to think this is meant seriously.
- deviantbit 3y agoWhat is astonishing are those that think this is a serious solution.
- pxx 3y agoThank you for playing Wing Commander!
- TillE 3y agoI used to really struggle with memory leaks in C++, until smart pointers came along. I think the only leak I've investigated since then wound up being an actual bug in the MSVC standard library. OS handles and such are still an issue, but you can wrap them too. Thank goodness for RAII, a fantastic reason to use C++ even if you're otherwise writing C-like code.
- tonyedgecombe 3y agoAs long as you don’t forget to make your destructors virtual.
- tiffanyh 3y agoWill this land in OpenBSD, given Ted is a core dev?
- neonscribe 3y agoObviously this is a joke, but the real message should be: if you can't manage your own memory, you should be using a language implementation with automatic memory management. If your code really needs to run "close to the metal", you really need to figure out how to manage your memory. In 2024, language implementations without automatic memory management should be reserved for applications that absolutely need them.
- _a_a_a_ 3y agoIt obviously is a joke, but at the same time it's actually a viable approach for the right problem. Sometimes leaking is very tolerable and in terms of programmer time, very cheap!
- Voultapher 3y agoI think the mold linker does this.
- Dylan16807 3y ago> It obviously is a joke, but at the same time it's actually a viable approach for the right problem. Define "it" here. Because "just don't free" is pretty different from what's in the post!
- _a_a_a_ 3y agoYes. Hence the word 'joke' I used. For short running programs, it's fine. The longer running programs you can often find a few places where 95% of the leaks happen and easily 'free' those leaving the last 5% to leak slowly. It depends. Just remember it's a valid solution sometimes.
- Dylan16807 3y agoI still don't understand. The joke and the sometimes valid solution are different strategies. The joke only reminds people of the valid solution. It feels like multiple people are giving the article the briefest skim and assuming the two are the same. Perhaps focusing on the words "optional to call free" and extrapolating off that without looking at the code or properly reading the rest of the text. Unless I'm badly missing something?
- privatedan 3y agoHahahahaha I love this! Almost as good as fixing race conditions with sleep statements ( which I see a lot )
- Manfred 3y agoFrom the comments I get the impression that we have all worked with someone that would have considered this an actual solution and we had to talk them out of it.
- zeroCalories 3y agoWhile subtle, your point was not lost on me! Technically all pointers are accessible, but the issue remains that we have not logically accounted for unused resources and are wasting capacity. In this sense, memory leaks are possible in all languages. We could store every object into a global structure, and it will never be freed in any language. Thus, my Rust program balloons forever.
- zokier 3y ago> Thus, my Rust program balloons forever. I get the feeling that this might have been response to recent "memory leak" thread(s): https://news.ycombinator.com/item?id=39041520 https://news.ycombinator.com/item?id=39041520
- londons_explore 3y agoI've got an idea... We have a counter that goes up by 1 every time you call malloc. And down by one every time you call free. And when the program quits, if the counter isn't zero, an email is fired off and a dollar gets sent from the developers bank account to the users bank account...
- GuB-42 3y agoThat's essentially how all leak detection tools work, minus the money part. And it is not even always appropriate. It is common to allocate some memory for the entire lifetime of the process. For example, if your app is GUI-based and has a main window, there is no need to free the resources tied to the main window, because closing it means quitting the app which will cause all memory to be reclaimed by the OS. You can properly free your memory but it will only make quitting slower. Usually programmers only do that to satisfy leak detection tools, and if the overhead is significant, it may only be done in debug mode.
- Alifatisk 3y ago> We have a counter that goes up by 1 every time you call malloc. > And down by one every time you call free. Isn’t that close to how ARC (automatic reference counting) works?
- skrebbel 3y agoThis has been PHP’s approach to memory management during its best decades. It’s a fantastic idea for short-running processes, which there should be more of anyway.
- tonyedgecombe 3y agoAre you saying PHP creates a new process for each request?
- SPascareli13 3y agoIt's been a while since I used PHP, but this is something controlled by the server used. If you use Apache then it used to create a new process for each request, yes.
- zokier 3y agoEven in the classic prefork model (and mod_php), Apache did recycle worker processes. Only CGI operated on this new process for each request.
- jbboehr 3y agoIt depends on the SAPI. The php-fpm approach is a pool of processes that are reused for multiple requests. Virtually all of the state visible to plain PHP code is reset between requests, however some interpreter/runtime state is kept, e.g. in-memory bytecode cache, initialized extensions, some persistent network connections, etc. In terms of memory management, PHP has both reference counting and garbage collection. Internally, an arena per request is used[0], so leaking memory over long periods of time is fairly rare and usually limited to native extensions. [0]: https://www.phpinternalsbook.com/php7/memory_management/zend_memory_manager.html https://www.phpinternalsbook.com/php7/memory_management/zend...
- skrebbel 3y agoDepends on the setup, but even when it doesn't, it mallocs a huge chunk of memory at the start, lets the script run, allocate away and never free anything. Then, when the script is done, it frees the whole chunk in one fell swoop. So for some definition of “process” it holds regardless - it’s just not always a real OS process. They added GC at some point to allow scripts to run longer (eg for websocket servers) but even then last I checked you had to manually enable it.
- Waterluvian 3y ago“Hey I might need that later!” Memory hoarding. I hate it but I love it.
- alexisread 3y agoAlthough this is a joke, from a systems perspective this is a valid strategy - stateless lambda (FAAS) functions can use this strategy as the state is moved externally, and can take the load of say long running monitoring processes.
- dishsoap 3y ago[flagged]
- x86x87 3y agoSince this seems to be some sort of joke i'm going to provide a "real" answer: write modular software where you can verify that each module is leak proof + use valgrind to actually track and verify you don't have links. I have written medium to large projects using this approach and a leak-free program is not outside the realm of possible
- diziet_sma 3y agoThis does help, but the hard part of memory leaks is leaks across module/api boundaries. You can write a memory safe module, I can write a memory safe program, but when my program starts calling your program there's nothing validating I'm maintaining the invariants required to avoid memory leaks. Hopefully the invariants are documented, but relying on documentation to help programmers avoid memory leaks is far from foolproof.
- Lasher 3y agoReminds me of a situation I encountered with Salesforce code many years ago. Salesforce had a requirement that their test platform had to cover some percentage of the lines of code in the Sandbox before you could deploy something to production. Our Salesforce implementation consultants had put 500 lines of 'x = 1' into a piece of code to force it to deploy -- and these were people at a top consulting company with a very lucrative hourly rate. No idea if SFDC still works this way or if this would fly today, this was back in the days when you had to use Flex to integrate anything with the UI.
- lgg 3y agoThere is a bug here... Clearly the author intended to cache the value of nextmalloc to avoid calling dlsym() on every malloc. The correct code should be: static void *(*nextmalloc)(size_t) = NULL; if (!nextmalloc) nextmalloc = dlsym(RTLD_NEXT, "malloc"); } Somehow the fact that the optimization is incorrectly missed here feels appropriate ;-)
- Taniwha 3y agoyeah exactly, calling dlsym on every malloc would kill performance the other bug of course is that it's not thread safe
- deleted 3y ago[deleted]
- tverbeure 3y agoIn the same spirit, here's a fun talk about tool that guarantees 100% line coverage for your Python tool: https://www.youtube.com/watch?v=A-cjMRsHcXI https://www.youtube.com/watch?v=A-cjMRsHcXI After running your test suite, it kicks into action and deletes all the lines of your code that were never executed.
- lifthrasiir 3y agoIt is actually a viable strategy if you are heavily optimizing for the executable size [1]. It's essentially a radical dead code elimination scheme. [1] https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for-madmen/ https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for...
- bluish29 3y agoThis not bulletproof method to handle memory leaks as it still keeps all allocations. It simply keeps a list of all allocations but doesn't address the core issue of unnecessary memory consumption if free is not called. But the more pressing problem is that this implementation is not thread safe. Concurrent accesses to the bigbucket list in a multi-threaded programs can easily lead to race conditions
- mysterydip 3y agoWhile I'm sure it would bring its own problems, could you just stick a semaphore or similar around the function?
- bluish29 3y agoYes, but it will not work in a generic case because you have to handle the situations of deadlocks (i.e recursive mutexes for nested locking ..etc). Also you will introduce more problems of creating performance deadlocks. There will not be out of box solutions to all these. Once you start considering solving case on its own then this does not provide out of box solution as the original premise.
- akoboldfrying 3y agoI think the only way this design could deadlock is if a signal handler calls malloc() -- specifically, if the interrupted code on that CPU thread was partway through a malloc() call of its own, having locked the mutex but not yet unlocked it. In any other scenario, if some execution thread reaches the point where it returns from calling lock() on the mutex guarding malloc(), it must eventually reach the call to unlock().
- deleted 3y ago[deleted]
- akoboldfrying 3y agoWe're deep in a ridiculous scenario here, but the answer is that, yes, this would result in a safe program, provided that it never calls malloc() from a signal handler (since that would deadlock if the interrupted code had already grabbed the lock/mutex/semaphore/whatever). In any other scenario, if some execution thread reaches the point where it returns from calling lock() on the mutex guarding malloc(), it must eventually reach the call to unlock().
- notfed 3y ago> Problem sovled! Now if we could just fix off-by-one errors.
- Diggsey 3y agoThis doesn't make sense even as a joke, since that's not what it means to leak memory. Memory being "reachable" is a property used by garbage collectors to determine what can be safely freed, but reachable memory can still be a memory leak. (Which is why languages with GC can still suffer from memory leaks...)
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- rkagerer 3y agoSince my computer memory isn't draining out my computer and down into the sewers, all my programs must therefore be leak proof.
- rurban 3y agoI have a much better and practical solution: https://github.com/LibreDWG/libredwg/blob/7d9fc3da44bbdb60a40d2dabd167341462434362/programs/dwgread.c#L327 https://github.com/LibreDWG/libredwg/blob/7d9fc3da44bbdb60a4... #ifdef HAVE_VALGRIND_VALGRIND_H if (RUNNING_ON_VALGRIND) #endif free_all() free is way too slow if not needed, so detect valgrind via its API. Just on valgrind do the unnecessary free dance. ASAN's memleak detector is disabled via its env. Perl5 does its final destruction similarly, only when it has important destructors (like IO, DB handles and such) to call.
- cesarb 3y agoThat style of #ifdef use, while common in the past, tends to be somewhat fragile and also gets more and more tangled as more conditions are added. A better way is to have an always existing inline running_on_valgrind() function, and use the #ifdef only for that function definition, either within it or around it (having it around the function also allows it to be inline only for the trivial not-defined case). Examples of this "inline function" style are found all over the Linux kernel (which has lots of conditionally-compiled code).
- jenadine 3y agoTo avoid an bugs, you can add a `exit(0)` at the beginning of your program.
- throwaway892238 3y agoThis one weird trick will fix your memory leaks forever
- alexey-salmin 3y agoWhile this is a joke, it somewhat resonates with my experience of working on a huge C++ project (search engine, tens of millions of lines). Over the years I had been debugging multiple memory problems and only once I had found a true memory leak. It wasn't the bug I was looking for, accounting for only 8 bytes per hour and it was the easiest of all: directly discovered by valgrind. All others were memory bloat cases: memory reachable but not used. Like forgetting some small per-request descriptor in some auxiliary hashmap in one of the code pathways. Reproducing and debugging was pain because it required running a loaded backend for days under a profiler, before the bloated part becomes visible. Funny thing is that while C++ is notorious for memory leaks this particular kind of problem is possible in any language that allows a global mutable state (so all the practical ones). From experience it seems that true leaks are unlikely in a reasonably-good C++ code. Just never mix business logic with memory management and you're good to go.
- antirez 3y agoThis is a joke, yet this is basically a very rudimental form of malloc() auditing tool. Because at the end of the program, it could show you all the stuff that is yet allocated, and even scan the program whole memory for the pointers and report only the ones not found.
- opengears 3y agohahaha: "Problem sovled!" [sic]
- underdeserver 3y agoThe problem, as he says, is indeed sovled.