4 ms·

This seems to be a malloc()/free() wrapper that keeps track of malloc()ed/realloc()ed/free()d pointers. I see that it's been submitted by the author, so one hop

This seems to be a malloc()/free() wrapper that keeps track of malloc()ed/realloc()ed/free()d pointers. I see that it's been submitted by the author, so one hopefully-helpful comment: it would be really helpful to highlight why one would use this instead of one of the well-established alternatives (Boehm GC, valgrind, ASan). The "run after every test case" example actually gives a fairly good reason, but note that realistic code bases probably have a few caches, freelist-optimizations or other exceptions to "deallocate all memory after every test".

ktRolster 10y ago
realistic code bases probably have a few caches, freelist-optimizations or other exceptions to "deallocate all memory after every test". If an open source library has memory allocated at finish, to me that's a warning to find a different project. If it's something I get assigned to work on at work, I still have to work on it; but it lets me know I'll probably be cleaning up many other things as well: the programmers who passed before were sloppy.
- ars 10y ago
  > If an open source library has memory allocated at finish, to me that's a warning to find a different project. You need to read this: https://news.ycombinator.com/item?id=8305283 https://news.ycombinator.com/item?id=8305283 Basically it's faster to simply exit and not try to walk datastructures deallocating everything.
  - ktRolster 10y ago
    That rarely becomes an issue, and should not be optimized for by default.
    - eutectic 10y ago
      What's the advantage to deallocating on exit?
      - ktRolster 10y ago
        You know your code is clean.
        luckydude 10y ago
        Hi, BitKeeper guy here (BK predates git, first distributed SCM). I built what I'm sure is a less good version of this, I called it purify, you included purify.h and it warned you about all the stuff that it could by redefining malloc/free/etc to ones that kept track of stuff. So some overlap with the system posted here. We were very careful to free everything before exit. It just felt cleaner. It was also a bad idea. Until a few years ago, BK used an ascii file format, which means it read in the graph with all the meta data, parsing that, malloc-ing each string, each struct, etc. At exit we carefully freed all that. Why was that a bad idea? If you wanted to link BK into your long lived IDE or wanted it to be a library, then it was a very good idea. But we never did that and BK doesn't really want to be used that way. So it became a performance problem. Consider bk changes -1 # same as git log -1 That used to parse the entire changeset graph, build an in memory one version of it, find the tip (HEAD), print out the comment, then recursively free everything. All for the holy grail of "knowing your code is clean". Clean is good unless it is pointless. Another way to say the same thing is you could make the claim that all code should be written to be thread safe. Even if you never multi thread it. That's a lot of work, maybe not so bad for people who are good at threading, but it's work for most people (I'm a former Sun kernel hack, I get threads, it's not that hard once you get over the hump but a lot of people don't). If you have no intention of multi threading the code then making it thread safe is just "clean code" masturbation. I'm all for clean code stuff where you are going to reap some value. In our case we did all that clean code memory management and eventually said screw that, changed the file format so we could mmap() it and page it in (read/write protect everything, take the fault, fault handler pages it in). No alloc, no free, way better performance. All those years of clean code bought us nothing. I get that if you are making a library the rules about clean are very very different. But for an app that is going to run and exit, don't push clean down people's throat, it isn't helping in a lot of cases.
        _0ffh 10y ago
    - mikepavone 10y ago
      Doing work that makes your library slower so that certain primitive memory checkers don't get upset doesn't seem like a good use of time to me. Especially when more sophisticated tools (Valgrind at least, probably others) can differentiate between actually leaked memory and stuff that's still reachable at program termination. Obviously there are scenarios where this is not appropriate, but it seems like a reasonable choice in many cases.
      - ktRolster 10y ago
        If your own code is clean other than de-allocating at exit, then I'm fine with that. If not, I'm asking you to kindly leave the industry, and not inflict your messy code on others.
  - mulle_nat 10y ago
    I think one should differentiate between app/tool code and library/plugin/daemon code. A typical tool like git starts, runs for a while and then exits. In this case worrying too much about memory cleanness is counterproductive. This changes when you are writing a data structure, like say a hashtable. You don't want a hashtable that leaks during operation. It changes when you write a daemon, that runs for a long time, you don't want it to consume all address space. It changes when you write plugins, that get repeatedly loaded and unloaded.
mulle_nat 10y ago
An advantage over valgrind f.e. is that the same test works on Windows to Linux to OS X without having to interface with different tools. The use of the "allocator" struct enables you to isolate your code to test, so that other caches do not obscure the result. The thing is, it's not just a wrapper, but also a decoupler.

ktRolster 10y ago

realistic code bases probably have a few caches, freelist-optimizations or other exceptions to "deallocate all memory after every test". If an open source library has memory allocated at finish, to me that's a warning to find a different project. If it's something I get assigned to work on at work, I still have to work on it; but it lets me know I'll probably be cleaning up many other things as well: the programmers who passed before were sloppy.

ars 10y ago
> If an open source library has memory allocated at finish, to me that's a warning to find a different project. You need to read this: https://news.ycombinator.com/item?id=8305283 https://news.ycombinator.com/item?id=8305283 Basically it's faster to simply exit and not try to walk datastructures deallocating everything.
- ktRolster 10y ago
  That rarely becomes an issue, and should not be optimized for by default.
  - eutectic 10y ago
    What's the advantage to deallocating on exit?
    - ktRolster 10y ago
      You know your code is clean.
      - luckydude 10y ago
        Hi, BitKeeper guy here (BK predates git, first distributed SCM). I built what I'm sure is a less good version of this, I called it purify, you included purify.h and it warned you about all the stuff that it could by redefining malloc/free/etc to ones that kept track of stuff. So some overlap with the system posted here. We were very careful to free everything before exit. It just felt cleaner. It was also a bad idea. Until a few years ago, BK used an ascii file format, which means it read in the graph with all the meta data, parsing that, malloc-ing each string, each struct, etc. At exit we carefully freed all that. Why was that a bad idea? If you wanted to link BK into your long lived IDE or wanted it to be a library, then it was a very good idea. But we never did that and BK doesn't really want to be used that way. So it became a performance problem. Consider bk changes -1 # same as git log -1 That used to parse the entire changeset graph, build an in memory one version of it, find the tip (HEAD), print out the comment, then recursively free everything. All for the holy grail of "knowing your code is clean". Clean is good unless it is pointless. Another way to say the same thing is you could make the claim that all code should be written to be thread safe. Even if you never multi thread it. That's a lot of work, maybe not so bad for people who are good at threading, but it's work for most people (I'm a former Sun kernel hack, I get threads, it's not that hard once you get over the hump but a lot of people don't). If you have no intention of multi threading the code then making it thread safe is just "clean code" masturbation. I'm all for clean code stuff where you are going to reap some value. In our case we did all that clean code memory management and eventually said screw that, changed the file format so we could mmap() it and page it in (read/write protect everything, take the fault, fault handler pages it in). No alloc, no free, way better performance. All those years of clean code bought us nothing. I get that if you are making a library the rules about clean are very very different. But for an app that is going to run and exit, don't push clean down people's throat, it isn't helping in a lot of cases.
        _0ffh 10y ago
  - mikepavone 10y ago
    Doing work that makes your library slower so that certain primitive memory checkers don't get upset doesn't seem like a good use of time to me. Especially when more sophisticated tools (Valgrind at least, probably others) can differentiate between actually leaked memory and stuff that's still reachable at program termination. Obviously there are scenarios where this is not appropriate, but it seems like a reasonable choice in many cases.
    - ktRolster 10y ago
      If your own code is clean other than de-allocating at exit, then I'm fine with that. If not, I'm asking you to kindly leave the industry, and not inflict your messy code on others.
- mulle_nat 10y ago
  I think one should differentiate between app/tool code and library/plugin/daemon code. A typical tool like git starts, runs for a while and then exits. In this case worrying too much about memory cleanness is counterproductive. This changes when you are writing a data structure, like say a hashtable. You don't want a hashtable that leaks during operation. It changes when you write a daemon, that runs for a long time, you don't want it to consume all address space. It changes when you write plugins, that get repeatedly loaded and unloaded.

mulle_nat 10y ago

An advantage over valgrind f.e. is that the same test works on Windows to Linux to OS X without having to interface with different tools. The use of the "allocator" struct enables you to isolate your code to test, so that other caches do not obscure the result. The thing is, it's not just a wrapper, but also a decoupler.