7 ms·
Why you should use talloc for your next C project
- TrygveThrontvei 15y agoWhy would you have a C project in the first place? Worst language ever. If you don't use C, you won't have to worry about memory allocation issues. By the way I went to Harvard and I know this stuff.
- alextp 15y agoThis feels slightly untrustworthy. What if there are still external references to subfields of a struct? I think it's probably safer to just write explicit constructors (that allocate and initialize) and destructors (to free properly) for the things you use.
- jpr 15y agoWhy does everyone conflate allocation and initialization?
- regularfry 15y agoRAII, at a guess.
- TrygveThrontvei 15y agoI want to emphasize that I went to Harvard, so I know what I'm talking about.
- bluetech 15y agoI have used talloc for a medium-sized project, and it was quite refreshing. Of course, it all depends on your program's structure. An hierarchical allocator naturally fits hierarchical data structures and containers, so parsers for example can benefit immensely in my experience. You are correct in that you can just write constructors and destructors, but then you still need to worry about error handling, double-frees and whatnot. With an allocator like talloc, you just state the relationships during the allocation, and afterwards only worry about the high-level structure. One thing which can be somewhat cumbersome is the need to pass around those "contexts"; for example, if you allocate an array of some struct, and you need to allocate additional memory inside that struct, your context will presumably be the array, and so you need to have access to it. One solution I have found which is helpful in most, but not all, situations is to use the wonderful container_of[1] macro, which enables you to do upcasting using some pointer trickery. And of course, it does cost some performance as they say on their homepage. [1] http://www.kroah.com/log/linux/container_of.html http://www.kroah.com/log/linux/container_of.html
- SeveredCross 15y agoI invariably find myself doing this in C projects because not doing it is a serious violation of DRY--writing possibly hundreds of lines of free (foo->bar); free (foo->baz); free (foo->quux->fnord); ... ; free (foo) is not only hugely error prone, but a waste of everyone's time.
- tanoku 15y agoWhy you shouldn't use talloc for your next C project: GPL v3.
- JoshTriplett 15y agoTalloc uses the LGPL, not the GPL.
- haberman 15y agoEven LGPL is a pretty steep price for the value being offered. The LGPL requires that is it possible to substitute a modified version of the library, which can be a logistical problem. Do you really want to have to link your malloc() as a shared library? How do you let the user substitute inline functions? What do you do when people start bugging you because they weren't able to substitute a modified version of the library due to some technical edge case? Too much legal baggage for such a small piece of functionality.
- JoshTriplett 15y agoI wouldn't call talloc a small piece of functionality. Also, the LGPL explicitly states that dynamic linking suffices, and also explicitly covers the inclusion of bits from header files. I do agree that the LGPL makes it non-trivial to statically link a proprietary application with the library; personally, I think it would help if the FSF had standard exception language libraries could use if they don't care about relinking and only care about requiring source for modified versions of the library. That said, I don't write proprietary software, so that doesn't affect me. :)
- haberman 15y ago> Also, the LGPL explicitly states that dynamic linking suffices Good point, I thought that the requirement was substitutability and dynamic linking was just given as an example of how a library can be substitutable, but it appears that dynamic linking is sufficient. > and also explicitly covers the inclusion of bits from header files. It's not clear to me exactly the meaning of these sections; if including bits from header files is truly allowed, then you could effectively circumvent the substitutability goal by simply inlining the whole library. > That said, I don't write proprietary software, so that doesn't affect me. :) I don't either, but I write a BSD-licensed library that I absolutely want to be usable with proprietary software without any difficulty.
- nviennot 15y agoWriting a pair of constructor/destructor is not a problem. tmalloc() adds a layer of complexity that is unhealthy to C programs. Are the few seconds you save by using tmalloc() really worth the hours spent to debug memory leaks 6 months later when the project got big and complex with multiple collaborators ? No. Introducing side-effets under the table is the best way to confuse other C programmers when they read your code. It's not worth it. Memory management is part of C, and will always be. If you don't like it, switch to another language.
- seabee 15y agotmalloc() is memory management. As are other, alternative memory allocation strategies, chosen for better performance for certain applications. The notion that there is only one right way to do things is quaint, even if you're talking about Python.
- palish 15y agoHi, welcome to HN. I'm curious, did you just find HN or did this particular article motivate you to create an account? (I wish there were a private message system.)
- nviennot 15y agoI lost my previous account password that I haven't used in a very long time...
- rayiner 15y agotmalloc() is a structured way of doing memory management within C. It's no different than all the other ad-hoc structured (or more often unstructured) memory management schemes that evolve within C programs, with the exception that it's semantics are at least documented.
- daxelrod 15y agoI understand that talloc adds complexity, but what large project with multiple collaborators does not already have some memory management policy? C forces you to explicitly communicate who is responsible for allocating and freeing each piece of memory. Isn't talloc just providing a mechanism that helps with this? If talloc is part of a project's memory management policy, contributors not using it consistently are by definition not following the policy consistenly. This would be a problem regardless of mechanism.
- malkia 15y agoHow does it deal with cycles, or leaning ownership?
- jemfinch 15y agoWhat's leaning ownership?
- chmike 15y agoThis is exactly the qestion I was asking myself. What if the child block needs to be deallocated before the parent block ? What if block dependency is changed ? These functions are to be used with particular conditions which aren't clearly specified. When the conditions are met then it is a clear benefit to use them. But these add complexity and new pitfalls to inexperienced programmers.
- JoachimSchipper 15y agoI don't think it deals with cycles at all (it's hierarchical, after all), but talloc does allow you to do funky stuff with object ownership - it uses reference counting and features a talloc_steal() function to move ownership.
- Locke1689 15y agoOne tricky thing here is that this does break the normal memory management semantics. That is, while one nice thing about malloc/free is that you can drop pretty much any implementation of a user-space memory allocator in there without changing your code, this is no longer the case. Whatever replacement allocator you use will have to change its interface, and probably its data structure, to fit the new model. Edit: Also, how is this better than doing something like this struct some_complex_struct { ... }; free_some_complex_struct(struct some_complex_struct * scs) { ... }
- daxelrod 15y agoI don't understand why talloc preclues a different malloc/free implementation. According to the article, talloc is a malloc wrapper. If that's the case, why shouldn't you be able to drop in whatever allocator you want? talloc should still use it under the covers.
- Locke1689 15y agoAh, I didn't see it's a wrapper. In that case, what's the point of using it at all? If it doesn't help to optimize the memory allocation, it seems more like a macro and less like a library.
- zbowling 15y agowhy I'm glad to be writing objective-c and getting free reference counting built it :-)
- hardy263 15y agoExcept apple advises you not to use reference counting when trying to debug. So when reference counting doesn't work, you still have to go back and manually count the references yourself...
- ryannielsen 15y agoApple doesn't advise developers to avoid ref counting, it advises them to avoid using -retainCount. -retainCount is a weak debugging tool, at best. It doesn't take into account autorelease pools, to start, and any framework-specific memory management is ignored. Its use is discouraged for good reason. For better tools, check out Instruments, MallocDebug or leaks. Or read Apple's Memory Usage Performance Guidelines, specifically the "Finding Memory Leaks" section. [1] Apple's Technical Note TN2124 "Mac OS X Debugging Magic" even has, in the Cocoa section, an example of how -retainCount can be misleading and confusing. [2] Nowhere does Apple advise Cocoa developers to avoid ref-counting. Cocoa's ref-counting/GC'd memory management system is considered a huge advantage for the platform. [1] http://developer.apple.com/library/mac/#documentation/Performance/Conceptual/ManagingMemory/Articles/FindingLeaks.html http://developer.apple.com/library/mac/#documentation/Perfor... [2] http://developer.apple.com/library/mac/#technotes/tn2004/tn2124.html http://developer.apple.com/library/mac/#technotes/tn2004/tn2...
- DougBTX 15y agoAre you mixing up garbage collection and reference counting? I can understand a suggestion to turn off garbage collection if you want to debug a non-garbage-collected, reference counted piece of code. Otherwise, a link to documentation which describes what you are talking about would clarify things.
- CountHackulus 15y agoOr you could use C++ and RAII. It's an excellent fit for the problem that's been presented. Though I do understand, there are some very special projects that can't use C++, and must use C. I'd say in most cases, the extra machinery you get with C++ outweighs the small performance penalty you get by going with straight C.
- copper 15y agoFWIW, with C++, the advantages of linking to tcmalloc make it the default for any code that I write that isn't intended to run as a daemon -- and even there, unless I hit tcmalloc's limit of not releasing memory back, it's still a viable choice.
- ballard 15y agoGoogle perftools are bloody awesomesauce.
- Eliezer 15y agoI look forward to seeing the rest of Common LISP implemented in C too.
- deleted 15y ago[deleted]
- kragen 15y agoEliezer, did somebody break into your account to post that comment? Since when does Common Lisp have any kind of manual deallocation, let alone manual deallocation of hierarchical memory pools?
- mtk 15y agowas surprised to see no mention of: http://www.gnu.org/s/hello/manual/libc/Obstacks.html http://www.gnu.org/s/hello/manual/libc/Obstacks.html seems like a very similar concept / means to an end?
- latitude 15y agotalloc is based on my library called halloc [1], which stands for hierarchical allocator. The idea is that every malloc'ed block can be assigned a parent block. Freeing the parent block also frees all its children in a recursive fashion, and that's about it. I cooked up halloc several years ago when implementing IKE (the IPsec keying protocol) library and it proved to be very useful for reducing the verbosity of the code without loosing any clarity. Tridge fluffed up halloc to make the API more fit and convenient for Samba code, and talloc came out. I still like my version better though because it is simpler and (subjectively) more elegant. [1] http://swapped.cc/halloc http://swapped.cc/halloc
- FooBarWidget 15y agoNo offense intended, but I have to say I was a little bit disappointed by halloc. I expected it to use a custom allocation scheme, like malloc()ing large chunks of memory in one go in order to reduce malloc() time and space overhead and to increase locality of reference. Instead it looks like it just malloc()s every object separately. I'm sure some people will find halloc useful, but in my C++ project where I already use smart pointers I didn't see any advantage in using halloc. It doesn't look like talloc is any better.
- rwmj 15y agoDo you have evidence that rolling your own suballocator under malloc is better than just using malloc. By "evidence" I mean you actually measured it, not arm-waving.
- FooBarWidget 15y agoYes. For example my app has a hash table which it fills upon every request. After this initial filling, not more hash table entries are set. At the end of the request the hash table is deleted, and a new one is created at the next request. I improved performance a bit by clearing the hash table instead of deleting and recreating it. What gave me an even bigger boost in performance was by using a custom allocator that allocates big chunks of memory in a single operation. The thing is my app is already not doing a lot of work per request so almost any speedup is noticeable.
- dhruvbird 15y agoHow does this work for a case like DOM nodes? Where each child is constantly re-assigned parents? Is there any way to reset the parent?
- bluetech 15y agoYes, and you can also have multiple parents with a reference-count semantics. void * talloc_steal(const void * new_ctx, const void * ptr); The talloc_steal() function changes the parent context of a talloc pointer. It is typically used when the context that the pointer is currently a child of is going to be freed and you wish to keep the memory for a longer time. NOTE: It is possible to produce loops in the parent/child relationship if you are not careful with talloc_steal(). No guarantees are provided as to your sanity or the safety of your data if you do this. There's also talloc_reparent where you explicitly choose the parent to change.
- dhruvbird 15y agoThis is nice!! Thanks!!
- dhruvbird 15y agoOn the whole, I think talloc() is targetting a very specific use-case and if you have that it would be great to use (just like obstack).