9 ms·

Show HN: Smart pointers for the C programming language

126 points by Snaipe 12y ago

Snaipe 12y ago
Article: http://snaipe.me/c/c-smart-pointers/ http://snaipe.me/c/c-smart-pointers/ Feedback appreciated, be it on the article, project or website ! Be wary that english is not my mother tongue, so there might be some grammatical weirdities and such.
- eatonphil 12y ago
  Wow, this is seriously great! This is exactly what I needed for my project, Viola[1], that provides an abstraction on top of libCello[2] to allow you to write really high-level C. This really helps make Viola a little more high-level. I am really grateful for your blog post! [1] - https://github.com/eatonphil/Viola https://github.com/eatonphil/Viola [2] - https://github.com/orangeduck/libCello https://github.com/orangeduck/libCello
  - Snaipe 12y ago
    I've read with great interest the source of libCello when I first saw it a few days ago on reddit, it seems like a fun experiment. Good luck with your project ! :)
- sepeth 12y ago
  I first thought about wrapping malloc to register all allocated data, and then free everything before exiting, but it wasn’t a satisfying solution. Quick question, Isn't this the same with just leak all the memory that you've allocated. Because, after a process died, operating system will just take it back the memory it used.
  - Snaipe 12y ago
    Yep, but if I just let the system get all the memory back, valgrind would have yelled at us for leaking memory, and it would have been detected by the testing system, slapping us in the face with a malus on our final grade.
falcolas 12y ago
Looks to be GCC specific, using several GCC specific variable attributes. Quite a bit of preprocessor macro magic as well. Also, my succinct and unprofessional response to apply.h[1]: <Screams and runs away> A slightly more professional response: What happens when you hit the implicit macro-based recursion limit? [1]https://github.com/Snaipe/c-smart-pointers/blob/master/src/apply.h https://github.com/Snaipe/c-smart-pointers/blob/master/src/a...
- Snaipe 12y ago
  GCC and Clang specific, as said in the readme page. I know, apply.h is ugly, but I did not want to lose too much time on that (and it was easy to dump these lines from a shell command), as it is only used for an optional macro. When the recursion limit is hit, every parameter after that is ignored, and I doubt someone will ever use a struct with 64 members needing destruction, considering of course they would use this, and use the completely optional "DESTRUCTOR" macro helper. But yes, I get it, I'll probably find the time to implement some kind of recursion based off OBSTRUCT/DEFER that we often see on stackoverflow.
  - minthd 12y ago
    This looks great. From the examples, it seems that it is fully compatible with standard pointers with no need of casting. Is it true ? If so, it could work well in simplifying development with embedded development libraries, say like the mbed.
    - Snaipe 12y ago
      Yup, these are just pointers, usage is still the same, and no cast/special functions are needed to manipulate them.
      - TheLoneWolfling 12y ago
        Well... you cannot call free on them, unless I'm mistaken. But other than that it's as normal. (This could be a problem if you want to call library code that frees a supplied pointer)
aninteger 12y ago
This is really neat, but I actually wonder if we aren't trying to solve a mostly solved problem. As a C programmer, yes, sometimes you forget to free memory that you've allocated, but then you run valgrind or some other leak checking tool and the problem is solved. When you add all these compiler specific attributes and introduce a lot of preprocessor macros I think you are creating a situation where you've created code that is easier to write, but potentially harder to read as there is now a lot more abstraction that a programmer has to wade through (assuming they need to debug that abstraction and not just trust that it just works). Now, if you're using a leak checker you can save that abstraction and switch back to using the standard free(3) library call and then 100% of C programmers will be able to follow the code.
- minthd 12y ago
  Isn't the whole point is to make sure the abstraction always work ?
  - Igglyboo 12y ago
    Isn't the whole point of C to have as minimal abstraction as possible?
    - Snaipe 12y ago
      Not really, otherwise even the standard library would not exist, and we would manually ask pages to the kernel every time we want memory. C, as a language, is indeed low level, but that does not mean that programs and libraries shouldn't build abstractions on top of the low level interface.
    - Retra 12y ago
      Not at all. The point of C is to have as much abstraction as possible while still allowing maximal control. And 'as much abstraction as possible' is a matter of historical context, since we learn all the time that there are better ways of doing abstraction than C does.
- Snaipe 12y ago
jheriko 12y ago
smart pointers are always dumb imo. GC or do it yourself - everything in between is always more of a headache in my experience. I've spent more time debugging reference counts in few code bases than debugging new/delete in many.
- Snaipe 12y ago
  Haha, I guess, but I think it's not about using them all over your code base -- there are always pros and cons in using RAII, smart pointers, GCs, and manual memory management, it's the programmer's job to balance it correctly.
- dicroce 12y ago
  smart pointers are better than GC or do it yourself imo. smart pointers and RAII allow me to explicitly model ownership and lifetime in a non imperative way. In addition, I am not subject to arbitrary GC runs with non deterministic amounts of work to do. Instead, objects simply go away as they are no longer needed in a completely deterministic way.
  - evincarofautumn 12y ago
    Naïve reference counting is deterministic, but it may also do an arbitrary amount of work on a deallocation because it reclaims objects eagerly. Deferred/incremental reference counting is much more suitable for, say, games, which is what it sounds like you’re thinking of.
  - pjmlp 12y ago
    I guess you never experienced a stop-the-world caused by cascading destructors.
- koenigdavidmj 12y ago
  This also provides unique_ptr, which doesn't use reference counting, and enforces rules very similar to those of Rust's borrow checker. They allow you to avoid memory leaks by ensuring that you either free any memory you allocate or pass it to another function (who must either free it or pass it to somebody else, ad infinitum).
kazinator 12y ago
This relies on support in the GCC infrastructure. That support is there thanks to the back-end support for C++. Calling it "for the C language" is misleading, insinuating portability at the language level. The title should be: "Smart pointers for the GNU C programming language". If you could have smart pointers in the C language via a couple of macros, it would have been done several decades ago. Of course, it can be done by a code transformation: make a C-like language which translates to C. Wait, that was done by someone named Bjarne Stroustrup in a project called "C with Classes". People didn't accept that was C though, even when he shortened the name to C++.
- Snaipe 12y ago
  I guess you're right. This was mostly an exercise on the attributes provided by gcc and clang/llvm, since I haven't seen anything really using RAII.
CardenB 12y ago
Forgive my ignorance, I'm still a bit of a beginner. Why is this particularly impressive? Has no one done smart pointers well for C? Smart pointers have been around a while, I thought there would be a ton of libraries for this by now.
- Snaipe 12y ago
  I've only been actively programming in C for the past three years, so I'm not that much informed either, but from my researches on the subject, I haven't seen any. The probable reason behind this is that most people programming in C wanting automatic memory management will use the Boehm-Demers-Weiser garbage collector (?)
  - asveikau 12y ago
    > The probable reason behind this is that most people programming in C wanting automatic memory management will use the Boehm-Demers-Weiser garbage collector The main reason behind it is this is not C.
- nanofortnight 12y ago
  You don't get RAII in standard C. This code uses the GCC-specific cleanup attribute to emulate RAII.
- tormeh 12y ago
  It is pretty impressive, actually. C, in general, does not have nice things. And it seems like this is on purpose. I think you're supposed to use C++ if you want nice things on top of C. That said, I've tried making green threads for C and it was a nightmare. I got something working though, so it's possible, but still a nightmare.
  - ANTSANTS 12y ago
    > That said, I've tried making green threads for C and it was a nightmare. I got something working though, so it's possible, but still a nightmare. By green threads, do you mean coroutines, or something more complicated? Try the libco library, it implements them in under a hundred lines of C + a small bit of assembly per architecture: https://gitorious.org/bsnes/bsnes/source/1a7bc6bb8767d6464e3ed7f1887398bfbd62dc3c:libco https://gitorious.org/bsnes/bsnes/source/1a7bc6bb8767d6464e3...
dahart 12y ago
I give it a thumbs up on the basis of the FAQ.
deleted 12y ago
[deleted]
- cbd1984 12y ago
  The FSF thinks the X11 License (what they call the MIT license) is compatible with the GPL, so there shouldn't be a legal problem: https://www.gnu.org/licenses/license-list.html#X11License https://www.gnu.org/licenses/license-list.html#X11License It's up to the Linux kernel team.
yason 12y ago
Running code when exiting scope is the key to many a things that would be useful, cool, or both. You can simulate some of the cases with a for loop. Other than that, you're out of luck unless you're willing to bite the bullet of compiler-specific features. If we forget about destructors and think about memory management only then (except for objects that are returned back) allocations could be done on the stack if only the stack was big enough. I think Linux could actually grow stack dynamically but it's usually limited to some minor size. Thus, I've sometimes emulated that with a chunk of heap-allocated memory that I use for carving new allocations one after another. Then, in some reasonable point in the execution I just reset the cursor back to the start of the big chunk, effectively "freeing" all allocations done by the functions called downwards from that point. It wastes memory but it's convenient as you basically have alloca() that uses heap. But you still need to manage any malloc()'d memory by yourself. Writing a simple garbage collector working on raw allocations is feasible, too, especially if you use the above heap allocated stack to limit the amount of longer-living allocations you do on the gc heap, and if you have a spot in the program that can stand a slight delay when the gc runs.
jahan-addison 12y ago
My trust in actually using this even for personal reasons would go 100x had you have written tests
- Snaipe 12y ago
  I have to polish a bit my python test suite, but it's coming ;)
endgame 12y ago
It's also a case study on how to make a tidy project using the autotools. The only thing I'd change is to commit a 0-byte m4/.keep file and then remove autogen.sh in favour of autoreconf. But why would you host autotooled projects on github, where they don't let you upload the release tarballs generated by make distcheck?
- Snaipe 12y ago
  Release tarballs can be uploaded on a third party, I usually don't like to commit binary blobs, especially releases, since you could just clone the release tag and run make distcheck. I'll commit a m4/.keep.
  - endgame 12y ago
    I'm griping about the fact that github removed its download functionality. Agreed that release tarballs shouldn't be committed in the repo.
mpu 12y ago
I think it is a nice idea, but I do not agree about the way to implement it. I would like to see more C extensions that work not only following C++'s ideas (dirty syntactic tricks) but are implemented using a proper C parser (like Cil) and principled program transformations (i.e. compilation). Maybe we need a classier framework than Gnu C and macros to toy around with C extensions...
rileymat 12y ago
Can someone explain the magic sauce that makes this work?
- Snaipe 12y ago
  I pretty much explain all that in the article[1], but here is the gist of it: * I implemented smalloc and sfree to respectively allocate and deallocate a memory block with prepended metadata * I use a GNU variable attribute called cleanup to run sfree when the variable goes out of scope * I made some macros to have some syntactic sugar on top of that [1] - http://snaipe.me/c/c-smart-pointers/ http://snaipe.me/c/c-smart-pointers/
  - rileymat 12y ago
    Thanks, the link I was clicking went straight to the Github repo. Did not see the article.
aidenn0 12y ago
Does it work with longjmp?
- Snaipe 12y ago
  Not tested, but since longjmp has __attribute__ ((noreturn)), it should.
  - aidenn0 12y ago
    ah, it uses function attributes to do that; nice. [edit] What about: fn A calls setjmp, calls... ...fn B allocates, calls... ...fn C which may or may not longjmp (so not marked noreturn).
    - Snaipe 12y ago
      If a function may or may not return, and it is not marked for inlining, even gcc cannot infer whenever the variable exits the scope, so no. You have to be extra careful anyways when using context switches.

126 points by Snaipe 12y ago

Article: http://snaipe.me/c/c-smart-pointers/ http://snaipe.me/c/c-smart-pointers/ Feedback appreciated, be it on the article, project or website ! Be wary that english is not my mother tongue, so there might be some grammatical weirdities and such.

eatonphil 12y ago
Wow, this is seriously great! This is exactly what I needed for my project, Viola[1], that provides an abstraction on top of libCello[2] to allow you to write really high-level C. This really helps make Viola a little more high-level. I am really grateful for your blog post! [1] - https://github.com/eatonphil/Viola https://github.com/eatonphil/Viola [2] - https://github.com/orangeduck/libCello https://github.com/orangeduck/libCello
- Snaipe 12y ago
  I've read with great interest the source of libCello when I first saw it a few days ago on reddit, it seems like a fun experiment. Good luck with your project ! :)
sepeth 12y ago
I first thought about wrapping malloc to register all allocated data, and then free everything before exiting, but it wasn’t a satisfying solution. Quick question, Isn't this the same with just leak all the memory that you've allocated. Because, after a process died, operating system will just take it back the memory it used.
- Snaipe 12y ago
  Yep, but if I just let the system get all the memory back, valgrind would have yelled at us for leaking memory, and it would have been detected by the testing system, slapping us in the face with a malus on our final grade.

falcolas 12y ago

Looks to be GCC specific, using several GCC specific variable attributes. Quite a bit of preprocessor macro magic as well. Also, my succinct and unprofessional response to apply.h[1]: <Screams and runs away> A slightly more professional response: What happens when you hit the implicit macro-based recursion limit? [1]https://github.com/Snaipe/c-smart-pointers/blob/master/src/apply.h https://github.com/Snaipe/c-smart-pointers/blob/master/src/a...

Snaipe 12y ago
GCC and Clang specific, as said in the readme page. I know, apply.h is ugly, but I did not want to lose too much time on that (and it was easy to dump these lines from a shell command), as it is only used for an optional macro. When the recursion limit is hit, every parameter after that is ignored, and I doubt someone will ever use a struct with 64 members needing destruction, considering of course they would use this, and use the completely optional "DESTRUCTOR" macro helper. But yes, I get it, I'll probably find the time to implement some kind of recursion based off OBSTRUCT/DEFER that we often see on stackoverflow.
- minthd 12y ago
  This looks great. From the examples, it seems that it is fully compatible with standard pointers with no need of casting. Is it true ? If so, it could work well in simplifying development with embedded development libraries, say like the mbed.
  - Snaipe 12y ago
    Yup, these are just pointers, usage is still the same, and no cast/special functions are needed to manipulate them.
    - TheLoneWolfling 12y ago
      Well... you cannot call free on them, unless I'm mistaken. But other than that it's as normal. (This could be a problem if you want to call library code that frees a supplied pointer)

aninteger 12y ago

This is really neat, but I actually wonder if we aren't trying to solve a mostly solved problem. As a C programmer, yes, sometimes you forget to free memory that you've allocated, but then you run valgrind or some other leak checking tool and the problem is solved. When you add all these compiler specific attributes and introduce a lot of preprocessor macros I think you are creating a situation where you've created code that is easier to write, but potentially harder to read as there is now a lot more abstraction that a programmer has to wade through (assuming they need to debug that abstraction and not just trust that it just works). Now, if you're using a leak checker you can save that abstraction and switch back to using the standard free(3) library call and then 100% of C programmers will be able to follow the code.

minthd 12y ago
Isn't the whole point is to make sure the abstraction always work ?
- Igglyboo 12y ago
  Isn't the whole point of C to have as minimal abstraction as possible?
  - Snaipe 12y ago
    Not really, otherwise even the standard library would not exist, and we would manually ask pages to the kernel every time we want memory. C, as a language, is indeed low level, but that does not mean that programs and libraries shouldn't build abstractions on top of the low level interface.
  - Retra 12y ago
    Not at all. The point of C is to have as much abstraction as possible while still allowing maximal control. And 'as much abstraction as possible' is a matter of historical context, since we learn all the time that there are better ways of doing abstraction than C does.
Snaipe 12y ago

jheriko 12y ago

smart pointers are always dumb imo. GC or do it yourself - everything in between is always more of a headache in my experience. I've spent more time debugging reference counts in few code bases than debugging new/delete in many.

Snaipe 12y ago
Haha, I guess, but I think it's not about using them all over your code base -- there are always pros and cons in using RAII, smart pointers, GCs, and manual memory management, it's the programmer's job to balance it correctly.
dicroce 12y ago
smart pointers are better than GC or do it yourself imo. smart pointers and RAII allow me to explicitly model ownership and lifetime in a non imperative way. In addition, I am not subject to arbitrary GC runs with non deterministic amounts of work to do. Instead, objects simply go away as they are no longer needed in a completely deterministic way.
- evincarofautumn 12y ago
  Naïve reference counting is deterministic, but it may also do an arbitrary amount of work on a deallocation because it reclaims objects eagerly. Deferred/incremental reference counting is much more suitable for, say, games, which is what it sounds like you’re thinking of.
- pjmlp 12y ago
  I guess you never experienced a stop-the-world caused by cascading destructors.
koenigdavidmj 12y ago
This also provides unique_ptr, which doesn't use reference counting, and enforces rules very similar to those of Rust's borrow checker. They allow you to avoid memory leaks by ensuring that you either free any memory you allocate or pass it to another function (who must either free it or pass it to somebody else, ad infinitum).

kazinator 12y ago

This relies on support in the GCC infrastructure. That support is there thanks to the back-end support for C++. Calling it "for the C language" is misleading, insinuating portability at the language level. The title should be: "Smart pointers for the GNU C programming language". If you could have smart pointers in the C language via a couple of macros, it would have been done several decades ago. Of course, it can be done by a code transformation: make a C-like language which translates to C. Wait, that was done by someone named Bjarne Stroustrup in a project called "C with Classes". People didn't accept that was C though, even when he shortened the name to C++.

Snaipe 12y ago
I guess you're right. This was mostly an exercise on the attributes provided by gcc and clang/llvm, since I haven't seen anything really using RAII.

CardenB 12y ago

Forgive my ignorance, I'm still a bit of a beginner. Why is this particularly impressive? Has no one done smart pointers well for C? Smart pointers have been around a while, I thought there would be a ton of libraries for this by now.

Snaipe 12y ago
I've only been actively programming in C for the past three years, so I'm not that much informed either, but from my researches on the subject, I haven't seen any. The probable reason behind this is that most people programming in C wanting automatic memory management will use the Boehm-Demers-Weiser garbage collector (?)
- asveikau 12y ago
  > The probable reason behind this is that most people programming in C wanting automatic memory management will use the Boehm-Demers-Weiser garbage collector The main reason behind it is this is not C.
nanofortnight 12y ago
You don't get RAII in standard C. This code uses the GCC-specific cleanup attribute to emulate RAII.
tormeh 12y ago
It is pretty impressive, actually. C, in general, does not have nice things. And it seems like this is on purpose. I think you're supposed to use C++ if you want nice things on top of C. That said, I've tried making green threads for C and it was a nightmare. I got something working though, so it's possible, but still a nightmare.
- ANTSANTS 12y ago
  > That said, I've tried making green threads for C and it was a nightmare. I got something working though, so it's possible, but still a nightmare. By green threads, do you mean coroutines, or something more complicated? Try the libco library, it implements them in under a hundred lines of C + a small bit of assembly per architecture: https://gitorious.org/bsnes/bsnes/source/1a7bc6bb8767d6464e3ed7f1887398bfbd62dc3c:libco https://gitorious.org/bsnes/bsnes/source/1a7bc6bb8767d6464e3...

dahart 12y ago

I give it a thumbs up on the basis of the FAQ.

deleted 12y ago

[deleted]

cbd1984 12y ago
The FSF thinks the X11 License (what they call the MIT license) is compatible with the GPL, so there shouldn't be a legal problem: https://www.gnu.org/licenses/license-list.html#X11License https://www.gnu.org/licenses/license-list.html#X11License It's up to the Linux kernel team.

yason 12y ago

Running code when exiting scope is the key to many a things that would be useful, cool, or both. You can simulate some of the cases with a for loop. Other than that, you're out of luck unless you're willing to bite the bullet of compiler-specific features. If we forget about destructors and think about memory management only then (except for objects that are returned back) allocations could be done on the stack if only the stack was big enough. I think Linux could actually grow stack dynamically but it's usually limited to some minor size. Thus, I've sometimes emulated that with a chunk of heap-allocated memory that I use for carving new allocations one after another. Then, in some reasonable point in the execution I just reset the cursor back to the start of the big chunk, effectively "freeing" all allocations done by the functions called downwards from that point. It wastes memory but it's convenient as you basically have alloca() that uses heap. But you still need to manage any malloc()'d memory by yourself. Writing a simple garbage collector working on raw allocations is feasible, too, especially if you use the above heap allocated stack to limit the amount of longer-living allocations you do on the gc heap, and if you have a spot in the program that can stand a slight delay when the gc runs.

jahan-addison 12y ago

My trust in actually using this even for personal reasons would go 100x had you have written tests

Snaipe 12y ago
I have to polish a bit my python test suite, but it's coming ;)

endgame 12y ago

It's also a case study on how to make a tidy project using the autotools. The only thing I'd change is to commit a 0-byte m4/.keep file and then remove autogen.sh in favour of autoreconf. But why would you host autotooled projects on github, where they don't let you upload the release tarballs generated by make distcheck?

Snaipe 12y ago
Release tarballs can be uploaded on a third party, I usually don't like to commit binary blobs, especially releases, since you could just clone the release tag and run make distcheck. I'll commit a m4/.keep.
- endgame 12y ago
  I'm griping about the fact that github removed its download functionality. Agreed that release tarballs shouldn't be committed in the repo.

mpu 12y ago

I think it is a nice idea, but I do not agree about the way to implement it. I would like to see more C extensions that work not only following C++'s ideas (dirty syntactic tricks) but are implemented using a proper C parser (like Cil) and principled program transformations (i.e. compilation). Maybe we need a classier framework than Gnu C and macros to toy around with C extensions...

rileymat 12y ago

Can someone explain the magic sauce that makes this work?

Snaipe 12y ago
I pretty much explain all that in the article[1], but here is the gist of it: * I implemented smalloc and sfree to respectively allocate and deallocate a memory block with prepended metadata * I use a GNU variable attribute called cleanup to run sfree when the variable goes out of scope * I made some macros to have some syntactic sugar on top of that [1] - http://snaipe.me/c/c-smart-pointers/ http://snaipe.me/c/c-smart-pointers/
- rileymat 12y ago
  Thanks, the link I was clicking went straight to the Github repo. Did not see the article.

aidenn0 12y ago

Does it work with longjmp?

Snaipe 12y ago
Not tested, but since longjmp has __attribute__ ((noreturn)), it should.
- aidenn0 12y ago
  ah, it uses function attributes to do that; nice. [edit] What about: fn A calls setjmp, calls... ...fn B allocates, calls... ...fn C which may or may not longjmp (so not marked noreturn).
  - Snaipe 12y ago
    If a function may or may not return, and it is not marked for inlining, even gcc cannot infer whenever the variable exits the scope, so no. You have to be extra careful anyways when using context switches.