9 ms·
Guide to Advanced Programming in C
- sdegutis 13y agoI like the idea behind this article, but I'm a little skeptical of some of its advice. For instance, I don't think it's a good idea to recommend using Boehm GC as a general solution to avoid memory management. It can't really tell the difference between pointers and pointer-sized integers, which means it usually leaks memory.
- pfacka 13y agoI see, but could you please provide better alternative? I couldn't find anything more alive and representative than Boehm GC.
- sdegutis 13y agoC is inherently not suitable for a GC. If you want automatic memory management, it's better to use something like Go. I'm not saying this is a bad GC library, just that I'm not sure I would recommend it for general use when writing C code.
- TheCoelacanth 13y agoSo to avoid using a conservative GC, you should instead use Go, a language which also has a conservative GC.
- 616c 13y agoThis is not to troll, but Nimrod (which has been mentioned a few times on list) is a language that looks something like mainly Pascal or Python and builds C code that will be subsequently compiled with a soft-realtime garbage collector or without (it does code elimination, GC included, if you are clever). If you are nuts you can opt to force using the Boehm GC with it, but do not expect great things out of it. There is also Rust. I am sure if you spend time on HN you have heard a lot about the latter.
- zurn 13y ago> it usually leaks memory Citation needed. Integers matching valid pointers to allocated objects are rare - only common on 32-bit when running near limits of virtual memory space. Which is a bad idea anyway. Language runtimes that eventually move off Boehm, such as Mono, usually cite other reasons.
- greenyoda 13y agoI'm not sure how malloc/free could be considered "advanced" C programming techniques. It's pretty hard to write any non-trivial program in C without using malloc/free and knowing the differences between static, stack-allocated and heap-allocated memory.
- pfacka 13y agoThe challenging part is to get details of using them correctly and avoid security and portability issues and I consider it neccesaty introduction for section dealing with memory management.
- spc476 13y agoAnd he didn't get all the details right. realloc(valid_pointer,0); IS defined (in C89 no less) to act like free(). It is NOT operating system dependent.
- pfacka 13y agoTrue, but return type of free() is void, thus it says nothing about return value. The POSIX says: "If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned."
- adultSwim 13y agoYou'd be surprised with how far you can get without dynamic memory or recursion... Still, I agree with your general sentiment. That some of these are "advanced" topics is troubling. Too many programmers today don't know how a computer works. Our ideas of "mastery" are way too low.
- erobbins 13y agoin the modern climate of bootcamped rails hackers, knowing that memory even needs to be allocated at some point is advanced.
- 13y ago
- antirez 13y agoBoehm is a bad advice in general... there are situations where it could work maybe, like if you implement an interpreter for the fun of it or something like that. For many kind of programs reference counting is the way to go for C, it still is manual, but an order of magnitude safer...
- freyrs3 13y agoIt also makes profiling much harder. Boehm doesn't play nice with valgrind without a fair bit of work.
- adultSwim 13y agoWorth reading.
- ghswa 13y agoI'm enjoying this so far although a couple of things have made me scratch my head in the example code allocating a vector. int create_vector(struct vector *vc, int size) { vc->data = 0; vc->size = 0; if (vc == NULL) { return VECTOR_NULL_ERROR; } /* check for integer and SIZE_MAX overflow */ if (size == 0 || size > SIZE_MAX) { errno = ENOMEM; return VECTOR_SIZE_ERROR; } Accessing the fields of vc before the NULL check seems like an error. Also, would it not be simpler to change type of the size parameter to size_t so that it's the same type as the size field of the vector struct?
- pfacka 13y agoYep, that is definitely error
- vinkelhake 13y agoYes, by having the NULL check after those members have been accessed, you're telling the compiler that vc cannot be NULL (because accessing the members if vc is NULL would be undefined behavior). The compiler can (and GCC will) eliminate the NULL check completely.
- ghswa 13y agoDoes gcc make any kind of assumptions about signed-overflow? For example, would it eliminate the comparison in this code: int8_t i; /* ... */ i += 1; if (i == 0) { /* ... */ }
- vinkelhake 13y agoYes, it'll exploit the fact that signed overflow is undefined. int incr(int i) { int old = i; i += 1; if (i < old) return 1; return 0; } GCC optimizes this to 'return 0;'. EDIT: Updated to a better example.
- silentbicycle 13y agoWould you mind elaborating on that? I don't see this behavior with GCC 4.2.1 on OSX or OpenBSD at -O3 when I look at the disassembly.
- bstamour 13y agoIs the phrase "locator value" defined by the C standard? I've never heard of it before. I always knew the 'l' in "lvalue" as meaning "this expression can appear on the left-hand side of an assignment operation." EDIT: Found the answer (thanks, draft C11 standard.) Section 6.3.2.1 contains the definition of lvalue, and it does not use the phrase "locator value" at all. However if you want to use it as a reminder that modifiable lvalues can be assigned to, then more power to ya :-) I can't help it - I'm a sucker for standardese.
- deletes 13y agoHow does this check for anything??: if ( size == 0 || size > SIZE_MAX ) {... size is not given a type, but if it is a size_t, then the comparison is constant and if statements is always false, unless size == 0. The second part is useless, since the maximum size of size_t == SIZE_MAX.
- jwise0 13y agoWorse, it is not even a certain check against integer overflow. A particularly large expression could have overflowed substantially enough that it becomes positive again! This, in fact, was the source of a vulnerability in PHP [1]. (The way they 'resolved' it, uh, isn't much better.) There is almost never a magic bullet for integer overflow. Programming secure systems requires thought. [1] http://use.perl.org/use.perl.org/_Aristotle/journal/33448.html http://use.perl.org/use.perl.org/_Aristotle/journal/33448.ht...
- deleted 13y ago[deleted]
- comex 13y agoNote that since it says 'size == 0', it's not merely "not a certain check", but a check that passes for almost all overflowing values :)
- tterrace 13y agoHmmm, that looks oddly familiar: http://use.perl.org/use.perl.org/_Aristotle/journal/33448.html http://use.perl.org/use.perl.org/_Aristotle/journal/33448.ht...
- pfacka 13y agoIt seems that I have totally messed up whole vector example by last minute changes. The check was meant to illustrate what eliteraspberrie described in previous comment.
- bch 13y agoThe article says two different things about how free() works: 1. In case of NULL pointer free does no action. 2. In "double free corruption" section, it says "Could be caused by calling free with pointer, which is [..] NULL pointer" So: which is it? Otherwise, there's no point in the NULL/assert() dance, you can freely free() with impunity: struct foo *a, *b, *c; a=NULL; b=NULL; c=NULL; a=malloc(sizeof *a); b=malloc(sizeof *b); c=malloc(sizeof *c); if(!(a && b && c)) {free(a); free(b); free(c); return 1;}
- yan 13y agoWhat the article intended to say is calling free() on the same pointer twice is considered 'double free'. Also an issue with the c++ delete operator. Calling free() on NULL is a no-op.
- EpicEng 13y agoBoth. free(null) is fine. free(something_already_freed_or_not_returned_by_malloc) (and friends) is undefined behavior.
- eliteraspberrie 13y agoGood advice. Integer arithmetic is one of the trickiest aspects of C, and dangerous in combination with the manual memory management. For more information, see the free chapter of TAOSSA: http://pentest.cryptocity.net/files/code_analysis/Dowd_ch06.pdf http://pentest.cryptocity.net/files/code_analysis/Dowd_ch06.... There are (were) a couple bugs in the example code. Here are a some guidelines that will help avoid those, and most problems with integer operations and the heap in general. First, don't mix unsigned and signed types in arithmetic; and always prefer the size_t type for variables representing the size of an object. Second, check for overflow before an operation, not after, like so: if (size > SIZE_MAX / 2) { goto error; } newsize = size * 2; Third, always double-check the arguments to memory allocation functions, especially for zero, because the result is not always well defined. if (size >= SIZE_MAX - n) { goto error; } foo = malloc(size + n); foo[size] = ...;
- sirclueless 13y agoIn particular, the code snippet from the blog post, if (size && size > SIZE_MAX) { errno = ENOMEM; err(1, "overflow"); } is a total no-op. It can't ever be true, the compiler might as well just remove the whole thing.
- nkurz 13y agoThe chapter sounded interesting, but your link didn't work. Here's a fixed up version: http://pentest.cryptocity.net/files/code_analysis/Dowd_ch06.pdf http://pentest.cryptocity.net/files/code_analysis/Dowd_ch06.... (Not your fault. I too miss the good-old-days when copying a PDF link from Google didn't involve multiple steps or URL decoding.)
- cjensen 13y agoWow. That's full of falsehoods like these: "What happens is that variable i is converted to unsigned integer." No: 'long i' is converted to 'unsigned long'. "Usually size_t corresponds with long of given architecture." No: For example, on Win64 size_t is 64 bits whereas long is 32 bits. If you're going to write about Advanced Programming, you should be careful to actually be correct.
- ahy1 13y ago> Wow. That's full of falsehoods like these: > "What happens is that variable i is converted to unsigned integer." No: 'long i' is converted to 'unsigned long'. Actually, unsigned long is an unsigned integer. He didn't write unsigned int. > "Usually size_t corresponds with long of given architecture." No: For example, on Win64 size_t is 64 bits whereas long is 32 bits. "Usually" is the keyword here. He could have said "Usually size_t has at least the same amount of bits as long" and it would be better related to the referred rule.
- zurn 13y agoThe "short" long is fantastically tasteless, it's not like source compatiblity with Win16 is has resulted in 64-bit builds of apps appearing.
- pfacka 13y agoI agree that wording is bit unclear. I will try to find better one, thanks.
- Scene_Cast2 13y agoThe sum() example does not work as the article says it does. Under Visual Studio 2013 compiling for x86 (C++ compiler, but C++ also has integer promotion rules), the function returns zero. The reason is: 65535 = 2^16-1. uint16_t is signed, therefore it has 15 bits to represent the value. When executing "int16_t a = 65535;" under a debugger, "a" is set to -1.
- EpicEng 13y agoIt's actually undefined behavior.
- pfacka 13y agoThanks, I tested with GCC and Clang on Linux on 64-bit x86, GCC+OpenBSD on 32-bit+x86 and GCC+Linux on PowerPC 603. The point I was aiming for is that described operation is indeed undefined.
- burstmode 13y agoIf uint16_t really is a signed 15bit value under VS2013, somebody in the compiler development department has a great sense of humor.
- Scene_Cast2 13y agoMy bad - that's a typo. I meant int16_t, the same as in the sample code.
- arunc 13y agoWondering why C++ style has been followed, at least, for the * association with the type and not with the variable.
- rquirk 13y agoWhat about that funky braces style? Code with wrong braces or whitespace messups indicate a lack of care. If the coder can't put a { in the right spot, what else has he screwed up?
- deleted 13y ago[deleted]
- aktau 13y agoJust a small question: __m128i c = _mm_set1_epi16(2) __attribute__((aligned(16))); Don't gcc/clang take care of aligning that type automatically? Isn't the attribute thus redundant?
- zurn 13y ago+1 for promoting Boehm GC. GCC uses it. edit: also Inkscape, w3m.
- jheriko 13y ago"Check for NULL at beginning of function or blocks which are dereferencing pointers to dynamically allocated memory" I really strongly disagree with this. Better for it to crash if you expect this memory to always have been allocated. This way you can fix your bug instead of putting your app into some potentially unexpected state... if allocations fail it doesn't make sense to just 'carry on anyway' in many situations.
- pfacka 13y agoNoted, thanks, failed allocation is definitely good candidate for fail fast scenario.
- dkersten 13y agoI disagree. Unless you can know for absolute certainty that there is no cleanup required, it is not a good idea to simply crash and leave resources in an inconsistent state. Otherwise, you need a way for the cleanup code to dispose of the resources and leave the system in a consistent state before crashing. Ok, its obviously good practice to develop your software in a way that a random outage doesn't leave anything in an inconsistent state, but a lot of software fails to do this. At the very least, you may end up with something like when a server crashes and cannot restart because the port is still considered in use by the OS.
- jheriko 13y agoif you are writing software which has to have some kind of guaranteed reliability then maybe there is a sort of argument for this - but unless you are handling the failure case in a deterministic and well behaved way then I still think that is a much worse state to be in than a crash. Even in production. In either case you are putting the system in a potentially weird or unrecoverable state - the difference with a hard crash is that you know straight away and its easier to debug, even after its shipped. Imagine the customer error report or imagine its a critical system and it starts making mistakes... For most software smoke testing heavily is enough to make it super rock solid. I know that most software does not do this - just using a web browser or a smartphone makes it painfully obvious that even the big software houses have some seriously shoddy practices and that testing gets seriously neglected (it may be that its impractically big... i stuggle to buy that tbh)