8 ms·
Inside the C Standard Library
- asveikau 8y ago> One interesting tidbit is that errno is sometimes not a global variable at all, but a macro for (*_Error()). Having to set a real data object immediately after performing hardware floating point ops would break the FPU pipeline. Allowing the check to be deferred until requested with this _Error() function doesn’t break the pipeline. The best reason I can think of to make errno a function rather than a global integer is due to threads. The function can handle thread local storage. With a simple declaration of "extern int errno;" all threads get the same errno and clobber each other. Nowadays there are things like __thread or c++11 thread_local but those mechanisms didn't exist when I first saw this pattern.
- aaronmdjones 8y agoIndeed that is the reason. On my system, I have: #define errno (*__errno_location ())
- kensai 8y agoIs this book worth having as the K&R one or is it too outdated considering we are upon C2X?
- simplicio 8y agoI read through it and found it interesting for the historical info and as a way to review the standard library. But if you're trying to get better at C, there are probably a lot of resources that are better uses of your time. It's pretty dated, the structure is a little hard to follow (he goes in alphabetical order of the header names, which given that a lot of the libraries reference each other, makes for somewhat confusing reading), and since he isn't targeting any particular architecture, the OS dependent stuff is pretty handwavy.
- pvitz 8y agoI think similar like Simplicio, but I would like to point you to Hanson's "C Interfaces and Implementations" which will show you how to organize your C code well and how to implement robustly ADTs. It is pretty educative and enjoyable to read.
- loeg 8y agoNo, but I don't think the K&R one is worthwhile either. It's far too outdated and teaches some bad practices. C99 in particular brought major improvements to the C language that you should be using. And C11 and C17/18 continued to refine the language in incremental ways.
- saagarjha 8y ago> I don't think the K&R one is worthwhile either. It's far too outdated and teaches some bad practices. It’s old, but I didn’t see anything horrible in the book. What bad practices does it teach?
- loeg 8y agohttps://news.ycombinator.com/item?id=9333520 https://news.ycombinator.com/item?id=9333520
- saagarjha 8y agoNot a fan of Learn C the Hard Way, but I will agree with you that K&R is a bit light on details of how/why to write safe C code. I stand by K&R being the best book for learning C that I’ve read, though it goes nicely with a good application security curriculum ;)
- rurban 8y agoHe really should have noted the misdefinition of the truncating versions in string.h. Eg. strncpy doesn't leave enough room for the final \0, so the string may be the unterminated. A serious POSIX error. This worse than the wchar_t iswalpha trouble with older UCS-2 implementations (Windows, AIX). Or the implementation trouble with memset, which either can be optimized away without a compiler barrier, or on modern hyperthreaded microarchitectures which violate load-store ordering and thus need a full memory barrier (mfence). no single libc does it properly, not even crypto libs, such as libsodium.
- deleted 8y ago[deleted]
- begriffs 8y agoThanks for pointing this out. Can you give an example of the strncpy problem? Also is there a precaution programs can take to use memset more safely?
- pjmlp 8y agostrncpy does not add a '\0' if the string does not fit into the target buffer, it just cuts it down to the max size. So to be on the safe side you need to wrap strncpy calls into a function that always puts a '\0' on the very last index of the destination string buffer. As for memset, the best workaround is to use OS specific variants that aren't subject to such issues, for example SecureZeroMemory() on Windows.
- rurban 8y agoSecureZeroMemory() is insecure on hyperthreaded systems, and probably on normal SMT systems also. It only uses a compiler-barrier, ensuring that it is not optimized away. This is not secure, it is just a basic precaution against wrong compiler optimizations. In reality the compiler should know about the libc memset and memzero and refuse to optimize it away, and memset_s/SecureZeroMemory needs a memory barrier.
- loeg 8y ago
- rgovostes 8y agoWell, I was not expecting to read a 10,000-word article about the C standard library this morning. I haven't been programming in C as long as many here, but over the past dozen years or so I've always felt that the standard library is a half-baked hodgepodge of inconsistencies and a minefield of gotchas that have caused innumerable security vulnerabilities. Why is open() in fcntl.h and close() in unistd.h? Why does bcopy go src,dst but memcpy go dst,src? Why did they make strlcpy return a different value than strncpy, so you can't improve a whole codebase with one recursive sed? There are many questions I still have, but I learned a bunch of new things from this article. Thanks to the author for putting the effort into digesting the source material for us. On a tangential topic, I don't know of a good reference for exploring what is included in the standard library. Yes, of course there are manpages for individual families of functions. But I don't know how anyone would find out about, say, strtok or strsep without encountering them in some existing code. I had a university assignment which would have been trivial if anyone in the class knew that fts.h existed.
- quietbritishjim 8y agoWhy is open() in fcntl.h and close() in unistd.h? Why does bcopy go src,dst I don't know about the rest, but open(), close() and bcopy() are not in the C standard library. They are either POSIX or Linux specific. The C standard library functions for files start with f e.g. fopen(), fclose(). On a tangential topic, I don't know of a good reference for exploring what is included in the standard library. The classic book The C Programming Language covers most, if not all, of the standard library. The whole thing can be found in an appendix. Admittedly reading an appendix isn't normally the best way to learn about something, but the library is so small and the amount not already covered in the text is even smaller so it wouldn't take long to skim. Of course it doesn't cover Unix-specific functions, like the unistd.h header; for that, I recommend Advanced Programming in the Unix Environment.
- tonysdg 8y agoI'd add that The C Programming Language should be seen as a tutorial (albeit an older one that no longer is fully up-to-date in best practices, in my experience) that can serve as a reference in a pinch (it skimps a bit on the corner cases if memory serves). Advanced Programming in the Unix Environment, conversely, should be seen as a reference book with fantastic examples and discussion on error handling -- yes, you can use it as a tutorial, but it's a tad dense and you'll likely never need a lot of it.
- nneonneo 8y agoWonderful exposition. One nitpick: please do NOT suggest seeding random functions with &main. Although ASLR is widespread, it is by no means guaranteed, and the default compilation options on many platforms will still produce non-PIE executables. This would render your seed completely static. A separate problem is that ASLR on some platforms has limited entropy. For example, on 32-bit Linux platforms the executable randomness has between 8 and 16 bits of entropy - which means your program would only ever have a small, finite range of possible random streams (16 bits correlating to less than 1 day of time() output taken once per second). Don’t rely on ASLR for randomness. At best it can be used as an additional source of random data. The C standard provides no mechanism for obtaining high quality entropy, as some environments (especially freestanding ones) may be unable to offer such randomness.
- loeg 8y agoOn recent unix-likes, use getrandom() or getentropy(). On Windows, use CryptGenRandom, RtlGenRandom, or BCryptGenRandom (I'm not very familiar with Windows, sorry).
- loup-vaillant 8y agoAnd on embedded systems with no RNG in sight, use a deterministic random number generator, with a seed generated from another system. Use fast key erasure to ensure forward secrecy. https://blog.cr.yp.to/20170723-random.html https://blog.cr.yp.to/20170723-random.html If the device ever powers down, you will need 32 bytes worth of persistent storage, so you don't repeat the random stream every time you boot up.
- loeg 8y agoI don't think this advice is really detailed enough to cover the breadth of the embedded space and/or comes with enough caveats about lack of writeable storage and reused keys. Also, sometimes people use 'embedded' to refer to things that are really full-sized computers (e.g., ARM with ordinary read/write storage), and on those you should just use ordinary CSPRNG implementations like Fortuna (which is not deterministic but does implement key erasure).
- andrepd 8y agoNitpick: text colour is a light gray (#3A4145). Even tough I have a good eyesight I was struggling. Please consider accessibility when designing your web pages :) Thankfully the web is hackable so I could just inspect element > change color to #000000.
- gwillz 8y agoThat's interesting because the contrast is around 12:1 - which is well above the AAA accessibility guidelines for small text. Perhaps the font and weight are contributing factors?
- deleted 8y ago[deleted]
- avar 8y agoThe author describes ctype.h as "kind of a letdown" and as "not sufficient for international text processing" and then proceeds to write an isalpha() that's not portable C because he assumes an ASCII derivative (the code won't work under e.g. an EBCDIC system). A lot of things in the C Standard Library look odd or pointless until you do some digging and realize that they were often designed to be portable to systems that are either really obscure today, or don't exist anymore.
- begriffs 8y agoYou mean where I'm comparing whether the character is between 'a' and 'z'? Yeah I'm aware that's an ASCII thing, and that the standard was designed to accommodate other encodings such as EBCDIC. My `isalpha` example was more about showing how such a macro ought to be careful to not evaluate its argument more than once. Did you read the rest of that section? I have arguments about why ctype appears to be insufficient.
- avar 8y agoI just meant to add a bit of info to an already excellent overview, not to detract from it. Yes, most of the rest of the section is about unrelated pitfalls. The non-ASCII use-case is something that's often forgotten these days, and until you mentioned it here there was no indication in the document that you were aware of it. I.e. the section discusses "non-English" non-8-bit languages etc. But the most basic use of ctype.h is e.g. checking if something is in the 0-9a-zA-Z range for the purposes of a basic parser, and you can't do that portably in C without ctype.h, or effectively writing your own ctype.h implementation.
- begriffs 8y ago> until you mentioned it here there was no indication in the document that you were aware of it Good point, and thanks for pointing it out. People can't read my mind and the example might give people the wrong idea. I'll add a note to the example.
- caf 8y agoThis snippet is not correct: #include <limits.h> #if ULONG_MAX != -1UL #error "This code requires 2s' complement arithmetic" #endif The #error case will never be called, because the behaviour of the unsigned types is completely defined in C - it doesn't matter whether the system uses 2s complement or not, the result of applying the negation operator to 1UL must always be ULONG_MAX.
- longcommonname 8y agoIf you use assertions you can combine the expression with & & "this error shouldn't happen because..." to get more context.
- mnaydin 8y ago"C99 provides a macro SIZE_MAX with the maximum value possible in size_t. C89 doesn’t have it, although you can obtain the value by casting (size_t)-1. This assumes a twos’ complement architecture,..." The architecture does not have to be two's-complement. The value of the expression (size_t)(-1) will always be SIZE_MAX, the largest number that can be represented in the size_t type. The bit pattern may change depending on the architecture, but the value shall equal to SIZE_MAX.