21 ms·

Rob Pike: Notes on Programming in C

122 points by ssp 16y ago

rcfox 16y ago
I disagree with the include files section. As long as each file has a proper include guard, it won't matter how many times you've included a file. I'd much rather have the preprocessor figure out when and where to include a file than do it myself. Maybe this was a significant performance issue in 1989, but it shouldn't be anymore. That said, you should only include files that are direct dependencies. There's no need to include stdio.h in every other file.
- meese_ 16y ago
  I think that advice is just outdated. It's a different situation now that it's a standard practice and there are optimizations supporting it (http://gcc.gnu.org/onlinedocs/gcc-2.95.3/cpp%5F1.html#SEC8 http://gcc.gnu.org/onlinedocs/gcc-2.95.3/cpp%5F1.html#SEC8).
- palish 16y ago
  That's a naive viewpoint. It's more of a concern in C++ ... if your codebase grows to massive proportions, and you #include headers within headers, then modifying a header can result in ~60% of your codebase being rebuilt. (At work, I deal with this pretty much every day.) Including header files within headers is simply unnecessary, especially in C. You need to forward declare in header files, then include in source files where you actually use that header file.
  - rcfox 16y ago
    Suppose you've written some key data structure in its own file. Let's say it's a binary tree. The binary tree's functions are all implemented in bintree.c, and its functions, defines, and struct are all declared in bintree.h. Now, you want to write some application-specific functionality that works on your binary tree. You need to pass a pointer to the binary tree to your new functions. Of course, you implement these new functions in app.c, and the functions are declared in app.h so that other parts of your application can call them. You're going to need to include bintree.h in app.h to ensure that the compiler knows about the binary tree's structure. Otherwise, you'll have to make sure to always include bintree.h before you include app.h. Is that really something you want to be bothered with?
    - palish 16y ago
      In app.h, you should forward-declare "typedef struct bintree_t bintree;" Then, in each source file that #includes "app.h" and also CALLS binary-tree-related functions, you #include "bintree.h". Like I said, this is more of a concern in C++ where compile times are way, way slower than C. And in C++, it's very much worth it to forward declare in header files. It makes a big difference once your source code base has grown to epic proportions.
      - frou_dh 16y ago
        It's a technique, but not so applicable to say "you should" do it.
  - edsrzf 16y ago
    I totally agree. Not only will headers including headers cause files to rebuild unnecessarily, but it can cause build times to increase simply because the compiler has to compile all those extra headers each time they're included in a source file. (Precompiled headers can help, but they aren't always an option.)
  - gte910h 16y ago
    C is not C++. In C you shouldn't be chaining header files because you don't make function calls in header files in C code (well most good C code at least, people abuse macros sometimes). In C you should only include header files in header files if you have types in the other header files. It's okay to forward declare if you must, but its often quite a bit clearer to not. Additionally, you should always include all the h files that have your undefined functions (only relying on include chaining for header files at most). So if you have: DogModule.c calling Sounds.c functions and exporting a public function in DogModule.c called "PlayBark(struct soundNode);", DogModule.h and things that use DogModule should both import Sounds.h. Good C habits and structure are very different than good C++ habits and structure. They're not the same language, not even close. Mangling your concepts of Good C and Good C++ is a great way to write both poorly.
    - palish 16y ago
      Insightful, thank you.
  - jbn 16y ago
    yes, and that's using the Pimpl idiom to decouple things can improve build times. If your whole codebase rebuilds everytime a header is touched, then your headers are probably not correctly organized. Having minimal headers (using forwards decls as much as possible) is vital to keeping rebuild times down.
- mwcremer 16y ago
  I think Pike's point about "thousands of needless lines of code" is that people do this: #include "foo.h" instead of this: #ifndef __FOO_H__ #include "foo.h" #endif where "foo.h" looks like this: #ifndef __FOO_H__ #define __FOO_H__ /* contents / #endif / __FOO_H__ */ which in the former case results in having to lex foo.h in order to skip the contents. However, as pointed out below, modern compilers will effectively do the same thing for you automatically. [Edited to make the code readable]
  - TheNewAndy 16y ago
    Just a quick pedantic point - __FOO_H__ is a reserved identifier (C90 reserved). You shouldn't be writing it in your code.
- ori_b 16y ago
  If you keep your header files independent, you have no reason to include a single header multiple times, and you can't do it accidentally through another header. If your header files don't transitively include other headers, then #include "a.h" #include "b.h" will never cause an error. Therefore, in well written code, the include guards will be no-ops, and for simplicity's sake, they may as well be left off.
  - rcfox 16y ago
    Sure, this isn't a problem in trivial projects. If you want to use types other than what the language gives you, you're going to have to introduce a dependency.
- jbn 16y ago
  even with internal include guards, just the time to read through (and skip) the whole files for later inclusions can become dominant in cpp processing time (i.e. it's mostly waiting on I/O reading the same headers many times over). That's why Lakos advises external include guards for large systems.
  - cdavid 16y ago
    Unless you have gigantic projects, your project source code is likely to be mostly in memory through kernel IO buffers nowadays, so even this is not true anymore IMO. EDIT: just for kicks, I compiled twice numpy (100 kLOC), first just after flushing IO buffers (echo 1 > /proc/sys/vm/drop_caches in recent linux kernels): - cold case: 24 seconds - host case: 16 seconds Of course, it is hard to say what contributes to the slowness (files themselves, loading in memory all the programs needed for compilation, etc...).
    - viraptor 16y ago
      If you include `ccache` (or similar), the recompilation time will drop to ~0. One-off compilation time doesn't matter that much and developers have many other tools they can use (pch?).
      - cdavid 16y ago
        Partial rebuilds are obviously quite important - but in that case, IO is even less of an issue, because it is the hot case assuming you have enough memory. I forgot about one case where you may not have enough memory: large C++ program with multiple compilation in // - this can easily takes GB of memory for template heavy code.
    - jbn 16y ago
      one easy way to slow down compiles on Windows is to include windows.h everywhere, this pulls down about 2 Mb of declarations that the cpp front-end has to parse just to reach in the internal guards... it all depends on how large your header files are (and recursively how large is all headers that they recursively pull in).
drblast 16y ago
Can anyone who's worked on a well-written, large C project comment on the best way to structure headers and include files? I don't like including stdio.h and stdint.h in every .c and .h file, but it seems like the "standard" way to compile with Makefiles is to compile each .h and .c pair as an object then link them together at the end. So you need the header files to use includes and such in every file or the compiler barfs. Every time I try to structure a C project I feel like I'm doing it wrong and that there must be an elegant solution that I'm missing.
- gte910h 16y ago
  (Apoligies, HN is mangling my comment when I try to include C style comments, so I'm using C99/C++ style ones) You should include whatever .h files you need in every file that uses them (things like stdlib and stdio for sure when you're using them). This makes it so you have fewer interfile dependencies which keeps your compile times down. So you only include other .h files in .h files if you need the types in them (or put a blank prototype in for it, aka struct foo;). Otherwise you include .h files only in .c files. Feels hairy, yes, but that's the correct way to do it. By doing it creative ways, you cause great pain down the road (especially compilation times, and hunting for simple bugs). If you're only using one or two functions, sometimes people just prototype the function: // some people just would stick this prototype in their .c //file they're writing if they need nothing else in math.h double sqrt(double x); I think you're missing something about C there though the way you talk about compiling things. h files aren't really compiled at all. They are substituted into files where they're #included. You can actually write your entire project without them if you really want to (not suggested, but you can) by just putting prototypes for the functions you call at the top of the c files that need to copy them. That's actually why its important to put guard include statements at the beginning and end of h files to make sure they aren't included more than once. A properly written h file shouldn't cause compiler errors when you do this to it: #include "foo.h" #include "foo.h" #include "foo.h" #include "foo.h" #include "foo.h"` The way you make it proper is you do this the top: #ifndef __FOO_H__ #define __FOO_H__ and this at the bottom #endif // __FOO_H__ Many IDEs do this automatically and vim and emacs both have plugins you can get to do this. Robert Pike's piece is from 1989, when computers ran much slower. The lexical analysis time from this is not worth talking about anymore. Structure: Generally C is structured with a main .c file which has a few functions related to start up in it and includes of header files in other modules which are needed. Each of the .c files includes the header files it needs, from either internal project modules, or from system level header files (aka <thingsInAlligatorBrackets.h>). Generally each .c file is a module, although in Object Oriented C, you may find modules built up of several different c files with each "class family" in a single .c files. Modules are purely conceptual in C. Some C people get macro happy. Do no do this. It is unclear to the rest of the C developing universe. Modern C compilers can do inline substitution quite well, often giving better performance than your macros. When people get macro happy, they end up making a file like "GlobalMacros.h" or even if they don't, they just have a lot of constants and end up with "LotsOfConstants.h" included in everything. Resist the urge to put your .h includes in there but NOT in the files that actually depend on it. If you do that you break code if that .h file is ever taken out. As far as final compilation goes: Every C file is compiled into an object file then linked using the linker. Alternatively, groups of .c files are compiled into .o files then linked into .a files (static libraries). These libraries are then linked in during the final compilation step.
wtracy 16y ago
"Thus I say maxphysaddr (not MaximumPhysicalAddress) for a global variable, but np not NodePointer for a pointer locally defined and used." Yes, because "maxphysaddr" is extremely readable for people who are not native English speakers. Wait, was that "maxiphysadd" or "maxmphyaddrss"? Better stop coding (breaking up my train of thought) and scroll back up to the variable declaration to double-check how I spelled it. (Yes, I know that modern IDEs can remember variable names for you, but there's always those emergency situations where you wind up trying to debug something with VIM over SSH via a slow-as-molasses hotel wifi service.)
- tedunangst 16y ago
  If you can't remember maxphysaddr vs maxiphysadd, I doubt you'll do much better remembering MaxPhysAddr vs MaxiPhysAdd.
  - chc 16y ago
    MaxPhysicalAddress is more readable than all the above. Pike simply had it wrong, IMO, probably from having been immersed in a culture of poor readability for so long that he had become blind to some of it.
    - tedunangst 16y ago
      In 1989, external symbols were (possibly) only unique wrt the first 6 letters. Trying to guess if anybody else would name a variable MaxPhy* isn't a fun game...
      - chc 16y ago
        That's true, but Pike didn't explain it that way. He wasn't talking about compatibility, but readability. That part of the article was explaining that longer and more complicated is not necessarily better in terms of readability, and there is a sweet spot where something is concise but not overabbreviated. That's all obviously true. Then he suggested that "maxphysaddr" is in this sweet spot for the idea of the highest point in physical memory that can be addressed, which I think is wrong.
rauljara 16y ago
I think it's helpful to have criteria like this out there, so that we can discuss them. That said, what's with the bias against capital letters? maxphysaddr is more readable than MaxPhysAddr? godisnowhere is more readable than GodIsNowHere? Or is that GodIsNowhere? I'm personally a fan of coding styles that cut down on ambiguity.
- amh 16y ago
  I think "godisnowhere" is more an example of a straight-up badly chosen name (identifiers should not be complete sentences) rather than a case where caps are better than lowercase. I'm not against camelcase and its variants, but there are a wide variety of naming standards using caps, it's quite possible to get mixed up between them and introduce ambiguity or worse (referring to the wrong variable).
  - TheNewAndy 16y ago
    I believe the intention of this name was the parsing ambiguity of the word "nowhere" when you don't have word delimiters, not as an example of an actual name, or a statement about the location of God :)
    - epo 16y ago
      Surely, that's his point, putting an ambiguity into a variable name is just asking for trouble, choose a better name.
      - TheNewAndy 16y ago
        The ambiguity isn't in the name on its own "God is nowhere". It is in the combination of that name, and a bad naming convention. If you had a naming convention which prohibits vowels, then the identifier "shrt" is ambiguous because it could be "short" or "shirt". That doesn't mean that "short" is a bad name for an identifier.
- jmulho 16y ago
  I prefer god_is_nowhere. The easiest thing to read is what you are used to. To fairly compare godisnowhere, GodIsNowhere, and god_is_nowhere, you need to stick with each for several days. I bet you'll discover, as I have, that god_is_nowhere is the easiest to read.
pgbovine 16y ago
''' The following data structures are a complete list for almost all practical programs: array linked list hash table binary tree ''' this is why it's such a pleasure to program in modern high-level languages (e.g, python, ruby) --- these basic data structures (essentially sequences and mappings) are baked into the language with simple, concise syntax
- chc 16y ago
  Except for one. Quick: What's the syntax for a binary tree in Python? OK, I'll give away the answer: It's not there. I don't think binary trees are even in the standard library for Python or Ruby. It's a sadly neglected data structure because, when exposed for what it is, it's a little bit too computer science-y for most people's taste.
  - archangel_one 16y ago
    I think it's simpler than that: when you have arrays, linked lists and hash tables there aren't _that_ many cases when you need a binary tree, so I guess the designers felt they weren't worth baking into the language at a syntax level. I agree it would have been nice to have in the standard library though.
    - chc 16y ago
      This is sort of a "when all you have is a hammer…" situation. You don't need binary trees, but only in the same sense most of these data structures can stand in for one or more of the others. Binary trees can actually be better than hash tables in many cases where hash tables are used in mainstream scripting languages simply due to language support. An informative post on the subject: http://enfranchisedmind.com/blog/posts/problems-with-hash-tables/ http://enfranchisedmind.com/blog/posts/problems-with-hash-ta...
      - rg3 16y ago
        Your point about the need of certain data structures reminds me of the famous humorous text "Real programmers don't use Pascal", which said: "As all Real Programmers know, the only useful data structure is the Array. Strings, Lists, Structures, Sets-- these are all special cases of arrays and can be treated that way just as easily without messing up your programming language with all sorts of complications." The full text is available here and is a recommended read if you have some spare minutes: http://www.pbm.com/~lindahl/real.programmers.html http://www.pbm.com/~lindahl/real.programmers.html
contol-m 16y ago
There is a lot more in his book, "The Practice of Programming" - http://cm.bell-labs.com/cm/cs/tpop/toc.html http://cm.bell-labs.com/cm/cs/tpop/toc.html
kqueue 16y ago
I disagree with him on the include part. It is very difficult to remember the order of includes + dependencies for each file you want to include. It becomes very messy.
- __david__ 16y ago
  I agree. I have better things to do than to memorize the random prerequisites of header files. Not to mention cluttering my C source with junk that isn't important to the source in that particular file. My rule is that I should be able to make an empty C file and put a single #include "header.h" line. If it doesn't compile then there is a bug in that .h file.
  - rbetts 16y ago
    One also sees the common advice to include the paired header (ie, foo.c should include foo.h) as the first #include to verify that foo.h is dependency-complete. The problem with dependency-complete headers is that once a .c file relies on that header for a symbol, if the header is changed to no longer require the symbol, all those .c files have to have their #includes fixed. This can take a lot of time to iterate through in a projects that require hours to compile.
yason 16y ago
I agree with about everything but the best part is the function pointer chapter. Function pointers are an insanely novel concept. They're not even a C thing per se but since C has first-class pointers they fit right it. Nevertheless, most design patterns, closures, higher-order functions, more complex dispatching, etc. are effectively just a function pointer and possibly some userdata. You can think in Lisp but do it all in C with function pointers.
- viraptor 16y ago
  > They're not even a C thing per se Really? from stdlib.h: void qsort(void *base, size_t nel, size_t width, int (*compar)(const void *, const void *));
  - yason 16y ago
    Not even a C thing per se, as in "C has function pointers but it's not really a C specific feature as some other languages have them too. Like assembly, very obviously." Many other languages don't. If they're sufficiently high-level they might have function objects or closures, which often cover everything you want. If they're Java they have neither but you can write a Visitor-style class or some other scheme from the design patterns book that emulates the behaviour of a function pointer. But fundamentally (and when the program is compiled down to machine code also hopefully) they're just function pointers.
    - eru 16y ago
      I agree. Though what it gets compiled down to depends on the compilation strategy. If your compiler is smart enough, and your use of functions as variables / function pointers is weak [1] enough, your functions may, say, just get inlined into a big case-statement-like construct. [1] In the sense of http://en.wikipedia.org/wiki/Strength_reduction http://en.wikipedia.org/wiki/Strength_reduction
- eru 16y ago
  I agree that the paragraphs about tables driven programs and function pointers are the best part. You can read "Combining datadriven programs with function pointers leads to an astonishingly expressive way of working, a way that, in my experience, has often led to pleasant surprises." as much as an endorsement of functional languages than the endorsement of OO-like techniques it was meant to. Though C is not really adequate to do Lisp-like programming. Because you can't capture free variables with C function pointers. Of course you can get around this restriction with lambda lifting [1], but why would you want to do the job of the compiler? As an example, it is quite hard to encode function composition in C, i.e. write a function that takes two function and returns their composition. Or the same specification in (non-idiomatic [2]) Haskell: composition :: (b -> c) -> (a -> b) -> a -> c composition f g = let h a = f (g a) in h I don't know of any way to make up new functions / function pointers in C at runtime. [1] http://en.wikipedia.org/wiki/Lambda_lifting http://en.wikipedia.org/wiki/Lambda_lifting [2] Idiomatic would be composition :: (b -> c) -> (a -> b) -> a -> c composition f g a = f (g a) but that's harder to understand for a C programmer who's not familiar with currying. Of course Haskell also has composition as a build-in. Or in Scheme (sorry, I am getting carried away with this): (define ((composition f g) a) (f (g a)))
basman 16y ago
The rule against nested includes seems dated — he doesn't like include guards in header files because "[t]he result is often thousands of needless lines of code passing through the lexical analyzer, which is (in good compilers) the most expensive phase." Surely not any more?
- cdavid 16y ago
  Indeed. I have read that parsing is the slowest part of compilation several times, but I guess this is very dated. For example, compiling numpy (a python package with ~ 100 kLOC of C code): - CFLAGS="-O0" -> ~ 20 sec - CFLAGS="-O1" -> ~ 35 sec - CFLAGS="-O3" -> ~ 60 sec Since the amount of parsed code for those modes is approximately the same (for numpy it is exactly the same, but one could imagine differences in glibc headers, etc...), this shows that parsing is not the bottleneck anymore if you build non-debug builds.
  - mfukar 16y ago
    No, that shows that the optimizer work takes a non-trivial amount of time. Optimizations (how often?) happen after parsing, type-checking and constructing the intermediate representation.
    - cdavid 16y ago
      sure, that's exactly the point I am trying to make: if it takes 75 % time more to compile @ O1 than O0, it means that parsing takes at most ~ 60 % of compilation time for quite conservative optimization. I think it is quite common to compile above the lowest level of optimization. To have an accurate estimation, one could look at clang, and only run the tokenizer.
  - gte910h 16y ago
    Yeah, the parsing argument is bunk anyhow. The pre-processor parsing is trivial and fast. Its the actual language which is the bulk of the time.
gte910h 16y ago
Notice the date folks: 1989. Lots of development has gone into C for the better since then.

122 points by ssp 16y ago

rcfox 16y ago

I disagree with the include files section. As long as each file has a proper include guard, it won't matter how many times you've included a file. I'd much rather have the preprocessor figure out when and where to include a file than do it myself. Maybe this was a significant performance issue in 1989, but it shouldn't be anymore. That said, you should only include files that are direct dependencies. There's no need to include stdio.h in every other file.

meese_ 16y ago
I think that advice is just outdated. It's a different situation now that it's a standard practice and there are optimizations supporting it (http://gcc.gnu.org/onlinedocs/gcc-2.95.3/cpp%5F1.html#SEC8 http://gcc.gnu.org/onlinedocs/gcc-2.95.3/cpp%5F1.html#SEC8).
palish 16y ago
That's a naive viewpoint. It's more of a concern in C++ ... if your codebase grows to massive proportions, and you #include headers within headers, then modifying a header can result in ~60% of your codebase being rebuilt. (At work, I deal with this pretty much every day.) Including header files within headers is simply unnecessary, especially in C. You need to forward declare in header files, then include in source files where you actually use that header file.
- rcfox 16y ago
  Suppose you've written some key data structure in its own file. Let's say it's a binary tree. The binary tree's functions are all implemented in bintree.c, and its functions, defines, and struct are all declared in bintree.h. Now, you want to write some application-specific functionality that works on your binary tree. You need to pass a pointer to the binary tree to your new functions. Of course, you implement these new functions in app.c, and the functions are declared in app.h so that other parts of your application can call them. You're going to need to include bintree.h in app.h to ensure that the compiler knows about the binary tree's structure. Otherwise, you'll have to make sure to always include bintree.h before you include app.h. Is that really something you want to be bothered with?
  - palish 16y ago
    In app.h, you should forward-declare "typedef struct bintree_t bintree;" Then, in each source file that #includes "app.h" and also CALLS binary-tree-related functions, you #include "bintree.h". Like I said, this is more of a concern in C++ where compile times are way, way slower than C. And in C++, it's very much worth it to forward declare in header files. It makes a big difference once your source code base has grown to epic proportions.
    - frou_dh 16y ago
      It's a technique, but not so applicable to say "you should" do it.
- edsrzf 16y ago
  I totally agree. Not only will headers including headers cause files to rebuild unnecessarily, but it can cause build times to increase simply because the compiler has to compile all those extra headers each time they're included in a source file. (Precompiled headers can help, but they aren't always an option.)
- gte910h 16y ago
  C is not C++. In C you shouldn't be chaining header files because you don't make function calls in header files in C code (well most good C code at least, people abuse macros sometimes). In C you should only include header files in header files if you have types in the other header files. It's okay to forward declare if you must, but its often quite a bit clearer to not. Additionally, you should always include all the h files that have your undefined functions (only relying on include chaining for header files at most). So if you have: DogModule.c calling Sounds.c functions and exporting a public function in DogModule.c called "PlayBark(struct soundNode);", DogModule.h and things that use DogModule should both import Sounds.h. Good C habits and structure are very different than good C++ habits and structure. They're not the same language, not even close. Mangling your concepts of Good C and Good C++ is a great way to write both poorly.
  - palish 16y ago
    Insightful, thank you.
- jbn 16y ago
  yes, and that's using the Pimpl idiom to decouple things can improve build times. If your whole codebase rebuilds everytime a header is touched, then your headers are probably not correctly organized. Having minimal headers (using forwards decls as much as possible) is vital to keeping rebuild times down.
mwcremer 16y ago
I think Pike's point about "thousands of needless lines of code" is that people do this: #include "foo.h" instead of this: #ifndef __FOO_H__ #include "foo.h" #endif where "foo.h" looks like this: #ifndef __FOO_H__ #define __FOO_H__ /* contents / #endif / __FOO_H__ */ which in the former case results in having to lex foo.h in order to skip the contents. However, as pointed out below, modern compilers will effectively do the same thing for you automatically. [Edited to make the code readable]
- TheNewAndy 16y ago
  Just a quick pedantic point - __FOO_H__ is a reserved identifier (C90 reserved). You shouldn't be writing it in your code.
ori_b 16y ago
If you keep your header files independent, you have no reason to include a single header multiple times, and you can't do it accidentally through another header. If your header files don't transitively include other headers, then #include "a.h" #include "b.h" will never cause an error. Therefore, in well written code, the include guards will be no-ops, and for simplicity's sake, they may as well be left off.
- rcfox 16y ago
  Sure, this isn't a problem in trivial projects. If you want to use types other than what the language gives you, you're going to have to introduce a dependency.
jbn 16y ago
even with internal include guards, just the time to read through (and skip) the whole files for later inclusions can become dominant in cpp processing time (i.e. it's mostly waiting on I/O reading the same headers many times over). That's why Lakos advises external include guards for large systems.
- cdavid 16y ago
  Unless you have gigantic projects, your project source code is likely to be mostly in memory through kernel IO buffers nowadays, so even this is not true anymore IMO. EDIT: just for kicks, I compiled twice numpy (100 kLOC), first just after flushing IO buffers (echo 1 > /proc/sys/vm/drop_caches in recent linux kernels): - cold case: 24 seconds - host case: 16 seconds Of course, it is hard to say what contributes to the slowness (files themselves, loading in memory all the programs needed for compilation, etc...).
  - viraptor 16y ago
    If you include `ccache` (or similar), the recompilation time will drop to ~0. One-off compilation time doesn't matter that much and developers have many other tools they can use (pch?).
    - cdavid 16y ago
      Partial rebuilds are obviously quite important - but in that case, IO is even less of an issue, because it is the hot case assuming you have enough memory. I forgot about one case where you may not have enough memory: large C++ program with multiple compilation in // - this can easily takes GB of memory for template heavy code.
  - jbn 16y ago
    one easy way to slow down compiles on Windows is to include windows.h everywhere, this pulls down about 2 Mb of declarations that the cpp front-end has to parse just to reach in the internal guards... it all depends on how large your header files are (and recursively how large is all headers that they recursively pull in).

drblast 16y ago

Can anyone who's worked on a well-written, large C project comment on the best way to structure headers and include files? I don't like including stdio.h and stdint.h in every .c and .h file, but it seems like the "standard" way to compile with Makefiles is to compile each .h and .c pair as an object then link them together at the end. So you need the header files to use includes and such in every file or the compiler barfs. Every time I try to structure a C project I feel like I'm doing it wrong and that there must be an elegant solution that I'm missing.

gte910h 16y ago
(Apoligies, HN is mangling my comment when I try to include C style comments, so I'm using C99/C++ style ones) You should include whatever .h files you need in every file that uses them (things like stdlib and stdio for sure when you're using them). This makes it so you have fewer interfile dependencies which keeps your compile times down. So you only include other .h files in .h files if you need the types in them (or put a blank prototype in for it, aka struct foo;). Otherwise you include .h files only in .c files. Feels hairy, yes, but that's the correct way to do it. By doing it creative ways, you cause great pain down the road (especially compilation times, and hunting for simple bugs). If you're only using one or two functions, sometimes people just prototype the function: // some people just would stick this prototype in their .c //file they're writing if they need nothing else in math.h double sqrt(double x); I think you're missing something about C there though the way you talk about compiling things. h files aren't really compiled at all. They are substituted into files where they're #included. You can actually write your entire project without them if you really want to (not suggested, but you can) by just putting prototypes for the functions you call at the top of the c files that need to copy them. That's actually why its important to put guard include statements at the beginning and end of h files to make sure they aren't included more than once. A properly written h file shouldn't cause compiler errors when you do this to it: #include "foo.h" #include "foo.h" #include "foo.h" #include "foo.h" #include "foo.h"` The way you make it proper is you do this the top: #ifndef __FOO_H__ #define __FOO_H__ and this at the bottom #endif // __FOO_H__ Many IDEs do this automatically and vim and emacs both have plugins you can get to do this. Robert Pike's piece is from 1989, when computers ran much slower. The lexical analysis time from this is not worth talking about anymore. Structure: Generally C is structured with a main .c file which has a few functions related to start up in it and includes of header files in other modules which are needed. Each of the .c files includes the header files it needs, from either internal project modules, or from system level header files (aka <thingsInAlligatorBrackets.h>). Generally each .c file is a module, although in Object Oriented C, you may find modules built up of several different c files with each "class family" in a single .c files. Modules are purely conceptual in C. Some C people get macro happy. Do no do this. It is unclear to the rest of the C developing universe. Modern C compilers can do inline substitution quite well, often giving better performance than your macros. When people get macro happy, they end up making a file like "GlobalMacros.h" or even if they don't, they just have a lot of constants and end up with "LotsOfConstants.h" included in everything. Resist the urge to put your .h includes in there but NOT in the files that actually depend on it. If you do that you break code if that .h file is ever taken out. As far as final compilation goes: Every C file is compiled into an object file then linked using the linker. Alternatively, groups of .c files are compiled into .o files then linked into .a files (static libraries). These libraries are then linked in during the final compilation step.

wtracy 16y ago

"Thus I say maxphysaddr (not MaximumPhysicalAddress) for a global variable, but np not NodePointer for a pointer locally defined and used." Yes, because "maxphysaddr" is extremely readable for people who are not native English speakers. Wait, was that "maxiphysadd" or "maxmphyaddrss"? Better stop coding (breaking up my train of thought) and scroll back up to the variable declaration to double-check how I spelled it. (Yes, I know that modern IDEs can remember variable names for you, but there's always those emergency situations where you wind up trying to debug something with VIM over SSH via a slow-as-molasses hotel wifi service.)

tedunangst 16y ago
If you can't remember maxphysaddr vs maxiphysadd, I doubt you'll do much better remembering MaxPhysAddr vs MaxiPhysAdd.
- chc 16y ago
  MaxPhysicalAddress is more readable than all the above. Pike simply had it wrong, IMO, probably from having been immersed in a culture of poor readability for so long that he had become blind to some of it.
  - tedunangst 16y ago
    In 1989, external symbols were (possibly) only unique wrt the first 6 letters. Trying to guess if anybody else would name a variable MaxPhy* isn't a fun game...
    - chc 16y ago
      That's true, but Pike didn't explain it that way. He wasn't talking about compatibility, but readability. That part of the article was explaining that longer and more complicated is not necessarily better in terms of readability, and there is a sweet spot where something is concise but not overabbreviated. That's all obviously true. Then he suggested that "maxphysaddr" is in this sweet spot for the idea of the highest point in physical memory that can be addressed, which I think is wrong.

rauljara 16y ago

I think it's helpful to have criteria like this out there, so that we can discuss them. That said, what's with the bias against capital letters? maxphysaddr is more readable than MaxPhysAddr? godisnowhere is more readable than GodIsNowHere? Or is that GodIsNowhere? I'm personally a fan of coding styles that cut down on ambiguity.

amh 16y ago
I think "godisnowhere" is more an example of a straight-up badly chosen name (identifiers should not be complete sentences) rather than a case where caps are better than lowercase. I'm not against camelcase and its variants, but there are a wide variety of naming standards using caps, it's quite possible to get mixed up between them and introduce ambiguity or worse (referring to the wrong variable).
- TheNewAndy 16y ago
  I believe the intention of this name was the parsing ambiguity of the word "nowhere" when you don't have word delimiters, not as an example of an actual name, or a statement about the location of God :)
  - epo 16y ago
    Surely, that's his point, putting an ambiguity into a variable name is just asking for trouble, choose a better name.
    - TheNewAndy 16y ago
      The ambiguity isn't in the name on its own "God is nowhere". It is in the combination of that name, and a bad naming convention. If you had a naming convention which prohibits vowels, then the identifier "shrt" is ambiguous because it could be "short" or "shirt". That doesn't mean that "short" is a bad name for an identifier.
jmulho 16y ago
I prefer god_is_nowhere. The easiest thing to read is what you are used to. To fairly compare godisnowhere, GodIsNowhere, and god_is_nowhere, you need to stick with each for several days. I bet you'll discover, as I have, that god_is_nowhere is the easiest to read.

pgbovine 16y ago

''' The following data structures are a complete list for almost all practical programs: array linked list hash table binary tree ''' this is why it's such a pleasure to program in modern high-level languages (e.g, python, ruby) --- these basic data structures (essentially sequences and mappings) are baked into the language with simple, concise syntax

chc 16y ago
Except for one. Quick: What's the syntax for a binary tree in Python? OK, I'll give away the answer: It's not there. I don't think binary trees are even in the standard library for Python or Ruby. It's a sadly neglected data structure because, when exposed for what it is, it's a little bit too computer science-y for most people's taste.
- archangel_one 16y ago
  I think it's simpler than that: when you have arrays, linked lists and hash tables there aren't _that_ many cases when you need a binary tree, so I guess the designers felt they weren't worth baking into the language at a syntax level. I agree it would have been nice to have in the standard library though.
  - chc 16y ago
    This is sort of a "when all you have is a hammer…" situation. You don't need binary trees, but only in the same sense most of these data structures can stand in for one or more of the others. Binary trees can actually be better than hash tables in many cases where hash tables are used in mainstream scripting languages simply due to language support. An informative post on the subject: http://enfranchisedmind.com/blog/posts/problems-with-hash-tables/ http://enfranchisedmind.com/blog/posts/problems-with-hash-ta...
    - rg3 16y ago
      Your point about the need of certain data structures reminds me of the famous humorous text "Real programmers don't use Pascal", which said: "As all Real Programmers know, the only useful data structure is the Array. Strings, Lists, Structures, Sets-- these are all special cases of arrays and can be treated that way just as easily without messing up your programming language with all sorts of complications." The full text is available here and is a recommended read if you have some spare minutes: http://www.pbm.com/~lindahl/real.programmers.html http://www.pbm.com/~lindahl/real.programmers.html

contol-m 16y ago

There is a lot more in his book, "The Practice of Programming" - http://cm.bell-labs.com/cm/cs/tpop/toc.html http://cm.bell-labs.com/cm/cs/tpop/toc.html

kqueue 16y ago

I disagree with him on the include part. It is very difficult to remember the order of includes + dependencies for each file you want to include. It becomes very messy.

__david__ 16y ago
I agree. I have better things to do than to memorize the random prerequisites of header files. Not to mention cluttering my C source with junk that isn't important to the source in that particular file. My rule is that I should be able to make an empty C file and put a single #include "header.h" line. If it doesn't compile then there is a bug in that .h file.
- rbetts 16y ago
  One also sees the common advice to include the paired header (ie, foo.c should include foo.h) as the first #include to verify that foo.h is dependency-complete. The problem with dependency-complete headers is that once a .c file relies on that header for a symbol, if the header is changed to no longer require the symbol, all those .c files have to have their #includes fixed. This can take a lot of time to iterate through in a projects that require hours to compile.

yason 16y ago

I agree with about everything but the best part is the function pointer chapter. Function pointers are an insanely novel concept. They're not even a C thing per se but since C has first-class pointers they fit right it. Nevertheless, most design patterns, closures, higher-order functions, more complex dispatching, etc. are effectively just a function pointer and possibly some userdata. You can think in Lisp but do it all in C with function pointers.

viraptor 16y ago
> They're not even a C thing per se Really? from stdlib.h: void qsort(void *base, size_t nel, size_t width, int (*compar)(const void *, const void *));
- yason 16y ago
  Not even a C thing per se, as in "C has function pointers but it's not really a C specific feature as some other languages have them too. Like assembly, very obviously." Many other languages don't. If they're sufficiently high-level they might have function objects or closures, which often cover everything you want. If they're Java they have neither but you can write a Visitor-style class or some other scheme from the design patterns book that emulates the behaviour of a function pointer. But fundamentally (and when the program is compiled down to machine code also hopefully) they're just function pointers.
  - eru 16y ago
    I agree. Though what it gets compiled down to depends on the compilation strategy. If your compiler is smart enough, and your use of functions as variables / function pointers is weak [1] enough, your functions may, say, just get inlined into a big case-statement-like construct. [1] In the sense of http://en.wikipedia.org/wiki/Strength_reduction http://en.wikipedia.org/wiki/Strength_reduction
eru 16y ago
I agree that the paragraphs about tables driven programs and function pointers are the best part. You can read "Combining datadriven programs with function pointers leads to an astonishingly expressive way of working, a way that, in my experience, has often led to pleasant surprises." as much as an endorsement of functional languages than the endorsement of OO-like techniques it was meant to. Though C is not really adequate to do Lisp-like programming. Because you can't capture free variables with C function pointers. Of course you can get around this restriction with lambda lifting [1], but why would you want to do the job of the compiler? As an example, it is quite hard to encode function composition in C, i.e. write a function that takes two function and returns their composition. Or the same specification in (non-idiomatic [2]) Haskell: composition :: (b -> c) -> (a -> b) -> a -> c composition f g = let h a = f (g a) in h I don't know of any way to make up new functions / function pointers in C at runtime. [1] http://en.wikipedia.org/wiki/Lambda_lifting http://en.wikipedia.org/wiki/Lambda_lifting [2] Idiomatic would be composition :: (b -> c) -> (a -> b) -> a -> c composition f g a = f (g a) but that's harder to understand for a C programmer who's not familiar with currying. Of course Haskell also has composition as a build-in. Or in Scheme (sorry, I am getting carried away with this): (define ((composition f g) a) (f (g a)))

basman 16y ago

The rule against nested includes seems dated — he doesn't like include guards in header files because "[t]he result is often thousands of needless lines of code passing through the lexical analyzer, which is (in good compilers) the most expensive phase." Surely not any more?

cdavid 16y ago
Indeed. I have read that parsing is the slowest part of compilation several times, but I guess this is very dated. For example, compiling numpy (a python package with ~ 100 kLOC of C code): - CFLAGS="-O0" -> ~ 20 sec - CFLAGS="-O1" -> ~ 35 sec - CFLAGS="-O3" -> ~ 60 sec Since the amount of parsed code for those modes is approximately the same (for numpy it is exactly the same, but one could imagine differences in glibc headers, etc...), this shows that parsing is not the bottleneck anymore if you build non-debug builds.
- mfukar 16y ago
  No, that shows that the optimizer work takes a non-trivial amount of time. Optimizations (how often?) happen after parsing, type-checking and constructing the intermediate representation.
  - cdavid 16y ago
    sure, that's exactly the point I am trying to make: if it takes 75 % time more to compile @ O1 than O0, it means that parsing takes at most ~ 60 % of compilation time for quite conservative optimization. I think it is quite common to compile above the lowest level of optimization. To have an accurate estimation, one could look at clang, and only run the tokenizer.
- gte910h 16y ago
  Yeah, the parsing argument is bunk anyhow. The pre-processor parsing is trivial and fast. Its the actual language which is the bulk of the time.

gte910h 16y ago

Notice the date folks: 1989. Lots of development has gone into C for the better since then.