4 ms·

> Compilers can easily memoize the position and parsed content of preprocessor directives, and do a very quick skip of an #include file, as a much better way to

by krig 12y ago

> Compilers can easily memoize the position and parsed content of preprocessor directives, and do a very quick skip of an #include file, as a much better way to resolve the problem. There are reasons why actual C/C++ compilers don't do this. Additionally, recursive includes is a major reason why C++ compilers are slow. For more on this, see http://clang.llvm.org/docs/Modules.html http://clang.llvm.org/docs/Modules.html

clarry 12y ago
> There are reasons why actual C/C++ compilers don't do this. http://gcc.gnu.org/onlinedocs/cpp/Once-Only-Headers.html#Once-Only-Headers http://gcc.gnu.org/onlinedocs/cpp/Once-Only-Headers.html#Onc... This construct is commonly known as a wrapper #ifndef. When the header is included again, the conditional will be false, because FILE_FOO_SEEN is defined. The preprocessor will skip over the entire contents of the file, and the compiler will not see it twice. CPP optimizes even further. It remembers when a header file has a wrapper ‘#ifndef’. If a subsequent ‘#include’ specifies that header, and the macro in the ‘#ifndef’ is still defined, it does not bother to rescan the file at all.
- pjmlp 12y ago
  Which breaks #includes written to be included multiple times and provide different outcomes based on defined macros. C based toolchains are just broken.
  - clarry 12y ago
    If a file is intended to be read multiple times with different results, then obviously you don't want to prevent it being read twice. However, we were talking about preventing the hit from reading a file twice unnecessarily. The quoted text says it doesn't bother rescanning if the macro is still defined. How do you get a different outcome from scanning the same file with the same define? Can you provide an example where this breaks things?
    - ambrop7 12y ago
      Yes, it's an optimization, it'll just make the common case (include guard) faster, without breaking anything.
      - pjmlp 12y ago
        How do you force the file to be re-read multiple times when the outcome is not idempotent?
        clarry 12y ago
        I do not understand why you are arguing about this. There are two scenarios: in one you want to prevent a file from being read multiple times because it wastes resources (and in some sad cases, might cause issues with redefinition). In this case, you use the include guards and the compiler can do a trick to optimize out unneeded scanning. You have this other hypothetical scenario in which you purposefully want a file to be read multiple times, with a different outcome (this kind of trickery isn't usually possible at all in other languages with module system so it is funny to argue about C being broken). Now I don't think this is a common scenario or even a good one to be in, but it is possible. The solution is simple: don't try to prevent the file from being read multiple times. Why would you try to prevent it, if it is meant to be read multiple times? You are arguing that a door is broken because you cannot walk in through it if you close the door. Don't close the door if you want to walk in...
        pjmlp 12y ago
        Because it is an optimization that relies on well behaved developers and we all know how good such assumptions end, specially in big teams with high attrition.
        dllthomas 12y ago
        As I read this, if the compiler determines that (with the #define set) the file is effectively empty, it knows it can just check the #define next time rather than re-reading. Without this optimization, you get the same result slower. It doesn't depend on well-behaved anything to get a result just as correct as without the optimization. With the single possible exception of a file that changes on disk between the multiple includes in the same build step. That is not a situation I have ever encountered and I am not at all sure it is supported and there aren't a bazillion other things that would break it in any given compiler chain (certainly at least line directives would be wrong, but that's arguably minor I guess).
- krig 12y ago
  Note that this only partially helps with a subset of the problem (and modules as proposed don't fix the issue, they just help a bit more): CPP still needs to make separate passes for each translation unit, for example. I'd recommend reading the link I put in my original post...
Someone 12y ago
But also: http://clang.llvm.org/doxygen/classclang_1_1MultipleIncludeOpt.html#details http://clang.llvm.org/doxygen/classclang_1_1MultipleIncludeO...: "Implements the simple state machine that the Lexer class uses to detect files subject to the 'multiple-include' optimization. The public methods in this class are triggered by various events that occur when a file is lexed, and after the entire file is lexed, information about which macro (if any) controls the header is returned." Modules improve things, but that doesn't mean one can do nothing to speed up lexing without them.

by krig 12y ago

clarry 12y ago

> There are reasons why actual C/C++ compilers don't do this. http://gcc.gnu.org/onlinedocs/cpp/Once-Only-Headers.html#Once-Only-Headers http://gcc.gnu.org/onlinedocs/cpp/Once-Only-Headers.html#Onc... This construct is commonly known as a wrapper #ifndef. When the header is included again, the conditional will be false, because FILE_FOO_SEEN is defined. The preprocessor will skip over the entire contents of the file, and the compiler will not see it twice. CPP optimizes even further. It remembers when a header file has a wrapper ‘#ifndef’. If a subsequent ‘#include’ specifies that header, and the macro in the ‘#ifndef’ is still defined, it does not bother to rescan the file at all.

pjmlp 12y ago
Which breaks #includes written to be included multiple times and provide different outcomes based on defined macros. C based toolchains are just broken.
- clarry 12y ago
  If a file is intended to be read multiple times with different results, then obviously you don't want to prevent it being read twice. However, we were talking about preventing the hit from reading a file twice unnecessarily. The quoted text says it doesn't bother rescanning if the macro is still defined. How do you get a different outcome from scanning the same file with the same define? Can you provide an example where this breaks things?
  - ambrop7 12y ago
    Yes, it's an optimization, it'll just make the common case (include guard) faster, without breaking anything.
    - pjmlp 12y ago
      How do you force the file to be re-read multiple times when the outcome is not idempotent?
      - clarry 12y ago
        I do not understand why you are arguing about this. There are two scenarios: in one you want to prevent a file from being read multiple times because it wastes resources (and in some sad cases, might cause issues with redefinition). In this case, you use the include guards and the compiler can do a trick to optimize out unneeded scanning. You have this other hypothetical scenario in which you purposefully want a file to be read multiple times, with a different outcome (this kind of trickery isn't usually possible at all in other languages with module system so it is funny to argue about C being broken). Now I don't think this is a common scenario or even a good one to be in, but it is possible. The solution is simple: don't try to prevent the file from being read multiple times. Why would you try to prevent it, if it is meant to be read multiple times? You are arguing that a door is broken because you cannot walk in through it if you close the door. Don't close the door if you want to walk in...
        pjmlp 12y ago
        Because it is an optimization that relies on well behaved developers and we all know how good such assumptions end, specially in big teams with high attrition.
        dllthomas 12y ago
        As I read this, if the compiler determines that (with the #define set) the file is effectively empty, it knows it can just check the #define next time rather than re-reading. Without this optimization, you get the same result slower. It doesn't depend on well-behaved anything to get a result just as correct as without the optimization. With the single possible exception of a file that changes on disk between the multiple includes in the same build step. That is not a situation I have ever encountered and I am not at all sure it is supported and there aren't a bazillion other things that would break it in any given compiler chain (certainly at least line directives would be wrong, but that's arguably minor I guess).
krig 12y ago
Note that this only partially helps with a subset of the problem (and modules as proposed don't fix the issue, they just help a bit more): CPP still needs to make separate passes for each translation unit, for example. I'd recommend reading the link I put in my original post...

Someone 12y ago

But also: http://clang.llvm.org/doxygen/classclang_1_1MultipleIncludeOpt.html#details http://clang.llvm.org/doxygen/classclang_1_1MultipleIncludeO...: "Implements the simple state machine that the Lexer class uses to detect files subject to the 'multiple-include' optimization. The public methods in this class are triggered by various events that occur when a file is lexed, and after the entire file is lexed, information about which macro (if any) controls the header is returned." Modules improve things, but that doesn't mean one can do nothing to speed up lexing without them.