4 ms·

I use ccache and love it, but I keep wondering: wouldn’t it make more sense for compilers to do caching themselves? That would allow them to cache the results o

I use ccache and love it, but I keep wondering: wouldn’t it make more sense for compilers to do caching themselves? That would allow them to cache the results of much finer-grained operations than whole translation units. If two (or 500!) translation units generate the same template instantiations and then each optimize that code, that has to happen in each translation unit and ccache can’t help; couldn’t a compiler with internal caching do it once and be done with it forever? I’ve considered trying to add this to clang, but haven’t prioritized it.

jcelerier 4y ago
zapcc used to work exactly like this. Sadly it was never merged back in clang even after being open sourced. https://github.com/yrnkrn/zapcc https://github.com/yrnkrn/zapcc
jansvoboda11 4y ago
You might be interested in this: https://discourse.llvm.org/t/rfc-add-an-llvm-cas-library-and-experiment-with-fine-grained-caching-for-builds/59864 https://discourse.llvm.org/t/rfc-add-an-llvm-cas-library-and...
Cloudef 4y ago
zig && zig cc does this + they are even planning to do incremental compilation with binary patching
- kristoff_it 4y ago
  Correct about the caching, but we don't plan to support in-place binary patching for C/C++ with the same level of granularity as with Zig.
  - Cloudef 4y ago
    That's understandable
deleted 4y ago
[deleted]
bjackman 4y ago
I think most of the requirements of this are fully orthogonal to the job of a given compiler, so it's best solved at the level of the build system. Bazel does this, so instead of needing to reimplement this for each compiler it's automatically done across all your languages, and even any random one-off build rules based on a shell script. You can share the cache with your team/build infra too: https://bazel.build/remote/caching https://bazel.build/remote/caching (Disclaimer: I've never used this with open-source Bazel, I work at Google and use the internal variant)
fatneckbeardz 4y ago
the time you spend pulling your hair out figuring out why your program stopped working due to some obscure problem with the caching mechanism, is alot more than the time you save by doing the caching. at least for a lot of people. for others it might make sense
tarranoth 4y ago
I think you are basically describing why people want modules to be implemented.
goombacloud 4y ago
I think the hard part is to know when caching is not allowed, and for that the compiler is a good point to manage such a cache because it could add metadata such as compiler version (maybe even patches?), compiler flags etc which a wrapper may miss. This disintegration of ccache and the compiler makes me not really trust it because these two could easily get out of sync and you would not easily notice when/why the result is wrong.

jcelerier 4y ago

zapcc used to work exactly like this. Sadly it was never merged back in clang even after being open sourced. https://github.com/yrnkrn/zapcc https://github.com/yrnkrn/zapcc

jansvoboda11 4y ago

You might be interested in this: https://discourse.llvm.org/t/rfc-add-an-llvm-cas-library-and-experiment-with-fine-grained-caching-for-builds/59864 https://discourse.llvm.org/t/rfc-add-an-llvm-cas-library-and...

Cloudef 4y ago

zig && zig cc does this + they are even planning to do incremental compilation with binary patching

kristoff_it 4y ago
Correct about the caching, but we don't plan to support in-place binary patching for C/C++ with the same level of granularity as with Zig.
- Cloudef 4y ago
  That's understandable

deleted 4y ago

[deleted]

bjackman 4y ago

I think most of the requirements of this are fully orthogonal to the job of a given compiler, so it's best solved at the level of the build system. Bazel does this, so instead of needing to reimplement this for each compiler it's automatically done across all your languages, and even any random one-off build rules based on a shell script. You can share the cache with your team/build infra too: https://bazel.build/remote/caching https://bazel.build/remote/caching (Disclaimer: I've never used this with open-source Bazel, I work at Google and use the internal variant)

fatneckbeardz 4y ago

the time you spend pulling your hair out figuring out why your program stopped working due to some obscure problem with the caching mechanism, is alot more than the time you save by doing the caching. at least for a lot of people. for others it might make sense

tarranoth 4y ago

I think you are basically describing why people want modules to be implemented.

goombacloud 4y ago

I think the hard part is to know when caching is not allowed, and for that the compiler is a good point to manage such a cache because it could add metadata such as compiler version (maybe even patches?), compiler flags etc which a wrapper may miss. This disintegration of ccache and the compiler makes me not really trust it because these two could easily get out of sync and you would not easily notice when/why the result is wrong.