5 ms·

Wait what? As long as the build process is reproducible, why do you care if it's incremental or not? If you don't trust bazel, people have been using distcc+cca

by yegle 7y ago

Wait what? As long as the build process is reproducible, why do you care if it's incremental or not? If you don't trust bazel, people have been using distcc+ccache since forever.

twic 7y ago
The concern is that an incremental build is not reproducible. In theory, it is, but in practice, sometimes it is not. Doing a clean build for the final artifact that is going to be released is pretty standard practice, IME.
- joshuamorton 7y ago
  Have you used bazel? The number of times I've even considered using `clean` (the drop some caches make everything mostly clean though still use the global cache command) in my 2+ years with it I can count one one hand, and most of those were unnecessary.
Silhouette 7y ago
Just as an anecdote, in all my time as a professional software developer, the number of projects I have worked on that have enjoyed reliable, reproducible, incremental builds is zero. The effects of global optimisations change. The output code winds up varying based on the order of incremental changes. Updated dependencies get overlooked because of race conditions or networking glitches or filesystem bugs or a hundred other unlikely but not impossible failures. Incremental builds are great for quick day-to-day development, but in my experience they are invariably unsuitable for robust production builds. For me, those should use a git clone or equivalent into an empty directory and then a completely clean build of everything, always.
lmm 7y ago
> As long as the build process is reproducible, why do you care if it's incremental or not? I wouldn't care if it was reliable. My experience is that it isn't. > If you don't trust bazel, people have been using distcc+ccache since forever. And it's been causing inconsistent builds all the while.
- robbrit 7y ago
  I've been using bazel (and the Google-internal version, blaze) for years now. The builds are extremely reliable and are used to deploy loads of projects across Google every day. I no longer work there, but my new company is also using bazel and we never run into issues with unreliable builds. I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools.
  - lmm 7y ago
    > I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools. Where's the breakthrough then? What's the magic that allows bazel to succeed where dozens of similar efforts have failed? Sometimes a new tool does fix the problems of past tools, but that's almost always due to some new insight.
    - robbrit 7y ago
      Not sure exactly, since I haven't used any of the dozens of similar efforts. It sounds like you're a lot more familiar with the area, so here's some resources: * Bazel's FAQ: https://bazel.build/faq.html https://bazel.build/faq.html * A description on how Blaze works: https://google-engtools.blogspot.com/2011/08/build-in-cloud-how-build-system-works.html https://google-engtools.blogspot.com/2011/08/build-in-cloud-... Unfortunately the design doc for Blaze is internal and AFAIK not available publicly. I imagine they took a look at a number of the similar efforts and made decisions based on what those approaches did wrong.
- mehrdada 7y ago
  While I can relate to your point, if you actually experience Blaze at Google (and Bazel too, but it has other ecosystem immaturity issues not related to reproducibility) you will quickly change your mind. I can totally see why prior tools you used may have put bad taste in your mouth.

by yegle 7y ago

Wait what? As long as the build process is reproducible, why do you care if it's incremental or not? If you don't trust bazel, people have been using distcc+ccache since forever.

twic 7y ago

The concern is that an incremental build is not reproducible. In theory, it is, but in practice, sometimes it is not. Doing a clean build for the final artifact that is going to be released is pretty standard practice, IME.

joshuamorton 7y ago
Have you used bazel? The number of times I've even considered using `clean` (the drop some caches make everything mostly clean though still use the global cache command) in my 2+ years with it I can count one one hand, and most of those were unnecessary.

Silhouette 7y ago

Just as an anecdote, in all my time as a professional software developer, the number of projects I have worked on that have enjoyed reliable, reproducible, incremental builds is zero. The effects of global optimisations change. The output code winds up varying based on the order of incremental changes. Updated dependencies get overlooked because of race conditions or networking glitches or filesystem bugs or a hundred other unlikely but not impossible failures. Incremental builds are great for quick day-to-day development, but in my experience they are invariably unsuitable for robust production builds. For me, those should use a git clone or equivalent into an empty directory and then a completely clean build of everything, always.

lmm 7y ago

> As long as the build process is reproducible, why do you care if it's incremental or not? I wouldn't care if it was reliable. My experience is that it isn't. > If you don't trust bazel, people have been using distcc+ccache since forever. And it's been causing inconsistent builds all the while.

robbrit 7y ago
I've been using bazel (and the Google-internal version, blaze) for years now. The builds are extremely reliable and are used to deploy loads of projects across Google every day. I no longer work there, but my new company is also using bazel and we never run into issues with unreliable builds. I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools.
- lmm 7y ago
  > I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools. Where's the breakthrough then? What's the magic that allows bazel to succeed where dozens of similar efforts have failed? Sometimes a new tool does fix the problems of past tools, but that's almost always due to some new insight.
  - robbrit 7y ago
    Not sure exactly, since I haven't used any of the dozens of similar efforts. It sounds like you're a lot more familiar with the area, so here's some resources: * Bazel's FAQ: https://bazel.build/faq.html https://bazel.build/faq.html * A description on how Blaze works: https://google-engtools.blogspot.com/2011/08/build-in-cloud-how-build-system-works.html https://google-engtools.blogspot.com/2011/08/build-in-cloud-... Unfortunately the design doc for Blaze is internal and AFAIK not available publicly. I imagine they took a look at a number of the similar efforts and made decisions based on what those approaches did wrong.
mehrdada 7y ago
While I can relate to your point, if you actually experience Blaze at Google (and Bazel too, but it has other ecosystem immaturity issues not related to reproducibility) you will quickly change your mind. I can totally see why prior tools you used may have put bad taste in your mouth.