4 ms·
I don't trust incremental builds (whether via Bazel or any other tool) for an actual deployable build - I've seen them go wrong too many times. For the local e
by lmm 8y ago
I don't trust incremental builds (whether via Bazel or any other tool) for an actual deployable build - I've seen them go wrong too many times.
For the local edit-test cycle incremental builds are great, but any decent IDE will do them in the multirepo case just as in the monorepo case.
- yegle 8y agoWait what? As long as the build process is reproducible, why do you care if it's incremental or not? If you don't trust bazel, people have been using distcc+ccache since forever.
- twic 8y agoThe concern is that an incremental build is not reproducible. In theory, it is, but in practice, sometimes it is not. Doing a clean build for the final artifact that is going to be released is pretty standard practice, IME.
- joshuamorton 8y agoHave you used bazel? The number of times I've even considered using `clean` (the drop some caches make everything mostly clean though still use the global cache command) in my 2+ years with it I can count one one hand, and most of those were unnecessary.
- Silhouette 8y agoJust as an anecdote, in all my time as a professional software developer, the number of projects I have worked on that have enjoyed reliable, reproducible, incremental builds is zero. The effects of global optimisations change. The output code winds up varying based on the order of incremental changes. Updated dependencies get overlooked because of race conditions or networking glitches or filesystem bugs or a hundred other unlikely but not impossible failures. Incremental builds are great for quick day-to-day development, but in my experience they are invariably unsuitable for robust production builds. For me, those should use a git clone or equivalent into an empty directory and then a completely clean build of everything, always.
- lmm 8y ago> As long as the build process is reproducible, why do you care if it's incremental or not? I wouldn't care if it was reliable. My experience is that it isn't. > If you don't trust bazel, people have been using distcc+ccache since forever. And it's been causing inconsistent builds all the while.
- robbrit 8y agoI've been using bazel (and the Google-internal version, blaze) for years now. The builds are extremely reliable and are used to deploy loads of projects across Google every day. I no longer work there, but my new company is also using bazel and we never run into issues with unreliable builds. I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools.
- lmm 8y ago> I think you're falling into the trap of "past solutions didn't work, therefore the problem is intractable", which is a special case of the "throwing the baby out with the bathwater" fallacy. Sometimes a new tool comes out that fixes the problems of past tools. Where's the breakthrough then? What's the magic that allows bazel to succeed where dozens of similar efforts have failed? Sometimes a new tool does fix the problems of past tools, but that's almost always due to some new insight.
- robbrit 8y agoNot sure exactly, since I haven't used any of the dozens of similar efforts. It sounds like you're a lot more familiar with the area, so here's some resources: * Bazel's FAQ: https://bazel.build/faq.html https://bazel.build/faq.html * A description on how Blaze works: https://google-engtools.blogspot.com/2011/08/build-in-cloud-how-build-system-works.html https://google-engtools.blogspot.com/2011/08/build-in-cloud-... Unfortunately the design doc for Blaze is internal and AFAIK not available publicly. I imagine they took a look at a number of the similar efforts and made decisions based on what those approaches did wrong.
- ceronman 8y agoMy point was about the the fact that by declaring your dependencies explicitly, Bazel allows you to compile only the things you depend on, not everything in the monorepo. And once you have built your dependencies once, you don't have to build them again. If you think about it, a package manager workflow (like maven, nuget, etc) is also a form of incremental compilation. You compile your libraries only once and publish them in binary form in some sort of repository and then import them. The difference is that if you want to make a change in both a library and an application this becomes hard to do with packages, while very easy with a monorepo and Bazel.
- lmm 8y ago> If you think about it, a package manager workflow (like maven, nuget, etc) is also a form of incremental compilation. You compile your libraries only once and publish them in binary form in some sort of repository and then import them. The difference is that that's explicit and visible, rather than being "magically" figured out by the build system. And the compiled binaries are 100% reproducible everywhere since it's literally the same file from the repository everywhere. So it's much more reliable, understandable, and debuggable. > The difference is that if you want to make a change in both a library and an application this becomes hard to do with packages, while very easy with a monorepo and Bazel. Having a library and application so tightly coupled that changes have to be simultaneous is poor practice, and will likely get you into trouble when the time comes to deploy them. Better to enforce that you test old application + new library (or vice versa) up front, than hit it for the first time in production.