3 ms·
That's like saying you should never build a house with concrete foundations because they take so long to dig and pour and the first layer of bricks doesn't need
by IshKebab 4y ago
That's like saying you should never build a house with concrete foundations because they take so long to dig and pour and the first layer of bricks doesn't need them anyway!
Get back to me when you work for a company with a monorepo that has to build and test everything in CI for every change (taking something like 200 CPU hours) because they didn't have the foresight to use Bazel.
- drewcoo 4y ago> company with a monorepo that has to build and test everything in CI for every change That smells like covering up an architecture problem (tight coupling) with a build tool. Not a recipe for success.
- deathanatos 4y agoI don't think that's actually the case. Bazel, AIUI, understands the entire dependency tree and can cache steps effectively, which a Makefile cannot (alone) do. (You could certainly build it into one.) CI tooling is orthogonal; your CI tooling can invoke Bazel or Makefile+custom caching, it wouldn't really matter. But the problem in CI is that you lack state; whereas a developer's machine might e.g., have a bunch of .o files from a previous build, CI will be starting from a clean slate¹. So builds take longer. My understanding of Bazel is that it has decent support for caching and can thus makes those rebuilds a lot faster, which it has b/c it understands exactly the inputs to the steps at hand. (Which, if you write your own caching layer, you'd have to figure out.) … but … I've also seen the same thing the top commenter has, with Bazel: endless suggestions to use Bazel … but migrating a large existing codebase is pain, and if you're not willing to put your work where your mouth is … then the status quo prevails. ¹While here CI lacking state is a sort of negative b/c of the time we need to rebuild it, normally, I think this is a good thing: it means you're constantly verifying you can build from scratch. Or, at least, something close to scratch, modulo things like vendoring. (I.e., if you do an "apt-get" from Ubuntu's servers in your CI pipeline … well … your build obviously isn't quite from scratch.)
- JustLurking2022 4y agoHaving experience FAANG monorepos, the other commenter was entirely correct about them papering over tight coupling with tools. It makes more sense when you realize it was really built to deal with statically linked C++ - there's no guarantee of ABI, so you really need to ensure that all your dependencies are built the same way as your project to ensure compatibility (and the dependency graphs in monorepos get DEEP). For any language with a stable ABI that utilizes dynamic linking, the complexity is far more then the benefit. The whole monorepo concept and all the tooling that goes with it really makes more sense when you think of it in that context of statically linked code that can't easily be packaged into something as portable as a JAR, so it's critical to build it all the same way.
- hot_gril 4y agoGoogle uses a monorepo intentionally, and for good reasons. They're also a massive company. For them, a complex build system makes sense... and even that system isn't exactly Bazel.
- Conan_Kudo 4y agoGoogle uses a monorepo because they were popular in 1999, not because they had a specific reason to do so[1]. As the codebase grew, they never changed and instead built tools around it. That's a perfectly fine choice if you have bodies to throw at the problem and can deal with the consequences of that choice, but most people can't. Google can and so it does. If it wasn't as successful as it is, it probably would have collapsed under the strain of the people and systems required to manage that code. [1]: https://qeunit.com/blog/how-google-does-monorepo/ https://qeunit.com/blog/how-google-does-monorepo/
- hot_gril 4y agoOk, could've been chosen for 1999 reasons back then, but there are good reasons today. It's easier to build things collaboratively, esp across teams, when there's just one repo and one branch (I don't count release branches). Yes it takes effort to support due to its size, but that's contained within a few teams, and everyone else benefits from it. Other tooling decisions place a burden on every SWE. Their config language is a weird trash that everyone has to sorta learn. Their deployment and integration testing tooling have similar qualities, and it's not cause of the monorepo. One consequence is that many teams cannot feasibly deploy microservices, even though it'd often make sense to, which is even worse if they're using C++.
- hot_gril 4y agoSoftware development isn't housing development. You lay a concrete foundation knowing exactly what is being built on top. The typical way of developing software is like adding stories to your house until it falls over, then rebuilding it stronger, which is perfectly fine in the software world. Otherwise things get overengineered. In practical terms, the makefile or whatever is usually good enough.
- IX-103 4y agoIf you're big enough to roll your own software ecosystem -- which means you have thousands of developers working on hundreds of closely coupled projects in a single code base -- and you need repeatable builds, then Bazel makes sense. Bazel isn't designed to make things easy. It isn't even designed to make things fast (though it is easy to add that on due to repeatable builds). Bazel is designed to prevent you from shooting yourself in the foot; providing strong controls on project dependencies, both dependencies of your project as well as other projects' dependence on your project. That means in 99% of companies the answer is no. You're better off with CMake or something thing like that.
- IshKebab 4y ago> then rebuilding it stronger That's a very rare occurrence in software. Honestly it's probably about as common as in housing. You can rebuild parts of a piece of software but the build system is generally really foundational and almost never changes in my experience. Especially if the software isn't a library so nobody except the main developers has to interact with the build system.
- hot_gril 4y agoThe build system can change with hardly any of the code changing. This is evident in how I've never even had to think about the build system at my SWE day job. It's evolved in the meantime. What's often impossible to replace without a full rewrite is a backend's DBMS, and people tend to underestimate how tight of a coupling that is. Only gets worse with abstraction.
- jayd16 4y agoWhat if I stick with companies that have enough foresight to not cause every change to build everything?
- IshKebab 4y agoIn CI? Then hooray! Your company is using Bazel (or Buck or Pants or similar).