3 ms·

I don’t see how using Waf as a base would help in any way. It seems like a massive mismatch for the problems that Facebook and Google are solving. You seem to b

I don’t see how using Waf as a base would help in any way. It seems like a massive mismatch for the problems that Facebook and Google are solving. You seem to be fond of Waf, maybe if you elaborated why you think that Waf would be a good base for compiling massive, multi-language code-bases, I could understand where you are coming from. Where I am coming from—it feels like Waf is kind of a better version of autotools, or something like that, and it’s just not in the same league. It’s like comparing a bicycle to a cargo ship. Like, “Why didn’t the people designing the cargo ship use the bicycle as a starting point?” I don’t want to abuse analogies here, but that’s what the question sounds like to me. This is based on my relatively limited experience using Waf (and SCons, which I know is different), and my experience using Bazel and Buck. Having spent a lot of time with Buck and Bazel, there are just so many little things you run into where you go, “Oh, that explains why Buck or Bazel is designed that way.” These design decisions permeate Buck and Bazel (Pants, Please, etc.) I just don’t see how Waf can be used as a base. I really do see this as a new “generation” of build systems, with Buck, Bazel, Please, and Pants, and everything else seems so much more primitive by comparison.

bogwog 4y ago
I’m coming from the perspective of someone who has been working with it for a while, and coincidentally very intensely hacking away at it recently. The thing about waf is that it’s more designed like a framework than a typical build tool. If you look at the code, it’s split into a core library (thats the <10k loc I estimated), and additional tools that do things like add C++ or Java build support. That’s one of the reasons I like Waf, since it becomes a powerful toolkit for creating a custom build system once you strip away the thin outer layer. There is no one-size-fits-all build system, so a tool that can be molded like waf is very powerful imo. I guess it’s hard to get that point across without experiencing it. There are just so many good design decisions everywhere. For example, extensibility comes easily because task generator methods are “flat”, and ordering is implemented via constraints. This means you can easily slip your own functions between any built in generator method to manipulate their inputs or outputs. It’s like a sub-build system just for creating Task objects. Also, I don’t want to give the impression that I think waf would have been a better choice for these companies. I’ve kind of been defending it a lot in this thread, but my original point/question was just to know if they evaluated waf/what they thought about it. After so many comments I feel like I might be coming off as hostile… which isn’t my intention.
- klodolph 4y ago
  I’m not trying to react to your comments as if they’re hostile, just hope to clear the air. I like defending Buck and Bazel a little bit, and at the same time, I really recognize that they are painful to adopt, don't solve everyone’s problems, etc. Waf does seem like a “do things as you like” framework, and I think that notion is antithetical to the Buck and Bazel design ethos. Buck and Bazel’s design are, “This is the correct way to do things, other ways are prohibited.” You fit your project into the Buck/Bazel system (which could be a massive headache for some) and in return you get a massive decrease in build times, as well as some other benefits like good cross-compilation support. One fundamental part of the Buck/Bazel design is that you can load any arbitrary subset of the build graph. Your repository has BUILD files in various directories. Those directories are packages, and you only load the subset of packages that you need for the targets you are actually evaluating during evaluation time. You can even load a child package without loading the parent—like, load //my/cool/package without loading //my/cool or //my. The build graph construction also looks somewhat different. There is an additional layer. In build systems like Waf, you have some set of configuration options, and the build scripts generate a graph of actions to perform which create the build using that configuration. In Buck/Bazel, there is an additional layer—you create a platform-agnostic build graph first (targets, which are specified using rules like cc_library), and then there’s a second analysis phase, which converts rules like “this is a cc_library” into actual actions like “run GCC on this file”. These extra layers are there, as far as I can tell, to support the goals of isolating different parts of your build system from each other. If they’re isolated, then you have better confidence that they will produce the same outputs every time, and you can make more of the build process parallelizable—not just the actual build actions, but the act of loading and analyzing build scripts. I do think that there is room to appreciate both philosophies—the “let’s make a flexible platform” philosophy, and the “let’s make a strict, my-way-or-the-highway build system” philosophy.

bogwog 4y ago

I’m coming from the perspective of someone who has been working with it for a while, and coincidentally very intensely hacking away at it recently. The thing about waf is that it’s more designed like a framework than a typical build tool. If you look at the code, it’s split into a core library (thats the <10k loc I estimated), and additional tools that do things like add C++ or Java build support. That’s one of the reasons I like Waf, since it becomes a powerful toolkit for creating a custom build system once you strip away the thin outer layer. There is no one-size-fits-all build system, so a tool that can be molded like waf is very powerful imo. I guess it’s hard to get that point across without experiencing it. There are just so many good design decisions everywhere. For example, extensibility comes easily because task generator methods are “flat”, and ordering is implemented via constraints. This means you can easily slip your own functions between any built in generator method to manipulate their inputs or outputs. It’s like a sub-build system just for creating Task objects. Also, I don’t want to give the impression that I think waf would have been a better choice for these companies. I’ve kind of been defending it a lot in this thread, but my original point/question was just to know if they evaluated waf/what they thought about it. After so many comments I feel like I might be coming off as hostile… which isn’t my intention.

klodolph 4y ago
I’m not trying to react to your comments as if they’re hostile, just hope to clear the air. I like defending Buck and Bazel a little bit, and at the same time, I really recognize that they are painful to adopt, don't solve everyone’s problems, etc. Waf does seem like a “do things as you like” framework, and I think that notion is antithetical to the Buck and Bazel design ethos. Buck and Bazel’s design are, “This is the correct way to do things, other ways are prohibited.” You fit your project into the Buck/Bazel system (which could be a massive headache for some) and in return you get a massive decrease in build times, as well as some other benefits like good cross-compilation support. One fundamental part of the Buck/Bazel design is that you can load any arbitrary subset of the build graph. Your repository has BUILD files in various directories. Those directories are packages, and you only load the subset of packages that you need for the targets you are actually evaluating during evaluation time. You can even load a child package without loading the parent—like, load //my/cool/package without loading //my/cool or //my. The build graph construction also looks somewhat different. There is an additional layer. In build systems like Waf, you have some set of configuration options, and the build scripts generate a graph of actions to perform which create the build using that configuration. In Buck/Bazel, there is an additional layer—you create a platform-agnostic build graph first (targets, which are specified using rules like cc_library), and then there’s a second analysis phase, which converts rules like “this is a cc_library” into actual actions like “run GCC on this file”. These extra layers are there, as far as I can tell, to support the goals of isolating different parts of your build system from each other. If they’re isolated, then you have better confidence that they will produce the same outputs every time, and you can make more of the build process parallelizable—not just the actual build actions, but the act of loading and analyzing build scripts. I do think that there is room to appreciate both philosophies—the “let’s make a flexible platform” philosophy, and the “let’s make a strict, my-way-or-the-highway build system” philosophy.