5 ms·

One question I have after all these articles and internal company presentations is: how exactly does Bazel track dependencies? Module A importing module B impor

One question I have after all these articles and internal company presentations is: how exactly does Bazel track dependencies? Module A importing module B importing module C is a language-level feature, not a build system-level feature.

iampims 4y ago
You have to declare them in each BUILD file.
teaearlgraycold 4y ago
You need to duplicate all dependency declarations of the language into the bazel build system.
keddad 4y ago
You basically have to declare all those dependencies in Bazel. For instance, if you look at rules for Go, they declare the whole dependency graph (Modules, their relations) in Bazel. There is tooling to generate that for you, so it's not really that bad. Where it gets harder is when this language feature is integrated with an existing build system. C# is normally build with it's own build system, and to build it with Bazel you have to essentially rewrite some of it's logic in Bazel, which can be hard, but doable.
- DC-3 4y ago
  Have you got any more information on how to use Bazel with C#?
  - keddad 4y ago
    There are open-source rules for that, but at my place of work, we found those lacking. We had to write our own set of rules which would basically take all the source files for a given target, generate MSbuild files depending on the target type and call it from Bazel to build the code in this isolated environment. Another problem are nugets: you have to wrap each in a Bazel target in order to make it available during build, and analyze them beforehand to understand what kind of outputs (like .dlls) will they generate during build. It's a bit of a mess, sadly.
    - DC-3 4y ago
      That's what I feared. Have you got any solution to make this play nicely with Visual Studio?
      - substation13 4y ago
        You will need to maintain MS Build files alongside the Bazel definitions. In theory the MS Build files can be generated from the Bazel definitions but someone has to write that tool.
- illiarian 4y ago
  > There is tooling to generate that for you, so it's not really that bad. So you need several layers of tooling just to get something built. It's bad
  - wrigby 4y ago
    I may be missing something, but it’s quite similar to pkgconfig+(cmake|autoconf). If you include a header, your build system probably needs to know where on the filesystem that header exists.
  - JeremyBanks 4y ago
    [dead]
  - zaphar 4y ago
    If you only ever need to build Rust code then cargo is all you should need. However if you need to build Rust, C, C++, python, and JavaScript all together in one build then Cargo is inevitably going to make you sad. That is when you reach for something like Bazel.
rtpg 4y ago
You have to declare in Bazel as well. But really, I think that people don't go extremely granular on this. For Rust projects that are spread over 10-100 crates, I think that most people would just model the crate dependencies rather than at a file level. Mostly for pragmatic reasons, but also because that will likely model your test suite closely and your build requirements. Bazel is a build tool, so one "module" (in Bazel you talk about target) per build artifact basically aligns with what people want. But you can in theory go really granular, making it so that tests only run if files it depend on change (less granular and that becomes "crates they depend on change")
- illiarian 4y ago
  > You have to declare in Bazel as well. > For Rust projects that are spread over 10-100 crates It's already 10-100 crates you need to manually declare and keep track of, or use a yet another tool to generate this for you. And for big projects you'd also want to go more granular and split by parts of your app I guess.
  - rtpg 4y ago
    I mean in the "real world" what happens is you do this once, and then you keep on using it and gradually make changes. Annoying, of course, but I think cargo is fortunately still relatively simple stuff, so the scope is much smaller than, say, Python setup.py shenanigans (though Rust build.rs stuff will definitely require some futzing around with environment variables at the Cargo level). The "obvious" thing would be to either generate cargo files from Bazel stuff, generate Bazel stuff from cargo files, or have a third source that generates both. And like... yeah, if you are suffering from compile time issues, Bazel can make that stuff order of magnitudes faster (especially if your dep tree is pretty horizontal), so there's a reason to do it!
malkia 4y ago
Not directly to your question, but basically bazel needs a static graph. So if you have a.cpp #include "a.h" - then you need to declare that "a.h" in your BUILD file. But fear not, (and I'm not sure if BAZEL does this yet, but does it for java). If you've declared only "a.cpp", but forgot "a.h" - then bazel would detect that (/showIncludes in MSVC, or file system sandbox in linux, etc.) - and it'll tell you - you need to add "a.h" to your BUILD file. Fear not again, it's possible to create a BUILDIFIER command that given what BAZEL told you it can do it automatically for you. Hence the BUILD files needs to be kept simple, for everyone to understand (even non-engineers), while the ugly parts go into the .bzl files and workspaces. So in tha java world (but any language could do it), anything that was discovered dynamically but not declared in, bazel can offer how to fix it (copy+paste the command offered, or tool makes it for you) - at worst report it and you have to fix it manually The important thing is - bazel can use only the information provided in the BUILD files to detect - oh hey, things have changed in the CI - so I need to rebuild. Without relying on "cl/gcc/clang" to run this through preproc step to discover actual headers. or keep horrible make-like deps.
pie_flavor 4y ago
It's in the language and it's in the manifest. The manifest usually ends in `.toml`, but with Bazel it ends in `.bazel`.

iampims 4y ago

You have to declare them in each BUILD file.

teaearlgraycold 4y ago

You need to duplicate all dependency declarations of the language into the bazel build system.

keddad 4y ago

You basically have to declare all those dependencies in Bazel. For instance, if you look at rules for Go, they declare the whole dependency graph (Modules, their relations) in Bazel. There is tooling to generate that for you, so it's not really that bad. Where it gets harder is when this language feature is integrated with an existing build system. C# is normally build with it's own build system, and to build it with Bazel you have to essentially rewrite some of it's logic in Bazel, which can be hard, but doable.

DC-3 4y ago
Have you got any more information on how to use Bazel with C#?
- keddad 4y ago
  There are open-source rules for that, but at my place of work, we found those lacking. We had to write our own set of rules which would basically take all the source files for a given target, generate MSbuild files depending on the target type and call it from Bazel to build the code in this isolated environment. Another problem are nugets: you have to wrap each in a Bazel target in order to make it available during build, and analyze them beforehand to understand what kind of outputs (like .dlls) will they generate during build. It's a bit of a mess, sadly.
  - DC-3 4y ago
    That's what I feared. Have you got any solution to make this play nicely with Visual Studio?
    - substation13 4y ago
      You will need to maintain MS Build files alongside the Bazel definitions. In theory the MS Build files can be generated from the Bazel definitions but someone has to write that tool.
illiarian 4y ago
> There is tooling to generate that for you, so it's not really that bad. So you need several layers of tooling just to get something built. It's bad
- wrigby 4y ago
  I may be missing something, but it’s quite similar to pkgconfig+(cmake|autoconf). If you include a header, your build system probably needs to know where on the filesystem that header exists.
- JeremyBanks 4y ago
  [dead]
- zaphar 4y ago
  If you only ever need to build Rust code then cargo is all you should need. However if you need to build Rust, C, C++, python, and JavaScript all together in one build then Cargo is inevitably going to make you sad. That is when you reach for something like Bazel.

rtpg 4y ago

You have to declare in Bazel as well. But really, I think that people don't go extremely granular on this. For Rust projects that are spread over 10-100 crates, I think that most people would just model the crate dependencies rather than at a file level. Mostly for pragmatic reasons, but also because that will likely model your test suite closely and your build requirements. Bazel is a build tool, so one "module" (in Bazel you talk about target) per build artifact basically aligns with what people want. But you can in theory go really granular, making it so that tests only run if files it depend on change (less granular and that becomes "crates they depend on change")

illiarian 4y ago
> You have to declare in Bazel as well. > For Rust projects that are spread over 10-100 crates It's already 10-100 crates you need to manually declare and keep track of, or use a yet another tool to generate this for you. And for big projects you'd also want to go more granular and split by parts of your app I guess.
- rtpg 4y ago
  I mean in the "real world" what happens is you do this once, and then you keep on using it and gradually make changes. Annoying, of course, but I think cargo is fortunately still relatively simple stuff, so the scope is much smaller than, say, Python setup.py shenanigans (though Rust build.rs stuff will definitely require some futzing around with environment variables at the Cargo level). The "obvious" thing would be to either generate cargo files from Bazel stuff, generate Bazel stuff from cargo files, or have a third source that generates both. And like... yeah, if you are suffering from compile time issues, Bazel can make that stuff order of magnitudes faster (especially if your dep tree is pretty horizontal), so there's a reason to do it!

malkia 4y ago

Not directly to your question, but basically bazel needs a static graph. So if you have a.cpp #include "a.h" - then you need to declare that "a.h" in your BUILD file. But fear not, (and I'm not sure if BAZEL does this yet, but does it for java). If you've declared only "a.cpp", but forgot "a.h" - then bazel would detect that (/showIncludes in MSVC, or file system sandbox in linux, etc.) - and it'll tell you - you need to add "a.h" to your BUILD file. Fear not again, it's possible to create a BUILDIFIER command that given what BAZEL told you it can do it automatically for you. Hence the BUILD files needs to be kept simple, for everyone to understand (even non-engineers), while the ugly parts go into the .bzl files and workspaces. So in tha java world (but any language could do it), anything that was discovered dynamically but not declared in, bazel can offer how to fix it (copy+paste the command offered, or tool makes it for you) - at worst report it and you have to fix it manually The important thing is - bazel can use only the information provided in the BUILD files to detect - oh hey, things have changed in the CI - so I need to rebuild. Without relying on "cl/gcc/clang" to run this through preproc step to discover actual headers. or keep horrible make-like deps.

pie_flavor 4y ago

It's in the language and it's in the manifest. The manifest usually ends in `.toml`, but with Bazel it ends in `.bazel`.