7 ms·

Except the Bazel syntax is really good. Compiling a few C++ files into a binary is just: cc_binary( name = "app", srcs = glob([ ".hpp", ".cpp

by adgasf 7y ago

Except the Bazel syntax is really good. Compiling a few C++ files into a binary is just: cc_binary( name = "app", srcs = glob([ "*.hpp", "*.cpp" ]), ) That's more concise than most build systems AND it's cross-platform.

joshuamorton 7y ago
Although I'd strongly suggest not doing that bazel works better if you have 1 library per header (or source file for non c langs). It helps to have tools to manage deps for your though.
- adgasf 7y ago
  > bazel works better if you have 1 library per header What do you mean?
  - klipt 7y ago
    You could have a cc_binary for your main foo_main.cc file that depends on cc_library targets separately specified for each bar_lib.h/bar_lib.cc. That makes the graph more granular, e.g. if you just update the foo_main.cc you don't need to recompile the libraries. Or you can reuse bar_lib in a different binary.
    - adgasf 7y ago
      Does Bazel break up these steps automatically under the hood? It could, in theory.
      - SolarNet 7y ago
        It does, this is testable a number of ways. The funniest one is to do a `#include "foo.bar"` (and make sure the file exists and is in `srcs`!) and watch bazel freak out with "I have no clue what kind of file that is, how do I analyze that!" Something to be aware of though is that Bazel does do some stupid things around this space. Like adding and removing files causes the build rule to change (if using a glob), which will proc a full rebuild of the library.
    - grandmczeb 7y ago
      > if you just update the foo_main.cc you don't need to recompile the libraries. Are you sure this is required? I just tried with a test repo and Bazel only performed two actions after I updated foo_main.cpp (compile foo_main.cpp and link app.)
      - SolarNet 7y ago
        The case in question is .h file include chains. And it does work correctly.
      - joshuamorton 7y ago
        Try two versions: Foo.cc that depends on a a lib that's glob of a.h, b.h, c.h, d.h. Foo.cc that depends on a_lib, b_lib, etc. Modify a. In the first case, you'll have to recompile all of abcd. In the second, only a.
        SolarNet 7y ago
        I just tested this and I think you are incorrect. Bazel will appear as if it's rebuilding all of them (it lists a dozen files), but it's really just checking the caches for them. Try running `bazel build -s` and watch what commands it actually runs. Note that adding and removing files (including through a glob) does always cause a rebuild though (because it's a change of the rule). This is a deficiency of bazel.
        joshuamorton 7y ago
        You can skip that if you use libraries, it won't even check the cache or need to relink. That's faster, much faster at scale. The other nice thing is you get dead code elimination for free.
        SolarNet 7y ago
        Sure, but my point was that Bazel is doing file/include analysis so medium sized multi-header libraries are not penalized.
SolarNet 7y ago
Yes but there is a hundred megabytes of system behind that. Tens of thousands of lines of starlark code, and tens (maybe hundreds) of thousands of lines of java code besides. A lot of work goes into those 4 lines. This is evidenced by the simple question of, how do you set up a custom compiler for that code properly? The answer being a couple hundred lines of starlark code, for what in other build systems is effectively `CC=clang` (to be fair bazel allows this but does not support or encourage it, especially for anything besides toy builds). I like Bazel (a lot!) but you are oversimplifying it here.
- HereBeBeasties 7y ago
  I have to ask, "so what?" Provided the abstraction doesn't leak I really don't care, in the same way that end users of Microsoft Word don't care about the algorithms that kern their text, about how DirectWrite does subpixel rendering, or anything else. Yes, it's all fiercely large and complex. The original user is pointing out that this is all hidden for simple projects, waiting in the wings for when you need it, and therefore doesn't matter.
  - SolarNet 7y ago
    Yes but it's minimizing the complexity of the cliff. The original author they were responding to was describing the static/dynamic discrepancy, and how it requires a lot of work. The author I was replying points out how simple the syntax is. But that's missing the point. I provided a direct counter example: Adding a new compiler to bazel is very complicated, and by new compiler I include something as simple as asserting "all of my C++ code is c++17" at a project level is (when done the correct way, and not one of the many hacky prone to failure ways) a 100+ line toolchain definition (because C++17 should be treated as an entirely new compiler from the perspective of bazel; treating it as a command line argument gets you in a bad place where you hit weird errors). To draw an analogy, it's like saying that C++ code like: some_matrix[i, j]; is great syntax and cross platform! When in reality it involves (in user space code) 3 complex classes (at a minimum!), overloading the comma operator(!), implicitly casting from builtin types, and probably hundreds of lines of template code (if you want any sort of extensibility). True it is cross platform and great syntax. But it obfuscates the amount of code and understanding required to do anything but what the most basic syntax allows, for example extending the system in any way.
flohofwoe 7y ago
Simple things like this are simple in all C/C++ build systems, the question is how the complex things are handled (multi-platform support, detecting and selecting different versions of the same compiler, cross-compiling, IDE support, multi-stage builds where source files are generated in the build which need to be inputs to later build stages, custom build steps, or bootstrapping tools which need to be used as compilers or code-generators for later build stages, etc etc etc...)
- adgasf 7y ago
  OP was suggesting that systems like Bazel are harder to get started with but pay off with larger, more complex builds. However, I am claiming that Bazel is also good for the simple cases.
  - cma 7y ago
    It might be that stuff (that is still beginner stuff) a bit beyond the first step is harder. Hello world is simple in python, but it is still hard to get started writing real-time code in it even if hello world might be your first step.
mzs 7y ago
I guess but this is also portable and more people know how to extend it: $ cat Makefile SRCS = foo.cpp bar.cpp baz.cpp OBJS = $(SRCS:.cpp=.o) PROG = app EXT = .PHONEY = all clean all : $(PROG) LDLIBS = $(OBJS) $(PROG) : $(PROG:$(EXT)=.cc) $(OBJS) clean : rm -f $(PROG) $(OBJS) $ make SRCS="`echo *.cpp`" clean all rm -f app a.o b.o c.o c++ -O2 -pipe -c a.cpp -o a.o c++ -O2 -pipe -c b.cpp -o b.o c++ -O2 -pipe -c c.cpp -o c.o c++ -O2 -pipe app.cc a.o b.o c.o -o app $
- pjmlp 7y ago
  If that was the case, autoconf and autotools would have never been a thing. The "beauty" of UNIX falls apart when trying to code across all of them. For example, I cannot even remember if that Makefile would work 100% with a POSIX make.
  - mzs 7y ago
    That Makefile works with BSD and GNU make, no extensions used. Bazel isn't really nicer than the autotools when you need something complicated, if it even supports your platform. Here's a fun bazel bug report: https://github.com/bazelbuild/bazel/issues/4350#issuecomment-353742540 https://github.com/bazelbuild/bazel/issues/4350#issuecomment... I've always managed to avoid autotools unless it was used before with things like LDFLAGS='-lsocket -lns' make ...
    - pjmlp 7y ago
      For me being POSIX compliant goes a little beyond BSD and GNU make.
      - mzs 7y ago
        What does posix compliant* even mean for make? The posix standard doesn't even require support for c++! That's a VERY portable Makefile. It would work on any make from the late seventies (assuming you also had a c++ compiler and system Makefiles that knew how to use it). It only uses a small subset of the rules in the standard. * https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html https://pubs.opengroup.org/onlinepubs/009695399/utilities/ma...
bsder 7y ago
The syntax is almost never the issue. The issue is that declarative build systems ALWAYS need an escape hatch for the exceptions--and the exceptions grow over time. We've been here before. It was called Make. And then it was called Ant. And then Maven. And Google still built their own system. And they will build another again. Nobody ever learns the lesson that a build system is a program--period. Sorry to burst your bubble: there will be circular dependencies--you will have to deal with it. There will be two different C compilers on two different architectures. You will have to deal with it. Libraries will be in the wrong place--you will have to deal with it. These libraries come from the system but these libraries come from an artifact--you will have to deal with it. The only way to deal with it is to have a genuine programming language underneath.
- EricBurnett 7y ago
  You say that like bazel hasn't already been proven out on a few billion LoC :). You're quite right that exceptions will exist - your job is to either fix them or squeeze them into the build system. Both are well trod paths. You're quite right that these edge cases aren't _easy_ - but in what system would they be? > there will be circular dependencies--you will have to deal with it. Add them both to the same compilation unit (cc_library, whatever). Or extract an ABI (e.g. Turbine) and compile against that. > There will be two different C compilers on two different architectures. You will have to deal with it. Poke around https://github.com/tensorflow/tensorflow/tree/master/third_party/toolchains https://github.com/tensorflow/tensorflow/tree/master/third_p... . I see arm and x86, Windows and many variants of Linux, multiple versions of GCC, multiple versions of clang, multiple versions of cuda, python 2 and 3, and names I don't even recognize. See also https://docs.bazel.build/versions/master/platforms-intro.html https://docs.bazel.build/versions/master/platforms-intro.htm.... > Libraries will be in the wrong place--you will have to deal with it. Just write the path to the file, or write a build rule that makes it visible in a standard location, or make it part of the platform definition, or use something like https://docs.bazel.build/versions/master/be/workspace.html#bind https://docs.bazel.build/versions/master/be/workspace.html#b... to alias it into your workspace. (Not recommended, but supported.) > Libraries will be in the wrong place--you will have to deal with it. Platform-specific library paths are very common. These days it's probably better specified as part of your "platform", but https://docs.bazel.build/versions/master/be/functions.html#select https://docs.bazel.build/versions/master/be/functions.html#s... is a convenient way to make in-place selectors. I won't pretend bazel can express _everything_, but there's little you can't hack in _somehow_ with sufficient motivation, and moving towards the bazel-recommended patterns brings growing peace-of-mind (and faster builds). (Disclaimer: Googler, work closely with the bazel team. I like bazel).

Except the Bazel syntax is really good. Compiling a few C++ files into a binary is just: cc_binary( name = "app", srcs = glob([ "*.hpp", "*.cpp

by adgasf 7y ago

joshuamorton 7y ago

Although I'd strongly suggest not doing that bazel works better if you have 1 library per header (or source file for non c langs). It helps to have tools to manage deps for your though.

adgasf 7y ago
> bazel works better if you have 1 library per header What do you mean?
- klipt 7y ago
  You could have a cc_binary for your main foo_main.cc file that depends on cc_library targets separately specified for each bar_lib.h/bar_lib.cc. That makes the graph more granular, e.g. if you just update the foo_main.cc you don't need to recompile the libraries. Or you can reuse bar_lib in a different binary.
  - adgasf 7y ago
    Does Bazel break up these steps automatically under the hood? It could, in theory.
    - SolarNet 7y ago
      It does, this is testable a number of ways. The funniest one is to do a `#include "foo.bar"` (and make sure the file exists and is in `srcs`!) and watch bazel freak out with "I have no clue what kind of file that is, how do I analyze that!" Something to be aware of though is that Bazel does do some stupid things around this space. Like adding and removing files causes the build rule to change (if using a glob), which will proc a full rebuild of the library.
  - grandmczeb 7y ago
    > if you just update the foo_main.cc you don't need to recompile the libraries. Are you sure this is required? I just tried with a test repo and Bazel only performed two actions after I updated foo_main.cpp (compile foo_main.cpp and link app.)
    - SolarNet 7y ago
      The case in question is .h file include chains. And it does work correctly.
    - joshuamorton 7y ago
      Try two versions: Foo.cc that depends on a a lib that's glob of a.h, b.h, c.h, d.h. Foo.cc that depends on a_lib, b_lib, etc. Modify a. In the first case, you'll have to recompile all of abcd. In the second, only a.
      - SolarNet 7y ago
        I just tested this and I think you are incorrect. Bazel will appear as if it's rebuilding all of them (it lists a dozen files), but it's really just checking the caches for them. Try running `bazel build -s` and watch what commands it actually runs. Note that adding and removing files (including through a glob) does always cause a rebuild though (because it's a change of the rule). This is a deficiency of bazel.
        joshuamorton 7y ago
        You can skip that if you use libraries, it won't even check the cache or need to relink. That's faster, much faster at scale. The other nice thing is you get dead code elimination for free.
        SolarNet 7y ago
        Sure, but my point was that Bazel is doing file/include analysis so medium sized multi-header libraries are not penalized.

SolarNet 7y ago

Yes but there is a hundred megabytes of system behind that. Tens of thousands of lines of starlark code, and tens (maybe hundreds) of thousands of lines of java code besides. A lot of work goes into those 4 lines. This is evidenced by the simple question of, how do you set up a custom compiler for that code properly? The answer being a couple hundred lines of starlark code, for what in other build systems is effectively `CC=clang` (to be fair bazel allows this but does not support or encourage it, especially for anything besides toy builds). I like Bazel (a lot!) but you are oversimplifying it here.

HereBeBeasties 7y ago
I have to ask, "so what?" Provided the abstraction doesn't leak I really don't care, in the same way that end users of Microsoft Word don't care about the algorithms that kern their text, about how DirectWrite does subpixel rendering, or anything else. Yes, it's all fiercely large and complex. The original user is pointing out that this is all hidden for simple projects, waiting in the wings for when you need it, and therefore doesn't matter.
- SolarNet 7y ago
  Yes but it's minimizing the complexity of the cliff. The original author they were responding to was describing the static/dynamic discrepancy, and how it requires a lot of work. The author I was replying points out how simple the syntax is. But that's missing the point. I provided a direct counter example: Adding a new compiler to bazel is very complicated, and by new compiler I include something as simple as asserting "all of my C++ code is c++17" at a project level is (when done the correct way, and not one of the many hacky prone to failure ways) a 100+ line toolchain definition (because C++17 should be treated as an entirely new compiler from the perspective of bazel; treating it as a command line argument gets you in a bad place where you hit weird errors). To draw an analogy, it's like saying that C++ code like: some_matrix[i, j]; is great syntax and cross platform! When in reality it involves (in user space code) 3 complex classes (at a minimum!), overloading the comma operator(!), implicitly casting from builtin types, and probably hundreds of lines of template code (if you want any sort of extensibility). True it is cross platform and great syntax. But it obfuscates the amount of code and understanding required to do anything but what the most basic syntax allows, for example extending the system in any way.

flohofwoe 7y ago

Simple things like this are simple in all C/C++ build systems, the question is how the complex things are handled (multi-platform support, detecting and selecting different versions of the same compiler, cross-compiling, IDE support, multi-stage builds where source files are generated in the build which need to be inputs to later build stages, custom build steps, or bootstrapping tools which need to be used as compilers or code-generators for later build stages, etc etc etc...)

adgasf 7y ago
OP was suggesting that systems like Bazel are harder to get started with but pay off with larger, more complex builds. However, I am claiming that Bazel is also good for the simple cases.
- cma 7y ago
  It might be that stuff (that is still beginner stuff) a bit beyond the first step is harder. Hello world is simple in python, but it is still hard to get started writing real-time code in it even if hello world might be your first step.

mzs 7y ago

I guess but this is also portable and more people know how to extend it: $ cat Makefile SRCS = foo.cpp bar.cpp baz.cpp OBJS = $(SRCS:.cpp=.o) PROG = app EXT = .PHONEY = all clean all : $(PROG) LDLIBS = $(OBJS) $(PROG) : $(PROG:$(EXT)=.cc) $(OBJS) clean : rm -f $(PROG) $(OBJS) $ make SRCS="`echo *.cpp`" clean all rm -f app a.o b.o c.o c++ -O2 -pipe -c a.cpp -o a.o c++ -O2 -pipe -c b.cpp -o b.o c++ -O2 -pipe -c c.cpp -o c.o c++ -O2 -pipe app.cc a.o b.o c.o -o app $

pjmlp 7y ago
If that was the case, autoconf and autotools would have never been a thing. The "beauty" of UNIX falls apart when trying to code across all of them. For example, I cannot even remember if that Makefile would work 100% with a POSIX make.
- mzs 7y ago
  That Makefile works with BSD and GNU make, no extensions used. Bazel isn't really nicer than the autotools when you need something complicated, if it even supports your platform. Here's a fun bazel bug report: https://github.com/bazelbuild/bazel/issues/4350#issuecomment-353742540 https://github.com/bazelbuild/bazel/issues/4350#issuecomment... I've always managed to avoid autotools unless it was used before with things like LDFLAGS='-lsocket -lns' make ...
  - pjmlp 7y ago
    For me being POSIX compliant goes a little beyond BSD and GNU make.
    - mzs 7y ago
      What does posix compliant* even mean for make? The posix standard doesn't even require support for c++! That's a VERY portable Makefile. It would work on any make from the late seventies (assuming you also had a c++ compiler and system Makefiles that knew how to use it). It only uses a small subset of the rules in the standard. * https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html https://pubs.opengroup.org/onlinepubs/009695399/utilities/ma...

bsder 7y ago

The syntax is almost never the issue. The issue is that declarative build systems ALWAYS need an escape hatch for the exceptions--and the exceptions grow over time. We've been here before. It was called Make. And then it was called Ant. And then Maven. And Google still built their own system. And they will build another again. Nobody ever learns the lesson that a build system is a program--period. Sorry to burst your bubble: there will be circular dependencies--you will have to deal with it. There will be two different C compilers on two different architectures. You will have to deal with it. Libraries will be in the wrong place--you will have to deal with it. These libraries come from the system but these libraries come from an artifact--you will have to deal with it. The only way to deal with it is to have a genuine programming language underneath.

EricBurnett 7y ago
You say that like bazel hasn't already been proven out on a few billion LoC :). You're quite right that exceptions will exist - your job is to either fix them or squeeze them into the build system. Both are well trod paths. You're quite right that these edge cases aren't _easy_ - but in what system would they be? > there will be circular dependencies--you will have to deal with it. Add them both to the same compilation unit (cc_library, whatever). Or extract an ABI (e.g. Turbine) and compile against that. > There will be two different C compilers on two different architectures. You will have to deal with it. Poke around https://github.com/tensorflow/tensorflow/tree/master/third_party/toolchains https://github.com/tensorflow/tensorflow/tree/master/third_p... . I see arm and x86, Windows and many variants of Linux, multiple versions of GCC, multiple versions of clang, multiple versions of cuda, python 2 and 3, and names I don't even recognize. See also https://docs.bazel.build/versions/master/platforms-intro.html https://docs.bazel.build/versions/master/platforms-intro.htm.... > Libraries will be in the wrong place--you will have to deal with it. Just write the path to the file, or write a build rule that makes it visible in a standard location, or make it part of the platform definition, or use something like https://docs.bazel.build/versions/master/be/workspace.html#bind https://docs.bazel.build/versions/master/be/workspace.html#b... to alias it into your workspace. (Not recommended, but supported.) > Libraries will be in the wrong place--you will have to deal with it. Platform-specific library paths are very common. These days it's probably better specified as part of your "platform", but https://docs.bazel.build/versions/master/be/functions.html#select https://docs.bazel.build/versions/master/be/functions.html#s... is a convenient way to make in-place selectors. I won't pretend bazel can express _everything_, but there's little you can't hack in _somehow_ with sufficient motivation, and moving towards the bazel-recommended patterns brings growing peace-of-mind (and faster builds). (Disclaimer: Googler, work closely with the bazel team. I like bazel).