4 ms·

You may need different versions of the same source files or targets, due to incompatibilities of the underlying runtime environment. For example, custom compile

• joshuamorton 8y ago
  You can define compiler flags in BUILD files. My_binary and my_binary_opt can live side by side. You can even select dependencies depending on the runtime[0]. Like I said, not hard. I can think of at least 3 situations where we swap flags or dependencies based on runtime that I've touched this week. [0]: https://docs.bazel.build/versions/master/be/functions.html#select https://docs.bazel.build/versions/master/be/functions.html#s...
  • mlthoughts2018 8y ago
    Managing something like this with the select functionality in bazel is a terrible way to solve it. The conditions necessary to make the decision of which build variant to choose can be arbitrarily complicated (for example, the logical test could be the execution of some arbitrary script like embodied in a .PHONY target in Make) and also rely on caching logic or details of whether the ambient environment performing the build is a CI environment or not, and cannot be limited to flag arguments. The identically same compiled version may be required by multiple consumers, even if they are distributed from each other. Forcing both consumers to rebuild some dependencies in such a case (as opposed to relying on pre-built artifacts from an artifact repository) is often an unacceptable performance overhead, and very wasteful even in cases when someone can afford to sit around and wait. It can also be affected by portability.. for example differences in Docker for Mac vs say Docker for Ubuntu when the underlying artifact is a container. Much better to ensure the container that is built passes all required portability tests as a publication criteria on a container repo, so that the different consumers on different platforms can be sure if it is part of their build workflow, they are getting the correct artifact and it cannot be subject to idiosyncrasies of the platform or ambient environment at local build time. The necessary build target for a given select condition may change over time, while still requiring backward compatibility with a former build of that same target from those same build conditions, effectively making a timestamp into an implicit version identifier for a backwards compatible version, meaning that re-running an identical build command with identical parameters at a later time will produce a build that succeeds for some consumers (who don’t need the backward compatibility or may even require a fundamental change that renders backward compatibility impossible) while failing for other consumers who require backward compatibility. I’m sure you have plenty of “not hard” rebuttals for how to mangle bazel to address these types of things, which is meaningless because solving it with versioned, permanent artifacts is simply better. It requires you to be extremely noseblind with diehard dogmatic insistence on using your “hammer for every nail” favorite to not see this. It’s also telling to me that you focus on what you perceive to be “not hard” in your preferred tooling and have said this in several comments as if you believe it means something. I believe you are missing the point.
    • joshuamorton 8y ago
      >The identically same compiled version may be required by multiple consumers, even if they are distributed from each other. Forcing both consumers to rebuild some dependencies in such a case (as opposed to relying on pre-built artifacts from an artifact repository) is often an unacceptable performance overhead, and very wasteful even in cases when someone can afford to sit around and wait. Bazel handles all of this caching internally (yes even in the distributed case), so you don't need a secondary artifact repository. The build system will make sure you don't rebuild things unless they've changed. This is like the first selling point of bazel[0]. From bazel.build: > With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds. Literally the first callout. >The conditions necessary to make the decision of which build variant to choose can be arbitrarily complicated If you cannot express the conditions of your builds statically, something in your CI process is horribly broken. Your build system should not be trying to guess whether its invoked on CI or locally or whatnot. I should get the same artifact, no matter what, no matter where I build it. The configuration of your build artifacts should be explicit. > Much better to ensure the container that is built passes all required portability tests as a publication criteria on a container repo, so that the different consumers on different platforms can be sure if it is part of their build workflow, they are getting the correct artifact and it cannot be subject to idiosyncrasies of the platform or ambient environment at local build time. This has nothing to do with the build system though. Yes, running tests under all the versions is a good idea. You can do that. You can set up a :mac and :ubuntu environment and make sure tests pass on both (by running your CI on both mac and ubuntu). >The necessary build target for a given select condition may change over time, while still requiring backward compatibility with a former build of that same target from those same build conditions, effectively making a timestamp into an implicit version identifier for a backwards compatible version, meaning that re-running an identical build command with identical parameters at a later time will produce a build that succeeds for some consumers (who don’t need the backward compatibility or may even require a fundamental change that renders backward compatibility impossible) while failing for other consumers who require backward compatibility. This doesn't make sense. Like, I can't even comprehend what you're doing without an example. Timestamp should never be relevant, revision will be. Again, you're only ever building from HEAD and a cache based on artifact hashes. Bazel doesn't use timestamps anywhere. Timestamp based builds are an antipattern. Make is bad. You shouldn't use it. You should use anything else. So in bazel land, when you check out a previous revision, you'll build a working version. When you check out the current version, you'll build a working version. Those builds will both be incremental, only rebuilding the modified dependencies, but will be byte for byte equivalent to a clean build made at any time. >It’s also telling to me that you focus on what you perceive to be “not hard” in your preferred tooling and have said this in several comments as if you believe it means something. I believe you are missing the point. Pretty much every complaint comes from you misusing bazel. That doesn't make bazel a bad tool. It means you don't appear to understand how it works or best practices. You misapplying a tool doesn't make the tool bad, it makes you a bad user. And you're going to read that as me saying bazel doesn't work for the things you're doing. Which is incorrect. I'm almost positive it does. You just are making (wrongheaded) assumptions about how dependency management is supposed to work and trying to make bazel fit your worldview instead of the opposite. Again, user error, not tooling issues. Stop trying to shoehorn an additional artifact repository in around bazel. Stop trying to hack all these make-style isms into a bazel workflow. Use the tool as it is meant to be used and perhaps your complaints about it being problematic will go away. As I'm reading your comments, all I see is you repeatedly trying to misuse the tool and then complain that its bad when your misuse turns out to be annoying. [0]: https://docs.bazel.build/versions/master/remote-caching.html https://docs.bazel.build/versions/master/remote-caching.html