4 ms·

I agree! But — - BigQuery is available on GCP - Bazel is an open source Blaze - google source formatting (for java at least) is open source There are probab

by mayank 4y ago

I agree! But — - BigQuery is available on GCP - Bazel is an open source Blaze - google source formatting (for java at least) is open source There are probably more…

metadat 4y ago
Blaze/Bazel actually sucks imo. The only good thing about it is that all of google uses that one piece of shit, making things nice and uniform and consistent. There's a reason it isn't popular outside of elgoog. IRL, every project tends to do a little bit or a lot of its own thing.
- dilyevsky 4y ago
  It isn’t popular? Like “uber and a dozen of similar sized companies and some major oss projects using it” not popular?
  - metadat 4y ago
    Uber engineering has a few gems but for every 1 gem they've hired 10 or more greedy idiots, so maybe not a great example? If you love fighting with your hands tied behind your back, choose Bazel. Otherwise, be pragmatic: Learn Make, Maven, and Gradle; then you'll be well-equipped for 95-99% of cases. Thankfully pip and npm are as straightforward as it gets.
    - ctvo 4y ago
      > Uber engineering has a few gems but for every 1 gem they've hired 10 or more greedy idiots, so maybe not a great example? What an odd sprinkling of something entirely personal. > Otherwise, be pragmatic: Learn Make, Maven, and Gradle; then you'll be well-equipped for 95-99% of cases. There's a time and a place for Bazel. Very large monorepos like those at Pinterest and Uber, with cross dependencies, and written in multiple languages benefit a lot from the remote backend and distributed cache of built artifacts. Make, Maven, and Gradle, even only for JVM based projects seem to not be entirely comparable.
    - dilyevsky 4y ago
      Can you explain why you bother with pip and npm if php and plain js already cover 95-99% of cases?
      - metadat 4y ago
        Because I do whatever it takes to get the job done. I also appreciate what Python and Javascript offer, there are some amazing libraries and tools tied to those ecosystems.
- dub 4y ago
  Bazel can be clunky, but not having some bazel equivalent can have very significant costs that are easy to get accustomed to or overlook. Things like engineers losing time wondering why their node dependencies weren't correctly installed, or dealing with a pre-commit check that reminds them they didn't manually regenerate the generated files, or having humans write machine-friendly configuration that's not actually human-friendly because there's no easy way to introduce custom file transformations during the build. Bazel doesn't spark joy for me and I wouldn't say I look forward to using it, but personally I would still always choose it for a codebase that's going to have multiple developers and last a long time. It's vastly easier to go with bazel from the beginning than to wish you could switch to it and realize people have already introduced a million circular dependencies and it's going to be a multi-month or multi-year process to migrate to it.
  - reissbaker 4y ago
    In my experience, Bazel is a net negative for most teams. Pretty much every JS engineer is familiar with npm; a tiny fraction are familiar with Bazel. Ditto with pip, cargo, etc. And it doesn't solve the hard part of the build process, which is distributed builds. Most of the user-perceptible value of Blaze is making builds fast by farming them out to a zillion machines — that's why it's called "blaze," because it's fast! — and Bazel doesn't do that for you. And it's clunky, and you need to teach every new hire how to use it. The juice just isn't worth the squeeze. Just write an adapter in your build infra for the well known tools and be done with it. You'll get much more value putting work into something else, like code review tools, testing, dev environments, staging...
    - dilyevsky 4y ago
      There are many remote build backends for bazel available - some open source and some proprietary (EngFlow and Google’s RBE). Even without a build farm it will still be a huge performance boost due to caching - something make and co (recommended in sibling thread) can not do along with a bunch of other stuff that bazel does for you.
      - 4y ago
- sleepydog 4y ago
  As a googler, I agree with this. Blaze itself is not that remarkable. What makes it good is that it is the only tool you need to use to build anything, in any language, in Google's monorepo, and that build failures are (usually) aggressively corrected. Plenty of build systems these days support varying levels of hermetic builds. The important thing to copy from Google here is the discipline and consistency around using one build tool and fixing build failures, not the specific tool itself.
dub 4y ago
For those of us stuck on AWS it's sad not having BigQuery, but the thing that really gets me is not having Dataflow Most of industry still seems unaware that no-knobs data query and pipeline systems even exist. If I only had a dollar for every time I saw a PR tweaking the memory settings of some Spark job or hive query that stopped running as the input data grew.... I'd love to see more people write their workflows using the Apache Beam API so they'll have the option to switch to a no-knobs, scalable pipeline engine in the future even if they're not using one today.

by mayank 4y ago

I agree! But — - BigQuery is available on GCP - Bazel is an open source Blaze - google source formatting (for java at least) is open source There are probably more…

metadat 4y ago

Blaze/Bazel actually sucks imo. The only good thing about it is that all of google uses that one piece of shit, making things nice and uniform and consistent. There's a reason it isn't popular outside of elgoog. IRL, every project tends to do a little bit or a lot of its own thing.

dilyevsky 4y ago
It isn’t popular? Like “uber and a dozen of similar sized companies and some major oss projects using it” not popular?
- metadat 4y ago
  Uber engineering has a few gems but for every 1 gem they've hired 10 or more greedy idiots, so maybe not a great example? If you love fighting with your hands tied behind your back, choose Bazel. Otherwise, be pragmatic: Learn Make, Maven, and Gradle; then you'll be well-equipped for 95-99% of cases. Thankfully pip and npm are as straightforward as it gets.
  - ctvo 4y ago
    > Uber engineering has a few gems but for every 1 gem they've hired 10 or more greedy idiots, so maybe not a great example? What an odd sprinkling of something entirely personal. > Otherwise, be pragmatic: Learn Make, Maven, and Gradle; then you'll be well-equipped for 95-99% of cases. There's a time and a place for Bazel. Very large monorepos like those at Pinterest and Uber, with cross dependencies, and written in multiple languages benefit a lot from the remote backend and distributed cache of built artifacts. Make, Maven, and Gradle, even only for JVM based projects seem to not be entirely comparable.
  - dilyevsky 4y ago
    Can you explain why you bother with pip and npm if php and plain js already cover 95-99% of cases?
    - metadat 4y ago
      Because I do whatever it takes to get the job done. I also appreciate what Python and Javascript offer, there are some amazing libraries and tools tied to those ecosystems.
dub 4y ago
Bazel can be clunky, but not having some bazel equivalent can have very significant costs that are easy to get accustomed to or overlook. Things like engineers losing time wondering why their node dependencies weren't correctly installed, or dealing with a pre-commit check that reminds them they didn't manually regenerate the generated files, or having humans write machine-friendly configuration that's not actually human-friendly because there's no easy way to introduce custom file transformations during the build. Bazel doesn't spark joy for me and I wouldn't say I look forward to using it, but personally I would still always choose it for a codebase that's going to have multiple developers and last a long time. It's vastly easier to go with bazel from the beginning than to wish you could switch to it and realize people have already introduced a million circular dependencies and it's going to be a multi-month or multi-year process to migrate to it.
- reissbaker 4y ago
  In my experience, Bazel is a net negative for most teams. Pretty much every JS engineer is familiar with npm; a tiny fraction are familiar with Bazel. Ditto with pip, cargo, etc. And it doesn't solve the hard part of the build process, which is distributed builds. Most of the user-perceptible value of Blaze is making builds fast by farming them out to a zillion machines — that's why it's called "blaze," because it's fast! — and Bazel doesn't do that for you. And it's clunky, and you need to teach every new hire how to use it. The juice just isn't worth the squeeze. Just write an adapter in your build infra for the well known tools and be done with it. You'll get much more value putting work into something else, like code review tools, testing, dev environments, staging...
  - dilyevsky 4y ago
    There are many remote build backends for bazel available - some open source and some proprietary (EngFlow and Google’s RBE). Even without a build farm it will still be a huge performance boost due to caching - something make and co (recommended in sibling thread) can not do along with a bunch of other stuff that bazel does for you.
    - 4y ago
sleepydog 4y ago
As a googler, I agree with this. Blaze itself is not that remarkable. What makes it good is that it is the only tool you need to use to build anything, in any language, in Google's monorepo, and that build failures are (usually) aggressively corrected. Plenty of build systems these days support varying levels of hermetic builds. The important thing to copy from Google here is the discipline and consistency around using one build tool and fixing build failures, not the specific tool itself.

dub 4y ago

For those of us stuck on AWS it's sad not having BigQuery, but the thing that really gets me is not having Dataflow Most of industry still seems unaware that no-knobs data query and pipeline systems even exist. If I only had a dollar for every time I saw a PR tweaking the memory settings of some Spark job or hive query that stopped running as the input data grew.... I'd love to see more people write their workflows using the Apache Beam API so they'll have the option to switch to a no-knobs, scalable pipeline engine in the future even if they're not using one today.