7 ms·

> code from Google. I spilled my coffee, I was just talking the other day to some coworkers how I don't trust google open source. Sure they open their code but

> code from Google. I spilled my coffee, I was just talking the other day to some coworkers how I don't trust google open source. Sure they open their code but they don't give a damn about contributions or making it easy for you to use the projects. I feel a lot of this sentiment extends to GCP as well. So many google projects are better than your average community one, but they never gain traction outside of google because it is just too damn hard to use them outside of google infra. The only Google project that seems to evade this rule that I know of is Go.

badpun 2y ago
Tensorflow is/was decent. It looked like they made a lot of effort for it to be accessible for outsiders.
- th2oi34234234 2y ago
  Have you tried building the damn thing ? Nix build is still stuck in the one from 3-4 y back because bazel doesn't play well. Debian too has some issues building the thing...
  - jimmaswell 2y ago
    As an industry we need to stop treating breaking changes as an acceptable thing. The rate of bit rot has accelerated to an absurd pace. I can't remember the package but I had to spend considerable time fixing a build because a package.. changed names.. for NO REASON. They just liked the new name better. This should be career death. You're wasting your fellow humans' time and energy on your vanity when you make a breaking change that is at all avoidable. I should be able to run a build script made 20 years ago and it should just work. No renamed package hunting, no WARNING WARNING DEPRECATED REWRITE ALL YOUR CODE FOR LEFTPAD 10.3 IMMEDIATELY in the console, no code changes, no fuss, we should expect it to just work. This state of affairs is a stain on our industry.
    - __MatrixMan__ 2y ago
      One day we will have bled enough and we'll switch to using cryptographic hashes of package contents (or of some recipe for deterministically building the thing on different architectures) instead of anything so flimsy as a name and version number. For the humans, we can render the hashes as something friendly, but there's no reason to confuse the machines with our human notions of friendliness.
      - School-Cotton 2y ago
        You’re basically describing nix and Guix.
        __MatrixMan__ 2y ago
        They use a hash of the derivation and its inputs as a memoization strategy: providing yesterday's answer to today's question since it was asked yesterday. But so far as I know nobody's actually using those hashes for the initial request. It's not like python will let you: import nix.numpy-hsbdjd...8r5z2 as np Such that the import mechanism ensures that the correct build of numpy is used. For that to work you'd have to change nix such that the hash did not digest parameters like `amd64-linux` witch indicated the system architecture (you'd want those to be satisfied at import time).
        School-Cotton 2y ago
        In Guix at least (I assume also nix) you can build things from source with a verified hash. I.e. write a numpy package definition that says download the Numpy source from this URL, and expect its hash to be equal to this string. You could then depend on that package from another package ensuring it uses a numpy built from that bit-for-bit exact source tree. Does that not amount to the same thing as what you want?
    - knome 2y ago
      this is why you build to a specific version of a library. drop your build script into a container with the versions of software it expects and it should do fine. containerization is the admittance that versioning environments is needed for most software. I expect the nix/guix crowds to win in the end.
  - p_l 2y ago
    Having tried on other platforms, it's not Bazwl, it's not even just Google. It's python packaging and the way the only really supported binary distribution method of Tensorflow for many many years was to use Pip and hope it doesn't crash. And it's reflected in how the TF build scripts only support building python lib as artefact, everything else at the very least involved dissecting bazel intermediate targets
humanrebar 2y ago
Googletest is the most widely used test library for C++. Googlemock is the only mocking library available that's reasonably feature complete.
- gpderetta 2y ago
  Google test and mock are quite powerful but are a big hit at both compile time and runtime, which matters for quick edit-compile-fix loops. I still go back and forth on whether google test and mock are worth it. Google benchmark is also nice.
  - rangestransform 2y ago
    > big hit at both compile time and runtime, which matters for quick edit-compile-fix loops honestly if you write C++ for work, there's no excuse for your company to not give you the beefiest dev machine that money can reasonably buy. given that rust exists, I think "get a faster computer" is a totally valid answer to build times, especially now that skylake malaise era is over and CPUs are getting faster
    - badsectoracula 2y ago
      > given that rust exists, I think "get a faster computer" is a totally valid answer to build times I find this amusing because one of the main reasons i avoid Rust (in the sense that i prefer to build things written in other languages if possible - i don't mind if someone else uses it and gives me a binary/library i can use - and it never went beyond "i might check this at some point, sometime, maybe" in my mind) is the build times compared to most other compilers :-P. Also, at least personally, if i get a faster computer i want my workflow to be faster.
    - tmtvl 2y ago
      You may want to add a '/s' at the end of your post there, because sarcasm doesn't really translate on the internet. The only way I can tell it's sarcasm is because nobody would really go 'throw away the old stuff, buy new stuff, waste more, pollute the oceans, consume, CONSUME!!!'.
kccqzy 2y ago
> but they don't give a damn about contributions Here is a concrete reason why Google open source sucks when it comes to contributions and I don't think it can be improved unless Google changes things drastically: (1) an external contributor makes a nice change and a PR on GitHub; (2) the change breaks internal use cases and their tests; (3) the team is unwilling to fix the PR or port the internal test (which may be a test several layers down the dependency tree) to open source. > making it easy for you to use the projects Google internally use Blaze, a version of Bazel. It's so ridiculously easy for one team to use another team's project that even just thinking about what the rest of us needs to do to use another project is unloved dreadful work. So people don't make that effort. I do not see either of these two points changing. Sure there are individuals at Google that really care about open source community, but most don't, and so their project is forever a cathedral not a bazaar.
- scrubs 2y ago
  Google wants to be technically persuasive -- a player in an engineering space (ergo open source) -- without being beset with the roles and responsibilities of a 3rd party being a bonafide customer when push comes to shove.
- DanielHB 2y ago
  It is not only that, but often when google uses an open source project not owned by them they either try to take ownership of the project or fork it instead of trying to contribute to the original.
  - jsnell 2y ago
    Which cases did you have in mind? Seems like it should be easy to find half a dozen examples since you claim it happens often.
    - Create 2y ago
      KHTML, officially discontinued in 2023. -- "Embrace, extend, and extinguish" (EEE) also known as "embrace, extend, and exterminate", is a phrase that the U.S. Department of Justice. It's also possible that President-elect Donald Trump may interfere with the DOJ's proposed remedies; he said on the campaign trail that a Google break-up may not be desirable since it could "destroy" a company that the US highly values.
__MatrixMan__ 2y ago
I blame monorepo culture. If it doesn't grow up in a context where it's expected to stand on its own, it crashes and burns when you kick it out of the nest.
- DanielHB 2y ago
  I heard that Meta also has a monorepo but most of their open source projects are very community driven. I think it is corporate mandate thing, no resources to be spent on open source and not tracking open source contributions as part of career development.
  - School-Cotton 2y ago
    Meta does have a monorepo but their open source stuff lives outside it. Or at least it did when I worked on PyTorch (2019). I did all my work in the separate open-source PyTorch repo and then commits got mirrored back to the monorepo by some automated process. You could also build and run it using completely standard tools; you didn’t need to download random internal source control software etc. like you do for e.g. Chromium.
    - DanielHB 2y ago
      Curious about the organizational dynamics around this kind of decisions. There is no reason why google couldn't do the same. I assume there is little will internally because everyone there is so focused on their performance reviews and helping external people using google open source projects is not tracked by that.
      - __MatrixMan__ 2y ago
        I think it's more of a strategic difference. Google seems like their long term planning involves thinking about open source less than Meta's. They're more wait-and-see about it. React must've been destined to be open source from the get go: gotta create a mountain of js to hide in so the users can't strip out the malicious parts. Kubernetes on the other hand could've been internal forever and still would've made sense. It just happened to later make sense to open source it (it feels lopsided to me, like they kept certain parts secret. It wouldn't feel that way if they had planned it as OSS from the get go).

badpun 2y ago

Tensorflow is/was decent. It looked like they made a lot of effort for it to be accessible for outsiders.

th2oi34234234 2y ago
Have you tried building the damn thing ? Nix build is still stuck in the one from 3-4 y back because bazel doesn't play well. Debian too has some issues building the thing...
- jimmaswell 2y ago
  As an industry we need to stop treating breaking changes as an acceptable thing. The rate of bit rot has accelerated to an absurd pace. I can't remember the package but I had to spend considerable time fixing a build because a package.. changed names.. for NO REASON. They just liked the new name better. This should be career death. You're wasting your fellow humans' time and energy on your vanity when you make a breaking change that is at all avoidable. I should be able to run a build script made 20 years ago and it should just work. No renamed package hunting, no WARNING WARNING DEPRECATED REWRITE ALL YOUR CODE FOR LEFTPAD 10.3 IMMEDIATELY in the console, no code changes, no fuss, we should expect it to just work. This state of affairs is a stain on our industry.
  - __MatrixMan__ 2y ago
    One day we will have bled enough and we'll switch to using cryptographic hashes of package contents (or of some recipe for deterministically building the thing on different architectures) instead of anything so flimsy as a name and version number. For the humans, we can render the hashes as something friendly, but there's no reason to confuse the machines with our human notions of friendliness.
    - School-Cotton 2y ago
      You’re basically describing nix and Guix.
      - __MatrixMan__ 2y ago
        They use a hash of the derivation and its inputs as a memoization strategy: providing yesterday's answer to today's question since it was asked yesterday. But so far as I know nobody's actually using those hashes for the initial request. It's not like python will let you: import nix.numpy-hsbdjd...8r5z2 as np Such that the import mechanism ensures that the correct build of numpy is used. For that to work you'd have to change nix such that the hash did not digest parameters like `amd64-linux` witch indicated the system architecture (you'd want those to be satisfied at import time).
        School-Cotton 2y ago
        In Guix at least (I assume also nix) you can build things from source with a verified hash. I.e. write a numpy package definition that says download the Numpy source from this URL, and expect its hash to be equal to this string. You could then depend on that package from another package ensuring it uses a numpy built from that bit-for-bit exact source tree. Does that not amount to the same thing as what you want?
  - knome 2y ago
    this is why you build to a specific version of a library. drop your build script into a container with the versions of software it expects and it should do fine. containerization is the admittance that versioning environments is needed for most software. I expect the nix/guix crowds to win in the end.
- p_l 2y ago
  Having tried on other platforms, it's not Bazwl, it's not even just Google. It's python packaging and the way the only really supported binary distribution method of Tensorflow for many many years was to use Pip and hope it doesn't crash. And it's reflected in how the TF build scripts only support building python lib as artefact, everything else at the very least involved dissecting bazel intermediate targets

humanrebar 2y ago

Googletest is the most widely used test library for C++. Googlemock is the only mocking library available that's reasonably feature complete.

gpderetta 2y ago
Google test and mock are quite powerful but are a big hit at both compile time and runtime, which matters for quick edit-compile-fix loops. I still go back and forth on whether google test and mock are worth it. Google benchmark is also nice.
- rangestransform 2y ago
  > big hit at both compile time and runtime, which matters for quick edit-compile-fix loops honestly if you write C++ for work, there's no excuse for your company to not give you the beefiest dev machine that money can reasonably buy. given that rust exists, I think "get a faster computer" is a totally valid answer to build times, especially now that skylake malaise era is over and CPUs are getting faster
  - badsectoracula 2y ago
    > given that rust exists, I think "get a faster computer" is a totally valid answer to build times I find this amusing because one of the main reasons i avoid Rust (in the sense that i prefer to build things written in other languages if possible - i don't mind if someone else uses it and gives me a binary/library i can use - and it never went beyond "i might check this at some point, sometime, maybe" in my mind) is the build times compared to most other compilers :-P. Also, at least personally, if i get a faster computer i want my workflow to be faster.
  - tmtvl 2y ago
    You may want to add a '/s' at the end of your post there, because sarcasm doesn't really translate on the internet. The only way I can tell it's sarcasm is because nobody would really go 'throw away the old stuff, buy new stuff, waste more, pollute the oceans, consume, CONSUME!!!'.

kccqzy 2y ago

> but they don't give a damn about contributions Here is a concrete reason why Google open source sucks when it comes to contributions and I don't think it can be improved unless Google changes things drastically: (1) an external contributor makes a nice change and a PR on GitHub; (2) the change breaks internal use cases and their tests; (3) the team is unwilling to fix the PR or port the internal test (which may be a test several layers down the dependency tree) to open source. > making it easy for you to use the projects Google internally use Blaze, a version of Bazel. It's so ridiculously easy for one team to use another team's project that even just thinking about what the rest of us needs to do to use another project is unloved dreadful work. So people don't make that effort. I do not see either of these two points changing. Sure there are individuals at Google that really care about open source community, but most don't, and so their project is forever a cathedral not a bazaar.

scrubs 2y ago
Google wants to be technically persuasive -- a player in an engineering space (ergo open source) -- without being beset with the roles and responsibilities of a 3rd party being a bonafide customer when push comes to shove.
DanielHB 2y ago
It is not only that, but often when google uses an open source project not owned by them they either try to take ownership of the project or fork it instead of trying to contribute to the original.
- jsnell 2y ago
  Which cases did you have in mind? Seems like it should be easy to find half a dozen examples since you claim it happens often.
  - Create 2y ago
    KHTML, officially discontinued in 2023. -- "Embrace, extend, and extinguish" (EEE) also known as "embrace, extend, and exterminate", is a phrase that the U.S. Department of Justice. It's also possible that President-elect Donald Trump may interfere with the DOJ's proposed remedies; he said on the campaign trail that a Google break-up may not be desirable since it could "destroy" a company that the US highly values.

__MatrixMan__ 2y ago

I blame monorepo culture. If it doesn't grow up in a context where it's expected to stand on its own, it crashes and burns when you kick it out of the nest.

DanielHB 2y ago
I heard that Meta also has a monorepo but most of their open source projects are very community driven. I think it is corporate mandate thing, no resources to be spent on open source and not tracking open source contributions as part of career development.
- School-Cotton 2y ago
  Meta does have a monorepo but their open source stuff lives outside it. Or at least it did when I worked on PyTorch (2019). I did all my work in the separate open-source PyTorch repo and then commits got mirrored back to the monorepo by some automated process. You could also build and run it using completely standard tools; you didn’t need to download random internal source control software etc. like you do for e.g. Chromium.
  - DanielHB 2y ago
    Curious about the organizational dynamics around this kind of decisions. There is no reason why google couldn't do the same. I assume there is little will internally because everyone there is so focused on their performance reviews and helping external people using google open source projects is not tracked by that.
    - __MatrixMan__ 2y ago
      I think it's more of a strategic difference. Google seems like their long term planning involves thinking about open source less than Meta's. They're more wait-and-see about it. React must've been destined to be open source from the get go: gotta create a mountain of js to hide in so the users can't strip out the malicious parts. Kubernetes on the other hand could've been internal forever and still would've made sense. It just happened to later make sense to open source it (it feels lopsided to me, like they kept certain parts secret. It wouldn't feel that way if they had planned it as OSS from the get go).