20 ms·
I cannot write software outside of Google anymore
- extheat 5y ago"anymore"? Was it really any better before? Not having a standardized packaging system in C++ has been a problem for as long as I can recall. It's reasons like this why header-only libraries are so popular. You can basically copy paste something into your project and (hopefully) call it a day. Of course, there's lots of caveats with that but I'm not sure there's anything that could be better. Maybe vcpkg manifests, but getting that into standardization/wide spread use is going to be tough.
- danielmarkbruce 5y agoI read it to be more "i can no longer be bothered". Ie, he's been spoiled by google's build infrastructure/code policies such that he just can't handle the crap anymore.
- phendrenad2 5y agoPart of the genius of FAANG is being big enough to do it all yourself. Want to make your aging PHP codebase faster? Just write your own PHP JIT (twice!!). Want to do RPC? Just invent your own protocol. Want servers? Just design your own chips. The list goes on and on. If you aren't FAANG, you suffer what you must at the hands of the open-source ecosystem. (Note that most things that FAANG invents are never open-sourced, or someone open-sources a knockoff of it, such as Google's Borg vs Kubernetes).
- rektide 5y agoOh and if you do go build awesome open source work, the best way to get attention is to do it inside a FAANG company & to release a corporate blog post on it. You'll have many orders of magnitude more attention & interest in your project, if comes from a tech giant.
- sjtindell 5y agoI thought you were about to say that if you do go build awesome work, you’ll just get bought by a FAANG anyway!
- SheinhardtWigCo 5y agoHa! I think many FAANG engineers would love nothing more but to "suffer" at the hands of the open-source ecosystem, instead of being force fed some half-baked not-invented-here thing that was cobbled together for political reasons.
- IshKebab 5y agoMaybe sometimes, but Bazel for example is light years ahead of CMake.
- mohanmcgeek 5y agoCalling these tools "knock offs" is not just insulting to the people building these tools but also far detached from reality since several FAANG employees would tell you how much better the OSS rewrite is compared to the original one.
- phendrenad2 5y agoInsult intended. It's insulting to me that people keep saying that k8s is "from Google" (as though it's a good thing) but then turn around and say that Borg is what Google actually uses internally (as though it's a bad thing).
- mohanmcgeek 5y agoI neither understand how that's insulting to you nor how that justifies you insulting these OSS projects. Did you write Borg or something?
- cybrexalpha 5y ago> …or someone open-sources a knockoff of it, such as Google's Borg vs Kubernetes Google made and open-sourced Kubernetes themselves. Yes it's based on Borg, but it was an internal project first.
- hutrdvnj 5y agoNowadays it's called MAMAA instead of FAANG.[0] [0] https://fortune.com/2021/10/29/faang-mamaa-jim-cramer-tech-facebook-meta/ https://fortune.com/2021/10/29/faang-mamaa-jim-cramer-tech-f...
- vnchr 5y agoJust because Jim Kramer says so doesn’t mean it’s true. If they’d have kept referring to Google instead of Alphabet, it could be MAGMA. I should tell Jim Kramer. Maybe he’ll make that change official.
- speedcoder 5y agoDid you mean NAGMA, as in they won't stop nagging you?
- phendrenad2 5y agoAlphabet, Apple, Amazon, Microsoft, Meta, Netflix. MAAMAN: "Wealth regarded as an evil influence or false object of devotion"
- theamk 5y agoThat's the big role of Linux distributions (debian, fedora, ubuntu, etc...) -- they have thousands of people packaging tens of thousands of C and C++ libraries. You choose whatever build system you like, create a single-line "install.sh" which runs some "apt install" command, and you are done -- things just work. The main downsides are that packages are somewhat old, and that your users need the have the same distribution. But containers help a lot with the latter problem.
- syntheweave 5y agoMaybe it's a pipe dream, but I'd love to see a project that tries to unify a lot of existing software under one robust, featureful build system, doing a job similar to packaging but with a more invasive method. Because, of course what will happen from there is that, while nearly everyone will grumble that it is not the best build system, they will use it in anger when they want to get their personal projects running. At that point it becomes the defacto standard and we can escape snowflake-build madness.
- Wowfunhappy 5y agoI don't think I understand how what you're describing is different than a Linux distribution. Edit: I suppose e.g. Debian is generally for binaries, but what I'm pretty sure source packages still work the same way? There's also Gentoo, and maybe Nix.
- JustLurking2022 5y agoBazel?
- smilliken 5y agoYou're looking for nixpkgs. Years ago I switched to it in anger for builds breaking, and I've had a much healthier relationship with computers since then.
- breakds 5y ago+1. I feel the same as OP after leaving Google. However, after switching to Nix, I had a greater experience working with projects (with mulitple languages), even better than the mono repo at Google.
- mrintellectual 5y agoFor a short period of time circa 2017/2018, Google was on the path toward monopolizing machine learning as well through Tensorflow. Thankfully, PyTorch overtook TF.
- dkarp 5y agoWhich was open sourced by another tech giant (FB)
- mrintellectual 5y agoThe original Lua-based Torch is actually from EPFL, not Facebook. Tensorflow originated from Google's internal DistBelief system, making TF a Google project through and through.
- torbTurret 5y agoBut nobody would care about PyTorch if it didn’t have big tech’s blessing, which is the point the above was making.
- disgruntledphd2 5y agoWow, and the paper was released in 2002!: https://infoscience.epfl.ch/record/82802?ln=en https://infoscience.epfl.ch/record/82802?ln=en To be fair though, the reason PyTorch exists is because Yann Le Cun used Torch, and presumably FB thought a Python version was better than teaching engineers Lua.
- rektide 5y agoThis point is somewhat more amusing in that Google is responsible for at least 2 of the major build tools in the field of options: Ninja (built to make faster Chrome compiles[1]) and Bazel. > You can use any 3rd party open source library you like, as long as you use their build, package management scheme, compiler, what have you. But you cannot then use any other open source library. Most everything I've seen will happily accept whatever I've installed into /usr/local/lib and /usr/local/include. Yes, you have to build the thing first. But it's gotten so much easier & better, generally. Sure there's configure, cmake, make, different toolchains... but most of the build tools are pretty flexible & configurable in pretty standard ways. Way more projects are toolchain neutral. (That said, webassembly/emsdk adds a whole new depth of complexity to everything.) And meson is kicking ass & taking names everywhere, is so much less of a "write it yourself" bag-of-tools and so much more "it just works" framework. [1] https://www.aosabook.org/en/posa/ninja.html https://www.aosabook.org/en/posa/ninja.html
- eternityforest 5y agoThe good thing is if you aren't FAANG scale, you probably don't need to write anything that involves significant build process hassle. If you do, you probably have enough people and a good process to kinda get by. Someone has almost certainly packaged the thing you want with Python bindings.
- xt00 5y agoI think most people get the sentiment but to say that somehow it’s better inside Google sounds hilarious to me.. try building chromium or similar things and you are greeted with a huge set of one off “Google solutions” that almost nobody else uses. oh don’t forget to edit the args.gn file and GYP defines and set your sysroot via a python script if you need to cross compile, download depot_tools and put it in your path so you can run “fetch” commands among others then finally now you have a ninja file and you can run ninja.. ok so all of those solutions somehow are better than writing a cmake file? It’s like one of the biggest problems that seems to be a plague related to build systems is that there is a point somewhere around the level of a makefile that a very large number of programmers can understand but is hard to maintain so various companies build infrastructure on top of the makefile that is semi custom.. then another engineer comes along and builds abstraction on top of that. So you end up with this house of cards that depends upon python 3.8, etc etc then finally there is a makefile. I do wish all of these whacky build systems would be required to have some choke point where it prints to stdout “generated makefile at path = /……, continuing in 3 seconds… running make -f xxxxxx”
- jsnell 5y agoThe author is referring to Google's internal monorepo, not to their open source projects.
- Brian_K_White 5y agoThis is not new. When AT&T dissolved the group that had been maintaining a bunch of unix software, and other people had to try to pick up maintenance of ksh, they discovered a little thing called AST.
- durnygbur 5y agoBut still you can post an article on the internet without Tw... 1/5
- torbTurret 5y ago“You can’t benefit from my awesome work because I’m too comfortable using tools at my job. Did I mention I work for Google? Listen to my music.” Not trying to start a flame war here but… who cares? I’ve published open source while working at faang myself and they all have internal build tools, version sets, etc. Plenty of employees still do tons of open-source.
- Nextgrid 5y ago> who cares? He's still making a valid point that dependency management & build systems for C & derivatives are a shit show.
- Zababa 5y agoHow many times this "valid point" has been made since the beginning of the year? Saying it more won't improve the situation.
- rurban 5y agothey are not. people who are using the wrong build system are a shit show instead. autotools is great, meson and bazel similary. but people quirking around with cmake alone, or conan or other toys cannot be helped much.
- hoseja 5y agoAre you trying to ignite a flamewar.
- rurban 5y agono, autotools critics just don't know what they are talking about.
- tetromino_ 5y ago> autotools is great I assume you are joking. Autotools is a slow-as-molasses system (running autoreconf could take half of your build's wall time!) designed for 1980s unix wars (with support for anything not resembling 1980s unix C programs being an awkward bolt-on), complete with an unusually hostile syntax and multiple levels of inefficient interpreters, all optimized to encourage cargo-cult copy-paste programming. It was barely tolerable at the turn of the century, when everyone used it for lack of a good alternative. For a while, its one saving grace was decent support for cross-compiling. But using it for a new project today is insane.
- durnygbur 5y ago...itter. 2/5
- perfopt 5y agoDoesn't this also serve as a nice lock in ? S/w engineers dont leave because they are too dependent on an internal ecosystem, or have years of knowledge built in and leaving means all that effort is wasted.
- FpUser 5y agoNot my experience. I write C++ backend servers among the other things and use few 3rd party libraries. With CMake all builds like a piece of cake. I of course might not represent general state of affairs but this is my 2c anyways.
- flohofwoe 5y agoThe other side of the coin is that it is almost impossible to use Google's open source libraries outside of Google, and I mean specifically the C++ stuff. Everything from the build system to the coding style oozes "NIH", even seemingly simple things are overengineered to fit the "Google way", and god forbid they use any standard tools or even ideas from outside Google. It wouldn't be a problem if the Google tools would be great, but they feel clunky and as if they've been frozen in time 10..15 years ago.
- dtx1 5y agohttps://imgflip.com/i/68q2i7 https://imgflip.com/i/68q2i7
- dTal 5y agoIt's funny but I have the same problem with digital music composition. Sure I can noodle on a physical musical instrument, and I can imagine elaborate compositions very easily, but the cognitive overhead of using pretty much any software, let alone the disjointed mess of open source music software available on Linux, completely takes me out of the headspace required to effectively compose. I know some people manage to do it, in the same way that some people manage to write complex software that uses many libraries in C++. Are they just smarter than me? Or have they simply {discovered, bought, created} a superior toolchain?
- auxfil 5y agoEveryone seems to be either just reading the 1st tweet/the title, or using it as a prompt to insert their own gripes about Google's insular technologies. The bigger takeaway here is that working at a large company, for any length of time, tends to sink you in (rather than lock you in) to their ways of working and it can be rather difficult to step into the wide open source world afterwards. It is not out of laziness, but out of learned ways of working and domain knowledge. It is difficult to extricate yourself from "The <Company> way"...sometimes with panic and dismay at the sheer amount of options out there, that simply aren't as interoperable as "The <Company> way" was. This is a legitimate and valid pain point that professional software engineers, working in companies, face in their careers. A smaller but equally important thing to note is mention of the number of people working on the build systems internally, that make and maintain interoperability between both internal and 3rd party libraries and tools, to create a smooth build system.
- mbrodersen 5y agoA good software developer can be productive with any set of tools/languages. If you can’t be productive, unless you have a very specific set of tools, then the problem is you.