7 ms·

Of course it does. Pretty much every FAANG, and many smaller ones, have teams for every programming language they consider important. Google has C++, Rust, Pyth

Of course it does. Pretty much every FAANG, and many smaller ones, have teams for every programming language they consider important. Google has C++, Rust, Python, Java, JavaScript, and others (although I've heard that several of these teams were hard it in this round.) To take a basic example: Google has a team that works inside LLVM and Clang to improve the code it produces for Google's specific workloads. (Often these help all of LLVM's clients.) But if this team is able to make a search query faster by even 0.01% a year, at Google scale that saves literally millions of hours in compute across Google's fleet. And millions of hours of compute is millions of dollars in power, space, and need to expand. This is true of all the other language teams at Google too. The Python team builds a better python by fixing upstream bugs, reducing the memory it consumes and so on. It upgrades the internal installation of Python to the newest versions, ensuring that the upgrade is smooth and doesn' break the using teams. It contributes proposals for language features to upstream, ensuring that Google's use cases are at least considered.

zem 2y ago
in addition to contributing to upstream python, we * maintained a stable version of python within google, and made sure that everything in the monorepo worked with it. in my time on the team we moved from 2.7 to 3.6, then incrementally to 3.11, each update taking months to over a year because the rule at google is if you check any code in, you are responsible for every single breakage it causes * maintained tools to keep thousands of third party packages constantly updated from their open source versions, with patch queues for the ones that needed google-specific changes * had highly customised versions of tools like pylint and black, targeted to google's style guide and overall codebase * contributed to pybind11, and maintained tools for c++ integration * developed and maintained build system rules for python, including a large effort to move python rules to pure starlark code rather than having them entangled in the blaze/bazel core engine * developed and maintained a typechecker (pytype) that would do inference on code without type annotations, and work over very large projects with a one-file-at-a-time architecture (this was my primary job at google, ama) * performed automated refactorings across hundreds of millions of lines of code and that was just the dev portion of our jobs. we also acted as a help desk of sorts for python users at google, helping troubleshoot tricky issues, and point newcomers in the right direction. plus we worked with a lot of other teams, including the machine learning and AI teams, the colaboratory and IDE teams, teams like protobuf that integrated with and generated python bindings, teams like google cloud who wanted to offer python runtimes to their customers, teams like youtube who had an unusually large system built in python and needed to do extraordinary things to keep it performant and maintainable. and we did all this for years with fewer than 10 people, most of whom loved the work and the team so much that we just stayed on it for years. also, despite the understaffing, we had managers who were extremely good about maintaining work/life balance and the "marathon, not sprint" approach to work. as i said in another comment, it's the best job i've ever had, and i'll miss it deeply.
- fluffyfe 2y ago
  That is a lot of great work. Sorry to hear that it is gone. Take some time to rest and hope find somewhere with better vision!
- deleted 2y ago
  [deleted]
- iandanforth 2y ago
  When was pytype invoked? Runtime? CI/CD? Did it output types to be added to files or did it infer and then enable downstream tasks based on those inferences?
  - zem 2y ago
    it was invoked manually at build time, and then at CI/CD time (there was no runtime type checking). we did output types to pyi files (the python equivalent of C header files), but mainly we used the inferred types to check the code for type correctness - one of the unique selling points of pytype, and why google was developing its own typechecker in the first place, was that the other available python type checkers would check annotated code for consistency, but would not do anything about type errors in unannotated code. if pytype did detect type errors, the code failed CI and had to be fixed before it was checked in. we did also have a separate tool that could merge the inferred types back into the source code as annotations, but it never got much uptake.
- zem 2y ago
  > the rule at google is if you check any code in, you are responsible for every single breakage it causes i can no longer edit my post to clarify this, but by "responsible for breakages" i meant that if your new check-in caused any CI tests anywhere within the codebase to fail, even if due to bugs in the other code, you had to stop and fix it, or get the owners to fix it, or find some principled way to temporarily disable those tests, before you could check your code in. this was a very real issue for things like the python runtime or widely used code checkers like pytype or pylint, because if e.g. the new version of python, or some improved check in pytype, started raising failures in a code pattern that was technically wrong but which the existing toolchain did not complain about, you could not release the new version until you had fixed all those new breakages. contrast this with non-monorepo codebases, where the other code would have had to deal with the fact that "oh, our code works under 3.10 but 3.11 broke it, guess we have to pin our own repo to 3.10 until we fix it", but as the python team we could just say "we support 3.11 now, you need to catch up"

zem 2y ago

in addition to contributing to upstream python, we * maintained a stable version of python within google, and made sure that everything in the monorepo worked with it. in my time on the team we moved from 2.7 to 3.6, then incrementally to 3.11, each update taking months to over a year because the rule at google is if you check any code in, you are responsible for every single breakage it causes * maintained tools to keep thousands of third party packages constantly updated from their open source versions, with patch queues for the ones that needed google-specific changes * had highly customised versions of tools like pylint and black, targeted to google's style guide and overall codebase * contributed to pybind11, and maintained tools for c++ integration * developed and maintained build system rules for python, including a large effort to move python rules to pure starlark code rather than having them entangled in the blaze/bazel core engine * developed and maintained a typechecker (pytype) that would do inference on code without type annotations, and work over very large projects with a one-file-at-a-time architecture (this was my primary job at google, ama) * performed automated refactorings across hundreds of millions of lines of code and that was just the dev portion of our jobs. we also acted as a help desk of sorts for python users at google, helping troubleshoot tricky issues, and point newcomers in the right direction. plus we worked with a lot of other teams, including the machine learning and AI teams, the colaboratory and IDE teams, teams like protobuf that integrated with and generated python bindings, teams like google cloud who wanted to offer python runtimes to their customers, teams like youtube who had an unusually large system built in python and needed to do extraordinary things to keep it performant and maintainable. and we did all this for years with fewer than 10 people, most of whom loved the work and the team so much that we just stayed on it for years. also, despite the understaffing, we had managers who were extremely good about maintaining work/life balance and the "marathon, not sprint" approach to work. as i said in another comment, it's the best job i've ever had, and i'll miss it deeply.

fluffyfe 2y ago
That is a lot of great work. Sorry to hear that it is gone. Take some time to rest and hope find somewhere with better vision!
deleted 2y ago
[deleted]
iandanforth 2y ago
When was pytype invoked? Runtime? CI/CD? Did it output types to be added to files or did it infer and then enable downstream tasks based on those inferences?
- zem 2y ago
  it was invoked manually at build time, and then at CI/CD time (there was no runtime type checking). we did output types to pyi files (the python equivalent of C header files), but mainly we used the inferred types to check the code for type correctness - one of the unique selling points of pytype, and why google was developing its own typechecker in the first place, was that the other available python type checkers would check annotated code for consistency, but would not do anything about type errors in unannotated code. if pytype did detect type errors, the code failed CI and had to be fixed before it was checked in. we did also have a separate tool that could merge the inferred types back into the source code as annotations, but it never got much uptake.
zem 2y ago
> the rule at google is if you check any code in, you are responsible for every single breakage it causes i can no longer edit my post to clarify this, but by "responsible for breakages" i meant that if your new check-in caused any CI tests anywhere within the codebase to fail, even if due to bugs in the other code, you had to stop and fix it, or get the owners to fix it, or find some principled way to temporarily disable those tests, before you could check your code in. this was a very real issue for things like the python runtime or widely used code checkers like pytype or pylint, because if e.g. the new version of python, or some improved check in pytype, started raising failures in a code pattern that was technically wrong but which the existing toolchain did not complain about, you could not release the new version until you had fixed all those new breakages. contrast this with non-monorepo codebases, where the other code would have had to deal with the fact that "oh, our code works under 3.10 but 3.11 broke it, guess we have to pin our own repo to 3.10 until we fix it", but as the python team we could just say "we support 3.11 now, you need to catch up"