5 ms·

Where is this extra code coming from? Do they do an immense amount of code generation?

by cplusplusfellow 4y ago

lupire 4y ago
300K person-years....
- dchftcs 4y ago
  That would still be 20-30k additional LOC per person year. Which is difficult to achieve especially in a corporate setting with a mature code base, without code generation, lots of blank lines, or including static data. Even in startup environments it's hard to get 20k-30k a year for several years from the average unless many of the engineers don't overlap.
  - mrwh 4y ago
    I don't know about that. 100 lines a day, 280-or-so days a year, is pretty doable for the average L4. You could do that in migrations alone, and people do.
    - dchftcs 4y ago
      L5 or above there's normally a lot more communication and planning, which pulls down the mean significantly. In migrations you typically need to subtract some old code in the end, and it's not uncommon to result in net negative LOC (if the migration is ever complete). For L4 maybe there are more days where one can do several hundred LOC in an afternoon than L5, but there are also many days that result in 100 or less over an entire week, because there's testing, debugging, profiling and figuring out what to code. But, maybe I'd be wrong about Google. Afterall, it might be easier to dump lots of code into the codebase if there are 10 independent chat apps in development at any one time.
    - xen0 4y ago
      Migrations tends to involve deleting lines too. A 200 line change typically doesn't grow the code base by 200 lines. I believe in my 2.5 years at Google, I've actually deleted more lines of code than I've added.
    - deleted 4y ago
      [deleted]
    - ahtihn 4y ago
      > 280-or-so days a year So working some week-ends and no vacations at all? There are about 260 week days in a year. Add bank holidays and vacations and you should ve closer to 230 working days in a year.
  - sudosysgen 4y ago
    Probably half of that is config or autogen, perhaps even more, so it seems fairly reasonable?
    - dchftcs 4y ago
      Yes but the point of contention was whether this can be done without code generation and static data. And probably both you and I agree it's quite difficult for a large organization to get 20k-30k net LOC additions per engineer without "conifg or autogen".
gravypod 4y ago
(Opinions are my own) > Do they do an immense amount of code generation? Blaze (aka Bazel [0]) has provisions that make it easy to generate code but this happens as a compile step rather than something that is checked into a git repo. [0] - https://bazel.build/ https://bazel.build/
kyrra 4y ago
Googler, opinions are my own. I believe the YouTube video linked from the wired article goes into this, but there's more machine generated code checked in than human generated code at this point. A lot of this is boilerplate or config data that is stored in source control. I feel like Google definitely believes that config is code, so we have a ton of configuration of our systems checked into Piper.
- brundolf 4y ago
  Why is the generated code checked in, vs generated as-needed later?
  - onei 4y ago
    Pure speculation on my part, but I've noticed Go has generate and build as separate steps and it's standard to check in generated code. This split no doubt makes compilation a lot simpler/faster, but perhaps it's a reflection of standard practices the language authors were used to.
  - azurezyq 4y ago
    Most of them are generated on the fly, like protos. But you know, there are tons of configs, all kinds of.
  - danpalmer 4y ago
    Traceability, simplicity, all sorts of reasons. If you commit all your codegen’d stuff you don’t need to worry about how to run codegen in most places. You can enforce a lot of security properties, you can run large scale analysis on the code in production because it’s all “statically” available, etc. That said, I think Google’s source control ecosystem is really quite different to what’s publicly available with systems like Git/Hub/Lab etc, even to Phabricator. These approaches work well at Google but don’t necessarily translate to other tools in the same way. Don’t take these sorts of things as gospel, or even as google advice about good engineering. Understand what they achieve, and then figure out a way to achieve that same result with the tools available to you.
  - exikyut 4y ago
deleted 4y ago
[deleted]
oh_sigh 4y ago
Sorry, I never checked the replies to this comment, but I just want to point out that I was just joking with my 10B statement and have no clue as to how much code is actually in google's repos. it may be 1T for all I know.

Where is this extra code coming from? Do they do an immense amount of code generation?

lupire 4y ago

300K person-years....

dchftcs 4y ago
That would still be 20-30k additional LOC per person year. Which is difficult to achieve especially in a corporate setting with a mature code base, without code generation, lots of blank lines, or including static data. Even in startup environments it's hard to get 20k-30k a year for several years from the average unless many of the engineers don't overlap.
- mrwh 4y ago
  I don't know about that. 100 lines a day, 280-or-so days a year, is pretty doable for the average L4. You could do that in migrations alone, and people do.
  - dchftcs 4y ago
    L5 or above there's normally a lot more communication and planning, which pulls down the mean significantly. In migrations you typically need to subtract some old code in the end, and it's not uncommon to result in net negative LOC (if the migration is ever complete). For L4 maybe there are more days where one can do several hundred LOC in an afternoon than L5, but there are also many days that result in 100 or less over an entire week, because there's testing, debugging, profiling and figuring out what to code. But, maybe I'd be wrong about Google. Afterall, it might be easier to dump lots of code into the codebase if there are 10 independent chat apps in development at any one time.
  - xen0 4y ago
    Migrations tends to involve deleting lines too. A 200 line change typically doesn't grow the code base by 200 lines. I believe in my 2.5 years at Google, I've actually deleted more lines of code than I've added.
  - deleted 4y ago
    [deleted]
  - ahtihn 4y ago
    > 280-or-so days a year So working some week-ends and no vacations at all? There are about 260 week days in a year. Add bank holidays and vacations and you should ve closer to 230 working days in a year.
- sudosysgen 4y ago
  Probably half of that is config or autogen, perhaps even more, so it seems fairly reasonable?
  - dchftcs 4y ago
    Yes but the point of contention was whether this can be done without code generation and static data. And probably both you and I agree it's quite difficult for a large organization to get 20k-30k net LOC additions per engineer without "conifg or autogen".

gravypod 4y ago

(Opinions are my own) > Do they do an immense amount of code generation? Blaze (aka Bazel [0]) has provisions that make it easy to generate code but this happens as a compile step rather than something that is checked into a git repo. [0] - https://bazel.build/ https://bazel.build/

kyrra 4y ago

Googler, opinions are my own. I believe the YouTube video linked from the wired article goes into this, but there's more machine generated code checked in than human generated code at this point. A lot of this is boilerplate or config data that is stored in source control. I feel like Google definitely believes that config is code, so we have a ton of configuration of our systems checked into Piper.

brundolf 4y ago
Why is the generated code checked in, vs generated as-needed later?
- onei 4y ago
  Pure speculation on my part, but I've noticed Go has generate and build as separate steps and it's standard to check in generated code. This split no doubt makes compilation a lot simpler/faster, but perhaps it's a reflection of standard practices the language authors were used to.
- azurezyq 4y ago
  Most of them are generated on the fly, like protos. But you know, there are tons of configs, all kinds of.
- danpalmer 4y ago
  Traceability, simplicity, all sorts of reasons. If you commit all your codegen’d stuff you don’t need to worry about how to run codegen in most places. You can enforce a lot of security properties, you can run large scale analysis on the code in production because it’s all “statically” available, etc. That said, I think Google’s source control ecosystem is really quite different to what’s publicly available with systems like Git/Hub/Lab etc, even to Phabricator. These approaches work well at Google but don’t necessarily translate to other tools in the same way. Don’t take these sorts of things as gospel, or even as google advice about good engineering. Understand what they achieve, and then figure out a way to achieve that same result with the tools available to you.
- exikyut 4y ago

deleted 4y ago

[deleted]

oh_sigh 4y ago

Sorry, I never checked the replies to this comment, but I just want to point out that I was just joking with my 10B statement and have no clue as to how much code is actually in google's repos. it may be 1T for all I know.