3 ms·

Apologies - read the ‘Ruby monolith’ up top and then glossed over the code without noticing the language mismatch. Mondays, am I right? I don’t understand how

Apologies - read the ‘Ruby monolith’ up top and then glossed over the code without noticing the language mismatch. Mondays, am I right? I don’t understand how bundling three sets of functionality into one binary reduces build time or artifact size. Surely the binary containing all the web and pubsub and scheduled tasks is strictly bigger than the binary for just the web would be, and takes longer to build and test that the binary for just the web?

tough 4y ago
Maybe the three binaries have shared deps, by bundling them on one you get to not duplicate those shared deps? I dunno as I have never used Go, but this would make sense with node Also thinking how all the related tasks a la CI/CD might be total time faster with one process than three, since they're doing less total work, both during tasks and to wake_up the process etc dunno just speculation on my part
lawrjone 4y ago
No problem, easy to skim by it! In terms of how this reduces artifact size: Go statically compiles everything, and the majority of that weight is from dependencies. Compiling into three different binaries would: 1. Increase build time, because even while the dependencies are cached from each subsequent build, linking isn't free. Takes about 20s to link each binary, so it adds about 40s+ to the build to create them twice. 2. Increases build size, because each binary is about 150MB, so we go from 150 -> 450MB, which increases time to upload to container registry and download into the infrastructure that runs things. In answer to the security surface, the code between cron/worker/web is very interlinked. There would be little practical reduction in surface area from splitting the binary, though perhaps for some applications there might.
- jameshart 4y ago
  You only have to rebuild the artifacts that are affected by each change. And each change (assuming CI/CD) only causes the changes build artifacts to be pushed to prod. So the amortized build/push cost depends on how often you are making changes that affect multiple artifacts versus fewer artifacts. If an individual change only affects either the web binary or one of the cron job binaries then those build times will be shorter than they would be to build the big binary, and the amount of data pushed to prod will be smaller. If an individual change affects multiple binaries then building all of them will take longer, and more data will need to be pushed (although - maybe it nets out the same in terms of fa out to multiple servers - your cron servers only need the cron binary, your web servers only need the web binary) So, if the vast majority of the time you are only impacting a single component build, you save build time and deploy bytes by building smaller artifacts. At the cost that making cross-system changes requires a slightly longer build time.

tough 4y ago

Maybe the three binaries have shared deps, by bundling them on one you get to not duplicate those shared deps? I dunno as I have never used Go, but this would make sense with node Also thinking how all the related tasks a la CI/CD might be total time faster with one process than three, since they're doing less total work, both during tasks and to wake_up the process etc dunno just speculation on my part

lawrjone 4y ago

No problem, easy to skim by it! In terms of how this reduces artifact size: Go statically compiles everything, and the majority of that weight is from dependencies. Compiling into three different binaries would: 1. Increase build time, because even while the dependencies are cached from each subsequent build, linking isn't free. Takes about 20s to link each binary, so it adds about 40s+ to the build to create them twice. 2. Increases build size, because each binary is about 150MB, so we go from 150 -> 450MB, which increases time to upload to container registry and download into the infrastructure that runs things. In answer to the security surface, the code between cron/worker/web is very interlinked. There would be little practical reduction in surface area from splitting the binary, though perhaps for some applications there might.

jameshart 4y ago
You only have to rebuild the artifacts that are affected by each change. And each change (assuming CI/CD) only causes the changes build artifacts to be pushed to prod. So the amortized build/push cost depends on how often you are making changes that affect multiple artifacts versus fewer artifacts. If an individual change only affects either the web binary or one of the cron job binaries then those build times will be shorter than they would be to build the big binary, and the amount of data pushed to prod will be smaller. If an individual change affects multiple binaries then building all of them will take longer, and more data will need to be pushed (although - maybe it nets out the same in terms of fa out to multiple servers - your cron servers only need the cron binary, your web servers only need the web binary) So, if the vast majority of the time you are only impacting a single component build, you save build time and deploy bytes by building smaller artifacts. At the cost that making cross-system changes requires a slightly longer build time.