6 ms·

It’s a Go app, not a Ruby one. Hence the Go code examples. And the rationale behind one binary rather than several is to reduce build time and deployment artef

It’s a Go app, not a Ruby one. Hence the Go code examples. And the rationale behind one binary rather than several is to reduce build time and deployment artefact size, along with benefits in development from one binary. All of which mean deploying is much quicker/easier, which means it’s less of a big deal that we deploy everything whenever any part changes (we deploy anywhere up to 30 times a day). Hope that helps it make more sense.

jameshart 3y ago
Apologies - read the ‘Ruby monolith’ up top and then glossed over the code without noticing the language mismatch. Mondays, am I right? I don’t understand how bundling three sets of functionality into one binary reduces build time or artifact size. Surely the binary containing all the web and pubsub and scheduled tasks is strictly bigger than the binary for just the web would be, and takes longer to build and test that the binary for just the web?
- tough 3y ago
  Maybe the three binaries have shared deps, by bundling them on one you get to not duplicate those shared deps? I dunno as I have never used Go, but this would make sense with node Also thinking how all the related tasks a la CI/CD might be total time faster with one process than three, since they're doing less total work, both during tasks and to wake_up the process etc dunno just speculation on my part
- lawrjone 3y ago
  No problem, easy to skim by it! In terms of how this reduces artifact size: Go statically compiles everything, and the majority of that weight is from dependencies. Compiling into three different binaries would: 1. Increase build time, because even while the dependencies are cached from each subsequent build, linking isn't free. Takes about 20s to link each binary, so it adds about 40s+ to the build to create them twice. 2. Increases build size, because each binary is about 150MB, so we go from 150 -> 450MB, which increases time to upload to container registry and download into the infrastructure that runs things. In answer to the security surface, the code between cron/worker/web is very interlinked. There would be little practical reduction in surface area from splitting the binary, though perhaps for some applications there might.
  - jameshart 3y ago
    You only have to rebuild the artifacts that are affected by each change. And each change (assuming CI/CD) only causes the changes build artifacts to be pushed to prod. So the amortized build/push cost depends on how often you are making changes that affect multiple artifacts versus fewer artifacts. If an individual change only affects either the web binary or one of the cron job binaries then those build times will be shorter than they would be to build the big binary, and the amount of data pushed to prod will be smaller. If an individual change affects multiple binaries then building all of them will take longer, and more data will need to be pushed (although - maybe it nets out the same in terms of fa out to multiple servers - your cron servers only need the cron binary, your web servers only need the web binary) So, if the vast majority of the time you are only impacting a single component build, you save build time and deploy bytes by building smaller artifacts. At the cost that making cross-system changes requires a slightly longer build time.
electroly 3y ago
> reduce build time and deployment artefact size I'm confused... doesn't this plan _maximize_ your build time and deployment size? You can't do worse than building and deploying every line of code you own on every build and deploy; it's the worst case scenario. Go is fast enough in compilation that it doesn't matter, right? But then what's the point?

jameshart 3y ago

Apologies - read the ‘Ruby monolith’ up top and then glossed over the code without noticing the language mismatch. Mondays, am I right? I don’t understand how bundling three sets of functionality into one binary reduces build time or artifact size. Surely the binary containing all the web and pubsub and scheduled tasks is strictly bigger than the binary for just the web would be, and takes longer to build and test that the binary for just the web?

tough 3y ago
Maybe the three binaries have shared deps, by bundling them on one you get to not duplicate those shared deps? I dunno as I have never used Go, but this would make sense with node Also thinking how all the related tasks a la CI/CD might be total time faster with one process than three, since they're doing less total work, both during tasks and to wake_up the process etc dunno just speculation on my part
lawrjone 3y ago
No problem, easy to skim by it! In terms of how this reduces artifact size: Go statically compiles everything, and the majority of that weight is from dependencies. Compiling into three different binaries would: 1. Increase build time, because even while the dependencies are cached from each subsequent build, linking isn't free. Takes about 20s to link each binary, so it adds about 40s+ to the build to create them twice. 2. Increases build size, because each binary is about 150MB, so we go from 150 -> 450MB, which increases time to upload to container registry and download into the infrastructure that runs things. In answer to the security surface, the code between cron/worker/web is very interlinked. There would be little practical reduction in surface area from splitting the binary, though perhaps for some applications there might.
- jameshart 3y ago
  You only have to rebuild the artifacts that are affected by each change. And each change (assuming CI/CD) only causes the changes build artifacts to be pushed to prod. So the amortized build/push cost depends on how often you are making changes that affect multiple artifacts versus fewer artifacts. If an individual change only affects either the web binary or one of the cron job binaries then those build times will be shorter than they would be to build the big binary, and the amount of data pushed to prod will be smaller. If an individual change affects multiple binaries then building all of them will take longer, and more data will need to be pushed (although - maybe it nets out the same in terms of fa out to multiple servers - your cron servers only need the cron binary, your web servers only need the web binary) So, if the vast majority of the time you are only impacting a single component build, you save build time and deploy bytes by building smaller artifacts. At the cost that making cross-system changes requires a slightly longer build time.

electroly 3y ago

> reduce build time and deployment artefact size I'm confused... doesn't this plan _maximize_ your build time and deployment size? You can't do worse than building and deploying every line of code you own on every build and deploy; it's the worst case scenario. Go is fast enough in compilation that it doesn't matter, right? But then what's the point?