4 ms·

30 minute docker builds? Crazy. I know it is out of style for some, but my microservice architecture, which has a dozen services, each takes about 1:30m to b

30 minute docker builds? Crazy. I know it is out of style for some, but my microservice architecture, which has a dozen services, each takes about 1:30m to build, maybe 2m at most (if there is a slow Next.js build in there and a few thousand npm packages), and that is just on a 4 core GitHub Actions worker. My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more services. (Open source template which shows how it works: https://github.com/bhouston/template-typescript-monorepo/actions/runs/10362188928 https://github.com/bhouston/template-typescript-monorepo/act... )

adityamaru 2y ago
> 30 minute docker builds? At Blacksmith we do see this pretty often! Rust services in particular are the most common offender.
- bhouston 2y ago
  Yikes. I would be so much less productive with a 30 minute build time.
- jeffparsons 2y ago
  I'm working on an ungodly pile of hacks (https://github.com/jeffparsons/hope https://github.com/jeffparsons/hope) to help with this. Coming Soon™: S3 backend and better tests.
  - adityamaru 2y ago
    `hope` is a good name for a service trying to solve this problem :D
    - jeffparsons 2y ago
      Haha, thanks. I chose it because: - Here's One (I? You?) Prepared Earlier - Sometimes Hope _is_ a (caching) strategy - And yeah, I really hope I can make this thing work well. I like silly puns. They bring me joy.
throw1948012309 2y ago
> My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more services. If you're deploying all your "microservices" in parallel, then what you might have built is a distributed monolith. A microservice can be tested and deployed independently.
- bhouston 2y ago
  Whether it is a distributed monolith or a set of microservices is independent of the speed of build.
- nine_k 2y ago
  I don't see a contradiction. I read it that the microservices are independent and thus can build in parallel, if several teams work on changes to several microservices. Spinning a build worker outright when a change us pushed is the fastest way, and may be expensive if the build process is prolonged. OTOH I've seen much faster image build times, with smart reuse of layers, so that you don't have to re-run that huge npm install if your packages.lock did not change.
jjayj 2y ago
I regularly build images where we install Python from source, which makes 30m seem quite normal...
- bhouston 2y ago
  Why?
  - xyst 2y ago
    Maybe to secure the supply chain
    - smallerize 2y ago
      If they can't trust their own build cache, they really have problems.
  - jjayj 2y ago
    We do development/builds in containers to make things easier on the devs. These are the Python build containers, so we have to rebuild every time there's a new version of Python or one of its dependencies.

adityamaru 2y ago

> 30 minute docker builds? At Blacksmith we do see this pretty often! Rust services in particular are the most common offender.

bhouston 2y ago
Yikes. I would be so much less productive with a 30 minute build time.
jeffparsons 2y ago
I'm working on an ungodly pile of hacks (https://github.com/jeffparsons/hope https://github.com/jeffparsons/hope) to help with this. Coming Soon™: S3 backend and better tests.
- adityamaru 2y ago
  `hope` is a good name for a service trying to solve this problem :D
  - jeffparsons 2y ago
    Haha, thanks. I chose it because: - Here's One (I? You?) Prepared Earlier - Sometimes Hope _is_ a (caching) strategy - And yeah, I really hope I can make this thing work well. I like silly puns. They bring me joy.

throw1948012309 2y ago

> My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more services. If you're deploying all your "microservices" in parallel, then what you might have built is a distributed monolith. A microservice can be tested and deployed independently.

bhouston 2y ago
Whether it is a distributed monolith or a set of microservices is independent of the speed of build.
nine_k 2y ago
I don't see a contradiction. I read it that the microservices are independent and thus can build in parallel, if several teams work on changes to several microservices. Spinning a build worker outright when a change us pushed is the fastest way, and may be expensive if the build process is prolonged. OTOH I've seen much faster image build times, with smart reuse of layers, so that you don't have to re-run that huge npm install if your packages.lock did not change.

jjayj 2y ago

I regularly build images where we install Python from source, which makes 30m seem quite normal...

bhouston 2y ago
Why?
- xyst 2y ago
  Maybe to secure the supply chain
  - smallerize 2y ago
    If they can't trust their own build cache, they really have problems.
- jjayj 2y ago
  We do development/builds in containers to make things easier on the devs. These are the Python build containers, so we have to rebuild every time there's a new version of Python or one of its dependencies.