3 ms·

How do you handle building changes to just one of those projects? Can Jenkins do that (easily)? I think that's the big thing that always puts me off monorepo..

How do you handle building changes to just one of those projects? Can Jenkins do that (easily)? I think that's the big thing that always puts me off monorepo... We'd basically be going from ten 5 minute builds to one 50 minute build if it wasn't possible to do incremental builds. IIRC Google and MS have purpose built tools that do impact detection to work out what to build for their monorepos to keep build times down.

kevan 7y ago
If you're doing a monorepo I think it's strongly implied that you'll also use a build system (Blaze/Brazil/BuildXL etc) that has granular compilation units and output caching so build time doesn't scale linearly with the company's total codebase. It's definitely important to consider before jumping in. Going from 5m to 50m compile times would be a major issue for me.
- sagarm 7y ago
  Err what is even the alternative? Even a makefile would provide incremental builds.
  - Too 7y ago
    A good makefile provides incremental builds in every possible scenario of source file changes. It's very easy to write bad makefiles that don't catch modified or removed header files, or even worse when code generation or other complicated build logic is involved. Because a lot of developers seem to think "random build fail? oh just do make clean" is an acceptable workaround.
- joshschreuder 7y ago
  Good to know, thanks. Something I'll have to do a bit more reading up on.
Too 7y ago
If you have to do make clean in a monorepo you are pretty much toast. Tooling for impact detection and reliable makefiles that always succeed incremental builds is absolutely crucial. In a way this is one of the hallmarks of a monorepo - Interfaces and dependencies changing so quickly it becomes too troublesome for humans to categorize (and re-categorize) them into repositories, so you let a machine (makefiles) do the work instead. And even without a monorepo you still have the same problem, eventually you will have to integrate all your mini repos into one final product, which you want to have tested. This is something you want to do as frequently as possible, ideally on every commit, not by doing major version-steps of sub-projects.
DanFeldman 7y ago
It was a bit hacky, but we've basically implemented some of the stuff in [1] to achieve incremental builds. If a pr changes projects a,b,c and not x,y,z then it will only build a,b,c. But it's not truly incremental right now, as it won't test things that depend on A/B/C. We have plans to use Bazel in the future, but you have to boil the ocean when moving to bazel and get everything ever inside bazel before you get any benefit out of it. Jenkins can't do it "easily" but it definitely can. I'd be happy to share our Jenkinsfile if you'd like. Our finding of changes is something like: #!/bin/bash set -euxo pipefail COMPARE_BRANCH=$1 MERGE_BASE=`git merge-base $COMPARE_BRANCH HEAD` FILES_CHANGED=$(git diff --name-only $MERGE_BASE | grep '/') echo ${FILES_CHANGED} | xargs dirname | cut -d "/" -f 1 | sort | uniq [1] blog.shippable.com/ci/cd-of-microservices-using-mono-repos

kevan 7y ago

If you're doing a monorepo I think it's strongly implied that you'll also use a build system (Blaze/Brazil/BuildXL etc) that has granular compilation units and output caching so build time doesn't scale linearly with the company's total codebase. It's definitely important to consider before jumping in. Going from 5m to 50m compile times would be a major issue for me.

sagarm 7y ago
Err what is even the alternative? Even a makefile would provide incremental builds.
- Too 7y ago
  A good makefile provides incremental builds in every possible scenario of source file changes. It's very easy to write bad makefiles that don't catch modified or removed header files, or even worse when code generation or other complicated build logic is involved. Because a lot of developers seem to think "random build fail? oh just do make clean" is an acceptable workaround.
joshschreuder 7y ago
Good to know, thanks. Something I'll have to do a bit more reading up on.

Too 7y ago

If you have to do make clean in a monorepo you are pretty much toast. Tooling for impact detection and reliable makefiles that always succeed incremental builds is absolutely crucial. In a way this is one of the hallmarks of a monorepo - Interfaces and dependencies changing so quickly it becomes too troublesome for humans to categorize (and re-categorize) them into repositories, so you let a machine (makefiles) do the work instead. And even without a monorepo you still have the same problem, eventually you will have to integrate all your mini repos into one final product, which you want to have tested. This is something you want to do as frequently as possible, ideally on every commit, not by doing major version-steps of sub-projects.

DanFeldman 7y ago

It was a bit hacky, but we've basically implemented some of the stuff in [1] to achieve incremental builds. If a pr changes projects a,b,c and not x,y,z then it will only build a,b,c. But it's not truly incremental right now, as it won't test things that depend on A/B/C. We have plans to use Bazel in the future, but you have to boil the ocean when moving to bazel and get everything ever inside bazel before you get any benefit out of it. Jenkins can't do it "easily" but it definitely can. I'd be happy to share our Jenkinsfile if you'd like. Our finding of changes is something like: #!/bin/bash set -euxo pipefail COMPARE_BRANCH=$1 MERGE_BASE=`git merge-base $COMPARE_BRANCH HEAD` FILES_CHANGED=$(git diff --name-only $MERGE_BASE | grep '/') echo ${FILES_CHANGED} | xargs dirname | cut -d "/" -f 1 | sort | uniq [1] blog.shippable.com/ci/cd-of-microservices-using-mono-repos