4 ms·

How do people do CI with monolithic repositories? Do they run ALL the tests or is there an easy way around that?

justinmk 11y ago
Your build agents should use a cached .git, and only pull updates. Different test groups can be a different build target in your build/make file, or you can have nested build files.
brandonbloom 11y ago
With SVN (or similary) you'd simply configure your CI to 1) checkout 1 or more files and subdirectories and 2) config your CI to run a particular build script. The notion of zero-config CI is a recent invention, thanks to conventions of small repos with well-known project file names per language/platform ecosystem.
QuercusMax 11y ago
At Google, the dependencies for everything are very well specified, so in most cases the code affected by a single change is quite small. Dependencies are often specified on a per file basis. It's kind of a pain in the ass, but being able to know for sure you won't break stuff is very powerful. Also, they have a ridiculous number of machines dedicated to running automated tests. My tech lead told me "feel free to add as many tests as long as they're under 30 seconds each". I was super skeptical of perforce before starting at Google, but it works really well, especially in conjunction with the code review tools and processes
- cpeterso 11y ago
  > Dependencies are often specified on a per file basis. It's kind of a pain in the ass, but being able to know for sure you won't break stuff is very powerful. How are the dependencies specified? Is this a language-specific tool?
  - gefh 11y ago
    Bazel is the open source version of the internal build tool, it is multi-language. BUILD files in each directory specify dependencies and more.
skybrian 11y ago
That's the job of the build system. After syncing to a new version, it should only rebuild and re-run the tests where a dependency changed. (It helps to have fine-grained dependencies to avoid running compiles or tests unnecessarily.) However, that's not enough when you touch a fundamental library and nearly everything is a downstream dependency. Then you really do need to run all the tests, so the build system should be able to run them in parallel on a build farm. Also, test results need to be cached by the build farm so every user doesn't need to run the same tests again for the same change. See Bazel [1] for how Google does it. (Well, it doesn't provide the build farm, but you can see what the build files look like.) [1] http://bazel.io/ http://bazel.io/

Your build agents should use a cached .git, and only pull updates. Different test groups can be a different build target in your build/make file, or you can have nested build files.

brandonbloom 11y ago

With SVN (or similary) you'd simply configure your CI to 1) checkout 1 or more files and subdirectories and 2) config your CI to run a particular build script. The notion of zero-config CI is a recent invention, thanks to conventions of small repos with well-known project file names per language/platform ecosystem.

QuercusMax 11y ago

At Google, the dependencies for everything are very well specified, so in most cases the code affected by a single change is quite small. Dependencies are often specified on a per file basis. It's kind of a pain in the ass, but being able to know for sure you won't break stuff is very powerful. Also, they have a ridiculous number of machines dedicated to running automated tests. My tech lead told me "feel free to add as many tests as long as they're under 30 seconds each". I was super skeptical of perforce before starting at Google, but it works really well, especially in conjunction with the code review tools and processes

cpeterso 11y ago
> Dependencies are often specified on a per file basis. It's kind of a pain in the ass, but being able to know for sure you won't break stuff is very powerful. How are the dependencies specified? Is this a language-specific tool?
- gefh 11y ago
  Bazel is the open source version of the internal build tool, it is multi-language. BUILD files in each directory specify dependencies and more.

skybrian 11y ago

That's the job of the build system. After syncing to a new version, it should only rebuild and re-run the tests where a dependency changed. (It helps to have fine-grained dependencies to avoid running compiles or tests unnecessarily.) However, that's not enough when you touch a fundamental library and nearly everything is a downstream dependency. Then you really do need to run all the tests, so the build system should be able to run them in parallel on a build farm. Also, test results need to be cached by the build farm so every user doesn't need to run the same tests again for the same change. See Bazel [1] for how Google does it. (Well, it doesn't provide the build farm, but you can see what the build files look like.) [1] http://bazel.io/ http://bazel.io/