3 ms·

I seriously wonder if the author ever worked on a project with more than around 10k LOC, multiple languages and tools, automatic testing, deployment, version co

I seriously wonder if the author ever worked on a project with more than around 10k LOC, multiple languages and tools, automatic testing, deployment, version control (merging this build script is going to be a huge PITA). No, you have to realize you're working on one side of the graph (source code), but your build produces something on the other side (your artifact). Without automatic dependency resolution of you DAG, you're constantly traversing from one side to the other, making the build 'work'.

antibjarne 9y ago
Maybe they're not using a language with insanely bloated compile times?
- edejong 9y ago
  It's not just about compile time. Couple of examples: - Parallel downloading / uploading of artifacts - Multi-step transformation (old-style: lexer to C to Object file to linker to deployment). You can turn this around, but generally you want each output of your lexer to be input to the C compiler and so on. - Testing (especially brute-force testing might take time) And in modern situations you have the same, with Typescript => Javascript => Minimisation => Combination => Deployment (roughly). It really helps to think of this as a dependency chain and use a partial-order planner to find a possible linearization and execute.
brudgers 9y ago
There are a lot of projects that don't meet those criteria. Scale is bidirectional. Tools designed for big projects can have features and designs that don't scale down well. Tools designed for big projects often depend on institutional type knowledge where expertise is spread among multiple individuals...at Google, there teams of engineers managing the single code tree is what allows a monorepository. Alternatives are not replacements.
- edejong 9y ago
  Yes, it's a team of engineers. But interestingly, Google started with a mono-repo. All through their explosive growth they maintained (mostly, except Android and Chromium) their vision of mono-repo. It clearly shows that their vision of mono-repo works both for small companies, as well as large ones. A build-script without dependency resolution clearly does not scale. It was the reason to design and implement Make, CMake, Maven, SBT and many others.
  - moosingin3space 9y ago
    To add to this, Google spent a lot of time and effort on building a custom build system (Bazel) to make this sort of dependency tracking in a mono-repo easy. It's pretty great, and I'd rather write BUILD files than Makefiles/shell scripts any day, but the OP appears to have been focusing on extremely simple projects, where shell scripts are okay.
    - brudgers 9y ago
      Right, "works for me" doesn't mean "you must do it.".
  - brudgers 9y ago
    My apologies for not expressing myself as clearly as I would like. Some systems don't need to scale. Make and SBT etc. have an impedance mismatch with some projects due to their learning curve and the current sophistication of the development team. The most important feature of any development tool is not whether it is best practice, it is whether it works. For example, Tarn Adams doesn't use version control for Dwarf Fortress. [1] That doesn't mean Microsoft should have switched the Windows codebase to zip files instead of Git any more than a single developer should start boiling the small lake of SBT just because it is better. Sometimes, it is easier and faster and better to just build the tool that is needed. And of course sometimes it isn't. [1]: https://www.reddit.com/r/IAmA/comments/1avszc/im_tarn_adams_of_bay_12_games_cocreator_of_dwarf/c919fo8/ https://www.reddit.com/r/IAmA/comments/1avszc/im_tarn_adams_...

antibjarne 9y ago

Maybe they're not using a language with insanely bloated compile times?

edejong 9y ago
It's not just about compile time. Couple of examples: - Parallel downloading / uploading of artifacts - Multi-step transformation (old-style: lexer to C to Object file to linker to deployment). You can turn this around, but generally you want each output of your lexer to be input to the C compiler and so on. - Testing (especially brute-force testing might take time) And in modern situations you have the same, with Typescript => Javascript => Minimisation => Combination => Deployment (roughly). It really helps to think of this as a dependency chain and use a partial-order planner to find a possible linearization and execute.

brudgers 9y ago

There are a lot of projects that don't meet those criteria. Scale is bidirectional. Tools designed for big projects can have features and designs that don't scale down well. Tools designed for big projects often depend on institutional type knowledge where expertise is spread among multiple individuals...at Google, there teams of engineers managing the single code tree is what allows a monorepository. Alternatives are not replacements.

edejong 9y ago
Yes, it's a team of engineers. But interestingly, Google started with a mono-repo. All through their explosive growth they maintained (mostly, except Android and Chromium) their vision of mono-repo. It clearly shows that their vision of mono-repo works both for small companies, as well as large ones. A build-script without dependency resolution clearly does not scale. It was the reason to design and implement Make, CMake, Maven, SBT and many others.
- moosingin3space 9y ago
  To add to this, Google spent a lot of time and effort on building a custom build system (Bazel) to make this sort of dependency tracking in a mono-repo easy. It's pretty great, and I'd rather write BUILD files than Makefiles/shell scripts any day, but the OP appears to have been focusing on extremely simple projects, where shell scripts are okay.
  - brudgers 9y ago
    Right, "works for me" doesn't mean "you must do it.".
- brudgers 9y ago
  My apologies for not expressing myself as clearly as I would like. Some systems don't need to scale. Make and SBT etc. have an impedance mismatch with some projects due to their learning curve and the current sophistication of the development team. The most important feature of any development tool is not whether it is best practice, it is whether it works. For example, Tarn Adams doesn't use version control for Dwarf Fortress. [1] That doesn't mean Microsoft should have switched the Windows codebase to zip files instead of Git any more than a single developer should start boiling the small lake of SBT just because it is better. Sometimes, it is easier and faster and better to just build the tool that is needed. And of course sometimes it isn't. [1]: https://www.reddit.com/r/IAmA/comments/1avszc/im_tarn_adams_of_bay_12_games_cocreator_of_dwarf/c919fo8/ https://www.reddit.com/r/IAmA/comments/1avszc/im_tarn_adams_...