3 ms·

How does a sparse checkout not defeat the purpose of a monorepo? I thought monorepos existed so it was easy to make changes that affect the whole codebase and

by krupan 7y ago

How does a sparse checkout not defeat the purpose of a monorepo? I thought monorepos existed so it was easy to make changes that affect the whole codebase and to test those changes. If you only checkout a portion of the files, how are you going to test against the whole repo? EDIT: my overall concern is that it looks like people are reinventing clearcase. Please speak to an older developer who worked at an HP/IBM type company in he late '90s/early 2000's before you do that. Please!

marcell 7y ago
Continuous integration tools still check out and test the whole repo. Google has used this approach for over a decade.
- andrewg 7y ago
  This would be impractical for really large monorepos like the ones Google and Microsoft have. They have virtual file system layers on top (MS open sourced theirs) to prevent checking out the whole repo. In fact, it’s not just useful for the CI/CD pipeline - any developers making significant changes to base libraries or core infrastructure should be able to use the VFS in combination with a system like Bazel to run all (or a significant sample of) affected tests across the company.
qznc 7y ago
They are hard to find. Do you know some? All I have is this thread: https://lobste.rs/s/fosip5/should_version_control_build_systems#c_bkq3ve https://lobste.rs/s/fosip5/should_version_control_build_syst...
- krupan 7y ago
  I used clearcase many years ago, and this thread on lobste.rs is pretty accurate and interesting. They point out that the biggest problems were exclusive checkouts, file versioning instead of changesets, and the baked in out of date assumptions about networking. Getting your configspec wrong was a common problem too. At HP we had some in-house perl-script wrappers around the raw clearcase tools that fixed many of these problems. The developers of those scripts had all left to go work for Rational (makers of clearcase), and I don't think anyone really knew how they worked. We also had a full-time clearcase engineer that kept the servers running. Fortunately our smallish projects didn't need the full power of clearcase and those perl scripts kept working fine for us. I did alway wonder what would happen if the one guy who understood the servers left the company. In short, it's a complex and powerful tool that very few people really understood. Very few projects need all that power and complexity. I'm sure Microsoft and Google benefit from complex version control tools and have engineers to spare for managing and understanding them, but I don't think any open source projects or smaller companies are really going to benefit from "clearcase for the modern age" type tools.
deleted 7y ago
[deleted]

by krupan 7y ago

marcell 7y ago

Continuous integration tools still check out and test the whole repo. Google has used this approach for over a decade.

andrewg 7y ago
This would be impractical for really large monorepos like the ones Google and Microsoft have. They have virtual file system layers on top (MS open sourced theirs) to prevent checking out the whole repo. In fact, it’s not just useful for the CI/CD pipeline - any developers making significant changes to base libraries or core infrastructure should be able to use the VFS in combination with a system like Bazel to run all (or a significant sample of) affected tests across the company.

qznc 7y ago

They are hard to find. Do you know some? All I have is this thread: https://lobste.rs/s/fosip5/should_version_control_build_systems#c_bkq3ve https://lobste.rs/s/fosip5/should_version_control_build_syst...

krupan 7y ago
I used clearcase many years ago, and this thread on lobste.rs is pretty accurate and interesting. They point out that the biggest problems were exclusive checkouts, file versioning instead of changesets, and the baked in out of date assumptions about networking. Getting your configspec wrong was a common problem too. At HP we had some in-house perl-script wrappers around the raw clearcase tools that fixed many of these problems. The developers of those scripts had all left to go work for Rational (makers of clearcase), and I don't think anyone really knew how they worked. We also had a full-time clearcase engineer that kept the servers running. Fortunately our smallish projects didn't need the full power of clearcase and those perl scripts kept working fine for us. I did alway wonder what would happen if the one guy who understood the servers left the company. In short, it's a complex and powerful tool that very few people really understood. Very few projects need all that power and complexity. I'm sure Microsoft and Google benefit from complex version control tools and have engineers to spare for managing and understanding them, but I don't think any open source projects or smaller companies are really going to benefit from "clearcase for the modern age" type tools.

deleted 7y ago

[deleted]