3 ms·
Been at Google for coming up 5 years and I don't think I'm giving away any secrets when I say don't recall there being any performance issues with the mono repo
by cmrdporcupine 10y ago
Been at Google for coming up 5 years and I don't think I'm giving away any secrets when I say don't recall there being any performance issues with the mono repository.
And coupling is what is explicitly being sought, not rejected. The whole point is to build everything off head, keep head sane at all times, and avoid version dependency hell.
- pierrebai 10y agoThen maybe you can answer a few questions: - Do you have zero external dependencies on 3rd party library not own by Google? These must still be managed anyway? How do you deal with project X is not ready to move to external library version N but project Y needs version N? - What about release branches? If you need to intergrate a bug-fix in a sub-system, the magic mono-repo now means merging a fix is harder as it may depend on other unrelated changes all over teh repo? The laissez-faire attitude of not having to care about details in HEAD would seem to bite back in release branches.
- cmrdporcupine 10y agoI am not sure how much of that I can answer in detail, sorry. But yes we have working strategies for dealing with both situations.
- tonfa 10y agohttp://m.cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext?mobile=true http://m.cacm.acm.org/magazines/2016/7/204032-why-google-sto... has a lot of answers (was published last summer).
- z3t4 10y agoI think its insane not to have dependencies and third party code in your scm. Although it seems facebook plan to move those to a package manager due to performance problems and libs touching ten thousand files in a path release.
- agentgt 10y agoYes but the question is where do you stop. Are you going to check in GNU Make and the compilers as well? How about whole operating systems? I also would say "only check in what you change" is a simpler rule than "check in all dependencies needed". I understand company source code needs to be different because of security reasons and convenience but should it really be that much different than any OSS project (which I guarantee would be pissed if you checked in all your dependencies). > I think its insane not to have dependencies and third party code in your scm Consequently there are some that might think the complete opposite (I wouldn't assign insanity but I would say it probably in most cases is not the right thing to do).
- z3t4 10y agoIt all depends. Google might want to keep track of the full stack, whereas your average small software company will quickly adapt when their platform pulls the rug.
- kyrra 10y agoFrom the ACM link someone else provided: > An area of the repository is reserved for storing open source code (developed at Google or externally). To prevent dependency conflicts, as outlined earlier, it is important that only one version of an open source project be available at any given time. Teams that use open source software are expected to occasionally spend time upgrading their codebase to work with newer versions of open source libraries when library upgrades are performed. So if an external library has to be updated, the entire codebase must be migrated all at once. Then Rosie is used to split the change into a lot of smaller changes (to be reviewed by all affected teams). Once all the smaller changes are LGTM'ed, it's submitted all at once.
- DannyBee 10y ago"- Do you have zero external dependencies on 3rd party library not own by Google? These must still be managed anyway? How do you deal with project X is not ready to move to external library version N but project Y needs version N? " I own third party policy, so i can answer this. I'll stick to public info. There are thousands of 3rd party libraries. The rules of the shared codebase is "third party libraries are not free headcount". You choose whether you use them or not (implicitly or explicitly). If you add a third party library, you get to maintain it, and stay within the support horizon of upstream. If you choose to use one, you get to stay up to date with the upgrades others make. If you need features not in the current version, you get to upgrade the library (and work with teams, who are not allowed to block you). This is pretty much the only way to make it all work in practice (i'm aware of how it sounds. In practice, even upgrading stuff literally every google target depensd on takes a week. It's only stuff where folks let it go for 6 years [we now have better detection of out-of-date code] that become a problem to upgrade). If you don't like it, don't use third party code. Note that these rules make it just like any other code, because your problem is not specific to third party code. Note that binary versioning, etc, is pretty much always a complete disaster in practice on a large scale "the magic mono-repo now means merging a fix is harder as it may depend on other unrelated changes all over teh repo?" This is rarely true in practice, because usually fixes are targeted.
- babyrainbow 10y agoMay be I am not understanding the problem correctly, but I wonder why you couldn't use separate repos, and build a script to record the current commit hash of every other repo when you make a commit in any one of the repo. So every commit in every repo has a map that maps the repo path to a commit hash. You can use this info to sync the versions of other repos when you update/checkout a version in any of the repos. And all of these can be scripted.
- jcranmer 10y agoWhy would you resort to a set of hacky scripts that everyone who wants to use the repository must follow when you can just let the VCS itself manage everything for you?
- babyrainbow 10y ago>Why would.. Because then you can keep the repos separated. Not saying that you should. But if this is the only problem that forces you to use a mono repo, then I am just asking if that problem can be solved in this manner? >hacky scripts that everyone who wants to use the repository must follow when you can just let the VCS itself manage everything for you?.. What hacky scripts? Do you think they (google, facebook etc) don't have enough resources to build a 'non hacky' script for what ever they needs done?