6 ms·
> in others it is flat out wrong. Git was designed for tens of thousands of developers (the Linux kernel), This is easy to check, and the article is right. The
by geofft 9y ago
> in others it is flat out wrong. Git was designed for tens of thousands of developers (the Linux kernel),
This is easy to check, and the article is right. The Linux kernel does not have tens of thousands of active developers. Over a little more than the last year:
titan:~/src/linux geofft$ git log v4.8..v4.14 --format='%aE' | sort -u | wc -l
4857
If you look at just the most recent release cycle:
titan:~/src/linux geofft$ git log v4.13..v4.14 --format='%aE' | sort -u | wc -l
1839
Alternatively, if you look over the last ~year, but at people who have more than five commits (chosen arbitrarily):
titan:~/src/linux geofft$ git log v4.8..v4.14 --format='%aE' | sort | uniq -c | awk '$1 > 5' | wc -l
1814
A codebase with 20,000 actively working on the code base is at least an order of magnitude more than the Linux kernel. If they're committing patches every single day, probably more than that. I have, last I checked, four commits in Linux: two for a summer internship in 2009 and two doc fixes. That certainly contributes to the scale of Linux the kernel in the sense of "there are so many people working on it!", but not in the sense of being an "active developer" of the git repo. I was fine sending in my patches to a mailing list and hearing back at some arbitrarily later point, not triggering any CI, not asking anyone else to collaborate on my work, etc. That doesn't work if you have 20,000 engineers on the same codebase writing code every day.
> Admittedly it is slow for files that are large in size, but it was designed for source code; what sane developer would have source files that are hundreds of MB in size?
Sometimes that's the best way to get things done? The best Debian workflow I've ever worked with (and I've worked with a lot) involved actually committing binary .debs to SVN alongside their source code, because that meant that the SVN revision number was the single source of truth. There wasn't some external artifact system, nor was there a risk of picking up the wrong packages from an apt repo when rebuilding an older SVN revision.
I'm not defending this as pretty. But I will defend this as sane. It got the job done reliably and let me work on actually shipping the product and not yak-shaving workflows.
- hitekker 9y ago+1 for the research and the distinction between sanity and purity.
- opmac 9y agoGood stuff. Came into this thread expecting the top comment to be a know-it-all answer trying to "debunk" the article, as so commonly seen on HN. This is a perfect, well-researched response.
- InclinedPlane 9y agohttps://news.ycombinator.com/item?id=4693920 https://news.ycombinator.com/item?id=4693920
- picklejay 9y agoInteresting read
- pfranz 9y ago> actually committing binary .debs to SVN alongside their source code, because that meant that the SVN revision number was the single source of truth This is totally armchair-devops, but it sounds like a more ideal case would be CI that runs based on your commit. In my experience, developers get lazy and are likely to commit binaries that dont match the code or miss some dependencies or steps. I completely agree with your point, using the tools as intended isn't always the best way to get things done. Video games are an easy example, code is often dependent and intermixed with binary assets and you want those to be versioned together.
- viraptor 9y agoIf you actually commit binaries to help with the CI, then those binaries should be installed by the CI. I'm not saying it's a good idea to do it this way, but if you really want to for some reason, then that's the way to enforce no steps or binaries are missing. (And if devs forget to add something, the build would just fail)
- pfranz 9y agoApologies if I wasn't clear, that's what I was getting at. Commit the code, CI generates the binaries. I guess you could commit the binaries if you wanted, but a separate caching repo would make things more manageable. You'd still be using the commit to reference the binary. But this is ideal, and there are good, practical reasons doing what he describes. For games the binary data isn't from code. They're textures and models generated in a package (like Photoshop). The bits of code are often tightly coupled with the textures--like a procedural waterfall that uses still images. You'd need to version both of those together.
- geofft 9y agoI mean, it's certainly true that at least once I found that another developer had updated the .debs and forgotten to update the sources, and I had to attempt to reconstruct what he did because I needed to add another patch on top. So you're absolutely right that some better CI would have helped here. But for the product we were building, it was important that developers were able to test their .debs before committing to the repo. That means that CI can't build post-commit: you have to have some way to take a local modification, turn it into a .deb, test it and see if it does what you want, and then commit that. I've thrown something together for my current job that supports both building a package interactively for testing and building one in the CI pipeline post-commit (using a git repo as the source in both cases) but it's definitely more cumbersome, and I haven't figured out quite how I want to fix it. (Also, yes, games are a more obvious example of this, but there are lots of purists who've never worked in game development who are somehow convinced that assets aren't code and shouldn't be in the code repo. Having not worked in game development myself either, I figured it was safer not to invite that argument.)
- kelnos 9y agoI think this is all missing the point. Do they want to store, say, the entirety of the base Windows OS in a single git repo? If that's the case, sure, git isn't a great fit for their use case, and they should either use something else, or find a way (as they're doing?) to change git to fit their needs. I can't see MS having tens of thousands of developers working on any single component of Windows, so I'm guessing they _do_ want a giant monorepo. If they were to break it up into separate repos per OS component, I doubt they'd have scaling issues (of course, breaking things up introduces coordination and dependency issues).
- KallDrexx 9y agoI can't see it being easy to take a 20 year massive code base and split it up into components that are so isolated enough they can live in different repositories smoothly. As they say in: https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-git-and-some-back-story/ https://blogs.msdn.microsoft.com/bharry/2017/02/03/scaling-g... The first big debate was – how many repos do you have – one for the whole company at one extreme or one for each small component? A big spectrum. Git is proven to work extremely well for a very large number of modest repos so we spent a bunch of time exploring what it would take to factor our large codebases into lots of tenable repos. Hmm. Ever worked in a huge code base for 20 years? Ever tried to go back afterwards and decompose it into small repos? You can guess what we discovered. The code is very hard to decompose. The cost would be very high. The risk from that level of churn would be enormous. And, we really do have scenarios where a single engineer needs to make sweeping changes across a very large swath of code. Trying to coordinate that across hundreds of repos would be very problematic.
- masklinn 9y ago> I can't see it being easy to take a 20 year massive code base and split it up into components that are so isolated enough they can live in different repositories smoothly. I can't see it being desirable either, they already have to deal with outdated external API, surely they don't want outdated internal API. A single repository lets you replace an API throughout the codebase at once and remove the old one entirely, if you start "modularising" then all "internal API" are suddenly external with all that entails.
- exikyut 9y agoQuestion: what's the `sort -u` for? You're piping to `wc -l`.
- trisimix 9y agoDude teach me how to pipe liKe you