12 ms·

I have this question for every proficient DVCS user. Would it scale? When I say scale, I refer to a codebase running into lakhs of sources with a minimum of 20

by raghava 15y ago

I have this question for every proficient DVCS user. Would it scale? When I say scale, I refer to a codebase running into lakhs of sources with a minimum of 20 filetypes including binary types, and at anytime more than 30 active branches, with frequency of merges/recons being one every week, and average volume of change in a source being 150 SLoC. I ask because am still unable to find a definitive answer and I want to know the things I might end up with, if I attempt migrating such a codebase from CVS to a DVCS like git/Mercurial. might be relevant: http://stackoverflow.com/questions/3320001/source-control-system-for-not-so-smart-programmers http://stackoverflow.com/questions/3320001/source-control-sy... Thanks.

patio11 15y ago
Just an FYI: most folks on the Internet don't use lakhs as a unit. The only reason I know what you mean is because I managed Indian outsourcing operations for a while. (FYI for everyone else: lakhs means 100,000 and a crore is 10 million.) I don't have your problems with source control. The Linux project probably has somewhere in the neighborhood of that, and since git was built for Linux, I would not bet on it being architecturally impossible for git to scale that far. The more serious problem I see coming down the pipe is getting your entire team to adopt DVCS workflows. For example, you'll probably end up moving away from having one person whose sole job is making sure merges don't explode into firey balls of death. Distribution of responsibility for making merges happen well could, conceivably, cause problems for a segment of your developers.
mhw 15y ago
I think the problem you need to tackle is not whether your source control system can scale to handle your huge project as a single entity, but whether your whole development process is being hampered because you're looking at it that way. I would recommend you break your system down into a number of separate modules and manage them separately, giving the modules their own release schedules and properly managing the dependencies between them. "The granule of reuse is the granule of release" [1], so if you don't have a formal release process around individual modules, you can't reliably reuse them. If you do that you can create separate repositories for each module, and you'll then find that your branches are more tightly scoped (to a single module), merges are simpler as a result, volume of change is localised to the less stable modules and weird file types can be restricted to specific modules. Once you've refactored your codebase you'll be in a position to make better choices about a source control system. [1] http://www.objectmentor.com/resources/articles/granularity.pdf http://www.objectmentor.com/resources/articles/granularity.p...

by raghava 15y ago

patio11 15y ago

Just an FYI: most folks on the Internet don't use lakhs as a unit. The only reason I know what you mean is because I managed Indian outsourcing operations for a while. (FYI for everyone else: lakhs means 100,000 and a crore is 10 million.) I don't have your problems with source control. The Linux project probably has somewhere in the neighborhood of that, and since git was built for Linux, I would not bet on it being architecturally impossible for git to scale that far. The more serious problem I see coming down the pipe is getting your entire team to adopt DVCS workflows. For example, you'll probably end up moving away from having one person whose sole job is making sure merges don't explode into firey balls of death. Distribution of responsibility for making merges happen well could, conceivably, cause problems for a segment of your developers.

mhw 15y ago

I think the problem you need to tackle is not whether your source control system can scale to handle your huge project as a single entity, but whether your whole development process is being hampered because you're looking at it that way. I would recommend you break your system down into a number of separate modules and manage them separately, giving the modules their own release schedules and properly managing the dependencies between them. "The granule of reuse is the granule of release" [1], so if you don't have a formal release process around individual modules, you can't reliably reuse them. If you do that you can create separate repositories for each module, and you'll then find that your branches are more tightly scoped (to a single module), merges are simpler as a result, volume of change is localised to the less stable modules and weird file types can be restricted to specific modules. Once you've refactored your codebase you'll be in a position to make better choices about a source control system. [1] http://www.objectmentor.com/resources/articles/granularity.pdf http://www.objectmentor.com/resources/articles/granularity.p...