3 ms·

Do you guys face issues with the increased mental complexities this approach brings? I am a bit shocked to know there are over 1000 repos in the account. I ca

by covi 12y ago

Do you guys face issues with the increased mental complexities this approach brings? I am a bit shocked to know there are over 1000 repos in the account. I can assume developers typically need to keep at most tens in their working set, but 1k does sound a lot to manage.

pkrein 12y ago
Big monolithic repos actually increase the mental complexity more in our opinion. Mental complexity really comes with how much of the system you're holding in your head at once. And with smaller repos you might be dealing with 10-20 repos on any given day, but that's only 1-2% of our codebase/system. The other 98% is ignorable. So we end up holding less in our heads, assuming we've abstracted things correctly. It did get annoying to deal with the mechanics of lots of repos. So we built tooling to make that easier. For example, CLI commands like "goto analytics.js" will clone and take us to the local copy of the repo. And "publish patch" handles all the mechanics of updating History.md from the git log, incrementing the version appropriately in package.json and component.json, tagging the commit and releasing to github and npm. Khaos, also mentioned in Sperandio's article, helps us template out new repos quickly. With a few pieces of tooling like that you can move pretty fast across lots of repos.
ianstormtaylor 12y ago
There are definitely a few drawbacks. For one GitHub is actually designed around a monolithic repository mindset, may be because it's all built on Rails :) which means that it gets harder to find things. So we needed to build directories for finding stuff, since no good option exists. There's also an interesting balance for things that technically should be their own repository, but shouldn't for workflow reasons since it actually means they will be more quality if they are grouped and updated at the same time. Where splitting things out can actually discourage updating if the cost is too high. There's also a big open question around what should be a separate repository, and what is separation just for separation's sake? A similar kind of question that has to be answered for individual functions in a codebase. We've made the wrong call sometimes and split things out that should have really been together. I tend to think that the drawbacks are worth it, since it means things are actually decoupled and generally more parts of the codebase are able to get to a "finished" state. Lots of the modules we've built haven't been touched in months or years now since they do exactly what they need to do without any bugs now. In general I think that in programming we talk a lot about building things that are decoupled and simple, but we haven't invested anywhere near enough in tooling to quell the drawbacks to things being more spread out. The pros are extremely valuable, and the cons are solvable, we just haven't done anything about it.
sperand_io 12y ago
Ian and Peter really nailed it with their comments. For me, the mental complexity of trying to hold a huge API in my head all at once greatly outweighs the cost associated with taking some time to find a well maintained, tested repo that does the exact thing I need really well :) Another great example came to me today: syntax highlighting. The readme on our solution [here](https://github.com/segmentio/highlight https://github.com/segmentio/highlight) hints at why the decision to break things apart made sense for us. As for both mental complexity and ease of use, I think this is a lot simpler than something more traditional like, for example, highlightjs. Rather than go to highlightjs's website to download a custom build of the script that doesn't have 22 languages (20 of which you probably don't need), adding it to your asset pipeline, then repeating whenever you need to add a new one, with highlight's plugin approach + a simple package manager and build process like duo, you can easily add and remove each language from your build ad hoc.

by covi 12y ago

pkrein 12y ago

Big monolithic repos actually increase the mental complexity more in our opinion. Mental complexity really comes with how much of the system you're holding in your head at once. And with smaller repos you might be dealing with 10-20 repos on any given day, but that's only 1-2% of our codebase/system. The other 98% is ignorable. So we end up holding less in our heads, assuming we've abstracted things correctly. It did get annoying to deal with the mechanics of lots of repos. So we built tooling to make that easier. For example, CLI commands like "goto analytics.js" will clone and take us to the local copy of the repo. And "publish patch" handles all the mechanics of updating History.md from the git log, incrementing the version appropriately in package.json and component.json, tagging the commit and releasing to github and npm. Khaos, also mentioned in Sperandio's article, helps us template out new repos quickly. With a few pieces of tooling like that you can move pretty fast across lots of repos.

ianstormtaylor 12y ago

There are definitely a few drawbacks. For one GitHub is actually designed around a monolithic repository mindset, may be because it's all built on Rails :) which means that it gets harder to find things. So we needed to build directories for finding stuff, since no good option exists. There's also an interesting balance for things that technically should be their own repository, but shouldn't for workflow reasons since it actually means they will be more quality if they are grouped and updated at the same time. Where splitting things out can actually discourage updating if the cost is too high. There's also a big open question around what should be a separate repository, and what is separation just for separation's sake? A similar kind of question that has to be answered for individual functions in a codebase. We've made the wrong call sometimes and split things out that should have really been together. I tend to think that the drawbacks are worth it, since it means things are actually decoupled and generally more parts of the codebase are able to get to a "finished" state. Lots of the modules we've built haven't been touched in months or years now since they do exactly what they need to do without any bugs now. In general I think that in programming we talk a lot about building things that are decoupled and simple, but we haven't invested anywhere near enough in tooling to quell the drawbacks to things being more spread out. The pros are extremely valuable, and the cons are solvable, we just haven't done anything about it.

sperand_io 12y ago

Ian and Peter really nailed it with their comments. For me, the mental complexity of trying to hold a huge API in my head all at once greatly outweighs the cost associated with taking some time to find a well maintained, tested repo that does the exact thing I need really well :) Another great example came to me today: syntax highlighting. The readme on our solution [here](https://github.com/segmentio/highlight https://github.com/segmentio/highlight) hints at why the decision to break things apart made sense for us. As for both mental complexity and ease of use, I think this is a lot simpler than something more traditional like, for example, highlightjs. Rather than go to highlightjs's website to download a custom build of the script that doesn't have 22 languages (20 of which you probably don't need), adding it to your asset pipeline, then repeating whenever you need to add a new one, with highlight's plugin approach + a simple package manager and build process like duo, you can easily add and remove each language from your build ad hoc.