4 ms·

Using multiple repositories seems like the most natural workflow to have. It's easy to make a new one; it's lightweight to do, and it allows your code base to s

Using multiple repositories seems like the most natural workflow to have. It's easy to make a new one; it's lightweight to do, and it allows your code base to scale naturally. You can set permissions for each repository, so if you did include some sensitive code within one repository or another, it's easy to narrow the access to them. (Yes, don't include sensitive code in a repository—but in an early-stage company, you or your engineers may not know that.) Multiple repositories make for a forgiving structure for your code base. You can tailor them however you like. But once you have a _lot_ of code, they become hard to manage. I see the utility of a monolithic repository there—now you know exactly where all your code is: it's in this one repository! Package managers mitigate a lot of the trouble with pulling in internal dependencies from other repositories. Nowadays, most languages have a package manager that can work with a private codebase, so monorepos aren't necessary to help with that. But monorepos can help if you have a ton of versions floating around and you don't want to support version 1.5.x of libjohnny when it's now on 4.3.x. Your code either works with libjohnny as it is right now, or it doesn't. (Which in turn makes it very clear to you how important it is to manage API-breaking changes!) This feels a little bit rambling, but my thought is that there is some analogy between monorepos and microservices; don't use them until you _need_ to use them! You'll know it when you get there.

mnm1 9y ago
With multiple repos, how do you solve the issue that the private dependencies installed by package managers are not themselves under source control when you edit them? They are in /vendor or /node_modules or whatever. It's easy to pull them down and of course, for other people's dependencies, this is fine. But say I pull out one component of my app that's used by multiple apps. I set up my private repo or maybe the package manager works straight with git. But when it pulls the dependencies down, it doesn't pull the git subdirectories and so that code is not under source control. I make edits, test, then I have to copy that code back to its original repo, resolve any conflicts, and check it in. If I need to make a change, I have to do that again. If someone worked on that code, I have to pull that and merge it in separately. Even if I manually pull it myself in the right place, I'm still dealing with a git repo inside a git repo but the two are unrelated so many tools won't work with the inner repo. Short of writing a bunch of custom scripts, is there a standard way to handle this situation that I assume anyone with multiple repos that share internal, private dependencies has?
- peterevans 9y ago
  Any changes you need to make to internal libraries you install via package manager would need to be made to those separate repositories that house the libraries, and then those changes would have a release made (generally this is done through a git tag, and using semantic versioning). Once you update the version, your package manager will allow you to install that update to the other places you're using it, so all you need to change is the package file. It might seem like a pain to do it this way, especially if you're rapidly iterating on a library—it might be that your library is not really mature, or even used by more than one repository, so it may not even make sense to have that library in a separate repository to begin with! But once you do have a mature code base, semantic versioning is a really sane way of managing dependency updates for the N number of other projects which use your library.
  - hinkley 9y ago
    And how do you test if a change in one of these repos fixes the problem you’re seeing? This is what we’re failing at with our multirepo. That and resectioning code to split or combine responsibilities in different ways. Something a monorepo makes trivial.
    - peterevans 9y ago
      Ideally, your library code that you're pulling in has some unit testing to demonstrate that things are working as they should be. (If not, consider adding unit testing! It's really useful!) If that is the case, then you can isolate the likelihood of an issue as either in the library (because unit tests fail there), or in the project consuming the library (because unit tests succeed in the library). Without testing, it's hard to have a ton of confidence in where the problem lies—which is exactly the problem you cite. And while a monorepo (or just a standalone repository with no separate libraries, which is frankly an easier setup to manage than monorepos or multirepos!) may make debugging a bit easier, it's not going to give you much more confidence in your code. Once your organization outgrows the paradigm where you just have N standalone projects with N completely separate code bases, and you do need to commit to either a multi-repo or a mono-repo configuration, it's really, really helpful to have unit testing to allow you to isolate where issues are.
draw_down 9y ago
You only create the repo once, whether that's easy to do is beside the point. Sure it's easy. The problem is deploying related changes together across repos. Every way of doing that sucks.

mnm1 9y ago

With multiple repos, how do you solve the issue that the private dependencies installed by package managers are not themselves under source control when you edit them? They are in /vendor or /node_modules or whatever. It's easy to pull them down and of course, for other people's dependencies, this is fine. But say I pull out one component of my app that's used by multiple apps. I set up my private repo or maybe the package manager works straight with git. But when it pulls the dependencies down, it doesn't pull the git subdirectories and so that code is not under source control. I make edits, test, then I have to copy that code back to its original repo, resolve any conflicts, and check it in. If I need to make a change, I have to do that again. If someone worked on that code, I have to pull that and merge it in separately. Even if I manually pull it myself in the right place, I'm still dealing with a git repo inside a git repo but the two are unrelated so many tools won't work with the inner repo. Short of writing a bunch of custom scripts, is there a standard way to handle this situation that I assume anyone with multiple repos that share internal, private dependencies has?

peterevans 9y ago
Any changes you need to make to internal libraries you install via package manager would need to be made to those separate repositories that house the libraries, and then those changes would have a release made (generally this is done through a git tag, and using semantic versioning). Once you update the version, your package manager will allow you to install that update to the other places you're using it, so all you need to change is the package file. It might seem like a pain to do it this way, especially if you're rapidly iterating on a library—it might be that your library is not really mature, or even used by more than one repository, so it may not even make sense to have that library in a separate repository to begin with! But once you do have a mature code base, semantic versioning is a really sane way of managing dependency updates for the N number of other projects which use your library.
- hinkley 9y ago
  And how do you test if a change in one of these repos fixes the problem you’re seeing? This is what we’re failing at with our multirepo. That and resectioning code to split or combine responsibilities in different ways. Something a monorepo makes trivial.
  - peterevans 9y ago
    Ideally, your library code that you're pulling in has some unit testing to demonstrate that things are working as they should be. (If not, consider adding unit testing! It's really useful!) If that is the case, then you can isolate the likelihood of an issue as either in the library (because unit tests fail there), or in the project consuming the library (because unit tests succeed in the library). Without testing, it's hard to have a ton of confidence in where the problem lies—which is exactly the problem you cite. And while a monorepo (or just a standalone repository with no separate libraries, which is frankly an easier setup to manage than monorepos or multirepos!) may make debugging a bit easier, it's not going to give you much more confidence in your code. Once your organization outgrows the paradigm where you just have N standalone projects with N completely separate code bases, and you do need to commit to either a multi-repo or a mono-repo configuration, it's really, really helpful to have unit testing to allow you to isolate where issues are.

draw_down 9y ago

You only create the repo once, whether that's easy to do is beside the point. Sure it's easy. The problem is deploying related changes together across repos. Every way of doing that sucks.