5 ms·

The basic problem with trying to solve the problems with versioned releases with monorepos is that the code in the repo and the code in production are not the s

The basic problem with trying to solve the problems with versioned releases with monorepos is that the code in the repo and the code in production are not the same. If you change your service’s API, just updating all of the other clients of your service in the repo is not sufficient to skirt the problem of different clients expecting different APIs. Re the tooling issue, I’ve frankly never seen it solved. I presume Google’s done it, because they’re a bit messianic about the whole thing, but outside of them, I suspect all monorepo implementations fit into the “Communism” model of working great in theory.

throwaways885 5y ago
> If you change your service’s API, just updating all of the other clients of your service in the repo is not sufficient to skirt the problem of different clients expecting different APIs. Most Google APIs are written in protobuf, which is backwards compatible over the wire. This makes most changes painless, although caveat emptor as things can still get messy as they do in the real world. With something like a Java interface, it doesn't really matter if those APIs change because all the code is built and released together at HEAD.
- majormajor 5y ago
  Backwards compatible from a parsing-this-message-won't-blow-up standpoint but not necessarily from a semantic meaning-of-each-field standpoint if your devs are ... clever ... enough. ;) That discipline to commit to an API and meaning for each field and not keep tinkering with it in messy ways is an under-valued and under-evangelized aspect.
- roughly 5y ago
  > With something like a Java interface, it doesn't really matter if those APIs change because all the code is built and released together at HEAD. Right, this is the thing that winds up following from the monorepo - the monorepo problem goes away if we just release everything, the whole fuckin’ system, into production every time we make any update to master. If that’s your system, hats off to you, I’m not nearly that good of an engineer.
  - barrkel 5y ago
    It's actually easier, in many ways -- if you can manage to release continuously. It forces you to stage your code incrementally and you should be able to roll back things easily if you see breakage.
  - deleted 5y ago
    [deleted]
  - throwaways885 5y ago
    If your services are backwards compatible to talk to each other, then you don't need to release instantly. Making breaking changes is tricky, but you don't need to release everything.
spicybright 5y ago
I don't understand the problem. Code structure that listens to dev/prod mode can handle the different APIs. Branches to mark prod vs. dev. Can you give a more concrete example of the problem? I don't think I'm understanding.
- roughly 5y ago
  The pitch I often hear with monorepos is “When you change your service/library/whatever, you can simultaneously change all of the clients of that library/service/whatever in one PR”. The problem is a PR is not a release - just because the code in the repo is up to date doesn’t mean the code in production is. So, for example, if your service changes to require an extra parameter in the API, sure, you can update all of the clients of that API to provide that extra parameter, but that still requires them to be built & released as well - just updating the code doesn’t actually fix the deploy/release coordination problem. Short version, you’re right, coordinating releases with feature flags, branches, etc. solves this problem, but it solves the problem in a multi-repo world too, and a monorepo doesn’t obviate the need for those solutions as many of its proponents seem to suggest. There are probably legitimate uses for monorepos, but an awful lot of people seem to position them as a silver bullet that lets developers stop having to worry about coordination in a multi-service world, and that’s just not how it works out.
  - floober 5y ago
    I agree that it doesn't improve things at the service interface level. What I have found is that I often have several internal libraries, and a monorepo ensures that I know if a change to my library breaks another service because they're all in one build. Keeping things consistent across services is very useful in my experience.
- thebean11 5y ago
  The issue comes if you are communicating between different binaries (for example, RPC requests or message handlers). Just because you can change the code atomically doesn't mean you can deploy the client and server atomically. As someone pointed out though, proto mitigates a lot of this.

throwaways885 5y ago

> If you change your service’s API, just updating all of the other clients of your service in the repo is not sufficient to skirt the problem of different clients expecting different APIs. Most Google APIs are written in protobuf, which is backwards compatible over the wire. This makes most changes painless, although caveat emptor as things can still get messy as they do in the real world. With something like a Java interface, it doesn't really matter if those APIs change because all the code is built and released together at HEAD.

majormajor 5y ago
Backwards compatible from a parsing-this-message-won't-blow-up standpoint but not necessarily from a semantic meaning-of-each-field standpoint if your devs are ... clever ... enough. ;) That discipline to commit to an API and meaning for each field and not keep tinkering with it in messy ways is an under-valued and under-evangelized aspect.
roughly 5y ago
> With something like a Java interface, it doesn't really matter if those APIs change because all the code is built and released together at HEAD. Right, this is the thing that winds up following from the monorepo - the monorepo problem goes away if we just release everything, the whole fuckin’ system, into production every time we make any update to master. If that’s your system, hats off to you, I’m not nearly that good of an engineer.
- barrkel 5y ago
  It's actually easier, in many ways -- if you can manage to release continuously. It forces you to stage your code incrementally and you should be able to roll back things easily if you see breakage.
- deleted 5y ago
  [deleted]
- throwaways885 5y ago
  If your services are backwards compatible to talk to each other, then you don't need to release instantly. Making breaking changes is tricky, but you don't need to release everything.

spicybright 5y ago

I don't understand the problem. Code structure that listens to dev/prod mode can handle the different APIs. Branches to mark prod vs. dev. Can you give a more concrete example of the problem? I don't think I'm understanding.

roughly 5y ago
The pitch I often hear with monorepos is “When you change your service/library/whatever, you can simultaneously change all of the clients of that library/service/whatever in one PR”. The problem is a PR is not a release - just because the code in the repo is up to date doesn’t mean the code in production is. So, for example, if your service changes to require an extra parameter in the API, sure, you can update all of the clients of that API to provide that extra parameter, but that still requires them to be built & released as well - just updating the code doesn’t actually fix the deploy/release coordination problem. Short version, you’re right, coordinating releases with feature flags, branches, etc. solves this problem, but it solves the problem in a multi-repo world too, and a monorepo doesn’t obviate the need for those solutions as many of its proponents seem to suggest. There are probably legitimate uses for monorepos, but an awful lot of people seem to position them as a silver bullet that lets developers stop having to worry about coordination in a multi-service world, and that’s just not how it works out.
- floober 5y ago
  I agree that it doesn't improve things at the service interface level. What I have found is that I often have several internal libraries, and a monorepo ensures that I know if a change to my library breaks another service because they're all in one build. Keeping things consistent across services is very useful in my experience.
thebean11 5y ago
The issue comes if you are communicating between different binaries (for example, RPC requests or message handlers). Just because you can change the code atomically doesn't mean you can deploy the client and server atomically. As someone pointed out though, proto mitigates a lot of this.