4 ms·

I'm struggling to understand the problem with shared code and the desire to fragment the code repo! Why can't you have both independently deployed microservice

I'm struggling to understand the problem with shared code and the desire to fragment the code repo! Why can't you have both independently deployed microservices and a shared code base? If the deployment lifecycle is different for each microservices and each deployment is self-contained, then they can be deployed with different versions of the code - even if they use the same source tree and share code. Obviously the shared code needs to be properly maintained and evolved, but it seems to me a lot of the software engineering problems occur when people move away from source code dependencies - with great tooling - versioning, diffs, debuggers - to other types of dependencies ( shared libs etc ) where the tools are non-existent or very simple. Now granted if you needed to fix a critical bug in the shared code - that would require a redeploy of everything, but that happens much less frequently than the need to deploy a single service with immunity as long as your keep your microservice contract. It also means the discipline of making sure every services is deploy-able at anytime is kept to. And if you didn't share code - you probably wouldn't be fixing a single bug once, you'd have much more code, with many more bugs.

gowld 7y ago
> Why can't you have both independently deployed microservices and a shared code base? This is what everyone does, so I can't even comprehend what Segment was doing. Maybe they were deploying a fleet of microservices inside a monolithic deployment? If so, there's no wonder it failed.
- zzbzq 7y ago
  We do separate code repos, my last place did separate repos, place before that did monolith(s) but still did separate repos for anything not in the same monolith. I'm pretty sure it's more common to do separate repos, rather than mono-repo, for separate services. Seems to me, though, the problem is people trying so hard to reuse code. That's the main problem cited in the article. People get really gung-ho about reusing code and creating shared libraries, but reusing code is actually bad most of the time. You should strive to only depend on things that you can reasonably expect to not change, and that you don't need to update even if a new version comes out. What you're supposed to do is take that code in the shared library, and make it a microservice, and obey the usual backward/forward compatibility rules. Using a monolith hides that problem because the code remains easy to update and build, but just as fragile and in need of heavy testing whenever you change code modules that have multiple consumers. That goes against the idea of mono-repos as well.
  - kelnos 7y ago
    > People get really gung-ho about reusing code and creating shared libraries, but reusing code is actually bad most of the time. Disagree here, in general. I'm not in the ruby hyper-DRY camp, but copypasta is not the solution to dependency management problems. Creating shared libraries does require discipline; you should do your best to just avoid breaking changes ever, and on the rare occasion you must, you need heavy communication and testing to ensure consumers find out about the change. And you can only change the API of the library; you can never incompatibly change how the library interacts with other services. I get that this is hard, but it's worthwhile if you can do it right. We have thousands (maybe even tens of thousands) of lines of share library code at this point. Some of it is probably not necessary, but most of it we'd be completely lost without. Reimplementing core logic and utility classes and auth code over and over again is a great way to burn out your developers and create bugs. And these bugs are even worse than your garden-variety bugs, because you have to track them down and fix them over and over, and each fix is slightly different because each reimplementation is slightly different. I agree that sometimes sharing code is a bad idea, but asserting it's bad "most of the time" is completely antithetical to my experience.
  - deleted 7y ago
    [deleted]
  - topher200 7y ago
    Segment's business in particular has them integrating with dozens of unique endpoints. There's an inherent desire for code-reuse in a system like that, along with customization required per endpoint.
  - mrits 7y ago
    "People get really gung-ho about reusing code and creating shared libraries, but reusing code is actually bad most of the time. You should strive to only depend on things that you can reasonably expect to not change" -- things changing is one of the main reasons you want to share code.
  - DrScientist 7y ago
    Ok I see - part of the root of the problem is you are probably using git rather than a version control system that works naturally with large shared repo's like svn? So it's easier to just have separate repo's - and then that makes sharing code sensibly a nightmare without additional tooling... etc etc because there isn't a single versioning system. Everything as a separate git repo has a lot to answer for in my opinion.

gowld 7y ago

> Why can't you have both independently deployed microservices and a shared code base? This is what everyone does, so I can't even comprehend what Segment was doing. Maybe they were deploying a fleet of microservices inside a monolithic deployment? If so, there's no wonder it failed.

zzbzq 7y ago
We do separate code repos, my last place did separate repos, place before that did monolith(s) but still did separate repos for anything not in the same monolith. I'm pretty sure it's more common to do separate repos, rather than mono-repo, for separate services. Seems to me, though, the problem is people trying so hard to reuse code. That's the main problem cited in the article. People get really gung-ho about reusing code and creating shared libraries, but reusing code is actually bad most of the time. You should strive to only depend on things that you can reasonably expect to not change, and that you don't need to update even if a new version comes out. What you're supposed to do is take that code in the shared library, and make it a microservice, and obey the usual backward/forward compatibility rules. Using a monolith hides that problem because the code remains easy to update and build, but just as fragile and in need of heavy testing whenever you change code modules that have multiple consumers. That goes against the idea of mono-repos as well.
- kelnos 7y ago
  > People get really gung-ho about reusing code and creating shared libraries, but reusing code is actually bad most of the time. Disagree here, in general. I'm not in the ruby hyper-DRY camp, but copypasta is not the solution to dependency management problems. Creating shared libraries does require discipline; you should do your best to just avoid breaking changes ever, and on the rare occasion you must, you need heavy communication and testing to ensure consumers find out about the change. And you can only change the API of the library; you can never incompatibly change how the library interacts with other services. I get that this is hard, but it's worthwhile if you can do it right. We have thousands (maybe even tens of thousands) of lines of share library code at this point. Some of it is probably not necessary, but most of it we'd be completely lost without. Reimplementing core logic and utility classes and auth code over and over again is a great way to burn out your developers and create bugs. And these bugs are even worse than your garden-variety bugs, because you have to track them down and fix them over and over, and each fix is slightly different because each reimplementation is slightly different. I agree that sometimes sharing code is a bad idea, but asserting it's bad "most of the time" is completely antithetical to my experience.
- deleted 7y ago
  [deleted]
- topher200 7y ago
  Segment's business in particular has them integrating with dozens of unique endpoints. There's an inherent desire for code-reuse in a system like that, along with customization required per endpoint.
- mrits 7y ago
  "People get really gung-ho about reusing code and creating shared libraries, but reusing code is actually bad most of the time. You should strive to only depend on things that you can reasonably expect to not change" -- things changing is one of the main reasons you want to share code.
- DrScientist 7y ago
  Ok I see - part of the root of the problem is you are probably using git rather than a version control system that works naturally with large shared repo's like svn? So it's easier to just have separate repo's - and then that makes sharing code sensibly a nightmare without additional tooling... etc etc because there isn't a single versioning system. Everything as a separate git repo has a lot to answer for in my opinion.