4 ms·
Hello there, have you heard of service oriented architecture? You must be joking to justify a single repository with "easier to change". Your problem is that t
by leccine 12y ago
Hello there, have you heard of service oriented architecture? You must be joking to justify a single repository with "easier to change". Your problem is that the code base must be tightly coupled if splitting the services out to different repos is not possible and you need to contribute to multiple repositories to get something done. I would say, the biggest change in Amazon's architecture was moving over to the service oriented way and it was worth the effort. Developers are forced to separate different functions to separate services and they are in charge of that service. If it goes down their are getting the alerts. All of the services are using 250ms timeouts so there is no cascading effect when a services goes down. The web page consists of few thousand service calls and it degrades gracefully. Facebook obviously have some tech depth that they need to fix. Using stupid design justified with some random crap that does not even make sense is not really acceptable (at least for me).
- indygreg2 12y agoSOA isn't a magic bullet. What if multiple services are utilizing a shared library? For each service to be independent in the way I think you are advocating for, you would need multiple copies of that shared library (either via separate copies in separate repos or a shared copy via something like subrepos). Multiple copies leads to copies getting out of sync. You (likely) lose the ability to perform a single atomic commit. Furthermore, you've increased the barrier to change (and to move fast) by introducing uncertainty. Are Service X and Service Y using the latest/greatest version of the library? Why did my change to this library break Service Z? Oh, it's because Service Z lags 3 versions behind on this library and can't talk with my new version. Unified repositories help eliminate the sync problem and make a whole class of problems that are detrimental to productivity and moving fast go away. Facebook isn't alone in making this decision. I believe Google maintains a large Perforce repository for the same reasons.
- nuser 12y agoThe advantages you ascribe to monolithic repos aren't due to monolithic repos, they're due to comprehensive tests. The multiple copies of a shared library argument is straight-up nonsense, because in the multiple repo scenario, there would be (one or more) repos of shared libraries. There wouldn't be copying. Unless the devs were morons. Builds in multiple repo environments are clearly identifiable, it's just by a combination of SHAs instead of a single SHA. In practice, this is a non-issue. Version clashes happen in every scenario. Unified repos create a horrific dependency hell, because it's impossible to have one service using version X of a lib while another service uses a version Y. Instead, if you want to update from X to Y, the entire codebase needs to get the update, no matter whether it needs it or not. It's a boondoggle. These decisions, as near as I can tell, are not made because they're good or bad, they're essentially arbitrary, related to what the first few engineers did. If it starts as a giant ball of code, it will always be a giant ball of code. If it starts out well-organized, it might remain well-organized.
- justinhj 12y agoWhilst not arguing against your other points which I think are interesting, git has the submodule system which we use for shared libraries.
- ulisesrmzroche 12y agoYou're coupling your 3rd party dependencies too tightly with your app logic, so that's why its so brittle. Start wrapping those functions.
- jzwinck 12y agoWhen you are done writing boilerplate to wrap all the third-party functions, don't forget to write equivalent documentation too. And tests. Or are you just going to use the same function names? If the latter it's a pointless exercise, because it's unlikely you can shoehorn A different vendor library into the exact same API later.
- Crito 12y ago> "What if multiple services are utilizing a shared library? For each service to be independent in the way I think you are advocating for, you would need multiple copies of that shared library (either via separate copies in separate repos or a shared copy via something like subrepos)." No, you have a notion of packages in your build system and deployment system. You want to use FooWizz framework for your new service BarQuxer? Include FooWiz+=2.0 as a dependency of your service. The build system will then get the suitable package FooWiz when building your BarQuxer. Another team on the other side of the company also wants to use FooWiz? They do the exact same thing. There is never a need for FooWiz to be duplicated, anybody can build with that package as a dependency.
- indygreg2 12y agoI think you are missing the point. Versioning and package management problems can largely go away when your entire code base is derived from a single repo. After all, library versioning and packaging are indirections to better solve common deployment and distribution requirements. These problems don't have to exist when you control all the endpoints. If you could build and distribute a 1 GB self-contained, statically linked binary, library versioning and packages become largely irrelevant.
- Crito 12y agoI'm telling you how corporations with extremely large codebases and a SOA do things. The problem that you described has been solved as I describe. SOA is beneficial over monolithic development for many other reasons unrelated to versioning. It just happens to enable saner versioning as one of it's benefits.
- leccine 12y agoI never said it was a magic bullet. What I am suggesting is to have separate services with independent software packages. This is all. Btw. you can't break a service with a commit, all the services are using versioned APIs so when you are building API version n+1 you are not ditching version n. Very simple, seamless upgrade, graceful downgrade. FYI such a system is powering Amazon and it works perfectly at scale.