5 ms·

> I currently use git submodules to track the application > as a whole . . . This is a conceptual error. There is no such thing as "the application as a w

> I currently use git submodules to track the application > as a whole . . . This is a conceptual error. There is no such thing as "the application as a whole". There is only the currently-deployed set of services, and their versions. You should have a build, test, and deploy pipeline (i.e. continuous deployment) which is triggered on any commit to master of any service. The "test" part should include system/integration tests for all services, deployed into a staging environment. If all tests pass, the service that triggered the commit can rolled out to production. Ideally that rollout should happen automatically, should be phased-in, and should be aborted and rolled back if production monitoring detects any problems.

supermatt 11y ago
> This is a conceptual error. There is no such thing as "the application as a whole". There is only the currently-deployed set of services, and their versions. Im using Martin Fowlers description (whom I admire greatly): "In short, the microservice architectural style is an approach to developing a single application as a suite of small services" http://martinfowler.com/articles/microservices.html http://martinfowler.com/articles/microservices.html My problem is knowing which microservice dependencies the application has. > You should have a build, test, and deploy pipeline (i.e. continuous deployment) which is triggered on any commit to master of any service. The "test" part should include system/integration tests for all services, deployed into a staging environment. If all tests pass, the service that triggered the commit can rolled out to production. Ideally that rollout should happen automatically, should be phased-in, and should be aborted and rolled back if production monitoring detects any problems. We have this, and it is what builds the manifest (as a git repo containing the submodules). The problems occur when you have multiple commits to different services. If a build is marked "red" because it fails a "future integration" then it just means that it is failing at that point in time. It may, following a commit of a dependency, become valid. However, it would need to have its build manually retriggered in order to be classified as such. This becomes cumbersome when you have a not insignificant number of services being commit to on a regular basis.
- sagichmal 11y ago
  > "In short, the microservice architectural style is an > approach to developing a single application as a suite > of small services" Sure, it's a single application from the perspective of the user, and that's a great abstraction to keep in mind when you're talking about things like uptime. But from the perspective of the operator, it's purely an abstraction -- a name you give to the collection of things you're manipulating. As such, I'm not sure building a single repo of git submodules makes much sense. Maybe for accountability reasons, but not for actual operations. The ·raison d'être· of microservices is to decouple the lifecycle of each service totally. By recombining them into a tagged release repo, you're arguably subverting that intent. > The problems occur when you have multiple commits to > different services. If a build is marked "red" because > it fails a "future integration" then it just means that > it is failing at that point in time. It may, following a > commit of a dependency, become valid. However, it would > need to have its build manually retriggered . . . Yep. To solve this problem, your CD system needs to automatically trigger rebuilds (and re-tests) of dependent (downstream) services when a dependency (upstream) changes. The easiest way to do that, in my experience, is to have a manifest in every repo that specifies dependencies as a flat list, which the CD system can parse to build a dependency graph. Tangential point, but -- with this, we're just scratching the surface of the incidental complexity that comes when you move to a microservices architecture. This is the stuff that isn't in all of those exalting blog posts -- beware! :)
  - supermatt 11y ago
    > The easiest way to do that, in my experience, is to have a manifest in every repo that specifies dependencies as a flat list, which the CD system can parse to build a dependency graph. Thats pretty much how we manage the contracts. > To solve this problem, your CD system needs to automatically trigger rebuilds (and re-tests) of dependent (downstream) services when a dependency (upstream) changes This is GOLD. I think you may have just solved the problem with something so obviously simple that it has made me feel like a fool... Many thanks!!!
  - threeseed 11y ago
    Something seems very wrong here. You shouldn't have services rebuilding/redeploying other services. We don't insist that Facebook rebuilds their applications if my Facebook app changes. There should be either an (a) API client, (b) stub version of that service's APIs or (c) deployed test service for any services you depend on. The architecture of microservices is pretty simple. It is a mini version of how the internet works.

supermatt 11y ago

> This is a conceptual error. There is no such thing as "the application as a whole". There is only the currently-deployed set of services, and their versions. Im using Martin Fowlers description (whom I admire greatly): "In short, the microservice architectural style is an approach to developing a single application as a suite of small services" http://martinfowler.com/articles/microservices.html http://martinfowler.com/articles/microservices.html My problem is knowing which microservice dependencies the application has. > You should have a build, test, and deploy pipeline (i.e. continuous deployment) which is triggered on any commit to master of any service. The "test" part should include system/integration tests for all services, deployed into a staging environment. If all tests pass, the service that triggered the commit can rolled out to production. Ideally that rollout should happen automatically, should be phased-in, and should be aborted and rolled back if production monitoring detects any problems. We have this, and it is what builds the manifest (as a git repo containing the submodules). The problems occur when you have multiple commits to different services. If a build is marked "red" because it fails a "future integration" then it just means that it is failing at that point in time. It may, following a commit of a dependency, become valid. However, it would need to have its build manually retriggered in order to be classified as such. This becomes cumbersome when you have a not insignificant number of services being commit to on a regular basis.

sagichmal 11y ago
> "In short, the microservice architectural style is an > approach to developing a single application as a suite > of small services" Sure, it's a single application from the perspective of the user, and that's a great abstraction to keep in mind when you're talking about things like uptime. But from the perspective of the operator, it's purely an abstraction -- a name you give to the collection of things you're manipulating. As such, I'm not sure building a single repo of git submodules makes much sense. Maybe for accountability reasons, but not for actual operations. The ·raison d'être· of microservices is to decouple the lifecycle of each service totally. By recombining them into a tagged release repo, you're arguably subverting that intent. > The problems occur when you have multiple commits to > different services. If a build is marked "red" because > it fails a "future integration" then it just means that > it is failing at that point in time. It may, following a > commit of a dependency, become valid. However, it would > need to have its build manually retriggered . . . Yep. To solve this problem, your CD system needs to automatically trigger rebuilds (and re-tests) of dependent (downstream) services when a dependency (upstream) changes. The easiest way to do that, in my experience, is to have a manifest in every repo that specifies dependencies as a flat list, which the CD system can parse to build a dependency graph. Tangential point, but -- with this, we're just scratching the surface of the incidental complexity that comes when you move to a microservices architecture. This is the stuff that isn't in all of those exalting blog posts -- beware! :)
- supermatt 11y ago
  > The easiest way to do that, in my experience, is to have a manifest in every repo that specifies dependencies as a flat list, which the CD system can parse to build a dependency graph. Thats pretty much how we manage the contracts. > To solve this problem, your CD system needs to automatically trigger rebuilds (and re-tests) of dependent (downstream) services when a dependency (upstream) changes This is GOLD. I think you may have just solved the problem with something so obviously simple that it has made me feel like a fool... Many thanks!!!
- threeseed 11y ago
  Something seems very wrong here. You shouldn't have services rebuilding/redeploying other services. We don't insist that Facebook rebuilds their applications if my Facebook app changes. There should be either an (a) API client, (b) stub version of that service's APIs or (c) deployed test service for any services you depend on. The architecture of microservices is pretty simple. It is a mini version of how the internet works.