12 ms·

Building Domain Driven Microservices

160 points by grzm 6y ago

donbox 6y ago
https://outline.com/ws4FFC https://outline.com/ws4FFC
banq 6y ago
anti-agile
- Redsquare 6y ago
  it is very agile, own your own shizzle, drop dependencies on others. What is anti-agile here?
- pragmaticdev 6y ago
  That's a strange claim to make. Care to elaborate on how making things easier and faster to change is less agile?
kcen 6y ago
One of the problems with forcing strictly disparate domain/service is that each one introduces its own security context and authZ handling. That's a lot of places for mistakes.
- Redsquare 6y ago
  just use auth0
- rumanator 6y ago
  You're just stating that distributed services need to have security.
  - elygre 6y ago
    I think they stated that distributed security is more work / harder than monolithic security. I would tend to agree.
    - Tsarbomb 6y ago
      In my experience, the reason it is more work is because you end up with something much more well defined and robust. Each service having well scoped and defined RBAC or AuthZ for its focused set of features makes the whole architecture as a whole much easier to reason about from a security standpoint. I've done successful pen testing and auditing of some monoliths in my time where the critical security issues arose out of untested and unexpected execution paths that were only possible because the surface area is so large. Maybe in theory a "well written" monolith would be superior but I'm only going but what I have seen in practice. I think the extra work is worth the trade off.
- jordanbeiber 6y ago
  My platform team supports devs with a proxy that handles authN & authZ. Service to service is secured using mTLS automatically. I’ve stated it previously in the thread - you need to enforce conventions through tooling - which is where “devops” (ugh, I know, but really...) comes in to play. We try to view the tooling platform as just another DDD domain.
asim 6y ago
Good article. This is what micro was built for. https://go-micro.dev https://go-micro.dev
eecc 6y ago
> Hide implementation detail and expose functionality through intention-revealing interfaces. Great, then a new US comes in that changes the intention and you enjoy the pleasure of changing the interface. Enter versioning. You start crying.
- rumanator 6y ago
  > Great, then a new US comes in that changes the intention and you enjoy the pleasure of changing the interface. That's not a microservices problem, not even a software architecture problem. That's at best a project management problem. In fact, pretty much every single engineering project there ever was would struggle to meet any form of neck-breaking change in design requirements.
  - quonn 6y ago
    But with microservices you have more supposed independent interfaces. With a more monolithic approach you might just change a system internally and deploy it. Internal interfaces are generally easy to adapt.
    - rumanator 6y ago
      > But with microservices you have more supposed independent interfaces. You don't. You have more services accessed over a network, and might not have control over all services. But that's already par for the course in nonoliths. If you were to refactor your components by changing classes around (say, a major version update in a dependency) while not updating the components that used them then you would also have problems. That's why versioning and service discovery and HATEOAS are a thing.
      - quonn 6y ago
        But the compiler will find that. It‘s very different from deploying services. (And I‘ve been doing both, for years.)
ivan_ah 6y ago
Off topic but wanted to flag: > [...] we intend to cover our experiences with Event Storming in a separate blog. In case anyone is not familiar with this usage, these days "blog" is often used instead of "blog post." I was thoroughly confused by this when I first ran into it. I tried fighting against this usage by colleagues, but it seems to be widespread enough now that it cannot be fought... I guess language evolves, and I have to get with the times. I suspect one or more of the following aspects might be at play: • general loss of identity (since lots of posts are on medium, and less on a personal and work blogs) • "blog" is short for "blog post" • desire to distance oneself from "post" which is more associated with facebook and instagram posts
- rumanator 6y ago
  It was indeed confusing.I honestly thought that the author was using this article as a clickbait setup to drive traffic to another blog. Blog: web log. A collection of entries that are used to log something and is made available to the internet. Blog post: a specific entry posted in a blog.
- hnick 6y ago
  I don't read any more into it than someone saying "put it on the USB" instead of spelling out USB stick/drive/etc. It's just a lazy abbreviation that caught on.
yeswecatan 6y ago
I've been interested and confused about DDD, hexagonal architecture, clean architecture etc. for a couple of years now but unfortunately have not had time to dedicate towards reading up on all of them. Has anyone actually implemented any of this as part of their jobs? I guess it all depends what you do. I mainly do web stuff, and Django/DRF lets us go fast. Out of the box, I think this goes against the aforementioned architecture as the ORM couples the domain model with the data model. Besides, gathering requirements (at least at my company) is unfortunately an ongoing thing which is just the nature of a startup I suppose.
- rumanator 6y ago
  > Out of the box, I think this goes against the aforementioned architecture as the ORM couples the domain model with the data model. It doesn't. You're confusing the persistence model with the domain model. Your persistence model is dictated by your need to persist your data while meeting the requirements of your domain model. Your domain model does not care about your persistence requirements. Your persistence model is bounded to your persistence service and should not leak out, while your domain model is ubiquitous to your system. Let's put it this way: with a Clean Architecture you are able to add/replace any persistence service you wish, and that has no impact on any other service.
  - yeswecatan 6y ago
    Can you provide a concrete example? I would assume if you wanted to decouple the persistence model from the domain model then you would need to write a separate domain model in Django since the domain and persistence model are tightly coupled.
    - rumanator 6y ago
      I'm not very familiar with Django thus my ability to provide relevant examples using this particular framework is limited. However I direct your attention to the fact that Django docs make it very clear that to them in general each model maps to a single database table. That's quite Spartan even for an ORM, and obviously the point of a domain model is to model a problem domain, not to model the database.
fmakunbound 6y ago
This is pretty frequently discussed at Java developer conferences for some reason.
remote_phone 6y ago
Bleah. I hate articles like this. They regurgitate all the well known talking points about enterprise software development but none of the realities of micro services. It sounds like they haven’t actually worked in the trenches supporting and coding real microservices at scale. The realities are that going from monolith to microservices is a huge investment, and only large engineering teams should do this migration. The reason is that once you go microservices You instantly require an immediate investment in devops which is something you didn’t need as much before with a monolith. Oncall rotations and pager duty schedules become imperative because services will go down. Also monitoring is another investment that you immediately need otherwise you’re completely blind. The article talks about things that are obvious, like designing the apis etc, but those are the same regardless of if it’s a module within a monolith or a microservice. All the characteristics they list at the beginning is no different than what a team that owns a module in a monolith would require as well. It feels like this article isn’t very useful because it regurgitates concepts from regular enterprise development and packages as new for microservices when it’s not. What the article lacks is the difference in architecting for a real microservice environment. One of the biggest missed issues is never mentioning dependencies. Sure it’s great to talk about how to model services with eventual consistency blah blah. But if you already have a large engineering team, which you must have otherwise why have microservices, then there will be a dependency between services and to believe otherwise is crazy. You will probably have one or two services that when they go down, your entire company is down. It’s imperative that every service is architected with that in mind and what happens when it’s down. Hand waving “eventual consistency” won’t cut it in the real world, it’s the biggest problem that each team needs to focus on early otherwise it’s a ticking time bomb.
- rumanator 6y ago
  > Bleah. I hate articles like this. They regurgitate all the well known talking points about enterprise software development but none of the realities of micro services. I disagree. The post is a good summary of all the key features and main concepts of a microservices architecture developed with domain-driven design. It does what it says on the tin, and you don't have to read three books to get there. > It sounds like they haven’t actually worked in the trenches supporting and coding real microservices at scale. I have no idea where you got that idea. It certainly wasn't from the blog post, though. > The realities are that going from monolith to microservices is a huge investment, and only large engineering teams should do this migration. Your rant makes no sense at all. You're just mindlessly repeating cliches about issues that have absolutely nothing to do with the subject. I mean, the blog post is about microservices software architecture with domain-driven design and clean code, but here you are whining about how you believe refactoring projects designed following an entirely different architectural style is hard.
deleted 6y ago
[deleted]
fulafel 6y ago
Wow, microservices sound like a very painful trade-off to make. Is it really worth it?
- oftenwrong 6y ago
  Microservices are for when you have no other choice. If you truly cannot build a monolith, then build microservices. If you have merely chosen to adopt a microservice architecture when a monolith would have been suitable, then you have made a huge mistake.
- jordanbeiber 6y ago
  It definitely can be, if it’s done for the “right reasons”. Mostly the reason would be to scale teams but still keep agility. It really couples nicely with DDD and I’m in the middle of my second project where these principles are used. I would not do it unless there’s a dedicated platform team that will take care of all the infra pieces you’ll require. This kind of points to the scale where it makes sense, I guess!
soonnow 6y ago
I think that many people look at Microservices from a purely technical standpoint. How much compute power do I get per dollar spent. What about the monitoring complexities and the logging problems? Microservices are actually often driven by business requirements. According to the Myhtical Man Month communication channels follow n(n − 1) / 2. So a team of 4 developers has 6 possible communication channels, a team of 5 already 10. This eincreases sharply as team sizes increase and is one of the reasons big enterprise projects suffer compare to startups. The business benefit of Microservices (done right) is that you build islands of communication, that communicate using only one outside channel. Now communication increases less sharply as teams are separated by their boundaries. According to the guardian the team size and structure is one of the secret for Amazons success. https://www.theguardian.com/technology/2018/apr/24/the-two-pizza-rule-and-the-secret-of-amazons-success https://www.theguardian.com/technology/2018/apr/24/the-two-p... For smaller teams the communication benefit will often not outweigh the additional complexity.
- oftenwrong 6y ago
  It is possible to keep teams "on islands" with a modular monolith. Each team builds a module. Logging, monitoring, and other shared infrastructure is there for all teams.
  - soonnow 6y ago
    Until you come to the deployment phase where all teams need to communicate and deploy in sync. This can be a significant factor for slow down. Also Microservices don't sync on the datastore, these are (in theory) not shared. So yes Microservices can be emulated in Monolithic architectures, as can Monolithic Microservices be built. Usually you end up with the worst of both worlds. The complexity of Microservices with the speed of Monoliths.
- hinkley 6y ago
  But what happens is you end up with a ton of them, fulfilling the same prophecy you were trying to avoid, or you keep trying to cram more “volume” behind the same surface area, until you eventually tear a hole in the fabric of space time and/or your business. The way to succeed is to do fewer things but make them count. Except that looks awful on your annual review, so the real way to succeed is to Defect. Be seen doing as much as you can and get out before they figure out you consumed all of the oxygen in the room.
Copenjin 6y ago
I'm really having a hard time finding the willpower to read articles that just even mention "microservices" in the title after working on such architectures (with Java) for a while. Nice idea but most of the times horribly implemented, with no care about monitoring and logging as someone has already said. It requires a degree of attention to details during design and development that few are willing to put in. It's easy to go for a prepackaged solution/framework thinking that you are building something that could be considered state of the art, when instead you end up with something that takes multiple tries to even boot correctly or 10 minutes or so to "stabilize" a garbled mess of inter-dependant components. Designing these systems require a whole new set of skills that were not required when designing monoliths.
- oftenwrong 6y ago
  I worked for a company that had tried to adopt microservices (without a good reason to) and failed miserably. Over time we were able to re-architect it into a "happy medium" of a few large components. It was a huge, costly effort... and I've never learned so much. Beyond technical concerns, I learned that you can architect a looming disaster and then leave for a higher position at a big company. Nobody watching your conference talk will know.
- jordanbeiber 6y ago
  You really point out that you’re going to require a dedicated team to manage all the things you’ll need. I you’re not at this scale there will be a lot of overhead to manage for the dev teams. I’m in a team like this, supporting a bunch of developers - we build a CLI, an API (in our domain! It’s just another domain with the devs as consumers) and juggle approx. 16 different tools (monitoring, metrics, tracing, service discovery, etc etc). Doing it through custom tooling that enforces conventions is, IMHO, the way to do it successfully.
  - Copenjin 6y ago
    There is a lot of very interesting work to be done in this area (amazing books coming out lately from O'Reilly about these topics) but I feel there is still some piece of the puzzle missing. We need more tools/practices for the developers to guide their development and understand how an architecture behave while they are still building it, to drive design decisions (and also check/correct them along the way).
tannhaeuser 6y ago
> Hide implementation detail and expose functionality through intention-revealing interfaces. Except "REST APIs" (what microservices expose almost always) don't reveal anything most of the time, wanting to hide internal state behind a beautiful and uniform facade essentially representing a two-tier model. For example, orders and order line items can't be added/deleted at will, but depending on whether the order has been processed or not, should expose a "cancel" action (with partial refund/with penalty for cancelling), etc. Sure you can hide that behind a DELETE method, or "reify" a "cancelled" status, but why? > Services are resilient to failures. In the case of "REST APIs", this means manually and labouriously attempting to reconcile state across multiple services in the client. Compare this with state changes though ordinary RPC, which can expose a meaningful state model, give contextual diagnostics/exceptions, and can have transaction context flowing with RPC calls.
jungoolist 6y ago
Domain Driven Microservices looks good on paper, but its a very very terrible idea. 1. You can't run join queries across domains 2. You can't migrate your schema across domains 3. You will need to sync and map a db with domain dbs for reporting 4. You need to write api/client server for every domain and maintain this with db schema changes
- UK-Al05 6y ago
  It seems like database schema and domain api are very coupled. They should have nothing to do with each other. You should be able to change your database schema without changing your api.... If you want to expose extra features or change the api design, you version your api. Reporting is a problem, but most large business have some kind of datalake/datawarehouse where all of it comes together anyway these days.
  - chrisan 6y ago
    > If you want to expose extra features or change the api design, you version your api. But you are still coupled. If you have to version your API that means it is relied on by something else and in order to use that extra feature still requires both (or more) services to all be on the same page. Microservices let you deploy and maintain at different times (if that is appealing to you) but in order to realize a new business feature you are still coupled. Now you have x tests for x versions as well, make sure the old version remains old and the new feature works in the new version.
    - UK-Al05 6y ago
      I mean yes, if you want a use a new feature using the new api. Then your going to have to move onto the new api. But that would happen regardless of using microservices. If you version it though, services that don't care can cary on regardless.
- pragmaticdev 6y ago
  I've never had to do these things. It's possible you're not applying domain boundaries correctly.
hestefisk 6y ago
I might be a bit cynical / neckbeard, but the tech industry tends to go in circles with good engineering principles sold as the newest fads / hype. Before all the microservices craze it was REST and APIs, before that web services, SOA and ESBs, and before that XML-RPC / CORBA, EAI, and enterprise Java beans. And before that we had, we someone we had RPC over the network (‘open systems’). Similarly DDD promotes separation of concerns and test driven development. That is exactly what eXtreme Programming told us in late 90ies when OOP and unit testing was in. Before that it was structured programming and software design specs, 4GL etc. Bottom line is I wish people would start calling a spade a spade and indulge in less fads. Let’s just call it structured programming over the network with clear separation of concerns and clean interfaces. That’s good engineering. Test early and often. Write good documentation.
- deleted 6y ago
  [deleted]
- eurekin 6y ago
  I'm completely on the same page with you on that one. I understand though the marketing needs. IT has this steady influx of beginners that don't have the insight yet. Since there are so many of them, at any given time, it's reasonable (or maybe it actually works) to use standard advertising techniques at them. "Secret sauce", "The missing puzzle piece", "10 things that make you a better dev", "Dev done correctly". Developers are kind of a dream target group: lots of disposable income, technically adept (i.e. at the cost of social skills), excited to experience a latest unicorn. I'm seeing this whole "hype cycle" thing as a competition among more senior, adept in marketing, developers. Since there is such a high demand, the supply is also rich.
  - jka 6y ago
    Could/should there be a 'wikipedia for software development practices'?
    - steve-s 6y ago
      What about https://wiki.c2.com https://wiki.c2.com
francasso 6y ago
Am I the only architect that hates microservices with gusto? I've only seen bad implementations out there (tens of products), and have yet to encounter one that looks reasonable. If your average developer has trouble implementing something sane when the only problem is splitting data and functions locally into different components, why in the world would you assume that putting in a network layer, exponentially increasing the failure rates, loosing good stack traces, making refactorings that have to move data from one part of the architecture to another a pain in the butt (because requirements never change right?), etc... would be a good thing? The horrors I've seen... oh boy, the horrors. The people I've heard praising microservice architectures so far fall into two categories: (1) Ex developers that moved into management and read a few articles/bought a book on the subject and think it's a great idea (but have no actual practical experience) (2) developers that have just started working on a new one and are still in the honeymoon phase (mind you, not developers that have started working on an __existing__ one that is a few years old, those figure out the mess they are in pretty quickly). You can screw up not-so-micro services too, but it's far easier to screw up microservice architectures and they are far, far harder to fix down the line when you figure out how you messed up (or re-architect when that thing that the PM told you was never going to be a use case is going to become a use case (after they tell you they are sorry of course)).
- andy_ppp 6y ago
  Totally agree - I'm not against services but I say as few services as humanly possible! The issues with micro services are just so unbelievably large that it'd take too long to enumerate them here, but for me the largest issue is you are putting arbirary network partitions into your application and you CANNOT possibly know when these network partitions will suddenly make an operation you need to be atomic impossible (or at best eventually consistent). In general it is excruciatingly difficult to iterate on micro services as well. Then there is deployment, service mesh, k8s etc. etc. that you have to wrangle with, all because people refused to make bounded contexts in their application without making them into different services. It is much simpler to just design your monolith into separate contexts than it is to introduce all the error handling you need for network partitions. I personally don't understand why Elixir isn't more loved by these micro services zealots - it allows you to get extremely far without adding network partitions and then allows you to switch to message passing when you need and then networked message passing all extremely smoothly as it makes sense for your app.
deleted 6y ago
[deleted]
karmakaze 6y ago
This talks about the thing that makes microservices fail most often, getting the bounded contexts wrong. It doesn't go into enough detail to show you how to do it right but does tell you what to watch for when you're doing it wrong. The biggest, most common mistake is when a bounded context is called "<noun> service". This should be a huge red flag. Unfortunately, most books (and this post) always use this bad naming starting everyone down the bad path. At the very minimum call it a "<noun> management service" if it is the primary/general create/update activity. e.g you might have a checkout service that creates most orders and a order management service that can make drafts or adjustments. Order management (being the decoupled source of truth) might even be downstream from checkout service. Other services that need this information should get it's own copy asynchronously, i.e. decoupled. The best way to learn how to do bounded contexts right is to study ones that worked well. Most writing tries to describe this in an abstract way using placeholder widget service things that helps no one. The BFF pattern is good but doesn't solve everything. If you're accessing multiple services for a synchronous mutation, it's likely not highly reliable but can get by for an interactive service that can tolerate some errors. A messaging service is not one that should, they should always accept, store and forward with the lowest possible error rates.
dvancouvering 6y ago
I have done a lot of work with microservices, and I agree they come with a ton of cognitive overhead and new ways for things to fail in unexpected and painful ways. In particular it now requires software engineers to understand principles and design patterns for distributed systems that when I was first starting out were relegated to theory and university papers. Can you guarantee ordering? Can you guarantee exactly once? How do you get transactional semantics? How do you handle cascading failures? It's all a bit overwhelming. I have also seen however that monoliths at some point lock up an organization. The key measures of software delivery performance are deploy frequency, lead time from commit to running in production, mean time to recovery and change failure rate. Monoliths impact all of these when you engineering org starts scaling past a handful of teams. A monolith also becomes a single point of failure. One person in one team checks in bad code, and the whole system falls down. Microservices can give you resilience. You can also scale microservices independently - a CPU intensive service can get a bunch of pods, a memory intensive service gets a beefier memory allocation, and other services can run idle most of the time or even be deployed as FAAS functions that almost never run. So I agree it's not a technical solution as much as it is about scaling an organization. At WeWork the team shrank so much that we began consolidating small services back into a single service because the operational overhead of having separate services didn't make sense any more when a single team was responsible for all of them. I do think choosing to split out a new service is a big decision to make. You have to really think through the cost and benefits. You also need to make sure you have the right tools in place so you can detect and recover quickly. Things like log aggregation, SLOs defined, measured and alerted on, canary deploys, progressive delivery, etc. Without that you're running completely blind. I also think you want to be thoughtful about keeping your coupling low. If each service depends on twenty other services then in my mind you now just have a distributed monolith.
- dmitryminkovsky 6y ago
  > Can you guarantee ordering? Can you guarantee exactly once? How do you get transactional semantics? How do you handle cascading failures? It's all a bit overwhelming. I don’t advocate DDD/microservices but if you absolutely must go down this road, Kafka Streams really neatly answers all these questions. Ben Stopford’s book[0] and blog posts[1] are really great if you want to know more. [0] http://www.benstopford.com/2018/04/27/book-designing-event-driven-systems/ http://www.benstopford.com/2018/04/27/book-designing-event-d... [1] https://www.confluent.io/blog/building-a-microservices-ecosystem-with-kafka-streams-and-ksql/ https://www.confluent.io/blog/building-a-microservices-ecosy...
corpMaverick 6y ago
Microservices are a good way to scale up development teams. So if you have 16 developers you can partition the work in two teams with one microservice each. If you have 80 devs you can have 10 teams with one microservice each team. Use Conwayś law to your advantage. Use DDD to decide where the boundaries are. The advantage of having each team own a microservice is that each team can deploy their software independently from the rest of the teams. And it works as long as the APIs are honored. So Microservices are way to decouple the work of different teams allowing them to work independently from each other.
- dakiol 6y ago
  This only works for the simplest cases. If the two microservices must share data and direct calls (API calls) are not allowed (because they make your services highly coupled to each other to the point where if one of them ia down, then the other cannot work properly, or simply because your use case cannot tolerate the latency overhead) then each service must 1) replicate the data needed locally by listening to interesting events of the other service and b) emit interesting events so that the other service can update its local db. In other words, async communication. While this let's you scale up teams, it's a huge overhead. So, unless you have over, let's say, 500 devs, it's not worth it.
  - corpMaverick 6y ago
    > This only works for the simplest cases. If the two microservices must share data and direct calls (API calls) are not allowed Actually, one assumption is that the microservices don't share data. Only API (Rest, SOAP, GraphQL, etc) calls are allowed. And the boundaries are created minimizing the coupling between microservices. > While this let's you scale up teams, it's a huge overhead. So, unless you have over, let's say, 500 devs, it's not worth it. As opposed to Monolith ? I think you will have problems collaborating in a Monolith at around 30 devs. What I am opposing is the idea that you can have 30 devs and 50 microservices. Which I have seen in real life.
- rched 6y ago
  Why do you need Microservices to create boundaries in your code? You trade the ability for teams to deploy independently for the ability to be able to make breaking changes. This is a good trade off in my experience and is better for team velocity. The approach we have taken is to keep a single codebase and deploy. We have applied the ideas from DDD to segment the code into contexts and assign ownership of each of those contexts to a team. This allows us to get some of the benefits of Microservices without losing all the benefits of a monolith. Having these boundaries in place also gives us the flexibility to move to Microservices if and when we decide we need to.
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
tveita 6y ago
I see the writer is from "WalmartLabs", so I assume this is advice for "Walmart scale" applications. For the rest of us it's probably good to remember that "you are not Walmart." > Now, we know that in any modern application, sacrificing availability is not a good idea either. I feel like this will apply to less than 10% of applications. Normally you'd rather deal with the occasional downtime than dig through the fallout of broken attempts at distributed transactions. Ask yourself "What amount of $X,000 are we losing per minute of downtime" and see if it rounds down to zero. > Another way to model this system is to separate, or group related models into separate microservices. In DDD, these models — Price, Priced Items, and Discounts — are called Aggregates. And this must be the case for less than 1% of readers - it would take a massive system before I'd consider having prices and discounts in separate databases. You're taking on the overhead of writing an additional service for what could have been a database join.

160 points by grzm 6y ago

donbox 6y ago

https://outline.com/ws4FFC https://outline.com/ws4FFC

banq 6y ago

anti-agile

Redsquare 6y ago
it is very agile, own your own shizzle, drop dependencies on others. What is anti-agile here?
pragmaticdev 6y ago
That's a strange claim to make. Care to elaborate on how making things easier and faster to change is less agile?

kcen 6y ago

One of the problems with forcing strictly disparate domain/service is that each one introduces its own security context and authZ handling. That's a lot of places for mistakes.

Redsquare 6y ago
just use auth0
rumanator 6y ago
You're just stating that distributed services need to have security.
- elygre 6y ago
  I think they stated that distributed security is more work / harder than monolithic security. I would tend to agree.
  - Tsarbomb 6y ago
    In my experience, the reason it is more work is because you end up with something much more well defined and robust. Each service having well scoped and defined RBAC or AuthZ for its focused set of features makes the whole architecture as a whole much easier to reason about from a security standpoint. I've done successful pen testing and auditing of some monoliths in my time where the critical security issues arose out of untested and unexpected execution paths that were only possible because the surface area is so large. Maybe in theory a "well written" monolith would be superior but I'm only going but what I have seen in practice. I think the extra work is worth the trade off.
jordanbeiber 6y ago
My platform team supports devs with a proxy that handles authN & authZ. Service to service is secured using mTLS automatically. I’ve stated it previously in the thread - you need to enforce conventions through tooling - which is where “devops” (ugh, I know, but really...) comes in to play. We try to view the tooling platform as just another DDD domain.

asim 6y ago

Good article. This is what micro was built for. https://go-micro.dev https://go-micro.dev

eecc 6y ago

> Hide implementation detail and expose functionality through intention-revealing interfaces. Great, then a new US comes in that changes the intention and you enjoy the pleasure of changing the interface. Enter versioning. You start crying.

rumanator 6y ago
> Great, then a new US comes in that changes the intention and you enjoy the pleasure of changing the interface. That's not a microservices problem, not even a software architecture problem. That's at best a project management problem. In fact, pretty much every single engineering project there ever was would struggle to meet any form of neck-breaking change in design requirements.
- quonn 6y ago
  But with microservices you have more supposed independent interfaces. With a more monolithic approach you might just change a system internally and deploy it. Internal interfaces are generally easy to adapt.
  - rumanator 6y ago
    > But with microservices you have more supposed independent interfaces. You don't. You have more services accessed over a network, and might not have control over all services. But that's already par for the course in nonoliths. If you were to refactor your components by changing classes around (say, a major version update in a dependency) while not updating the components that used them then you would also have problems. That's why versioning and service discovery and HATEOAS are a thing.
    - quonn 6y ago
      But the compiler will find that. It‘s very different from deploying services. (And I‘ve been doing both, for years.)

ivan_ah 6y ago

Off topic but wanted to flag: > [...] we intend to cover our experiences with Event Storming in a separate blog. In case anyone is not familiar with this usage, these days "blog" is often used instead of "blog post." I was thoroughly confused by this when I first ran into it. I tried fighting against this usage by colleagues, but it seems to be widespread enough now that it cannot be fought... I guess language evolves, and I have to get with the times. I suspect one or more of the following aspects might be at play: • general loss of identity (since lots of posts are on medium, and less on a personal and work blogs) • "blog" is short for "blog post" • desire to distance oneself from "post" which is more associated with facebook and instagram posts

rumanator 6y ago
It was indeed confusing.I honestly thought that the author was using this article as a clickbait setup to drive traffic to another blog. Blog: web log. A collection of entries that are used to log something and is made available to the internet. Blog post: a specific entry posted in a blog.
hnick 6y ago
I don't read any more into it than someone saying "put it on the USB" instead of spelling out USB stick/drive/etc. It's just a lazy abbreviation that caught on.

yeswecatan 6y ago

I've been interested and confused about DDD, hexagonal architecture, clean architecture etc. for a couple of years now but unfortunately have not had time to dedicate towards reading up on all of them. Has anyone actually implemented any of this as part of their jobs? I guess it all depends what you do. I mainly do web stuff, and Django/DRF lets us go fast. Out of the box, I think this goes against the aforementioned architecture as the ORM couples the domain model with the data model. Besides, gathering requirements (at least at my company) is unfortunately an ongoing thing which is just the nature of a startup I suppose.

rumanator 6y ago
> Out of the box, I think this goes against the aforementioned architecture as the ORM couples the domain model with the data model. It doesn't. You're confusing the persistence model with the domain model. Your persistence model is dictated by your need to persist your data while meeting the requirements of your domain model. Your domain model does not care about your persistence requirements. Your persistence model is bounded to your persistence service and should not leak out, while your domain model is ubiquitous to your system. Let's put it this way: with a Clean Architecture you are able to add/replace any persistence service you wish, and that has no impact on any other service.
- yeswecatan 6y ago
  Can you provide a concrete example? I would assume if you wanted to decouple the persistence model from the domain model then you would need to write a separate domain model in Django since the domain and persistence model are tightly coupled.
  - rumanator 6y ago
    I'm not very familiar with Django thus my ability to provide relevant examples using this particular framework is limited. However I direct your attention to the fact that Django docs make it very clear that to them in general each model maps to a single database table. That's quite Spartan even for an ORM, and obviously the point of a domain model is to model a problem domain, not to model the database.

fmakunbound 6y ago

This is pretty frequently discussed at Java developer conferences for some reason.

remote_phone 6y ago

Bleah. I hate articles like this. They regurgitate all the well known talking points about enterprise software development but none of the realities of micro services. It sounds like they haven’t actually worked in the trenches supporting and coding real microservices at scale. The realities are that going from monolith to microservices is a huge investment, and only large engineering teams should do this migration. The reason is that once you go microservices You instantly require an immediate investment in devops which is something you didn’t need as much before with a monolith. Oncall rotations and pager duty schedules become imperative because services will go down. Also monitoring is another investment that you immediately need otherwise you’re completely blind. The article talks about things that are obvious, like designing the apis etc, but those are the same regardless of if it’s a module within a monolith or a microservice. All the characteristics they list at the beginning is no different than what a team that owns a module in a monolith would require as well. It feels like this article isn’t very useful because it regurgitates concepts from regular enterprise development and packages as new for microservices when it’s not. What the article lacks is the difference in architecting for a real microservice environment. One of the biggest missed issues is never mentioning dependencies. Sure it’s great to talk about how to model services with eventual consistency blah blah. But if you already have a large engineering team, which you must have otherwise why have microservices, then there will be a dependency between services and to believe otherwise is crazy. You will probably have one or two services that when they go down, your entire company is down. It’s imperative that every service is architected with that in mind and what happens when it’s down. Hand waving “eventual consistency” won’t cut it in the real world, it’s the biggest problem that each team needs to focus on early otherwise it’s a ticking time bomb.

rumanator 6y ago
> Bleah. I hate articles like this. They regurgitate all the well known talking points about enterprise software development but none of the realities of micro services. I disagree. The post is a good summary of all the key features and main concepts of a microservices architecture developed with domain-driven design. It does what it says on the tin, and you don't have to read three books to get there. > It sounds like they haven’t actually worked in the trenches supporting and coding real microservices at scale. I have no idea where you got that idea. It certainly wasn't from the blog post, though. > The realities are that going from monolith to microservices is a huge investment, and only large engineering teams should do this migration. Your rant makes no sense at all. You're just mindlessly repeating cliches about issues that have absolutely nothing to do with the subject. I mean, the blog post is about microservices software architecture with domain-driven design and clean code, but here you are whining about how you believe refactoring projects designed following an entirely different architectural style is hard.

deleted 6y ago

[deleted]

fulafel 6y ago

Wow, microservices sound like a very painful trade-off to make. Is it really worth it?

oftenwrong 6y ago
Microservices are for when you have no other choice. If you truly cannot build a monolith, then build microservices. If you have merely chosen to adopt a microservice architecture when a monolith would have been suitable, then you have made a huge mistake.
jordanbeiber 6y ago
It definitely can be, if it’s done for the “right reasons”. Mostly the reason would be to scale teams but still keep agility. It really couples nicely with DDD and I’m in the middle of my second project where these principles are used. I would not do it unless there’s a dedicated platform team that will take care of all the infra pieces you’ll require. This kind of points to the scale where it makes sense, I guess!

soonnow 6y ago

I think that many people look at Microservices from a purely technical standpoint. How much compute power do I get per dollar spent. What about the monitoring complexities and the logging problems? Microservices are actually often driven by business requirements. According to the Myhtical Man Month communication channels follow n(n − 1) / 2. So a team of 4 developers has 6 possible communication channels, a team of 5 already 10. This eincreases sharply as team sizes increase and is one of the reasons big enterprise projects suffer compare to startups. The business benefit of Microservices (done right) is that you build islands of communication, that communicate using only one outside channel. Now communication increases less sharply as teams are separated by their boundaries. According to the guardian the team size and structure is one of the secret for Amazons success. https://www.theguardian.com/technology/2018/apr/24/the-two-pizza-rule-and-the-secret-of-amazons-success https://www.theguardian.com/technology/2018/apr/24/the-two-p... For smaller teams the communication benefit will often not outweigh the additional complexity.

oftenwrong 6y ago
It is possible to keep teams "on islands" with a modular monolith. Each team builds a module. Logging, monitoring, and other shared infrastructure is there for all teams.
- soonnow 6y ago
  Until you come to the deployment phase where all teams need to communicate and deploy in sync. This can be a significant factor for slow down. Also Microservices don't sync on the datastore, these are (in theory) not shared. So yes Microservices can be emulated in Monolithic architectures, as can Monolithic Microservices be built. Usually you end up with the worst of both worlds. The complexity of Microservices with the speed of Monoliths.
hinkley 6y ago
But what happens is you end up with a ton of them, fulfilling the same prophecy you were trying to avoid, or you keep trying to cram more “volume” behind the same surface area, until you eventually tear a hole in the fabric of space time and/or your business. The way to succeed is to do fewer things but make them count. Except that looks awful on your annual review, so the real way to succeed is to Defect. Be seen doing as much as you can and get out before they figure out you consumed all of the oxygen in the room.

Copenjin 6y ago

I'm really having a hard time finding the willpower to read articles that just even mention "microservices" in the title after working on such architectures (with Java) for a while. Nice idea but most of the times horribly implemented, with no care about monitoring and logging as someone has already said. It requires a degree of attention to details during design and development that few are willing to put in. It's easy to go for a prepackaged solution/framework thinking that you are building something that could be considered state of the art, when instead you end up with something that takes multiple tries to even boot correctly or 10 minutes or so to "stabilize" a garbled mess of inter-dependant components. Designing these systems require a whole new set of skills that were not required when designing monoliths.

oftenwrong 6y ago
I worked for a company that had tried to adopt microservices (without a good reason to) and failed miserably. Over time we were able to re-architect it into a "happy medium" of a few large components. It was a huge, costly effort... and I've never learned so much. Beyond technical concerns, I learned that you can architect a looming disaster and then leave for a higher position at a big company. Nobody watching your conference talk will know.
jordanbeiber 6y ago
You really point out that you’re going to require a dedicated team to manage all the things you’ll need. I you’re not at this scale there will be a lot of overhead to manage for the dev teams. I’m in a team like this, supporting a bunch of developers - we build a CLI, an API (in our domain! It’s just another domain with the devs as consumers) and juggle approx. 16 different tools (monitoring, metrics, tracing, service discovery, etc etc). Doing it through custom tooling that enforces conventions is, IMHO, the way to do it successfully.
- Copenjin 6y ago
  There is a lot of very interesting work to be done in this area (amazing books coming out lately from O'Reilly about these topics) but I feel there is still some piece of the puzzle missing. We need more tools/practices for the developers to guide their development and understand how an architecture behave while they are still building it, to drive design decisions (and also check/correct them along the way).

tannhaeuser 6y ago

> Hide implementation detail and expose functionality through intention-revealing interfaces. Except "REST APIs" (what microservices expose almost always) don't reveal anything most of the time, wanting to hide internal state behind a beautiful and uniform facade essentially representing a two-tier model. For example, orders and order line items can't be added/deleted at will, but depending on whether the order has been processed or not, should expose a "cancel" action (with partial refund/with penalty for cancelling), etc. Sure you can hide that behind a DELETE method, or "reify" a "cancelled" status, but why? > Services are resilient to failures. In the case of "REST APIs", this means manually and labouriously attempting to reconcile state across multiple services in the client. Compare this with state changes though ordinary RPC, which can expose a meaningful state model, give contextual diagnostics/exceptions, and can have transaction context flowing with RPC calls.

jungoolist 6y ago

Domain Driven Microservices looks good on paper, but its a very very terrible idea. 1. You can't run join queries across domains 2. You can't migrate your schema across domains 3. You will need to sync and map a db with domain dbs for reporting 4. You need to write api/client server for every domain and maintain this with db schema changes

UK-Al05 6y ago
It seems like database schema and domain api are very coupled. They should have nothing to do with each other. You should be able to change your database schema without changing your api.... If you want to expose extra features or change the api design, you version your api. Reporting is a problem, but most large business have some kind of datalake/datawarehouse where all of it comes together anyway these days.
- chrisan 6y ago
  > If you want to expose extra features or change the api design, you version your api. But you are still coupled. If you have to version your API that means it is relied on by something else and in order to use that extra feature still requires both (or more) services to all be on the same page. Microservices let you deploy and maintain at different times (if that is appealing to you) but in order to realize a new business feature you are still coupled. Now you have x tests for x versions as well, make sure the old version remains old and the new feature works in the new version.
  - UK-Al05 6y ago
    I mean yes, if you want a use a new feature using the new api. Then your going to have to move onto the new api. But that would happen regardless of using microservices. If you version it though, services that don't care can cary on regardless.
pragmaticdev 6y ago
I've never had to do these things. It's possible you're not applying domain boundaries correctly.

hestefisk 6y ago

I might be a bit cynical / neckbeard, but the tech industry tends to go in circles with good engineering principles sold as the newest fads / hype. Before all the microservices craze it was REST and APIs, before that web services, SOA and ESBs, and before that XML-RPC / CORBA, EAI, and enterprise Java beans. And before that we had, we someone we had RPC over the network (‘open systems’). Similarly DDD promotes separation of concerns and test driven development. That is exactly what eXtreme Programming told us in late 90ies when OOP and unit testing was in. Before that it was structured programming and software design specs, 4GL etc. Bottom line is I wish people would start calling a spade a spade and indulge in less fads. Let’s just call it structured programming over the network with clear separation of concerns and clean interfaces. That’s good engineering. Test early and often. Write good documentation.

deleted 6y ago
[deleted]
eurekin 6y ago
I'm completely on the same page with you on that one. I understand though the marketing needs. IT has this steady influx of beginners that don't have the insight yet. Since there are so many of them, at any given time, it's reasonable (or maybe it actually works) to use standard advertising techniques at them. "Secret sauce", "The missing puzzle piece", "10 things that make you a better dev", "Dev done correctly". Developers are kind of a dream target group: lots of disposable income, technically adept (i.e. at the cost of social skills), excited to experience a latest unicorn. I'm seeing this whole "hype cycle" thing as a competition among more senior, adept in marketing, developers. Since there is such a high demand, the supply is also rich.
- jka 6y ago
  Could/should there be a 'wikipedia for software development practices'?
  - steve-s 6y ago
    What about https://wiki.c2.com https://wiki.c2.com

francasso 6y ago

Am I the only architect that hates microservices with gusto? I've only seen bad implementations out there (tens of products), and have yet to encounter one that looks reasonable. If your average developer has trouble implementing something sane when the only problem is splitting data and functions locally into different components, why in the world would you assume that putting in a network layer, exponentially increasing the failure rates, loosing good stack traces, making refactorings that have to move data from one part of the architecture to another a pain in the butt (because requirements never change right?), etc... would be a good thing? The horrors I've seen... oh boy, the horrors. The people I've heard praising microservice architectures so far fall into two categories: (1) Ex developers that moved into management and read a few articles/bought a book on the subject and think it's a great idea (but have no actual practical experience) (2) developers that have just started working on a new one and are still in the honeymoon phase (mind you, not developers that have started working on an __existing__ one that is a few years old, those figure out the mess they are in pretty quickly). You can screw up not-so-micro services too, but it's far easier to screw up microservice architectures and they are far, far harder to fix down the line when you figure out how you messed up (or re-architect when that thing that the PM told you was never going to be a use case is going to become a use case (after they tell you they are sorry of course)).

andy_ppp 6y ago
Totally agree - I'm not against services but I say as few services as humanly possible! The issues with micro services are just so unbelievably large that it'd take too long to enumerate them here, but for me the largest issue is you are putting arbirary network partitions into your application and you CANNOT possibly know when these network partitions will suddenly make an operation you need to be atomic impossible (or at best eventually consistent). In general it is excruciatingly difficult to iterate on micro services as well. Then there is deployment, service mesh, k8s etc. etc. that you have to wrangle with, all because people refused to make bounded contexts in their application without making them into different services. It is much simpler to just design your monolith into separate contexts than it is to introduce all the error handling you need for network partitions. I personally don't understand why Elixir isn't more loved by these micro services zealots - it allows you to get extremely far without adding network partitions and then allows you to switch to message passing when you need and then networked message passing all extremely smoothly as it makes sense for your app.

deleted 6y ago

[deleted]

karmakaze 6y ago

This talks about the thing that makes microservices fail most often, getting the bounded contexts wrong. It doesn't go into enough detail to show you how to do it right but does tell you what to watch for when you're doing it wrong. The biggest, most common mistake is when a bounded context is called "<noun> service". This should be a huge red flag. Unfortunately, most books (and this post) always use this bad naming starting everyone down the bad path. At the very minimum call it a "<noun> management service" if it is the primary/general create/update activity. e.g you might have a checkout service that creates most orders and a order management service that can make drafts or adjustments. Order management (being the decoupled source of truth) might even be downstream from checkout service. Other services that need this information should get it's own copy asynchronously, i.e. decoupled. The best way to learn how to do bounded contexts right is to study ones that worked well. Most writing tries to describe this in an abstract way using placeholder widget service things that helps no one. The BFF pattern is good but doesn't solve everything. If you're accessing multiple services for a synchronous mutation, it's likely not highly reliable but can get by for an interactive service that can tolerate some errors. A messaging service is not one that should, they should always accept, store and forward with the lowest possible error rates.

dvancouvering 6y ago

I have done a lot of work with microservices, and I agree they come with a ton of cognitive overhead and new ways for things to fail in unexpected and painful ways. In particular it now requires software engineers to understand principles and design patterns for distributed systems that when I was first starting out were relegated to theory and university papers. Can you guarantee ordering? Can you guarantee exactly once? How do you get transactional semantics? How do you handle cascading failures? It's all a bit overwhelming. I have also seen however that monoliths at some point lock up an organization. The key measures of software delivery performance are deploy frequency, lead time from commit to running in production, mean time to recovery and change failure rate. Monoliths impact all of these when you engineering org starts scaling past a handful of teams. A monolith also becomes a single point of failure. One person in one team checks in bad code, and the whole system falls down. Microservices can give you resilience. You can also scale microservices independently - a CPU intensive service can get a bunch of pods, a memory intensive service gets a beefier memory allocation, and other services can run idle most of the time or even be deployed as FAAS functions that almost never run. So I agree it's not a technical solution as much as it is about scaling an organization. At WeWork the team shrank so much that we began consolidating small services back into a single service because the operational overhead of having separate services didn't make sense any more when a single team was responsible for all of them. I do think choosing to split out a new service is a big decision to make. You have to really think through the cost and benefits. You also need to make sure you have the right tools in place so you can detect and recover quickly. Things like log aggregation, SLOs defined, measured and alerted on, canary deploys, progressive delivery, etc. Without that you're running completely blind. I also think you want to be thoughtful about keeping your coupling low. If each service depends on twenty other services then in my mind you now just have a distributed monolith.

dmitryminkovsky 6y ago
> Can you guarantee ordering? Can you guarantee exactly once? How do you get transactional semantics? How do you handle cascading failures? It's all a bit overwhelming. I don’t advocate DDD/microservices but if you absolutely must go down this road, Kafka Streams really neatly answers all these questions. Ben Stopford’s book[0] and blog posts[1] are really great if you want to know more. [0] http://www.benstopford.com/2018/04/27/book-designing-event-driven-systems/ http://www.benstopford.com/2018/04/27/book-designing-event-d... [1] https://www.confluent.io/blog/building-a-microservices-ecosystem-with-kafka-streams-and-ksql/ https://www.confluent.io/blog/building-a-microservices-ecosy...

corpMaverick 6y ago

Microservices are a good way to scale up development teams. So if you have 16 developers you can partition the work in two teams with one microservice each. If you have 80 devs you can have 10 teams with one microservice each team. Use Conwayś law to your advantage. Use DDD to decide where the boundaries are. The advantage of having each team own a microservice is that each team can deploy their software independently from the rest of the teams. And it works as long as the APIs are honored. So Microservices are way to decouple the work of different teams allowing them to work independently from each other.

dakiol 6y ago
This only works for the simplest cases. If the two microservices must share data and direct calls (API calls) are not allowed (because they make your services highly coupled to each other to the point where if one of them ia down, then the other cannot work properly, or simply because your use case cannot tolerate the latency overhead) then each service must 1) replicate the data needed locally by listening to interesting events of the other service and b) emit interesting events so that the other service can update its local db. In other words, async communication. While this let's you scale up teams, it's a huge overhead. So, unless you have over, let's say, 500 devs, it's not worth it.
- corpMaverick 6y ago
  > This only works for the simplest cases. If the two microservices must share data and direct calls (API calls) are not allowed Actually, one assumption is that the microservices don't share data. Only API (Rest, SOAP, GraphQL, etc) calls are allowed. And the boundaries are created minimizing the coupling between microservices. > While this let's you scale up teams, it's a huge overhead. So, unless you have over, let's say, 500 devs, it's not worth it. As opposed to Monolith ? I think you will have problems collaborating in a Monolith at around 30 devs. What I am opposing is the idea that you can have 30 devs and 50 microservices. Which I have seen in real life.
rched 6y ago
Why do you need Microservices to create boundaries in your code? You trade the ability for teams to deploy independently for the ability to be able to make breaking changes. This is a good trade off in my experience and is better for team velocity. The approach we have taken is to keep a single codebase and deploy. We have applied the ideas from DDD to segment the code into contexts and assign ownership of each of those contexts to a team. This allows us to get some of the benefits of Microservices without losing all the benefits of a monolith. Having these boundaries in place also gives us the flexibility to move to Microservices if and when we decide we need to.

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

tveita 6y ago

I see the writer is from "WalmartLabs", so I assume this is advice for "Walmart scale" applications. For the rest of us it's probably good to remember that "you are not Walmart." > Now, we know that in any modern application, sacrificing availability is not a good idea either. I feel like this will apply to less than 10% of applications. Normally you'd rather deal with the occasional downtime than dig through the fallout of broken attempts at distributed transactions. Ask yourself "What amount of $X,000 are we losing per minute of downtime" and see if it rounds down to zero. > Another way to model this system is to separate, or group related models into separate microservices. In DDD, these models — Price, Priced Items, and Discounts — are called Aggregates. And this must be the case for less than 1% of readers - it would take a massive system before I'd consider having prices and discounts in separate databases. You're taking on the overhead of writing an additional service for what could have been a database join.