11 ms·
Is a shared database in microservices actually an anti-pattern?
- deleted 8y ago[deleted]
- jzoch 8y agoThis is more likely to be a problem if you split your services up by noun (Users,Orders) rather than by function (makeOrder, loginService, etc). This isn't bulletproof either but I've seen it helps reduce this occurrence a lot. Pardon the poor examples
- ChicagoDave 8y agoIn other words, data store (domain), behavior (createOrder), and event (orderCreated). I feel the OPs pain but I’ve built MSAs from DDD principals and it really is a better architecture for complex systems.
- karmakaze 8y agoThis is the most important point when making microservices. There's lots of talk about bounded contexts but very little about how to draw the boundaries. Most are superficial and repeat bad examples like OrderService (order is both a verb and boun, OrderingService sounds ok). Using two part service names (ContentComposition, MessageDelivery) usually works out well. As to the shared db aspect, while spitting a miniservice into micro ones I had this situation. All the code was split first and the db changes being the most difficult to was done last as a point.of we don't expect to revert this.choice. We weren't at a performance limit so that was fine, any schema changes had to be clearly communicated and coordinated. In all not too bad. I don't think I'd want to leave it in that state as a normal state. The point of micro is Independence and isolation of changes and sharing a db leaves a sensitive area. Don't let that stop you if it's only a temporary state. Just get commitment as to how temporary that is, since in absolute terms everything's temporary.
- sigi45 8y agoWhy would you make Miniservices?
- karmakaze 8y agoWe never had the intention to make miniservices. Some were microservices that acquired abilities that grew to become separate microservices. On other projects they were prototypes that lasted long enough and became small monoliths to get split up.
- taeric 8y agoMy take is yes. Shared infrastructure is always a risk. It seems to encourage other bad choices. Yes, you can design around it. Good discipline around libraries with schemas, and such. These are almost always more code than letting a service own all communication with an infrastructure.
- taffer 8y agoCompletely agree with the article. If each piece of code has its own data store, you lose all the advantages of the DBMS: you have to handwrite your joins, your transaction system and get all sorts of problems with cache invalidation, data inconsistencies and n+1 fetching. Encapsulation is important, but there are better ways to do it, such as using the authorization system of your DBMS.
- ChicagoDave 8y agoThe only thing that needs a relational data store are reports. Operational data stores should only be concerned with their own domain and expose behaviors and events. There are exceptions, but if you’re building a complex system, monolithic architectures have a known lifetime while MSA’s tend to mitigate long term coderot. It’s hard to see this until you’ve built domain driven micro service based systems properly.
- taffer 8y ago> monolithic architectures have a known lifetime while MSA’s tend to mitigate long term coderot I'd say it's exactly the other way around: the programming languages and technologies change every five years or so, but the data stays forever, so why couple the data with the technology du jour? I'd rather have my data managed in one place so I can adapt the technology layers that sit on top whenever requirements change, rather than writing joins and transactions in my application code. I'd rather not bother with all the complexity and keep it simple as long as I don't need an Amazon or Netflix scale.
- ChicagoDave 8y agoBecause business complexity modeled relationally becomes stale over time, reduces agility, and develops massive amounts of code rot/tech debt. Relational databases are tools to be used where it makes sense. NoSQL tables are tools. Graph databases are tools. Configuration files are tools. Given your statement, Linux’s entire configuration footprint should be in a relational database. As any good architect will say in response to a design question, “It depends.” MSA’s aren’t a panacea either. Event Streams are an interesting development and new patterns are likely to emerge. But today? I’d focus on reducing complexity with domain-driven design and choosing patterns that enable agility and support separation of concerns. MSA fits very well into that philosophy.
- eweise 8y ago"Then it sends request to users service to get missing information about users" What information would an order need to know about the user except for its ID? Shipping address? That could have been stored in the order db. Usually there is an orchestration layer above these services such as graphql, that merges all the data together for presenting to the UI. Somewhere there needs to be a defined contract between service boundaries so that internals can be changed without affecting all dependent systems. The db is not a good place for this contract.
- ris 8y agoMicroservices are an anti-pattern.
- he0001 8y agoI’d say microservices for the sake of microservices is an anti pattern. Microservices does have their usages.
- HelloNurse 8y agoI don't understand the problem. If the report needs order details and user details, and current data isn't available because some piece of the system has a problem, there's going to be no report today regardless of what the database is like and what concerns are separated or not. C'est la vie. Moreover, the real dependency is from the report to the orders and the users; the article takes for granted that the report is shoehorned into the orders component, but it's clearly arbitrary.
- ozim 8y agoLooks like author does not know you can split business domain without making microservices...
- pritambarhate 8y agoYes, shared database is an anti-pattern in micro services architecture. If one shared database can serve your system well then you don't need microservices. You should build a monolith instead. The main reason for using microservices is scale. First ability to scale development teams and secondly ability to scale the infrastructure. With microservices you get vertical sharding out of the box. Yes it means dealing with eventual consistency for at least a few usecases. But any system that wants to serve millions of concurrent users needs to deal with eventual consistency as some sort of vertical or horizontal sharding is necessary at that scale anyways.
- dehrmann 8y ago> The main reason for using microservices is scale What enables this scale is the encapsulation you get with microservices. You lose this with a shared database. You get a similar problem with multitenancy because "isolated" microservices are impacted by what unrelated services do, so there's a loss of resiliency.
- eweise 8y agoThe issue isn't about microservices, its about isolating dependencies between various components of the system. When any part of the system can reach into the user table then you've created a big ball of mud. Even with a monolith, you should have a defined contract between the user service and the rest of the system. That way you can start with a monolith and move to microservices later as needed. You can also change the underlying persistence layer without disrupting the rest of the system. You could for instance start with mongodb for user information while other system components use postgres. If sometime later you think postgres handle json just as well, no problem switch user over to postgres and the rest of the system is unaffected. You should really build your monolith in a similar fashion to microservices, just leave out the remoting layer.
- ebiester 8y agoFor Insert/update/delete, that makes sense. However, many times it makes sense to just do the join in the database rather than make an n^2 join in user space. This becomes problematic in collection cases - 100 cases belonging to 70 users. It's a quick left hand join but a slowdown in application space.
- rejschaap 8y agoTLDR: How to get all of the disadvantages of microservices and none of the benefits
- alanfranz 8y agoShared database is an antipattern. Microservices can be an antipattern by themselves, btw.
- sigi45 8y agoIf you don't even need to split out the database you probably don't need Microservices.
- jrochkind1 8y agoWhat if microservices are actually an anti-pattern?
- acdha 8y agoAnti-pattern is strong but I’d go with premature optimization because most developers grossly overestimate their ability to create the correct architecture ahead of time and underestimate the performance, maintenance, reliability, and security costs. Most of the times I’ve seen a $$$ app struggle to match 90s single-server app performance it’s been because the team was struggling under the weight of unnecessary architecture.
- barrkel 8y agoIf your problem could be solved with buy rather than build, for services that you buy (e.g. could conceivably exist via AWS in your perfect world), then microservices may be a smart idea - if you can fork off a team to own that ideal service, you can eat the integration costs because you don't need to deal with the complexities of keeping that service running, it doesn't add to the total cost of ownership of getting your code running in production, it's just another remote API with an SLA. In that situation, the other services sharing your database is madness. Of course they shouldn't use your database. If they did, it would mean you're responsible for their load; you'd need to balance your needs with their needs. When your database goes down, their service and its clients (that you don't need to know about) would also go down. No good at all. Don't do that. And if you're not in a situation where different teams are responsible for different microservices, you probably shouldn't be using microservices.
- etaioinshrdlu 8y agoIf you have database that can scale its QPS linearly with number of machines in the cluster, maybe it stops being a bad idea? That is basically what is happening when using a service such as S3 or many distributed databases...
- mbrodersen 8y agoDon't ask if X is an "anti-pattern". Ask if X makes sense for YOUR project in YOUR context.
- eouw0o83hf 8y agoYes, yes it is.
- thoman23 8y agoMy understanding is that the point of microservices isn't to just move what would have been a relational join in SQL to an equivalent join in the service layer, which is what the author is implying. Instead, the order service should already have the subset of user data it needs to perform its function. If a user name changes, for example, that would trigger an event that gets consumed by a handler in the order domain. You embrace some denormalization and duplication of data in order for each service to have as much of what it needs to perform its function without a ton of other dependencies. I'm not saying this is necessarily the way to go, I'm just saying that this is my reading of the microservices design pattern.
- rorykoehler 8y agoThis is exactly correct and it took me a good while to understand this due to many people either not implementing this pattern or not knowing about it and then giving their 2 cents on microservices. Event sourcing and CQRS is the way to go.
- FrankyHollywood 8y agoyou're right, I'm in the middle of a big project where the lead engineer decided 'data duplication is forbidden'. The mess it created is terrible, both in complexity and in performance. I can't imagine microservice architecture will ever work in a complex domain without every service having most of its dependent data in its own store. What would help if people don't look at it as 'duplication' but rather 'caching'.
- cynicalreason 8y agoWithout sounding condescending, but as a very late 'bloomer' - only working in development/software for 3 years and aiming towards an application architect role - I always thought this was the consensus.
- Izkata 8y agoIt's similar to the "unit" in "unit testing": people will relate the most basic meaning to something they already understand, resulting in different, subtle distinctions. In this case the breakdown actually seems the same to me as with the common unit testing misunderstanding - breaking down by syntactic form instead of conceptual form.
- deleted 8y ago[deleted]
- est 8y agoDatabase is just another service which does not speak protobuf. Use it as a dependency or use multiple ones depending on your context.
- quizotic 8y agoThere's a bit of an emperor's clothes problem with microservices. If microservices are never combined together, they are essentially monoliths under a cooler name. But as soon as you combine them, you run into the same problems that are solved by traditional shared database systems. Here are two: cross-microservice referential integrity, and cross-microservice transaction coordination, but there are plenty more. Take the Orders/Customers example of the article. Assume the Customers microservice has some notion of identity, and that the Orders microservice contains a "foreign" reference to Customers identity. If the microservices are truly independent and autonomous, then the Customer microservice should be able to change its scheme for Customer identities, or a delete a Customer. But then what happens to the Orders that reference that now old and outdated Customer identity? If changing the Customer microservice breaks the Orders microservice, then what's the point of the separate encapsulation? Traditional shared database systems have ways to deal with this kind of referential integrity. As far as I can tell, this issue is ignored by microservice architectures. As is the larger issue of cross-microservice constraints (of which referential integrity is just one instance). The thing that really gets my goat is the lack of cross-microservice transaction coordination. In its place, we get all sorts of hand-waving about "eventual consistency" and "compensating transactions". Growl. Eventual consistency has a meaning that's related to distributed/replicated storage and the CAP theorem. Maybe its principles can apply to microservices, but the onus is on the microservice proponents to actually connect those dots. And most importantly, eventual consistency is implemented and supported by the DBMS, not by application developers. The thought that each microservice will independently implement the transactional guarantees of consistency and isolation should fill everyone with dread. That job should belong to the overarching system, not to individual microservices. It's too hard to get right. So for now, shared database in microservicew is not an anti-pattern. It may be the only workable pattern. When microservice frameworks grow up and offer the capabilities of shared database systems to microservice developers, then we can talk about anti-patterns.
- ZenPsycho 8y agoI'm just an ignorant nobody, but why would you split a microservice off along a facet that would require transaction and referential integraty? Wouldn't you want to pick off pieces that could be truly independent and not need to care about transactions or referential integrity? otherwise, what problem are you actually solving by splitting the piece off?
- ChicagoDave 8y agoI’ll take the hit on my karma and just suggest readers of this thread read the comments of the original post on hackernoon. There’s a clear demonstration that MSA and domain oriented data stores are based on a decade of service oriented architecture design and development.
- zahreeley 8y agoNo
- gigatexal 8y agoA shared database offers the allure of transactions and persistent state management. But using a traditional RDBMS is not very scalable. Schema changes have hurt us many times. And then what of foreign keys and such. It is nice to be able to consistently open a single transaction and get an answer from the one true source of truth but it quickly becomes the single biggest bottleneck. I prefer the database per service model and leaving a global transaction manager to be implemented if needed. I think it adds much more in flexibility and scalability.
- suff 8y agoYes, because any artificial linkage that prevents one service from scaling independently from another violates the separation of concerns principal.
- siliconc0w 8y agoOne pattern is to subscribe to an event bus and retain a read-only replica of your dependency's data. So the CustomerService publishes a CustomerModify operation which is picked up by the OrderService which then knows the new value of the customer's shipping address. If CustomerService wants to make backwards incompatible changes to its schema it should be handled by versioning so the CustomerService has to broadcast both versions of its message until such a time as all subscribers are using the updated version and then it can 'deprecate' the old message version. Ideally OrderService is using a library provided by CustomerService to handle applying CustomerModify messages so the next time OrderService is deployed (which should be continuously right?) it automatically picks up the new schema so this is a painless and automatic affair.
- stunt 8y agoThe real answer: It depends. But since most of the devs and architects fail to identify when you are allowed to do it, the rule of thumb is to not do it as it is a safer option.
- barrystaes 8y ago> Is a shared database in microservices actually an anti-pattern? No, unless you dont want (or care about having) a single source of truth. If state is distributed its harder to backup/restore/rewind or even query atomically and reproducibly. You could still make the single database sharded and duplicated though, but still: in most cases its still one shared database. Even storing Files outside of a DB is just sharding, important thing is the DB refers to the file and still is the single source of truth. And when you want vertical segmentation see designs like multi-tenant, but its still not a database per microsevice: quite the opposite. When you dont want a single source of truth you could do without it ofcourse.
- LaserToy 8y agoFrom my expirience, at some scale (system and organization) it became a easier to solve technical issues of having multiple DBs than dealing with bottlenecks on 1 shared DB. We had a fleet mucroservices that used 3 semishared DBs (1 for 1 domain, 1 for another domain and 1 shared). It worked ok for a year and now we run 40 DBs. It is harder to maintain and harder to develop against, but we are not constantly stuck in redeploying half of the stack because scheme just changed. So, no silver bullet. You need to use something that makes sense, not what is fancy today.
- virgakwolfw 8y agoNobody splits databases because of encapsulation. DBMSes have encapsulation functionality built in. Databases are split because a single DB instance (Oracle RAC explodes and causes billions in lost sales) cannot support the workload. Most other reasons is because of fashion without understand Why. Most companies don’t have this problem. They’re inventing shiny problems and come up with excuses to work on them.
- Oras 8y agoIf you can't split your databases then maybe you don't have a correctly established domain? There is no point of doing microservices when you still have tightly coupled data and you don't know how to split them efficiently. Monolith is not bad if it is well structured, tested and maintainable. Otherwise, you'll have to change multiple services for adding a new field of data.