4 ms·

lol good luck without centralized data

hot_gril 3y ago
Each service is authoritative on its own data, and it works neatly. Any medium to large size system is going to have natural places to separate that, or else will buckle under complexity or even hardware constraints if you try to keep it all in one DB. I've seen it repeatedly.
- corethree 3y ago
  Nah this doesn't work out. Because requirements are so chaotic and modular design isn't an axiomatic science things will for sure go wrong with the design both because you got the design wrong and because the requirements shifted to a point where another design was better. You encapsulated data under one service and you find that it's actually utilized much more in another service. This happens a lot. The longer you can centralize your storage the better and easier everything is.
  - aledalgrande 3y ago
    Exactly. And having more than one source of truth for your data complicates a lot of things. Where I work we run a massive MySQL database that has a pretty simple sharding rule, so I don't believe that you need to split the database even at scale.
    - hot_gril 3y ago
      It's not primarily about the size of the DB, it's about what's in it. If your massive MySQL DB is only used by one team, then that's fine. If separate teams' services are speaking through shared tables on it, that becomes a mess of an API easily.
      - aledalgrande 3y ago
        We have thousands of developers on it. I’m not saying it’s a silver bullet, but it works and it works well. Also, the service writing on it is only one (monolith).
        hot_gril 3y ago
        Depending on what the service is, I can see how this would work fine, but I don't think this is a common case.
  - leetharris 3y ago
    Except I've seen this work at dozens of companies? This thread is very painful, feels like CS101 coming in to tell us why we don't need relational databases and that we can just shove it all inside mongo.
    - corethree 3y ago
      It's not the thread it's most likely you. I don't know if people tell you but dealing with you is likely to be extremely painful if you say rude garbage like this to peoples faces. Multiple databases can "work" but you get the problem I described above. You know when you go to a company and eventually you see idiot practices or technical debt? Unfortunately this belongs to that category. You seeing it at dozens of companies doesn't validate anything except for the fact that bad patterns are common. It also validates the existence of people who often do things poorly and don't know it (aka you).
      - hot_gril 3y ago
        Multiple DBs does work, and large orgs do it this way with good results. But I'm not here to insult people for disagreeing. It's just software, nothing to get heated over.
        corethree 3y ago
        No worries. You didn't call the thread painful. That kind of comment causes heat. It doesn't belong here. Of course it "works". But you can do better. Splitting data into several dbs when one db suffices is called "extra work". Good results can arise from "extra work". But the extra work wasn't necessary. You have to think like this: Sure multiple dbs work. But would 1 db in it's place work just as well? The top commenter here from Netflix one of the people who promoted the whole microservices thing is telling startups to go monolith for a reason.
        hot_gril 3y ago
        Our large org has been familiar with the 1 DB approach for over a decade. At some point the need for microservices became clear, and years after that, they actually switched. Even our subteam is feeling the need to divide up our service because its scope has kept expanding. Startups shouldn't start with microservices. I've worked for (or led) a few different ones, and I've always created a monolith while avoiding other kinds of premature scaling (sometimes even "folder, bro" is overkill). Doesn't mean I'd run things that way with 200 people owning systems with evolving requirements for 10+ years, or even three separate dev teams of 5 people.
jedberg 3y ago
If you have centralized data then you have broken the abstraction and are in for a world of hurt. You shouldn't need centralized data, just a service front door where that service is the master of that part of the data.
- aledalgrande 3y ago
  So what are you gonna do when you need data migrations? PS: I'm not saying using one DB with microservices, it's still with the monolith
  - leetharris 3y ago
    Migrations per service are exactly why you separate them out in the first place. Reading your comments in this thread has been painful. If you have multiple things sharing one data store, having ownership over the structure of the data is very difficult.
    - aledalgrande 3y ago
      Did you read the PS? According to your condescending tone you did not.
  - hot_gril 3y ago
    What kind of data migration did you have in mind, and what's the problem with it?
threeseed 3y ago
You mean like having user authentication data in AD/LDAP and everything else in an RDBMS. Because this is the standard architecture you will find for most platforms.
gotimo 3y ago
inb4 the separate "data consistency service"

Each service is authoritative on its own data, and it works neatly. Any medium to large size system is going to have natural places to separate that, or else will buckle under complexity or even hardware constraints if you try to keep it all in one DB. I've seen it repeatedly.

corethree 3y ago
Nah this doesn't work out. Because requirements are so chaotic and modular design isn't an axiomatic science things will for sure go wrong with the design both because you got the design wrong and because the requirements shifted to a point where another design was better. You encapsulated data under one service and you find that it's actually utilized much more in another service. This happens a lot. The longer you can centralize your storage the better and easier everything is.
- aledalgrande 3y ago
  Exactly. And having more than one source of truth for your data complicates a lot of things. Where I work we run a massive MySQL database that has a pretty simple sharding rule, so I don't believe that you need to split the database even at scale.
  - hot_gril 3y ago
    It's not primarily about the size of the DB, it's about what's in it. If your massive MySQL DB is only used by one team, then that's fine. If separate teams' services are speaking through shared tables on it, that becomes a mess of an API easily.
    - aledalgrande 3y ago
      We have thousands of developers on it. I’m not saying it’s a silver bullet, but it works and it works well. Also, the service writing on it is only one (monolith).
      - hot_gril 3y ago
        Depending on what the service is, I can see how this would work fine, but I don't think this is a common case.
- leetharris 3y ago
  Except I've seen this work at dozens of companies? This thread is very painful, feels like CS101 coming in to tell us why we don't need relational databases and that we can just shove it all inside mongo.
  - corethree 3y ago
    It's not the thread it's most likely you. I don't know if people tell you but dealing with you is likely to be extremely painful if you say rude garbage like this to peoples faces. Multiple databases can "work" but you get the problem I described above. You know when you go to a company and eventually you see idiot practices or technical debt? Unfortunately this belongs to that category. You seeing it at dozens of companies doesn't validate anything except for the fact that bad patterns are common. It also validates the existence of people who often do things poorly and don't know it (aka you).
    - hot_gril 3y ago
      Multiple DBs does work, and large orgs do it this way with good results. But I'm not here to insult people for disagreeing. It's just software, nothing to get heated over.
      - corethree 3y ago
        No worries. You didn't call the thread painful. That kind of comment causes heat. It doesn't belong here. Of course it "works". But you can do better. Splitting data into several dbs when one db suffices is called "extra work". Good results can arise from "extra work". But the extra work wasn't necessary. You have to think like this: Sure multiple dbs work. But would 1 db in it's place work just as well? The top commenter here from Netflix one of the people who promoted the whole microservices thing is telling startups to go monolith for a reason.
        hot_gril 3y ago
        Our large org has been familiar with the 1 DB approach for over a decade. At some point the need for microservices became clear, and years after that, they actually switched. Even our subteam is feeling the need to divide up our service because its scope has kept expanding. Startups shouldn't start with microservices. I've worked for (or led) a few different ones, and I've always created a monolith while avoiding other kinds of premature scaling (sometimes even "folder, bro" is overkill). Doesn't mean I'd run things that way with 200 people owning systems with evolving requirements for 10+ years, or even three separate dev teams of 5 people.

jedberg 3y ago

If you have centralized data then you have broken the abstraction and are in for a world of hurt. You shouldn't need centralized data, just a service front door where that service is the master of that part of the data.

aledalgrande 3y ago
So what are you gonna do when you need data migrations? PS: I'm not saying using one DB with microservices, it's still with the monolith
- leetharris 3y ago
  Migrations per service are exactly why you separate them out in the first place. Reading your comments in this thread has been painful. If you have multiple things sharing one data store, having ownership over the structure of the data is very difficult.
  - aledalgrande 3y ago
    Did you read the PS? According to your condescending tone you did not.
- hot_gril 3y ago
  What kind of data migration did you have in mind, and what's the problem with it?

threeseed 3y ago

You mean like having user authentication data in AD/LDAP and everything else in an RDBMS. Because this is the standard architecture you will find for most platforms.

gotimo 3y ago

inb4 the separate "data consistency service"