8 ms·

I think > "Isolate applications. Keep different applications away from other applications’ tables." is only good advice if the tables are application specifi

I think > "Isolate applications. Keep different applications away from other applications’ tables." is only good advice if the tables are application specific data and you don't do microservices in that stupid braindead way that makes it so that everything from the admin panel to data visualization are their own "applications" with their own databases and doing things that would be even the simplest of queries becomes a project in writing what are effectively bad performance joins via random http APIs. IE have a data model and understand where the most painless boundaries are, don't throw up dozens of DBs for the hell of it.

harlanji 6y ago
There’s a model of software based around shipping events around, and subscriptions between systems. The purposes of separation are at least a couple important, perhaps you know. Each has a DB, often embedded, that is suitable and materialized from the subscriptions and its own; mutated predictably.
- Glyptodon 6y ago
  Transmitting events is fine, I like transmitting and receiving events. But if you end up having a bunch of DBs with a lot of overlap I don't know how much you're gaining as it pretty much turns into another permutation of DB sharding. Of course microservices done wrong doesn't mean event-based systems done right are bad.
  - rualca 6y ago
    > But if you end up having a bunch of DBs with a lot of overlap I don't know how much you're gaining You're gaining component isolation and separation of concerns, and the ability to scale services horizontally without bothering about shared state. Also, you gain the ability to develop each system without having any constraint imposed by third parties. In other words, you're making your service not only simpler but also more robust, easier to operate and maintain.
    - danmaz74 6y ago
      Each individual microservice becomes simpler, but the whole system becomes much more complex than an equivalent monolith with a single DB. Microservices have their advantages, but simplicity and testability aren't part of them.
      - rualca 6y ago
        > Each individual microservice becomes simpler, but the whole system becomes much more complex than an equivalent monolith with a single DB. Indeed a distributed system forces developers to think through how dependencies communicate, which otherwise would be taken for granted. However, that misses the whole point of microservices. Services are peeled off monoliths because the monolith no longer meets operational requirements, which is trivial to provide with a distributed system. I'm referring to aspects such as reliability, or ceasing to be able to scale vertically. In addition, some people simply mindlessly develop monoliths not realizing they already have a distributed system, once we account for the fact that managed services and even running a database already opens that door wide open. > Microservices have their advantages, but simplicity and testability aren't part of them. Not really. Contary to your claim, microservices make systems more easy to test. The only thing that changes is that you're no longer able to drop a breakpoint in a request and track it all the way through a database. However, you get cleanly separated components with very crisp boundaries that make it trivial to test interactions. Perhaps the main cultural clash that is added by moving from a monolith to a distributed system is that developers start to need to thing things through when they put up an interface.
ianpurton 6y ago
> dont' do microservices in that stupid braindead way Best advice I've read on HN this year.
bpodgursky 6y ago
> even the simplest of queries becomes a project in writing what are effectively bad performance joins via random http APIs Don't forget the part where the queries are impossible to test, because you can't spin up real instances of all 15 APIs in a test environment, so all the HTTP calls are mocked and the responses are meaningless!
- rualca 6y ago
  > Don't forget the part where the queries are impossible to test, because you can't spin up real instances of all 15 APIs in a test environment, Why? What's stopping you? > so all the HTTP calls are mocked and the responses are meaningless! Why are you talking about mocks in the context of an integration tests?
  - bpodgursky 6y ago
    You're the one who said "integration test". A single DB query is normally something that _can_ be trivially unit-tested. But if the DB query is split across half a dozen complex services, suddenly it can _only_ be integration tested. Which usually means slow iteration and no local testing.
    - rualca 6y ago
      > You're the one who said "integration test". Yes, I was. And that's the problem, because it seems you didn't realize that tests that involve checking how 15 APIs interact are indeed tests that check how different services are integrated. Do you disagree with anything? > A single DB query is normally something that _can_ be trivially unit-tested. Well, not really. You unit test the call to the repository, not the call to the DB. Calls to the DB are tested in integration tests. Think about it. > But if the DB query is split across half a dozen complex services, Aren't you trying to tests how half a dozen complex services integrate? I mean, take a step back and read your own description of the problem. As soon as you talk about interacting "complex services" is already a clear tell that you're way beyond unit testing territory. > Which usually means slow iteration and no local testing. No, not really. It still means you have to add unit tests that cover how your individual clients and repositories and business logic works under the expected scenarios of a distributed system, and after you pass all of those you check how your service interacts with other services in integration tests.
JMTQp8lwXL 6y ago
> doing things that would be even the simplest of queries becomes a project in writing what are effectively bad performance joins via random http APIs In some ways, is GraphQL trying to slap a fresh coat of paint on what is otherwise is a bad problem under the hood? At my employer, we're trying to adopt it, but we've never addressed the underlying performance issues of the microservices, and now the slowness is quite apparent with the GraphQL wrapper tier. This isn't a diss to GraphQL. The queries are awesome to write. The developer experience of writing / being the client of a GraphQL server is extremely excellent. But I feel I might not be alone in a journey that is playing out to be a mediocre implementation because we are avoiding the core problem.
- an_opabinia 6y ago
  It does beg the question, if the developer authors a GraphQL server to do joins, why didn't the application just do that in the first place? Is it always a symptom of a dead piece of engineering?
  - rienbdj 6y ago
    The alternative is an explosion of REST API routes as each application has different requirements. It is much simpler to have one graphql endpoint that is flexible enough for all consumers.
    - GordonS 6y ago
      I've seen the explosion of REST endpoints you mention, several times. But isn't GraphQL just adding a thin veneer over that, giving the appearance of a single URL, but actually kind of containing all those REST routes under the hood? If you have an explosion of REST routes, you'll likely end up with an explosio6of GraphQL queries too.
      - valenterry 6y ago
        No. It contains only a subset of the rest routes and allows to combine them efficiently. That is what avoids the explosion. You don't have to add any new "rest route" equivalent in graphQL if some user just wants a new combination of data but doesn't want to make 10 requests to get it.
rualca 6y ago
> is only good advice if the tables are application specific data Isn't that the whole point? > and you don't do microservices in that stupid braindead way that (...) I feel that this cynical jab is not only set as a flimsy straw man but also adds absolutely nothing to the discussion. Independent services need to have their own independent database. Simple. Each service has its bounded context, persists it in its database, and just runs. It matters nothing what data the bounded context holds or what your opinions are on microservices.
- jayd16 6y ago
  A lot of posters here seem to have been deeply burned from microservices designed along the wrong lines. I mean, sure, it happens. You're going to make mistakes just like you can misjudge how to separate concerns in a set of classes. It shouldn't be an issue to fix it. Maybe some teams focus on pure separation before they have a solid design? Maybe its just a culture of feeling like service boundaries can never change? I don't really get it either.
  - etripe 6y ago
    I think what we're seeing here is Conway's law in action. Most places I've worked don't define their bounded contexts based on separation of concerns, but rather on separation of reporting. That is: team A has been given arbitrary stewardship of generic app functions T, U, V based on management purview. Team B has been given stewardship of generic functions W, X, Y, Z. So the bounded context is along those lines. But wait, both V and W are related to billing, you say? No matter! Since the cliché is you can't solve people problems with technology, the same applies to micro-services: it's much harder to convince managers to shift accountability than it is to move the code from one micro-service to another. Even if you do move the code, you get to have your shit and eat it, too.
    - rualca 6y ago
      > I think what we're seeing here is Conway's law in action. Most places I've worked don't define their bounded contexts based on separation of concerns, but rather on separation of reporting. You're not wrong. I mean, one of the goals of microservices is to peel off responsibilities to a service that can be owned and managed by a small team. However, what I've been seeing in these anti-microservices rant is developers struggling to acknowledge the need to reason with loosely coupled interfaces, reason about partial system failure modes that were always there in monoliths but we're either ignored or turned into critical system failures, and the microservices tax.
nine_k 6y ago
If you need microservice architecture, it's likely because either the size or the rate of change of your data prevent you from using a single RDBMS with synchronous transactions. So your microservices talk to something like Cassandra. You might want some of the advantages of microservices without yet reaching this scale. Then you likely have most microservices stateless, and can admit you have a single database where you can do joins (but not updates) across microservice borders.
- deleted 6y ago
  [deleted]
- sixdimensional 6y ago
  CQRS

harlanji 6y ago

There’s a model of software based around shipping events around, and subscriptions between systems. The purposes of separation are at least a couple important, perhaps you know. Each has a DB, often embedded, that is suitable and materialized from the subscriptions and its own; mutated predictably.

Glyptodon 6y ago
Transmitting events is fine, I like transmitting and receiving events. But if you end up having a bunch of DBs with a lot of overlap I don't know how much you're gaining as it pretty much turns into another permutation of DB sharding. Of course microservices done wrong doesn't mean event-based systems done right are bad.
- rualca 6y ago
  > But if you end up having a bunch of DBs with a lot of overlap I don't know how much you're gaining You're gaining component isolation and separation of concerns, and the ability to scale services horizontally without bothering about shared state. Also, you gain the ability to develop each system without having any constraint imposed by third parties. In other words, you're making your service not only simpler but also more robust, easier to operate and maintain.
  - danmaz74 6y ago
    Each individual microservice becomes simpler, but the whole system becomes much more complex than an equivalent monolith with a single DB. Microservices have their advantages, but simplicity and testability aren't part of them.
    - rualca 6y ago
      > Each individual microservice becomes simpler, but the whole system becomes much more complex than an equivalent monolith with a single DB. Indeed a distributed system forces developers to think through how dependencies communicate, which otherwise would be taken for granted. However, that misses the whole point of microservices. Services are peeled off monoliths because the monolith no longer meets operational requirements, which is trivial to provide with a distributed system. I'm referring to aspects such as reliability, or ceasing to be able to scale vertically. In addition, some people simply mindlessly develop monoliths not realizing they already have a distributed system, once we account for the fact that managed services and even running a database already opens that door wide open. > Microservices have their advantages, but simplicity and testability aren't part of them. Not really. Contary to your claim, microservices make systems more easy to test. The only thing that changes is that you're no longer able to drop a breakpoint in a request and track it all the way through a database. However, you get cleanly separated components with very crisp boundaries that make it trivial to test interactions. Perhaps the main cultural clash that is added by moving from a monolith to a distributed system is that developers start to need to thing things through when they put up an interface.

ianpurton 6y ago

> dont' do microservices in that stupid braindead way Best advice I've read on HN this year.

bpodgursky 6y ago

> even the simplest of queries becomes a project in writing what are effectively bad performance joins via random http APIs Don't forget the part where the queries are impossible to test, because you can't spin up real instances of all 15 APIs in a test environment, so all the HTTP calls are mocked and the responses are meaningless!

rualca 6y ago
> Don't forget the part where the queries are impossible to test, because you can't spin up real instances of all 15 APIs in a test environment, Why? What's stopping you? > so all the HTTP calls are mocked and the responses are meaningless! Why are you talking about mocks in the context of an integration tests?
- bpodgursky 6y ago
  You're the one who said "integration test". A single DB query is normally something that _can_ be trivially unit-tested. But if the DB query is split across half a dozen complex services, suddenly it can _only_ be integration tested. Which usually means slow iteration and no local testing.
  - rualca 6y ago
    > You're the one who said "integration test". Yes, I was. And that's the problem, because it seems you didn't realize that tests that involve checking how 15 APIs interact are indeed tests that check how different services are integrated. Do you disagree with anything? > A single DB query is normally something that _can_ be trivially unit-tested. Well, not really. You unit test the call to the repository, not the call to the DB. Calls to the DB are tested in integration tests. Think about it. > But if the DB query is split across half a dozen complex services, Aren't you trying to tests how half a dozen complex services integrate? I mean, take a step back and read your own description of the problem. As soon as you talk about interacting "complex services" is already a clear tell that you're way beyond unit testing territory. > Which usually means slow iteration and no local testing. No, not really. It still means you have to add unit tests that cover how your individual clients and repositories and business logic works under the expected scenarios of a distributed system, and after you pass all of those you check how your service interacts with other services in integration tests.

JMTQp8lwXL 6y ago

> doing things that would be even the simplest of queries becomes a project in writing what are effectively bad performance joins via random http APIs In some ways, is GraphQL trying to slap a fresh coat of paint on what is otherwise is a bad problem under the hood? At my employer, we're trying to adopt it, but we've never addressed the underlying performance issues of the microservices, and now the slowness is quite apparent with the GraphQL wrapper tier. This isn't a diss to GraphQL. The queries are awesome to write. The developer experience of writing / being the client of a GraphQL server is extremely excellent. But I feel I might not be alone in a journey that is playing out to be a mediocre implementation because we are avoiding the core problem.

an_opabinia 6y ago
It does beg the question, if the developer authors a GraphQL server to do joins, why didn't the application just do that in the first place? Is it always a symptom of a dead piece of engineering?
- rienbdj 6y ago
  The alternative is an explosion of REST API routes as each application has different requirements. It is much simpler to have one graphql endpoint that is flexible enough for all consumers.
  - GordonS 6y ago
    I've seen the explosion of REST endpoints you mention, several times. But isn't GraphQL just adding a thin veneer over that, giving the appearance of a single URL, but actually kind of containing all those REST routes under the hood? If you have an explosion of REST routes, you'll likely end up with an explosio6of GraphQL queries too.
    - valenterry 6y ago
      No. It contains only a subset of the rest routes and allows to combine them efficiently. That is what avoids the explosion. You don't have to add any new "rest route" equivalent in graphQL if some user just wants a new combination of data but doesn't want to make 10 requests to get it.

rualca 6y ago

> is only good advice if the tables are application specific data Isn't that the whole point? > and you don't do microservices in that stupid braindead way that (...) I feel that this cynical jab is not only set as a flimsy straw man but also adds absolutely nothing to the discussion. Independent services need to have their own independent database. Simple. Each service has its bounded context, persists it in its database, and just runs. It matters nothing what data the bounded context holds or what your opinions are on microservices.

jayd16 6y ago
A lot of posters here seem to have been deeply burned from microservices designed along the wrong lines. I mean, sure, it happens. You're going to make mistakes just like you can misjudge how to separate concerns in a set of classes. It shouldn't be an issue to fix it. Maybe some teams focus on pure separation before they have a solid design? Maybe its just a culture of feeling like service boundaries can never change? I don't really get it either.
- etripe 6y ago
  I think what we're seeing here is Conway's law in action. Most places I've worked don't define their bounded contexts based on separation of concerns, but rather on separation of reporting. That is: team A has been given arbitrary stewardship of generic app functions T, U, V based on management purview. Team B has been given stewardship of generic functions W, X, Y, Z. So the bounded context is along those lines. But wait, both V and W are related to billing, you say? No matter! Since the cliché is you can't solve people problems with technology, the same applies to micro-services: it's much harder to convince managers to shift accountability than it is to move the code from one micro-service to another. Even if you do move the code, you get to have your shit and eat it, too.
  - rualca 6y ago
    > I think what we're seeing here is Conway's law in action. Most places I've worked don't define their bounded contexts based on separation of concerns, but rather on separation of reporting. You're not wrong. I mean, one of the goals of microservices is to peel off responsibilities to a service that can be owned and managed by a small team. However, what I've been seeing in these anti-microservices rant is developers struggling to acknowledge the need to reason with loosely coupled interfaces, reason about partial system failure modes that were always there in monoliths but we're either ignored or turned into critical system failures, and the microservices tax.

nine_k 6y ago

If you need microservice architecture, it's likely because either the size or the rate of change of your data prevent you from using a single RDBMS with synchronous transactions. So your microservices talk to something like Cassandra. You might want some of the advantages of microservices without yet reaching this scale. Then you likely have most microservices stateless, and can admit you have a single database where you can do joins (but not updates) across microservice borders.

deleted 6y ago
[deleted]
sixdimensional 6y ago
CQRS