4 ms·

I know, right? It's also perfectly possible to split those things up, and to have them share the same database. This causes microservice extremists' heads to

I know, right? It's also perfectly possible to split those things up, and to have them share the same database. This causes microservice extremists' heads to pop off..

withinboredom 4y ago
We do this at work with multiple entry points. New engineers sometimes don’t even know it exists until the 6th month when they eventually run into shenanigans.
- cheradenine_uk 4y ago
  Hehe "shenanigans" ;-) The truth is, most things can be made to work - though there are various trade-offs ("shenanigans") involved. e.g: In my current project, there's a bit that loads large-ish files, parses/converts the data, and loads them (order of tens of thousands of rwos) into SQL tables. For reasons that best escape me - since after the files are parsed the entire dataset is _literally_ in memory, instead of doing a bulk sql update there- and then, thousands of messages are enqueued to be consumed by a practically unlimited number of lambda functions that then... bomb out as the SQL database hits it's connection limit and keels over. I guess those shenanigans have better buzzword compliance!
  - withinboredom 4y ago
    Sounds like one of my favorite shenanigans when the stakeholders tell you it needs to handle billions, so you engineer it that way. Reality: it needs to handle thousands. Sounds like you got the reverse shenanigans.
    - nicoburns 4y ago
      Except that there's only one database, you it wouldn't even billions. In fact, I suspect it will handle significantly less load than a batched model will. I've been able to get 100x speedups by batching SQL queries.
      - withinboredom 4y ago
        I miss the days of just one database sometimes… Batching is the way to go, 100%. Remove the network as much as you possibly can.
  - toomanydoubts 4y ago
    You should really put a connection manager in between your lambda functions and database, like HAProxy or AWS RDS Proxy. (still, that doesn't change the fact the code is badly designed)
WJW 4y ago
It's possible, but also a really poor idea. The whole point of microservices is to separate concerns. Reintegrating those concerns through the database is an excellent way to run into problems. Changes to the database schema will alter how all the services that touch it work, and now you have to manage schema migrations across multiple services and probably multiple teams. So much nicer to expose a documented and versioned API (and/or published event stream), so that the consumers of data your service manages have some flexibility at which moment to migrate to the new schema.

I know, right? It's also perfectly possible to split those things up, and to have them share the same database. This causes microservice extremists' heads to pop off..

withinboredom 4y ago

We do this at work with multiple entry points. New engineers sometimes don’t even know it exists until the 6th month when they eventually run into shenanigans.

cheradenine_uk 4y ago
Hehe "shenanigans" ;-) The truth is, most things can be made to work - though there are various trade-offs ("shenanigans") involved. e.g: In my current project, there's a bit that loads large-ish files, parses/converts the data, and loads them (order of tens of thousands of rwos) into SQL tables. For reasons that best escape me - since after the files are parsed the entire dataset is _literally_ in memory, instead of doing a bulk sql update there- and then, thousands of messages are enqueued to be consumed by a practically unlimited number of lambda functions that then... bomb out as the SQL database hits it's connection limit and keels over. I guess those shenanigans have better buzzword compliance!
- withinboredom 4y ago
  Sounds like one of my favorite shenanigans when the stakeholders tell you it needs to handle billions, so you engineer it that way. Reality: it needs to handle thousands. Sounds like you got the reverse shenanigans.
  - nicoburns 4y ago
    Except that there's only one database, you it wouldn't even billions. In fact, I suspect it will handle significantly less load than a batched model will. I've been able to get 100x speedups by batching SQL queries.
    - withinboredom 4y ago
      I miss the days of just one database sometimes… Batching is the way to go, 100%. Remove the network as much as you possibly can.
- toomanydoubts 4y ago
  You should really put a connection manager in between your lambda functions and database, like HAProxy or AWS RDS Proxy. (still, that doesn't change the fact the code is badly designed)

WJW 4y ago

It's possible, but also a really poor idea. The whole point of microservices is to separate concerns. Reintegrating those concerns through the database is an excellent way to run into problems. Changes to the database schema will alter how all the services that touch it work, and now you have to manage schema migrations across multiple services and probably multiple teams. So much nicer to expose a documented and versioned API (and/or published event stream), so that the consumers of data your service manages have some flexibility at which moment to migrate to the new schema.