3 ms·

I have found that its more the dogma to put up microservices for everything that spoils the pot. My take on it is that we should have called is "distributed se

I have found that its more the dogma to put up microservices for everything that spoils the pot. My take on it is that we should have called is "distributed services" because micro implies that they have to be small. Usually there are clear borders in your app of stuff that can me compartementalized, but they usually should be called medium services instead. Also - your database is a microservice - design your data acces layer accordingly, you dont need an api endpoint, where a stored procedure (or the like) will do.

KronisLV 4y ago
> My take on it is that we should have called is "distributed services" because micro implies that they have to be small. > Usually there are clear borders in your app of stuff that can me compartmentalized, but they usually should be called medium services instead. These are excellent points and probably loosely coincide with how many teams you might have working on different parts of the greater system as well. > ...you dont need an api endpoint, where a stored procedure (or the like) will do. I'm not sure about this. Outside of niche cases and domains (like batch processes), stored procedures are harder to develop, harder to test, harder to debug, harder to audit (or at least less convenient to, when you want to ship your logs with something like GELF) and also have worse tooling for working with them, ensuring code style rules and so on. Just look at the reality in many places: https://www.jetbrains.com/lp/devecosystem-2021/databases/#Databases_do-you-debug-stored-procedures https://www.jetbrains.com/lp/devecosystem-2021/databases/#Da... Do you debug stored procedures? 47% Never 44% Rarely 9% Frequently Do you have tests in your database? 14% Yes 70% No 15% I don't know Do you keep your database scripts in a version control system? 54% Yes 37% No 9% I don't know Do you write comments for the database objects? 49% No 27% Yes, for many types of objects 24% Yes, only for tables How would you feel about code that about half of people have never debugged properly, almost tree quarters of people don't bother testing, only about half uses version control for and about half doesn't even document? Because as far as I can tell, this matches my experience when working with the majority of databases out there, either the development culture or tooling (or the entire architecture of those) isn't there yet, and it might never be. While database design should be a major consideration that begs attention, I personally maintain that the majority of the functionality should go into the app, regardless of whether you use an ORM or not. An exception to this would be using plenty of database views to make querying easier and putting consideration towards how one could handle enums and such, so you don't end up with a database that's not possible to understand without reading the app source code in another window.
- jhoelzel 4y ago
  While I see what you mean, I think, personally, the point is moot and also applies to any scripting language. If you have someone in your org specialized on SQL debugging stored procedures is their job. By now there are also many sufficient tools, that will help you actually debug the code and it has been a first class citizen in visual studio for years now. Also keep in mind, that most APIs are crud, not complex businness logic and that there is a middle ground: build your own libraries instead of microservices and share those. This of course all is heavily debateable and only based on my personal experience, but your statistics are missing the value for the amount of people who are actually "good at sql". Also a lot of people dont understand the value of relational data at all. So yeah, if you never learned to code you cant program. Sql is not just a second class citizien of your stack that "stores stuff" and of course you have to optimize your use cases, but again this is exactly what i mean with my top comment: you sir are dogmatic too :D SQL is AMAZING and can be directly conected to your frontend. That does not mean its a smart thing to do.
  - KronisLV 4y ago
    There are some pretty good points in your comment and I'll agree that context is key here - the project, the team and even particular tech choices will have a lot of impact on what exactly you'll be dealing with and what might be the correct tool for the job. > This of course all is heavily debatable and only based on my personal experience, but your statistics are missing the value for the amount of people who are actually "good at sql". That said, one cannot just optimize for people who are good at a particular technology and instead one needs to look at the market averages, for the locale that they're in (unless they have that many choices of what culture to work in in the first place). In my case, where I am, we are still mostly on board with ACID and RDBMSes, however DBAs are, while not exactly a fleeting career, on the downturn (probably about 20 years away from being as obsolete as "GlassFish Server Manager", much like the DevOps trend digs into Ops engineer positions). Sure, most people are okay with running EXPLAIN PLAN occasionally, but putting significant amounts of logic in the DB is usually a pain to deal with. I've seen first hand how much of a dumpster fire a system that has like 90% of the logic in the DB eventually gets: stored procedures for rendering forms, stored procedures for data validations, stored procedures for persisting data, with absolutely ALL of the issues that I mentioned before. Making changes in it was a mess, figuring out which of the hundred or so packages even needed the changes was a pain, the naming was nonsensical (due to length limitations for DB objects), it's like someone tried coding in a language that wasn't meant for writing proper code. Of course, I've also seen some bad projects in the more traditionally tiered architectures, but none were as bad as the DB-centric design. That's my anecdotal experience, on which I'll base most of my future tech stack decisions. Of course, sometimes in-database processing is just the right way to go about things, especially in something like PostGIS where you don't have many options app-side for handling certain kinds of operations on the data, or need to do them without N+1 issues or too much network I/O due to latency.

KronisLV 4y ago

> My take on it is that we should have called is "distributed services" because micro implies that they have to be small. > Usually there are clear borders in your app of stuff that can me compartmentalized, but they usually should be called medium services instead. These are excellent points and probably loosely coincide with how many teams you might have working on different parts of the greater system as well. > ...you dont need an api endpoint, where a stored procedure (or the like) will do. I'm not sure about this. Outside of niche cases and domains (like batch processes), stored procedures are harder to develop, harder to test, harder to debug, harder to audit (or at least less convenient to, when you want to ship your logs with something like GELF) and also have worse tooling for working with them, ensuring code style rules and so on. Just look at the reality in many places: https://www.jetbrains.com/lp/devecosystem-2021/databases/#Databases_do-you-debug-stored-procedures https://www.jetbrains.com/lp/devecosystem-2021/databases/#Da... Do you debug stored procedures? 47% Never 44% Rarely 9% Frequently Do you have tests in your database? 14% Yes 70% No 15% I don't know Do you keep your database scripts in a version control system? 54% Yes 37% No 9% I don't know Do you write comments for the database objects? 49% No 27% Yes, for many types of objects 24% Yes, only for tables How would you feel about code that about half of people have never debugged properly, almost tree quarters of people don't bother testing, only about half uses version control for and about half doesn't even document? Because as far as I can tell, this matches my experience when working with the majority of databases out there, either the development culture or tooling (or the entire architecture of those) isn't there yet, and it might never be. While database design should be a major consideration that begs attention, I personally maintain that the majority of the functionality should go into the app, regardless of whether you use an ORM or not. An exception to this would be using plenty of database views to make querying easier and putting consideration towards how one could handle enums and such, so you don't end up with a database that's not possible to understand without reading the app source code in another window.

jhoelzel 4y ago
While I see what you mean, I think, personally, the point is moot and also applies to any scripting language. If you have someone in your org specialized on SQL debugging stored procedures is their job. By now there are also many sufficient tools, that will help you actually debug the code and it has been a first class citizen in visual studio for years now. Also keep in mind, that most APIs are crud, not complex businness logic and that there is a middle ground: build your own libraries instead of microservices and share those. This of course all is heavily debateable and only based on my personal experience, but your statistics are missing the value for the amount of people who are actually "good at sql". Also a lot of people dont understand the value of relational data at all. So yeah, if you never learned to code you cant program. Sql is not just a second class citizien of your stack that "stores stuff" and of course you have to optimize your use cases, but again this is exactly what i mean with my top comment: you sir are dogmatic too :D SQL is AMAZING and can be directly conected to your frontend. That does not mean its a smart thing to do.
- KronisLV 4y ago
  There are some pretty good points in your comment and I'll agree that context is key here - the project, the team and even particular tech choices will have a lot of impact on what exactly you'll be dealing with and what might be the correct tool for the job. > This of course all is heavily debatable and only based on my personal experience, but your statistics are missing the value for the amount of people who are actually "good at sql". That said, one cannot just optimize for people who are good at a particular technology and instead one needs to look at the market averages, for the locale that they're in (unless they have that many choices of what culture to work in in the first place). In my case, where I am, we are still mostly on board with ACID and RDBMSes, however DBAs are, while not exactly a fleeting career, on the downturn (probably about 20 years away from being as obsolete as "GlassFish Server Manager", much like the DevOps trend digs into Ops engineer positions). Sure, most people are okay with running EXPLAIN PLAN occasionally, but putting significant amounts of logic in the DB is usually a pain to deal with. I've seen first hand how much of a dumpster fire a system that has like 90% of the logic in the DB eventually gets: stored procedures for rendering forms, stored procedures for data validations, stored procedures for persisting data, with absolutely ALL of the issues that I mentioned before. Making changes in it was a mess, figuring out which of the hundred or so packages even needed the changes was a pain, the naming was nonsensical (due to length limitations for DB objects), it's like someone tried coding in a language that wasn't meant for writing proper code. Of course, I've also seen some bad projects in the more traditionally tiered architectures, but none were as bad as the DB-centric design. That's my anecdotal experience, on which I'll base most of my future tech stack decisions. Of course, sometimes in-database processing is just the right way to go about things, especially in something like PostGIS where you don't have many options app-side for handling certain kinds of operations on the data, or need to do them without N+1 issues or too much network I/O due to latency.