3 ms·

We need to stop using Stored Procedures

2 points by snikolaev 19d ago

gregw2 18d ago
Just learn how to use Liquibase (or flyway, or... ) in your git repo and CICD pipeline deployments alongside your code. I did it with production Scala apps over a decade ago. I even built sproc TDD test suites.
- joeblubaugh 15d ago
  Besides CI, what benefits were you getting from writing the logic as a stored procedure?
  - murphomatic 15d ago
    I have this question as well .. granted there are a few targeted cases where SPs make sense, I generally agree with the headache perspective of SPs - such as: the business logic that inevitably ends up in SPs - how do you go about unit testing that in isolation, and particularly from a behavioral standpoint?
  - mgkimsal 15d ago
    I haven't looked in to this in ... decades, but I seem to remember stored procs executed measurably faster 20+ years ago. That was one reason a project I was on pushed them. And the benchmarks they showed... it wasn't unreasonable. But we have faster networks, more memory, ssd, etc - I'm unsure if speed is a big reason any more. IIRC cached execution plans was a big win back then, but may not be as a win today. I'm thinking about sybase and mssql from the late 90s when I was first getting in to this topic. But... over the years, I've seen push to sprocs where the logic was never version controlled or subject to the same level of testing the regular app source code was. The DBA folks had total control and were treated as a separate class of engineer than mere 'app' engineers, which caused all sorts of problems...
joeblubaugh 15d ago
Maybe they’re not properly called sprocs, but there’s at least one good use for logic stored in the database: data retention. I like CREATE EVENT / pg_cron for trimming append-only tables when the data has aged out.
saxenaabhi 15d ago
This is very poorly written and is wrong on basic facts. > So what does a stored procedure get us? > Absolutely nothing! Well, I mean, headache for one. > ... we have to deploy migrations to update our queries, and we have to run diff migration_for_my_sproc migration_for_my_sproc_n to see how things changed 1) You can version sql functions in your repo alongside your code and deploy sql function alongside your db migrations(even in the same transaction). Have one file per sql function and you can also compute checksums to speed it up if like me you have a repo with 600 stored procedures. 2) With stored procedures you have no need to to db.startTransaction on server when executing multiple statements and wait for db round trips. That's often the biggest reason for preferring stored procedures. 3) I have seen systems in healthcare/finance where different teams have no access to underlying tables and the db only exposes sql procedures. Every-time a procedure is called it also adds a log entry to an audit table. Databases are a great piece of technology! Learning how to use them properly can have huge payoff in terms of business value generation. EDIT: not mentioned in this article but people often mention testing difficulties with stored procedures. You can have normal vitest tests testing your postgres functions with in-memory pglite.
- jeremyjh 15d ago
  > I have seen systems in healthcare/finance where different teams have no access to underlying tables and the db only exposes sql procedures. Every-time a procedure is called it also adds a log entry to an audit table. For a large system that has many different teams working on it, it is better to have a core API layer with clear ownership, than to not have one. But why would you choose to build that with database stored procedures? If this was built 25+ years ago, then that is all the answer that is needed.
  - saxenaabhi 15d ago
    Why wouldn't you use stored procedures for it? For internal teams why is it better to have a API? I can see usecases in which API could make sense, but it doesn't matter in most cases. SQL already has authorization/authentication built in. For rate limiting you can use something like planetscale's traffic control.
deleted 15d ago
[deleted]
calvinmorrison 15d ago
I am uneasy about databases being used in general how they are today. Nobody writes SQL to read and write records directly. it's all pumped through business logic. Then you write a stored procedure to update x-on-y and the business logic breaks, in a scenario that is impossible for the business logic to produce. additionally the logic is very far away from the data in many cases, obfuscated through layers of data modelling. If there was a way to bring these closer, that would be nice.
est 15d ago
I think stored procedures are just strong typed "serverless lambdas" which runs really, really close do your data storage.
- degamad 15d ago
  Yes, this! The ancient wisdom which advocated for stored procedures, which modern developers find distasteful, were really advocating for microservices close to your data, which encapsulated security, business logic, and data persistence so that multiple consumers could share the same data without repeating the logic and code. The fact that some people write those microservices in PL/SQL and some in JavaScript doesn't change the relevance of the encapsulation.
  - jeremyjh 15d ago
    But PL/SQL is a horrifying language. You have barely any facilities for modularity, encapsulation or composition. Every procedure is just one name in a huge flat name space. Development tools are primitive. The language is absurdly verbose - it was state of the art in the 1970s when COBOL reigned supreme.
    - est 15d ago
      This is an implementation problem, you can introduce a JIT'd language plus some "container" with "versioned deployments" in modern DB clusters.
- laughing_man 15d ago
  That's the theory. I never ran across a project that used stored procedures which didn't have a code base split between stored procedures and whatever server language, with no rhyme or reason for what goes where. It's just a maintenance headache.
dxxvi 15d ago
I don't have any concrete examples at this moment to show the advantages of using stored procedures. So, suppose that we have to update 2 tables A, B if data in table C satisfies some conditions. Then stored procedures allow us to send only 1 request to the db while with the update commands we have to send at least 2 requests.
JeffRosenberg 15d ago
> our application code and DB access are separately versioned, our sprocs can change right under our feet from aberrant (i.e. extremely rare, insane) DBAs Your database (and its migrations) should be source controlled along with the rest of your code. Problem solved, now you can take advantage of some of the legitimate benefits stored procedures have to offer!
lowbloodsugar 15d ago
Apparently “we” need to learn how to use them properly.
noddingham 15d ago
(In our on-prem SQL server environment) We expose stored procedures to web developers like an API contract to ensure a level of integrity and robustness in how they interact with the database. We find SQL knowledge among web developers these days is a toss up and I'd rather have the DBA design the schema and provide stored procedures.
bunderbunder 15d ago
I used to be pro-sproc, but I agree that making queries part of the database schema can be burdensome and creates some development friction that I’d rather avoid. I’ve mostly moved on to storing queries as *.sql files in the application’s repository. You still get the good query caching and predictable plans. But you also get some other nice perks, like queries and the code that uses it being versioned together, and making it easy for developers to test queries in a SQL console. Most editors even have plugins that give you autocompletion and basic error checking in exchange for a connection string to the dev database. That latter bit is why I don’t love inline SQL in string literals or ORMs’ querying DSLs. Both discourage tinkering with queries to observe how they work. I believe that’s a major reason why it’s so common for developers to commit awful performance sins like computing aggregations on the client side. It’s hard to expect people to get comfortable with SQL window functions when the project’s database interaction is set up in a way that actively discourages doing so.
hbarka 15d ago
This is an opinion where the author is obviously coming from an application developer’s bias. If you’re a data engineer you love the idea of stored procedures. A data engineer is often more comfortable putting logic closer to the data, especially for bulk transformations, set-based processing, ETL/ELT, data quality rules, and involving large volumes of data. Doing this in an application just to process them would be inefficient.
mkotlikov 15d ago
Tell that to Spacetime DB which achieves amazing performance by making everything a stored procedure!
twellborn 14d ago
[flagged]

2 points by snikolaev 19d ago

Just learn how to use Liquibase (or flyway, or... ) in your git repo and CICD pipeline deployments alongside your code. I did it with production Scala apps over a decade ago. I even built sproc TDD test suites.

joeblubaugh 15d ago
Besides CI, what benefits were you getting from writing the logic as a stored procedure?
- murphomatic 15d ago
  I have this question as well .. granted there are a few targeted cases where SPs make sense, I generally agree with the headache perspective of SPs - such as: the business logic that inevitably ends up in SPs - how do you go about unit testing that in isolation, and particularly from a behavioral standpoint?
- mgkimsal 15d ago
  I haven't looked in to this in ... decades, but I seem to remember stored procs executed measurably faster 20+ years ago. That was one reason a project I was on pushed them. And the benchmarks they showed... it wasn't unreasonable. But we have faster networks, more memory, ssd, etc - I'm unsure if speed is a big reason any more. IIRC cached execution plans was a big win back then, but may not be as a win today. I'm thinking about sybase and mssql from the late 90s when I was first getting in to this topic. But... over the years, I've seen push to sprocs where the logic was never version controlled or subject to the same level of testing the regular app source code was. The DBA folks had total control and were treated as a separate class of engineer than mere 'app' engineers, which caused all sorts of problems...

joeblubaugh 15d ago

Maybe they’re not properly called sprocs, but there’s at least one good use for logic stored in the database: data retention. I like CREATE EVENT / pg_cron for trimming append-only tables when the data has aged out.

saxenaabhi 15d ago

This is very poorly written and is wrong on basic facts. > So what does a stored procedure get us? > Absolutely nothing! Well, I mean, headache for one. > ... we have to deploy migrations to update our queries, and we have to run diff migration_for_my_sproc migration_for_my_sproc_n to see how things changed 1) You can version sql functions in your repo alongside your code and deploy sql function alongside your db migrations(even in the same transaction). Have one file per sql function and you can also compute checksums to speed it up if like me you have a repo with 600 stored procedures. 2) With stored procedures you have no need to to db.startTransaction on server when executing multiple statements and wait for db round trips. That's often the biggest reason for preferring stored procedures. 3) I have seen systems in healthcare/finance where different teams have no access to underlying tables and the db only exposes sql procedures. Every-time a procedure is called it also adds a log entry to an audit table. Databases are a great piece of technology! Learning how to use them properly can have huge payoff in terms of business value generation. EDIT: not mentioned in this article but people often mention testing difficulties with stored procedures. You can have normal vitest tests testing your postgres functions with in-memory pglite.

jeremyjh 15d ago
> I have seen systems in healthcare/finance where different teams have no access to underlying tables and the db only exposes sql procedures. Every-time a procedure is called it also adds a log entry to an audit table. For a large system that has many different teams working on it, it is better to have a core API layer with clear ownership, than to not have one. But why would you choose to build that with database stored procedures? If this was built 25+ years ago, then that is all the answer that is needed.
- saxenaabhi 15d ago
  Why wouldn't you use stored procedures for it? For internal teams why is it better to have a API? I can see usecases in which API could make sense, but it doesn't matter in most cases. SQL already has authorization/authentication built in. For rate limiting you can use something like planetscale's traffic control.

deleted 15d ago

[deleted]

calvinmorrison 15d ago

I am uneasy about databases being used in general how they are today. Nobody writes SQL to read and write records directly. it's all pumped through business logic. Then you write a stored procedure to update x-on-y and the business logic breaks, in a scenario that is impossible for the business logic to produce. additionally the logic is very far away from the data in many cases, obfuscated through layers of data modelling. If there was a way to bring these closer, that would be nice.

est 15d ago

I think stored procedures are just strong typed "serverless lambdas" which runs really, really close do your data storage.

degamad 15d ago
Yes, this! The ancient wisdom which advocated for stored procedures, which modern developers find distasteful, were really advocating for microservices close to your data, which encapsulated security, business logic, and data persistence so that multiple consumers could share the same data without repeating the logic and code. The fact that some people write those microservices in PL/SQL and some in JavaScript doesn't change the relevance of the encapsulation.
- jeremyjh 15d ago
  But PL/SQL is a horrifying language. You have barely any facilities for modularity, encapsulation or composition. Every procedure is just one name in a huge flat name space. Development tools are primitive. The language is absurdly verbose - it was state of the art in the 1970s when COBOL reigned supreme.
  - est 15d ago
    This is an implementation problem, you can introduce a JIT'd language plus some "container" with "versioned deployments" in modern DB clusters.
laughing_man 15d ago
That's the theory. I never ran across a project that used stored procedures which didn't have a code base split between stored procedures and whatever server language, with no rhyme or reason for what goes where. It's just a maintenance headache.

dxxvi 15d ago

I don't have any concrete examples at this moment to show the advantages of using stored procedures. So, suppose that we have to update 2 tables A, B if data in table C satisfies some conditions. Then stored procedures allow us to send only 1 request to the db while with the update commands we have to send at least 2 requests.

JeffRosenberg 15d ago

> our application code and DB access are separately versioned, our sprocs can change right under our feet from aberrant (i.e. extremely rare, insane) DBAs Your database (and its migrations) should be source controlled along with the rest of your code. Problem solved, now you can take advantage of some of the legitimate benefits stored procedures have to offer!

lowbloodsugar 15d ago

Apparently “we” need to learn how to use them properly.

noddingham 15d ago

(In our on-prem SQL server environment) We expose stored procedures to web developers like an API contract to ensure a level of integrity and robustness in how they interact with the database. We find SQL knowledge among web developers these days is a toss up and I'd rather have the DBA design the schema and provide stored procedures.

bunderbunder 15d ago

I used to be pro-sproc, but I agree that making queries part of the database schema can be burdensome and creates some development friction that I’d rather avoid. I’ve mostly moved on to storing queries as *.sql files in the application’s repository. You still get the good query caching and predictable plans. But you also get some other nice perks, like queries and the code that uses it being versioned together, and making it easy for developers to test queries in a SQL console. Most editors even have plugins that give you autocompletion and basic error checking in exchange for a connection string to the dev database. That latter bit is why I don’t love inline SQL in string literals or ORMs’ querying DSLs. Both discourage tinkering with queries to observe how they work. I believe that’s a major reason why it’s so common for developers to commit awful performance sins like computing aggregations on the client side. It’s hard to expect people to get comfortable with SQL window functions when the project’s database interaction is set up in a way that actively discourages doing so.

hbarka 15d ago

This is an opinion where the author is obviously coming from an application developer’s bias. If you’re a data engineer you love the idea of stored procedures. A data engineer is often more comfortable putting logic closer to the data, especially for bulk transformations, set-based processing, ETL/ELT, data quality rules, and involving large volumes of data. Doing this in an application just to process them would be inefficient.

mkotlikov 15d ago

Tell that to Spacetime DB which achieves amazing performance by making everything a stored procedure!

twellborn 14d ago

[flagged]