4 ms·

Wayyyy back in the day, after the dotcom crash, I got moved from a SWE role in my company to the consulting (customer implementation) side to try to bring more

Wayyyy back in the day, after the dotcom crash, I got moved from a SWE role in my company to the consulting (customer implementation) side to try to bring more rigor to their process. One of the first things I did was replace a several thousand line stored procedure full of pivots, transforms, cursors, etc. with a few hundred lines of code. As a bonus, performance improved by a couple orders of magnitude as well.

forgetfulness 3y ago
Knowing what to process in memory, what to delegate to the database, to the datawarehouses, or other heavyweight data-processing engines like Spark, is its own subfield, data engineering. Finding data engineers that can actually do it will become difficult in just a bit, as there's a goldrush to take on that role, and lots of people want in with some rudimentary knowledge of SQL and Python. Anyway, I'm always suspicious of people advocating for stored procedures, because those are version controlled if you're lucky, and I've yet to see them subject to automated testing.
- javcasas 3y ago
  Random google search for "migrations stored procedures" https://stackoverflow.com/questions/14139445/code-first-migrations-and-stored-procedures#16589152 https://stackoverflow.com/questions/14139445/code-first-migr... shows that EF6 already supports them. And pgTap as an example of testing them https://pgtap.org/ https://pgtap.org/ . What I'm suspicious is that, having seen untested stored procedures, you haven't bothered to try unit test them. I mean, you can do a lot with BEGIN; set db in good state; call procedure; ROLLBACK; but you have to try.
  - forgetfulness 3y ago
    That's an unmaintained Perl library, I highly doubt it sees significant use. I haven't seen untested stored procedures in years, because I don't use them, no team I've been in uses them. Complex prepared statements, that's common, but stored procedures, no.

forgetfulness 3y ago

Knowing what to process in memory, what to delegate to the database, to the datawarehouses, or other heavyweight data-processing engines like Spark, is its own subfield, data engineering. Finding data engineers that can actually do it will become difficult in just a bit, as there's a goldrush to take on that role, and lots of people want in with some rudimentary knowledge of SQL and Python. Anyway, I'm always suspicious of people advocating for stored procedures, because those are version controlled if you're lucky, and I've yet to see them subject to automated testing.

javcasas 3y ago
Random google search for "migrations stored procedures" https://stackoverflow.com/questions/14139445/code-first-migrations-and-stored-procedures#16589152 https://stackoverflow.com/questions/14139445/code-first-migr... shows that EF6 already supports them. And pgTap as an example of testing them https://pgtap.org/ https://pgtap.org/ . What I'm suspicious is that, having seen untested stored procedures, you haven't bothered to try unit test them. I mean, you can do a lot with BEGIN; set db in good state; call procedure; ROLLBACK; but you have to try.
- forgetfulness 3y ago
  That's an unmaintained Perl library, I highly doubt it sees significant use. I haven't seen untested stored procedures in years, because I don't use them, no team I've been in uses them. Complex prepared statements, that's common, but stored procedures, no.