2 ms·
Knowing what to process in memory, what to delegate to the database, to the datawarehouses, or other heavyweight data-processing engines like Spark, is its own
by forgetfulness 3y ago
Knowing what to process in memory, what to delegate to the database, to the datawarehouses, or other heavyweight data-processing engines like Spark, is its own subfield, data engineering.
Finding data engineers that can actually do it will become difficult in just a bit, as there's a goldrush to take on that role, and lots of people want in with some rudimentary knowledge of SQL and Python.
Anyway, I'm always suspicious of people advocating for stored procedures, because those are version controlled if you're lucky, and I've yet to see them subject to automated testing.
- javcasas 3y agoRandom google search for "migrations stored procedures" https://stackoverflow.com/questions/14139445/code-first-migrations-and-stored-procedures#16589152 https://stackoverflow.com/questions/14139445/code-first-migr... shows that EF6 already supports them. And pgTap as an example of testing them https://pgtap.org/ https://pgtap.org/ . What I'm suspicious is that, having seen untested stored procedures, you haven't bothered to try unit test them. I mean, you can do a lot with BEGIN; set db in good state; call procedure; ROLLBACK; but you have to try.
- forgetfulness 3y agoThat's an unmaintained Perl library, I highly doubt it sees significant use. I haven't seen untested stored procedures in years, because I don't use them, no team I've been in uses them. Complex prepared statements, that's common, but stored procedures, no.