4 ms·
SQL+Python is extremely powerful, and the author makes some good points (multivariate regression should be done in python and not SQL for example), but the quer
by oarabbus_ 7y ago
SQL+Python is extremely powerful, and the author makes some good points (multivariate regression should be done in python and not SQL for example), but the query example in the blog is not a good one. Every modern DB has aggregate and statistical functions like ntile, percent_Rank, median, min, max, etc. I’d honestly rather run these functions against the database than do it in python. Especially if you're working with billion+ row datasets. In fact if all I'm doing is these kinds of statisical/aggregate functions then I definitely wouldn't use python at all.
- n4r9 7y agoSQL Server doesn't make it easy to calculate median values. There are a number of slightly convoluted ways to do it but it's far from straightforward and usually I end up calculating it in C# if I can.
- bshipp 7y agoI wonder if people do timings for the variances between a function in the DB vs Python? Unless your queries are so small (like individual records) that a round-trip to the DB is impacted by network lag, I find it's almost always faster to ask the DB to do as much of the lifting as possible. The downside, of course, is that your code potentially becomes unreadable, harder to migrate between database backends, and more difficult to debug.
- teej 7y agoPushing computation back into the database has been a major trend in the last few years. Tools like DBT have made that transition easy in terms of clean code and testable SQL.