3 ms·
I think if your SQL becomes spaghetti, it is time to fetch the data out of the database and then process it. SQL databases aren't a data-science studio that len
by systemvoltage 4y ago
I think if your SQL becomes spaghetti, it is time to fetch the data out of the database and then process it. SQL databases aren't a data-science studio that lends itself for extreme processing capabilities just because it is fast. Furthermore, it is the last thing that scales typically. Relational processing is what its good at. Doing things like bayesian/statistical analysis on the data in SQL is asking for trouble.
- setr 4y agoRelational databases are perfect for data analysis scenarios, especially when it fits in a single server. Why are you pumping data over the network, and serializing/deserializing it from a table structure usually back into a table structure, when you could just process it , often using only the data in the datastore, and pipe the final computed results over? Hell they even have a name for databases specialized for this usecase — OLAP The problem is that databases, and SQL, are such a pain to extend and define sufficiently general operations that you don’t just have scipy in the database, and thus your complicated analytics have to be fully specified with the few rudimentary analytic functions that are actually provided to you.
- ttfkam 4y agohttps://madlib.apache.org/ https://madlib.apache.org/ https://docs.timescale.com/timescaledb/latest/how-to-guides/hyperfunctions/function-pipelines/ https://docs.timescale.com/timescaledb/latest/how-to-guides/... https://duckdb.org/2022/05/04/friendlier-sql.html https://duckdb.org/2022/05/04/friendlier-sql.html It's a shame that so many people regard SQL using the reference SQL92 spec when in fact that it is continually evolving and improving like any other language still in use.