4 ms·
SQL is indeed a powerful too. But it's just one of many. Just today I used: SQL to dump data to csv, python to modify it, and gRPCurl to upload it to another se
by gigatexal 7y ago
SQL is indeed a powerful too. But it's just one of many. Just today I used: SQL to dump data to csv, python to modify it, and gRPCurl to upload it to another service. Different tools, different problems, use the right one.
That being said ... RDBMs still suffer from scale and there are many efforts to fix this with sharding to disaggregating the data layer from the query layer with examples like AWS's Aurora and GCP's BigQuery etc -- in all likelihood data will likely live in many different places and will need to be adapted into things like a datastore or queried from things like Hive and Presto, or even a traditional ETL + EDW setup -- my point being there's no one thing that will solve everything as most single solutions break down when the number of rows is measured in the billions or more.
- nomel 7y ago> SQL to dump data Is this another way to say you use SQL to query specific data from a larger dataset?
- gigatexal 7y agoYeah sorry. I used the cli front end to my database from a container and had it dump the results to a file that I then massaged with python
- chrisjc 7y agoI imagine they mean SQL to unload from whatever SQL storage engine out to blob(s3,azure,etc), HDFS or local-file system. COPY INTO s3://mybucket/foobar/ from (select a, b, c from ...);