3 ms·
In data science it has been an uphill battle between the people who only think csv exists and those who switch to a new nosql flavor every 6 months
by mnky9800n 7y ago
In data science it has been an uphill battle between the people who only think csv exists and those who switch to a new nosql flavor every 6 months
- overcast 7y agoCSV/Excel has run the world since the beginning, that's going to be a never ending battle. Large orgs are still entrenched into SQL when their enterprise ERP/MRP systems run off Oracle, and their analysts are extracting reports to Excel.
- tartoran 7y agoRight after flat files which are still in use on mainframes.
- prepend 7y agoI think it’s artificially presented as an exclusive option between sql and csv/nosql/whatever in data science. I know plenty of people who are comfortable in both. But I don’t know any “data science” people as good at sql as a good dba. But that’s ok. I think a good analysis requires both sql and some flat file, if only for reproducibility. I’ve encountered many point in time analyses that can’t be recreated because they were just sql against db and there’s no way to reproduce the result months later. Are their methods accurate? Or auditable? It’s much harder because the source data is changed. So I like talking about only-sqlers about the importantance of incorporating the data management of extracts and archives into the overall solution. And doing so in a way that doesn’t try to redesign the database trying to avoid the need for users to file away point in time snapshots of data.