3 ms·
I worked at an organization that ran an energy market and we had a massive amount of data stored in SQL databases and this was how you did large-scale data anal
by 7thaccount 2y ago
I worked at an organization that ran an energy market and we had a massive amount of data stored in SQL databases and this was how you did large-scale data analysis. Most electrical engineers there knew the schema backwards and forwards and could write multi page queries (there are a LOT of tables and many many join operations) pretty quick to go through TB, probably at least a PB of data. It wasn't considered a niche skill...more like bread and butter (equally useful as Python). Every now and then we'd hire someone that had a "data science" background that wouldn't know SQL (just some stats and R or Python+Pandas) that would try to convince us we were doing things wrong, but it was generally because the "big data" they learned in school could fit into a dataframes. I'd say SQL is now more important than ever.