4 ms·

No offense, but you sounds like someone who doesn't 'get' SQL. I'm reasonable at Excel. I've been using it in anger for close to 20 years. My most serious dres

No offense, but you sounds like someone who doesn't 'get' SQL. I'm reasonable at Excel. I've been using it in anger for close to 20 years. My most serious dressing down I've every received at work was related to a column of incorrect data in Excel. (I built a spreadsheet based on incorrect data. Management expected me to build correct spreadsheets from incorrect data.) I'm reasonable at SQL. I've been using it casually for 10 years. My cutoff at this point is that I'll use Excel if I expect my analysis to take less than five minutes. I'll use pandas or SQL for anything more complicated than that, because it's that much more powerful. Excel stops scaling at about a million rows. SQL stops scaling when your indices no longer fit in RAM.

enitihas 7y ago
I have written a lot of SQL over the years, and consider myself to be reasonably well versed with SQL and relational databases. SQL doesn't allow you to act on your data while seeing it. The interaction modes are totally different. In SQL: Type query -> See some results -> Repeat Excel: Everything in front of your eyes. You are manipulating live data. That is not something you do in SQL. (SQL doesn't mean much though, as it is a standard and not a tool. Maybe you had some specific SQL based tooling in mind) > Excel stops scaling at about a million rows. A large amount of real world data has less than a million rows. Every tool will stop at some point. At that point the benefits of Excel fade away, even if it could handle the data, as 1M rows are too large for our brain to make a sense of live. >My cutoff at this point is that I'll use Excel if I expect my analysis to take less than five minutes A lot of real world analysis takes 5 minutes. Of course, Excel is not perfect, and I too will never use Excel for anything that has to be done repeatedly (pandas or similar solutions are perfect here), but for quick and dirty data exploration, Excel is very good.
liamwestray 7y ago
Excel is hands down the best financial analysis and modeling tool ever made. SQL is not really comparable here. Any attempt to use it as a database or for data analysis should be met with extreme caution. As soon as someone suggests a VB macro for anything they should be shot. Excel is not the right tool for those kids of jobs. Python and R have just destroyed excel for data analysis and statistics (and most visualizations now) and are free. You can’t go back once getting used to that flexibility. Jupyter Notebooks reproducibility and traceability put the nail in the coffin for excel as a data analysis tool.

enitihas 7y ago

I have written a lot of SQL over the years, and consider myself to be reasonably well versed with SQL and relational databases. SQL doesn't allow you to act on your data while seeing it. The interaction modes are totally different. In SQL: Type query -> See some results -> Repeat Excel: Everything in front of your eyes. You are manipulating live data. That is not something you do in SQL. (SQL doesn't mean much though, as it is a standard and not a tool. Maybe you had some specific SQL based tooling in mind) > Excel stops scaling at about a million rows. A large amount of real world data has less than a million rows. Every tool will stop at some point. At that point the benefits of Excel fade away, even if it could handle the data, as 1M rows are too large for our brain to make a sense of live. >My cutoff at this point is that I'll use Excel if I expect my analysis to take less than five minutes A lot of real world analysis takes 5 minutes. Of course, Excel is not perfect, and I too will never use Excel for anything that has to be done repeatedly (pandas or similar solutions are perfect here), but for quick and dirty data exploration, Excel is very good.

liamwestray 7y ago

Excel is hands down the best financial analysis and modeling tool ever made. SQL is not really comparable here. Any attempt to use it as a database or for data analysis should be met with extreme caution. As soon as someone suggests a VB macro for anything they should be shot. Excel is not the right tool for those kids of jobs. Python and R have just destroyed excel for data analysis and statistics (and most visualizations now) and are free. You can’t go back once getting used to that flexibility. Jupyter Notebooks reproducibility and traceability put the nail in the coffin for excel as a data analysis tool.