5 ms·

I could've sworn I've seen people who work in the financial and maybe data science industries who've praised Excel to the heavens on HN. Like on the same level

I could've sworn I've seen people who work in the financial and maybe data science industries who've praised Excel to the heavens on HN. Like on the same level as Python, R, etc. I don't work in finance (or data science really) so I can't comment, but it seems merely a tool to me.

lordnacho 10y ago
No way, not data science. How are you going to load in millions, possibly billions of individual numbers? With finance there's a lot of things that seem like good fits, but only if you keep them small. Something like a personal budget is fine, where everything fits on a screen.
- blahi 10y ago
  I have 2.4 billion data points loaded in Excel at the moment. Not that it matters. "Data science" is not something that magically kicks in after you go beyond some "big data" threshold. Excel is heavily used in managerial science type of positions and I can assure you those are rather heavy on the "data science" workflows.
  - ramblenode 10y ago
    The maximum number of rows in the most recent version is 1,048,576 [0]. I'm deducing then that your data is in wide format. What would happen if you wanted it in long format? It seems you would just be out of luck with Excel, which is not a problem in any other major statistics software. [0] https://support.office.com/en-us/article/Excel-specifications-and-limits-ca36e2dc-1f09-4620-b726-67c00b05040f https://support.office.com/en-us/article/Excel-specification...
    - blahi 10y ago
      Except that there is Power Pivot which is xVelocity - the same columnar store database use in SQL Server.
  - goatlover 10y ago
    Why on Earth would you use Excel for that many data points?
    - blahi 10y ago
      Quik self-service OLAP. Blazing fast slice & dice and multidimensional expressions (cube formulas). I would need a corporate BI solution to get that and it won't be self-service. Nothing beats Excel + Power BI in reporting.
- dandermotj 10y ago
  The "big data vs. small data" argument isn't the issue for data scientists. It's the total lack of decent statistical libraries, visualisation tools, programming capabilities and on and on. Don't think anything I've worked on would come close to hitting the excel memory ceiling, but without a doubt completing it in excel would be nigh on impossible.
- PeterisP 10y ago
  A simple excel pivotchart on 10k-100k financial entries often works better than the best BI / MIS packages. For most companies, all their data science fits in excel easily. Full transaction history with each individual item sold since founding the company? No problem. Every single visitor/page view on their website? I've seen such logs imported to excel to do some rough aggregation. Manufacturing data about each particular widget sub-part that ever went off your conveyor belt? Again, for many companies with hundreds of employees that would still fit in excel without issues. There are so many companies who speak about big data while their largest datasets can fit into RAM of a cheap laptop. That doesn't mean that data science and analysis is worthless to them, quite contrary.
throw_away_777 10y ago
There are lots of people who are data analysts who use Excel. Data scientists would more generally use more powerful tools, but Excel is still useful for some things. For example if you have to manually enter in a small amount of data and then perform some simple calculations on this data, Excel is a fine tool. If your data is stored in a machine readable format or you want to do complex analysis with it you generally shouldn't be using Excel.
- huac 10y ago
  +1 for manual entry; or if you need to fix some cells in the data.
Nicholas_C 10y ago
I'm probably one of those people. You can't build a model in R or Python and walk a sales or finance VP through it. The other day I walked some auditors through a regression model built in excel. I could not imagine doing that in R or Python, their eyes would glaze over and think it's a black box. That's the beauty of excel, very easy for everyone to understand as the numbers are right in front of you.

lordnacho 10y ago

No way, not data science. How are you going to load in millions, possibly billions of individual numbers? With finance there's a lot of things that seem like good fits, but only if you keep them small. Something like a personal budget is fine, where everything fits on a screen.

blahi 10y ago
I have 2.4 billion data points loaded in Excel at the moment. Not that it matters. "Data science" is not something that magically kicks in after you go beyond some "big data" threshold. Excel is heavily used in managerial science type of positions and I can assure you those are rather heavy on the "data science" workflows.
- ramblenode 10y ago
  The maximum number of rows in the most recent version is 1,048,576 [0]. I'm deducing then that your data is in wide format. What would happen if you wanted it in long format? It seems you would just be out of luck with Excel, which is not a problem in any other major statistics software. [0] https://support.office.com/en-us/article/Excel-specifications-and-limits-ca36e2dc-1f09-4620-b726-67c00b05040f https://support.office.com/en-us/article/Excel-specification...
  - blahi 10y ago
    Except that there is Power Pivot which is xVelocity - the same columnar store database use in SQL Server.
- goatlover 10y ago
  Why on Earth would you use Excel for that many data points?
  - blahi 10y ago
    Quik self-service OLAP. Blazing fast slice & dice and multidimensional expressions (cube formulas). I would need a corporate BI solution to get that and it won't be self-service. Nothing beats Excel + Power BI in reporting.
dandermotj 10y ago
The "big data vs. small data" argument isn't the issue for data scientists. It's the total lack of decent statistical libraries, visualisation tools, programming capabilities and on and on. Don't think anything I've worked on would come close to hitting the excel memory ceiling, but without a doubt completing it in excel would be nigh on impossible.
PeterisP 10y ago
A simple excel pivotchart on 10k-100k financial entries often works better than the best BI / MIS packages. For most companies, all their data science fits in excel easily. Full transaction history with each individual item sold since founding the company? No problem. Every single visitor/page view on their website? I've seen such logs imported to excel to do some rough aggregation. Manufacturing data about each particular widget sub-part that ever went off your conveyor belt? Again, for many companies with hundreds of employees that would still fit in excel without issues. There are so many companies who speak about big data while their largest datasets can fit into RAM of a cheap laptop. That doesn't mean that data science and analysis is worthless to them, quite contrary.

throw_away_777 10y ago

There are lots of people who are data analysts who use Excel. Data scientists would more generally use more powerful tools, but Excel is still useful for some things. For example if you have to manually enter in a small amount of data and then perform some simple calculations on this data, Excel is a fine tool. If your data is stored in a machine readable format or you want to do complex analysis with it you generally shouldn't be using Excel.

huac 10y ago
+1 for manual entry; or if you need to fix some cells in the data.

Nicholas_C 10y ago

I'm probably one of those people. You can't build a model in R or Python and walk a sales or finance VP through it. The other day I walked some auditors through a regression model built in excel. I could not imagine doing that in R or Python, their eyes would glaze over and think it's a black box. That's the beauty of excel, very easy for everyone to understand as the numbers are right in front of you.