3 ms·

This article seems to be a bunch of random claims with no evidence. If you want to demonstrate how Excel is a poor choice, show how someone would do data analy

This article seems to be a bunch of random claims with no evidence. If you want to demonstrate how Excel is a poor choice, show how someone would do data analysis using excel, then show how your alternative system, whatever it is, would do it "better", for some definition of better you want to use.

t_fatus 10y ago
Although the reasons aren’t quite known, MS Excel doesn’t have a good reputation among data analysts. Just stopped there : + not working on linux, paying on MacOS and Windows + proprietary format: hard to get data in / out from other tools (for instance, but not limited to, python lib, R lib, matlab (if some of us still want to use matlab) and all tools built upon those libs) + sucks for any real-sized dataset (try to handle 10k+ rows, not even thinking about 1M+ rows) Those 3 reasons alone mean that you need to pay for a tool which would hardly do the job a team ask for, if only it works on their OSs, and won't be able to share it with other teams.
- CJefferson 10y ago
  While there are good reasons for not using excel, be careful not to dump too fast. Excel's format is publicly available (if a little horrible to read XML), and it imports/exports CSV fine. Yes you do have to pay for it, but I find lots of R extensions don't work well on Windows, so OS-lock isn't a solved problem. You can use Office 365 in the web browser if you want to access from linux. The biggest issue is, as you say, don't try to put 1M+ rows into Excel. I agree. But I've worked with lots of people who have 20-1000 data points, and there Excel is great. I'd love to have an alternative to suggest to people, that scales well with them, but we aren't there yet, and Excel still fills an important niche. Assuming people only use it because they are idiots (not saying you said that, but many do) and not looking at why they use it is what is stopping other tools replacing it.
dkersten 10y ago
"This article seems to be a bunch of random claims with no evidence." I was part of a startup that was essentially trying to replace excel for a specific market and we made a very similar list (with sources). Sadly I don't have access to it anymore... Anecdotally, from what I hear from friends who work in the industry, many banks have a rather error-prone practice of emailing excel sheets to each other resulting in version mismatches, versions getting lost, latency etc. Here are a few reports for you to look at: http://www.businessinsider.com/excel-partly-to-blame-for-trading-loss-2013-2?utm_source=scoopinion&IR=T http://www.businessinsider.com/excel-partly-to-blame-for-tra... http://www.cio.com/article/2438188/enterprise-software/eight-of-the-worst-spreadsheet-blunders.html http://www.cio.com/article/2438188/enterprise-software/eight... http://blogs.wsj.com/moneybeat/2014/10/16/spreadsheet-mistake-costs-tibco-shareholders-100-million/ http://blogs.wsj.com/moneybeat/2014/10/16/spreadsheet-mistak... But I guess the question you are asking isn't "is excel error prone", but rather, "are the alternatives less error prone", and while I don't have any supporting evidence, I would think that when using a "real" programming language, you would have version control, unit tests and so on.
- TeMPOraL 10y ago
  > I would think that when using a "real" programming language, you would have version control, unit tests and so on. You would if and only if you employed professional programmers at those tasks, which with Excel (apparently) could be done with less/differently trained people. Version control and unit tests are not features of environment, but of professional training and discipline.
  - dkersten 10y ago
    True, although availability of tools and ease of use does encourage certain practices. You "could" do these things with Excel, but it doesn't encourage it at all. But you're right, without professional training and discipline even if the tools exist, they're not used...
    - TeMPOraL 10y ago
      I agree. My point being, when thinking about viable Excel replacement, one also needs to consider if and how the skillset differs to get the most of the benefits of the new tool. A company may not be willing to switch if the alternative solution requires extended specialized training for the staff.
      - dkersten 10y ago
        Yes, indeed, you are correct.
- ellius 10y ago
  I work for a bank, and literally almost everyone I work with does this. By comparison, I use R for pretty much everything (minus some formatting in Excel). I have a far easier time identifying errors because than my coworkers, because instead of getting version 328 of Book 1.xlsx with no information about structure, edge cases, etc., I have well-documented code explaining exactly what came in, how I transformed and analyzed it, and exactly what went out. I will say the one downside to this practice is that I get stuck editing a bunch of connector code because no one can send the same information in the same format twice.

t_fatus 10y ago

Although the reasons aren’t quite known, MS Excel doesn’t have a good reputation among data analysts. Just stopped there : + not working on linux, paying on MacOS and Windows + proprietary format: hard to get data in / out from other tools (for instance, but not limited to, python lib, R lib, matlab (if some of us still want to use matlab) and all tools built upon those libs) + sucks for any real-sized dataset (try to handle 10k+ rows, not even thinking about 1M+ rows) Those 3 reasons alone mean that you need to pay for a tool which would hardly do the job a team ask for, if only it works on their OSs, and won't be able to share it with other teams.

CJefferson 10y ago
While there are good reasons for not using excel, be careful not to dump too fast. Excel's format is publicly available (if a little horrible to read XML), and it imports/exports CSV fine. Yes you do have to pay for it, but I find lots of R extensions don't work well on Windows, so OS-lock isn't a solved problem. You can use Office 365 in the web browser if you want to access from linux. The biggest issue is, as you say, don't try to put 1M+ rows into Excel. I agree. But I've worked with lots of people who have 20-1000 data points, and there Excel is great. I'd love to have an alternative to suggest to people, that scales well with them, but we aren't there yet, and Excel still fills an important niche. Assuming people only use it because they are idiots (not saying you said that, but many do) and not looking at why they use it is what is stopping other tools replacing it.

dkersten 10y ago

"This article seems to be a bunch of random claims with no evidence." I was part of a startup that was essentially trying to replace excel for a specific market and we made a very similar list (with sources). Sadly I don't have access to it anymore... Anecdotally, from what I hear from friends who work in the industry, many banks have a rather error-prone practice of emailing excel sheets to each other resulting in version mismatches, versions getting lost, latency etc. Here are a few reports for you to look at: http://www.businessinsider.com/excel-partly-to-blame-for-trading-loss-2013-2?utm_source=scoopinion&IR=T http://www.businessinsider.com/excel-partly-to-blame-for-tra... http://www.cio.com/article/2438188/enterprise-software/eight-of-the-worst-spreadsheet-blunders.html http://www.cio.com/article/2438188/enterprise-software/eight... http://blogs.wsj.com/moneybeat/2014/10/16/spreadsheet-mistake-costs-tibco-shareholders-100-million/ http://blogs.wsj.com/moneybeat/2014/10/16/spreadsheet-mistak... But I guess the question you are asking isn't "is excel error prone", but rather, "are the alternatives less error prone", and while I don't have any supporting evidence, I would think that when using a "real" programming language, you would have version control, unit tests and so on.

TeMPOraL 10y ago
> I would think that when using a "real" programming language, you would have version control, unit tests and so on. You would if and only if you employed professional programmers at those tasks, which with Excel (apparently) could be done with less/differently trained people. Version control and unit tests are not features of environment, but of professional training and discipline.
- dkersten 10y ago
  True, although availability of tools and ease of use does encourage certain practices. You "could" do these things with Excel, but it doesn't encourage it at all. But you're right, without professional training and discipline even if the tools exist, they're not used...
  - TeMPOraL 10y ago
    I agree. My point being, when thinking about viable Excel replacement, one also needs to consider if and how the skillset differs to get the most of the benefits of the new tool. A company may not be willing to switch if the alternative solution requires extended specialized training for the staff.
    - dkersten 10y ago
      Yes, indeed, you are correct.
ellius 10y ago
I work for a bank, and literally almost everyone I work with does this. By comparison, I use R for pretty much everything (minus some formatting in Excel). I have a far easier time identifying errors because than my coworkers, because instead of getting version 328 of Book 1.xlsx with no information about structure, edge cases, etc., I have well-documented code explaining exactly what came in, how I transformed and analyzed it, and exactly what went out. I will say the one downside to this practice is that I get stuck editing a bunch of connector code because no one can send the same information in the same format twice.