4 ms·

People have been saying this for over 20 years now. Excel was, is and will remain a foundational aspect of compute tasks for programmers and non-programmers ali

People have been saying this for over 20 years now. Excel was, is and will remain a foundational aspect of compute tasks for programmers and non-programmers alike. It is not by chance that Excel is popular.

socialdemocrat 6y ago
A lot of that is also simply momentum. Many people are utterly unaware of the alternatives outside the MS Office bubble. I have met so many people who thought the only program that exist for writing text is MS Word. Seriously. It is not even funny. But that is how people often get trained early on. You get taught to use MS Word at school and other context and never get exposed to alternative tools or ways of doing things. Here is a simple anecdote. Have on two occasions had long term Excel users get stuck on some task, looking for a solution. Then I ask them to use Apple Numbers. See if they can figure it out. Despite never having use it they figure out their task in no time. My point is that MS Excel isn't even all that good of a spreadsheet application yet almost everybody use it despite better alternatives existing.
- fermienrico 6y ago
  I've been writing software for many years. I still use Excel for quick personal finance stuff, calculator and many other random lists, etc. It takes 5 seconds to launch and you're already productive. It has its place. Not in a data science pipeline obviously.
  - socialdemocrat 6y ago
    Could you give some examples of some concrete tasks? Because I use Julia for exactly those same tasks, and I don't get why Excel would be any benefit. Julia launches faster. It is easy to just keep around in a Terminal window all day. Doesn't consume much resources. Any command I have previously typed is quickly accessible. Excel in contrast doesn't know what you did previously or can easily bring that kind of functionality back.
    - fermienrico 6y ago
      Are you serious? "Can you give concrete examples for how you use a knife in the kitchen?" It's an absurd question.
    - wenc 6y ago
      I'm deep into the dataframe/SQL way of doing things but I also use Excel a lot. In my opinion, spreadsheets provide a different paradigm that dataframes do not supplant. In theory anything in Excel can be done in a programming language (that's obvious). In practice, certain tasks are simply easier do in Excel. Excel is essentially a functional reactive calculator on a visual grid. It lets people quickly create models to test ad-hoc ideas. Because it works at a cell-level, ad-hoc calculations are much simpler. Because it is reactive, you can test different scenarios without recompiling or re-running. Sure, it has its limitations -- which is why multidimensional modeling tools like Quantrix exist. Also it's true that many people use Excel far beyond its intended purposes (I've come across some really complex Excel sheets with VBA that really should be in a SQL database). And yes, the dataframe paradigm is powerful for structured/systematic and reproducible/replayable transformations of data, e.g. column/row operations, aggregations, filtering. However, the very structure that makes dataframes so powerful and consistent also constrains it. In a spreadsheet, you can break a formula at specific cells (for exceptions etc). You can do that programmatically in a dataframe too, but it's a less natural operation which requires ad-hoc coding, whereas in a spreadsheet, you just head over to the cell and change the formula. Or take operations that are easily done via copy-and-pasting, say, like lagging a column by 2 but only between rows 23-65 and skipping certain rows because they're say, weekends/holidays. In Excel, you can carry this out by just manipulating the data directly i.e. physically moving cells around. In a dataframe, you have to extract the required rows, apply the lag operator, and then reinsert the lagged data (and reset indices, or do vertical concatenate) etc. I manage my personal cash flow and do scenario analyses of different kinds (financial, probabilistic, etc.) in Google sheets (and not a database or dataframe) precisely because I can introduce exceptions in the table and perform ad-hoc tasks like test scenarios in real-time easily. If I want to check an idea out, say the calculation of airflow in an aircraft cabin, a spreadsheet (and not a dataframe) is what I'd reach for first. Navigationally, a spreadsheet also lets you "touch" data and get a feel for it in a way that dataframes don't (in a programming language, you're typically always working with a subset view of a dataframe, e.g. SELECT TOP 1000 * FROM x, or df.head(3) -- these paradigms are the default because of the inherent assumption that all operations have to scale to the largest of datasets). In exploratory data analysis, it's super important to be able to see and feel the data because that's how you discover inconsistencies, sentinel values, exceptions, etc. Spreadsheets give you a low-friction interface (like filter/sort and freeze rows) and a set of powerful functional tools out of the box that generally beats any CSV viewer.
- prower 6y ago
  In all honesty, no one (at least, no one in the real actual world) wants alternatives to things that already work. Give me a standard that's good enough and we'll see how to deal with edge-cases. That's what internet is for, dammit. Yes, I know a million different better pieces of software exists for my peculiar task. No, i don't care. I have other things to do in my life than babysitting a damn computer.
  - socialdemocrat 6y ago
    What you should care about is using the software that makes your life the most easy, and that is not necessarily what you are currently using. What you want is easy to use software, that is flexible and can do what you want to do. The stuff you learned 10 years ago isn't necessarily that software.

socialdemocrat 6y ago

A lot of that is also simply momentum. Many people are utterly unaware of the alternatives outside the MS Office bubble. I have met so many people who thought the only program that exist for writing text is MS Word. Seriously. It is not even funny. But that is how people often get trained early on. You get taught to use MS Word at school and other context and never get exposed to alternative tools or ways of doing things. Here is a simple anecdote. Have on two occasions had long term Excel users get stuck on some task, looking for a solution. Then I ask them to use Apple Numbers. See if they can figure it out. Despite never having use it they figure out their task in no time. My point is that MS Excel isn't even all that good of a spreadsheet application yet almost everybody use it despite better alternatives existing.

fermienrico 6y ago
I've been writing software for many years. I still use Excel for quick personal finance stuff, calculator and many other random lists, etc. It takes 5 seconds to launch and you're already productive. It has its place. Not in a data science pipeline obviously.
- socialdemocrat 6y ago
  Could you give some examples of some concrete tasks? Because I use Julia for exactly those same tasks, and I don't get why Excel would be any benefit. Julia launches faster. It is easy to just keep around in a Terminal window all day. Doesn't consume much resources. Any command I have previously typed is quickly accessible. Excel in contrast doesn't know what you did previously or can easily bring that kind of functionality back.
  - fermienrico 6y ago
    Are you serious? "Can you give concrete examples for how you use a knife in the kitchen?" It's an absurd question.
  - wenc 6y ago
    I'm deep into the dataframe/SQL way of doing things but I also use Excel a lot. In my opinion, spreadsheets provide a different paradigm that dataframes do not supplant. In theory anything in Excel can be done in a programming language (that's obvious). In practice, certain tasks are simply easier do in Excel. Excel is essentially a functional reactive calculator on a visual grid. It lets people quickly create models to test ad-hoc ideas. Because it works at a cell-level, ad-hoc calculations are much simpler. Because it is reactive, you can test different scenarios without recompiling or re-running. Sure, it has its limitations -- which is why multidimensional modeling tools like Quantrix exist. Also it's true that many people use Excel far beyond its intended purposes (I've come across some really complex Excel sheets with VBA that really should be in a SQL database). And yes, the dataframe paradigm is powerful for structured/systematic and reproducible/replayable transformations of data, e.g. column/row operations, aggregations, filtering. However, the very structure that makes dataframes so powerful and consistent also constrains it. In a spreadsheet, you can break a formula at specific cells (for exceptions etc). You can do that programmatically in a dataframe too, but it's a less natural operation which requires ad-hoc coding, whereas in a spreadsheet, you just head over to the cell and change the formula. Or take operations that are easily done via copy-and-pasting, say, like lagging a column by 2 but only between rows 23-65 and skipping certain rows because they're say, weekends/holidays. In Excel, you can carry this out by just manipulating the data directly i.e. physically moving cells around. In a dataframe, you have to extract the required rows, apply the lag operator, and then reinsert the lagged data (and reset indices, or do vertical concatenate) etc. I manage my personal cash flow and do scenario analyses of different kinds (financial, probabilistic, etc.) in Google sheets (and not a database or dataframe) precisely because I can introduce exceptions in the table and perform ad-hoc tasks like test scenarios in real-time easily. If I want to check an idea out, say the calculation of airflow in an aircraft cabin, a spreadsheet (and not a dataframe) is what I'd reach for first. Navigationally, a spreadsheet also lets you "touch" data and get a feel for it in a way that dataframes don't (in a programming language, you're typically always working with a subset view of a dataframe, e.g. SELECT TOP 1000 * FROM x, or df.head(3) -- these paradigms are the default because of the inherent assumption that all operations have to scale to the largest of datasets). In exploratory data analysis, it's super important to be able to see and feel the data because that's how you discover inconsistencies, sentinel values, exceptions, etc. Spreadsheets give you a low-friction interface (like filter/sort and freeze rows) and a set of powerful functional tools out of the box that generally beats any CSV viewer.
prower 6y ago
In all honesty, no one (at least, no one in the real actual world) wants alternatives to things that already work. Give me a standard that's good enough and we'll see how to deal with edge-cases. That's what internet is for, dammit. Yes, I know a million different better pieces of software exists for my peculiar task. No, i don't care. I have other things to do in my life than babysitting a damn computer.
- socialdemocrat 6y ago
  What you should care about is using the software that makes your life the most easy, and that is not necessarily what you are currently using. What you want is easy to use software, that is flexible and can do what you want to do. The stuff you learned 10 years ago isn't necessarily that software.