5 ms·
I think there is scope to replace Excel, but it’s hard. You’re seeing a gradual spread of “data science” tools in finance, as the more technical analysts start
by theptip 5y ago
I think there is scope to replace Excel, but it’s hard. You’re seeing a gradual spread of “data science” tools in finance, as the more technical analysts start to use Python over Excel. But as you say, the management layer still uses Excel to poke and prod a model.
I think that there will be a generational shift here - you are not going to train an SVP to use Python, but the next generation of SVPs might have more exposure and be willing to use Numpy in a Jupyter notebook.
And in the other direction - there is definitely scope to come up with an “Excel isomorphic” Python framework for data science. It’s fairly easy to generate an Excel sheet from Python computations, but maintaining bi-directionality is Hard, and would require restrictions on the Excel side. I think with the right UI, you could do this though.
- civilized 5y agoI have a colleague who is a PhD in Applied Math. Pretty bright guy, huge Python/Jupyter lover with many years of experience. Loves to use git, loves to write dozens of unit tests. A true Man of the Future, according to Excel haters. He wrote some Python to solve some mildly complex business problem. I told him to translate it into Excel for stakeholders. He did, and the answer came out completely different. It turned out he had made multiple catastrophic errors in the Python. This is not the first, second, or third time this has happened. Python, and tech-beyond-Excel in general, just isn't the silver bullet software types often seem to think it is. Even experts sometimes seem to do a worse job in it than in Excel.
- pwnna 5y agoI love both spreadsheets and Python/Jupyter. But neither is Excel the silver bullet? The reason the error is caught (and the real lesson) is more because he tried to reproduce his work as opposed to a particular technology stack.
- catgary 5y agoThat is hardly special to Python->Excel translation, and happens any time someone reimplements a piece of software.
- civilized 5y agoI think Excel has something to do with it. In Excel, you're usually forced to do computations in small steps and look at the intermediate results. In traditional coding, you don't see anything but the final result (unless you ask to see it). It's much easier to assume everything is working as intended when it's not. Excel has its problems too. Different tools for different jobs. Tech boosters need to understand this and not just cynically assume that spreadsheet lovers are old fogeys who are afraid of their jobs being automated away.
- catgary 5y agoI think you’re just describing poor coding practices
- dkersten 5y agoCertainly if you do traditional TDD or something like REPL-driven development, you see the intermediate results and validate the correctness of your code as you go.
- civilized 5y agoThose are great! But they are coding disciplines one must choose to follow, and continue following every step of the way. You aren't inherently following them because that's how the tool works. Sometimes the tool-enforced discipline has advantages.
- cpeterso 5y agoHow would one write tests for a complex, mission-critical Excel spreadsheet? Or use version control? Spreadsheets often mix the data and code/formulas and the formulas are hidden behind the sheet view and sprinkled across many cells. At least Python scripts separate the code from the data so you can write tests using known good or fuzzing data. And you can use version control to track and review code changes.
- arrosenberg 5y agoI have an equal number of anecdotes where a few lines of easily readable python replace complex Excel calcs that invariably have a mistake somewhere. The tools aren’t the issue.
- cutler 5y agoYou said he "loves to write dozens of unit tests" so why so many errors?
- civilized 5y agoSometimes it's hard to think of the right tests, especially when you're solving a mathematical model that hasn't been solved before. Even for things that are intuitive and have been implemented thousands of times before, like web logins and shopping carts, where the tests one should do are not hard to think of... even so, software engineers rarely develop tests that catch all possible bugs on the first try.
- ljw1001 5y agoReplacing excel is like replacing paper. People always talk about formulas, but most spreadsheets don’t have any. They’re todo lists, or more commonly, static database exports. Or they have formulas, but the author calculates a few numbers and throws it away. In this case they’re like a scratch pad, or a calculator with a visible memory. I think it’s certainly possible to make better business modeling tools, and have played with some designs in that space. But they’ll never be Excel. And that’s ok