3 ms·
I think these are all fair points and a good reflection of the downside of Python. But there are also some pretty huge upsides. My view is that for any model
by RobinL 6y ago
I think these are all fair points and a good reflection of the downside of Python. But there are also some pretty huge upsides. My view is that for any model of significant complexity, the pros of Python outweigh the cons from a technical point of view.
- Abstraction. It's very difficult to effectively abstract parts of a model in Excel. It's a bit like a doctor having to 'model' a human being as a collection of atoms, rather than having abstractions like organs, cells etc. This makes it very hard to build re-usable components, so analysts end up reinventing the wheel. You also quickly hit a 'complexity ceiling' in Excel, above which mistakes and errors becoming much more likely, and complexity is very difficult to manage.
- Existing libraries provide a huge range of sophisticated calculations and operations for which we don't need to write any code.
- Separation of concerns - particularly separating data from model. Easy in Python, hard in Excel. Another aspect of this is that using data science software promotes the use of tidy data[0] (i.e. clear thinking about how data should be structured).
- Unit/integration tests. For complex models, these are essential. Users of Excel (even extremely clever/competent people) don't have have a great reputation for producing error-free spreadsheets, and I think this is an important reason why, alongside copy-paste errors. The tools for testing in Excel/VBA are rudimentary.
- Version control. This is particularly important for historical reproducibility because it allows us to run past models, and also understand what has changed in the codebase since.
I appreciate some of the above is also possible in VBA, but if you're writing an entire model in code and not really using Excel at all, my view is it's better to use a more sophisticated programming language.
There is also an important cultural point of having to re-skill everyone, and I can see that in some context that means in the short run at least, Excel/VBA may still be better overall.
I've written a bit more about all of this here:
https://www.robinlinacre.com/transforming_analytical_functions/ https://www.robinlinacre.com/transforming_analytical_functio...
[0] https://vita.had.co.nz/papers/tidy-data.pdf https://vita.had.co.nz/papers/tidy-data.pdf
- pjmlp 6y ago> I appreciate some of the above is also possible in VBA, but if you're writing an entire model in code and not really using Excel at all, my view is it's better to use a more sophisticated programming language. Which is why many VBA experts eventually adopt VB.NET instead of jumping into a complete foreign language, with the benefit that is actually compiled to native code (JIT/NGEN), if performance is ever an issue.
- cutler 6y agoYes, I was wondering why Python was the automatic choice considering C#.Net has typesafe native APIs for Excel on Windows.
- RobinL 6y agoI was more thinking about migrating away from Excel fully rather than interfacing with Excel from Python. I agree that to interact with Excel programmatically VBA is a better choice (and no doubt C#/VB.NET as well, but I have no direct experience). For what it's worth, for interacting with Excel and Office more generally, I've always though VBA is extremely well designed.