2 ms·

Excel is/was a popular tool at the highest level in various industries, including finance. A frequently shared anecdote: https://baselinescenario.com/2013/02/0

by danso 7y ago

Excel is/was a popular tool at the highest level in various industries, including finance. A frequently shared anecdote: https://baselinescenario.com/2013/02/09/the-importance-of-excel/ https://baselinescenario.com/2013/02/09/the-importance-of-ex... > JPMorgan’s Chief Investment Office needed a new value-at-risk (VaR) model for the synthetic credit portfolio (the one that blew up) and assigned a quantitative whiz (“a London-based quantitative expert, mathematician and model developer” who previously worked at a company that built analytical models) to create it. The new model “operated through a series of Excel spreadsheets, which had to be completed manually, by a process of copying and pasting data from one spreadsheet to another.” The internal Model Review Group identified this problem as well as a few others, but approved the model, while saying that it should be automated and another significant flaw should be fixed.* After the London Whale trade blew up, the Model Review Group discovered that the model had not been automated and found several other errors...

alexpotato 7y ago
That's why this is such a great article: Manual Work is a Bug (https://queue.acm.org/detail.cfm?id=3197520 https://queue.acm.org/detail.cfm?id=3197520) It creates the mindset of, let me do a little everyday to make this a more automated process.
- froindt 7y ago
  I did this when I started my first full time role out of college with great success. I was responsible for tons of reports that were manual and tedious. My goal was to improve something about the process every time I did the report. It could be as little as making the label I looked for bold, or as big as refactoring and automating the entire thing. In the same way, I made standard work improvements. That turned ~4 hrs/day of reports into an automated system, gave me a good understanding of how the various systems and reports work together, and helped me get my first promotion. I've moved on to a new role where I'm doing the same thing on a process which has been void of substantial improvements for 3+ years. There's always something, and I can remove a bit more time with every automation iteration.
  - danso 7y ago
    When I was working in academia, early on I decided I would finally buckle down and learn proper devops and scripting (e.g. how to do things at the command-line) and had the luxury of having all the time I wanted to bikeshed on automating the collation process (never did master Make syntax though!). The csvkit toolset – particularly `in2csv` [0] and `csvsql`, which, respectively, can be used to extract CSV from Excel sheets and import/insert directly into sqlite – make frequent appearances in my daily work these days. Though admittedly, I don't know what options there are to distangle spreadsheets that aren't just data (e.g. intertwined formulas). [0] https://csvkit.readthedocs.io/en/1.0.3/scripts/in2csv.html https://csvkit.readthedocs.io/en/1.0.3/scripts/in2csv.html [1] https://csvkit.readthedocs.io/en/1.0.3/scripts/csvsql.html https://csvkit.readthedocs.io/en/1.0.3/scripts/csvsql.html

by danso 7y ago

alexpotato 7y ago

That's why this is such a great article: Manual Work is a Bug (https://queue.acm.org/detail.cfm?id=3197520 https://queue.acm.org/detail.cfm?id=3197520) It creates the mindset of, let me do a little everyday to make this a more automated process.

froindt 7y ago
I did this when I started my first full time role out of college with great success. I was responsible for tons of reports that were manual and tedious. My goal was to improve something about the process every time I did the report. It could be as little as making the label I looked for bold, or as big as refactoring and automating the entire thing. In the same way, I made standard work improvements. That turned ~4 hrs/day of reports into an automated system, gave me a good understanding of how the various systems and reports work together, and helped me get my first promotion. I've moved on to a new role where I'm doing the same thing on a process which has been void of substantial improvements for 3+ years. There's always something, and I can remove a bit more time with every automation iteration.
- danso 7y ago
  When I was working in academia, early on I decided I would finally buckle down and learn proper devops and scripting (e.g. how to do things at the command-line) and had the luxury of having all the time I wanted to bikeshed on automating the collation process (never did master Make syntax though!). The csvkit toolset – particularly `in2csv` [0] and `csvsql`, which, respectively, can be used to extract CSV from Excel sheets and import/insert directly into sqlite – make frequent appearances in my daily work these days. Though admittedly, I don't know what options there are to distangle spreadsheets that aren't just data (e.g. intertwined formulas). [0] https://csvkit.readthedocs.io/en/1.0.3/scripts/in2csv.html https://csvkit.readthedocs.io/en/1.0.3/scripts/in2csv.html [1] https://csvkit.readthedocs.io/en/1.0.3/scripts/csvsql.html https://csvkit.readthedocs.io/en/1.0.3/scripts/csvsql.html