45 ms·

If this spreadsheet had this type of business impact why was it in excel in the first place? Where was the version control? Why did a contractor have administr

If this spreadsheet had this type of business impact why was it in excel in the first place? Where was the version control? Why did a contractor have administrative password to such a heavily business dependent process? This article speaks volumes about their internal IT Governance and process.

coldtea 7y ago
>If this spreadsheet had this type of business impact why was it in excel in the first place? You'd be surprised. Excel files are used to handle trillions (not billions) of dollars in businesses, even for the most critical processes...
danso 7y ago
Excel is/was a popular tool at the highest level in various industries, including finance. A frequently shared anecdote: https://baselinescenario.com/2013/02/09/the-importance-of-excel/ https://baselinescenario.com/2013/02/09/the-importance-of-ex... > JPMorgan’s Chief Investment Office needed a new value-at-risk (VaR) model for the synthetic credit portfolio (the one that blew up) and assigned a quantitative whiz (“a London-based quantitative expert, mathematician and model developer” who previously worked at a company that built analytical models) to create it. The new model “operated through a series of Excel spreadsheets, which had to be completed manually, by a process of copying and pasting data from one spreadsheet to another.” The internal Model Review Group identified this problem as well as a few others, but approved the model, while saying that it should be automated and another significant flaw should be fixed.* After the London Whale trade blew up, the Model Review Group discovered that the model had not been automated and found several other errors...
- alexpotato 7y ago
  That's why this is such a great article: Manual Work is a Bug (https://queue.acm.org/detail.cfm?id=3197520 https://queue.acm.org/detail.cfm?id=3197520) It creates the mindset of, let me do a little everyday to make this a more automated process.
  - froindt 7y ago
    I did this when I started my first full time role out of college with great success. I was responsible for tons of reports that were manual and tedious. My goal was to improve something about the process every time I did the report. It could be as little as making the label I looked for bold, or as big as refactoring and automating the entire thing. In the same way, I made standard work improvements. That turned ~4 hrs/day of reports into an automated system, gave me a good understanding of how the various systems and reports work together, and helped me get my first promotion. I've moved on to a new role where I'm doing the same thing on a process which has been void of substantial improvements for 3+ years. There's always something, and I can remove a bit more time with every automation iteration.
    - danso 7y ago
      When I was working in academia, early on I decided I would finally buckle down and learn proper devops and scripting (e.g. how to do things at the command-line) and had the luxury of having all the time I wanted to bikeshed on automating the collation process (never did master Make syntax though!). The csvkit toolset – particularly `in2csv` [0] and `csvsql`, which, respectively, can be used to extract CSV from Excel sheets and import/insert directly into sqlite – make frequent appearances in my daily work these days. Though admittedly, I don't know what options there are to distangle spreadsheets that aren't just data (e.g. intertwined formulas). [0] https://csvkit.readthedocs.io/en/1.0.3/scripts/in2csv.html https://csvkit.readthedocs.io/en/1.0.3/scripts/in2csv.html [1] https://csvkit.readthedocs.io/en/1.0.3/scripts/csvsql.html https://csvkit.readthedocs.io/en/1.0.3/scripts/csvsql.html
michannne 7y ago
Work in fields like healthcare, governmemt, finance, law or defense and you will see Excel spreadsheets are used for just about everything. They are simple to create, simple to use and offer very advanced controls for a fraction of the time and cost it requires to development a sufficient framework for the data in-house or configure an existing solution. It is very easy to train people on how to use excel if they are willing to invest the time to learn formulas, and once you learn VB, you will be tempted to do most data presentation in Excel. That said, just because most people know how to use Excel does not mean they are technically capable of setting up things like version control, backups, validation or security. A lot of users I have worked with in addition get their files from someone else, who may not even work at the company anymore and they have no idea what is or could be running in the background or how exactly the calculation in certain cells is done. Further, it is very common that one technically capable person creates an excel that becomes a crux of an internal process simply because it is the easiest thing to use, once IT governence policies are put in place, it doesn't help because they damage lies in the fundamental data they started with. Tl;dr it is more common and innocent than it seems.
- amerine 7y ago
  Word. Excel is super powerful in expert user hands.
  - Djvacto 7y ago
    Powerpoint. I feel like I've only scratched the surface of Excel and it's definitely the most interesting part of Office software. You can do pretty much anything, and it's amazing. I've seen PDFs imitated for stuff like D&D Character Sheets, where all the cells are formatted and colored to provide whitespace around the actual fields.
AnIdiotOnTheNet 7y ago
Excel is probably the most widely used programming environment on the planet.
closeparen 7y ago
The whole point of a spreadsheet is that it's a programming environment outside the control of IT processes and governance. If you don't want people to have that, don't ship Excel to their machines. (But good luck to any CIO who tries that).

coldtea 7y ago

>If this spreadsheet had this type of business impact why was it in excel in the first place? You'd be surprised. Excel files are used to handle trillions (not billions) of dollars in businesses, even for the most critical processes...

danso 7y ago

Excel is/was a popular tool at the highest level in various industries, including finance. A frequently shared anecdote: https://baselinescenario.com/2013/02/09/the-importance-of-excel/ https://baselinescenario.com/2013/02/09/the-importance-of-ex... > JPMorgan’s Chief Investment Office needed a new value-at-risk (VaR) model for the synthetic credit portfolio (the one that blew up) and assigned a quantitative whiz (“a London-based quantitative expert, mathematician and model developer” who previously worked at a company that built analytical models) to create it. The new model “operated through a series of Excel spreadsheets, which had to be completed manually, by a process of copying and pasting data from one spreadsheet to another.” The internal Model Review Group identified this problem as well as a few others, but approved the model, while saying that it should be automated and another significant flaw should be fixed.* After the London Whale trade blew up, the Model Review Group discovered that the model had not been automated and found several other errors...

alexpotato 7y ago
That's why this is such a great article: Manual Work is a Bug (https://queue.acm.org/detail.cfm?id=3197520 https://queue.acm.org/detail.cfm?id=3197520) It creates the mindset of, let me do a little everyday to make this a more automated process.
- froindt 7y ago
  I did this when I started my first full time role out of college with great success. I was responsible for tons of reports that were manual and tedious. My goal was to improve something about the process every time I did the report. It could be as little as making the label I looked for bold, or as big as refactoring and automating the entire thing. In the same way, I made standard work improvements. That turned ~4 hrs/day of reports into an automated system, gave me a good understanding of how the various systems and reports work together, and helped me get my first promotion. I've moved on to a new role where I'm doing the same thing on a process which has been void of substantial improvements for 3+ years. There's always something, and I can remove a bit more time with every automation iteration.
  - danso 7y ago
    When I was working in academia, early on I decided I would finally buckle down and learn proper devops and scripting (e.g. how to do things at the command-line) and had the luxury of having all the time I wanted to bikeshed on automating the collation process (never did master Make syntax though!). The csvkit toolset – particularly `in2csv` [0] and `csvsql`, which, respectively, can be used to extract CSV from Excel sheets and import/insert directly into sqlite – make frequent appearances in my daily work these days. Though admittedly, I don't know what options there are to distangle spreadsheets that aren't just data (e.g. intertwined formulas). [0] https://csvkit.readthedocs.io/en/1.0.3/scripts/in2csv.html https://csvkit.readthedocs.io/en/1.0.3/scripts/in2csv.html [1] https://csvkit.readthedocs.io/en/1.0.3/scripts/csvsql.html https://csvkit.readthedocs.io/en/1.0.3/scripts/csvsql.html

michannne 7y ago

Work in fields like healthcare, governmemt, finance, law or defense and you will see Excel spreadsheets are used for just about everything. They are simple to create, simple to use and offer very advanced controls for a fraction of the time and cost it requires to development a sufficient framework for the data in-house or configure an existing solution. It is very easy to train people on how to use excel if they are willing to invest the time to learn formulas, and once you learn VB, you will be tempted to do most data presentation in Excel. That said, just because most people know how to use Excel does not mean they are technically capable of setting up things like version control, backups, validation or security. A lot of users I have worked with in addition get their files from someone else, who may not even work at the company anymore and they have no idea what is or could be running in the background or how exactly the calculation in certain cells is done. Further, it is very common that one technically capable person creates an excel that becomes a crux of an internal process simply because it is the easiest thing to use, once IT governence policies are put in place, it doesn't help because they damage lies in the fundamental data they started with. Tl;dr it is more common and innocent than it seems.

amerine 7y ago
Word. Excel is super powerful in expert user hands.
- Djvacto 7y ago
  Powerpoint. I feel like I've only scratched the surface of Excel and it's definitely the most interesting part of Office software. You can do pretty much anything, and it's amazing. I've seen PDFs imitated for stuff like D&D Character Sheets, where all the cells are formatted and colored to provide whitespace around the actual fields.

AnIdiotOnTheNet 7y ago

Excel is probably the most widely used programming environment on the planet.

closeparen 7y ago

The whole point of a spreadsheet is that it's a programming environment outside the control of IT processes and governance. If you don't want people to have that, don't ship Excel to their machines. (But good luck to any CIO who tries that).