8 ms·

One huge reason why Finance pros in general, and Fund Managers, Brokers, Traders in particular, love Microsoft Excel is because all your formulas and calculatio

One huge reason why Finance pros in general, and Fund Managers, Brokers, Traders in particular, love Microsoft Excel is because all your formulas and calculations update almost instantly (unless you explicitly disable this feature). So they can apply their business and process knowledge to create tables and formulas, pull data from various sources, and create something that is self-contained, stored locally (inside the lan, security reasons), is blazing fast, and helps them make money and generate profits for the company. I've seen very sophisticated OMSs (Order Management Systems) built completely using excel workbooks, with 1 sheet containing orders and formula's to pull realtime pricing from bloomberg, reuters, another sheet for pre-trade analysis, another for the actual trades, and a summary sheet with pivot tables and charts to show the trade progress and summary. As and when the last_price changes, the formulas update, so the trader is looking at a real-time view of his situation. The same excel workbook that let's the traders do their job would take months / years to convert to a web app, and even then, the speed may not match up. I know devs hate Excel, in part, because you are the "IT guy" they call (been there many times) when the spreadsheet gets out of hand, or slows down, or keeps crashing excel after a certain inflection point.

toomuchtodo 9y ago
Someone on HN said this once, and I found it profound and have stolen it: Excel is the most powerful REPL in the finance industry, and it's ubiquitous.
- deleted 9y ago
  [deleted]
maxxxxx 9y ago
I can see this in my company. IT has for some reason decided to put some data into Hadoop. Now the users need to write spec documents for each little thing and every trivial report takes months to get done. Before they had Excel and could create whatever they needed in minutes. Us programmers don't always make life easier for end users. Often we create a level of bureaucracy. EDIT: I like a lot of the responses here. They clearly illustrate the disconnect between end users and IT/devs in many companies.
- adambard 9y ago
  If you're putting Excel-sized data in Hadoop, something has gone seriously wrong in your IT department's decision-making process. It does make for a nice parable about the dangers of shiny-new-thing-syndrome though.
  - maxxxxx 9y ago
    Yep. this is going on everywhere. How many solid but boring Java applications have been replaced by a microservice, cloud, NoSQL, NoieJS monster?
    - sogen 9y ago
      http://www.zdnet.com/article/disgruntled-developer-breaks-thousands-of-javascript-node-js-apps/ http://www.zdnet.com/article/disgruntled-developer-breaks-th...
  - jordanb 9y ago
    Not really. They could be trying to build a company-wide data lake. Some files in the lake will be very small, others will be very large. Because it's in HDFS it won't matter if the very small ones suddenly become very large. End users groan because people groan every time anything changes and they have to learn something new.
    - ghostly_s 9y ago
      If users were previously empowered to do a task themselves, and now need to request another team do that task on their behalf, that's a very valid reason to be groaning IMO.
      - wil421 9y ago
        You have to understand that enterprise class companies don’t want Access databases under people desks. It might be valid reason for the sales or ordering guy who loses control to complain. For the IT professionals at the company it allows them to create a scalable solution that can be used at an enterprise level.
alecco 9y ago
> The same excel workbook that let's the traders do their job would take months / years to convert to a web app, and even then, the speed may not match up. Sorry but I have to call nonsense on that part. I've seen recently a whole trading operation converted to a system very close to a web app within 6 months. The hard part of the job was untangling the mess they created. And knowledgeable quants who actually program told us it's typical. The financial pros have horrible, horrible stuff in their spreadsheets. They are used to it and they want (perhaps unconsciously) to keep their dirty hacks out of sight. Lest they fall from heaven.
- EpicEng 9y ago
  Except excel is a generic solution. If you have to build a specific solution for each processes your costs skyrocket and people lose the ability to quickly implement something on their own. Everything takes forever to get done because you're constantly haggling with development and writing requirements instead of just opening excel and getting work done.
  - osrec 9y ago
    Not really. In the long term, reg fines and dev costs balance out.
    - philipov 9y ago
      Citation please.
      - osrec 9y ago
        Pretty much every rogue trading scandal in the last few years is caused by some guy coming to the front office from the back and gaming the system with knowledge of its weaknesses. In more or less every case, poor front office controls, systems or data practices were identified as the root cause. Example https://www.google.co.uk/amp/s/amp.theguardian.com/business/nils-pratley-on-finance/2012/nov/26/ubs-failure-control-rogue-trader-kweku-adoboli https://www.google.co.uk/amp/s/amp.theguardian.com/business/...
sgt101 9y ago
Errm, I've seen spreadsheets that take an hour to open and "overnight" to update.
threatofrain 9y ago
What I don't understand is when shops start stressing the scalability of Excel, such as accountancy shops that share their spreadsheets around through primitive systems of locking (even some of the big 10 accountancies do this), why they never considered anything like Access. The jump is straight from Excel to full-on DB backend, with no transition in the middle.
- fny 9y ago
  The transition isn't free. Additional software means more technical overhead. Now you need to worry about training your teams to understand and maintain an Access DB instead of pure spreadsheets. You'll now also have to worry about hiring people who'll be able to adapt to the new system. You'll probably need to hire a full time DBA too. Also, if it ain't broke, why fix it? I feel like a transition to a new system at times is akin to a software rewrite.
  - deleted 9y ago
    [deleted]
- Nullabillity 9y ago
  If there's something the world needs even less than more Excel, it's more Access.
turc1656 9y ago
Yes. I use Excel every day and while there are some serious cons (like a large data sets eating up tons of memory, crashing as a result, etc.) it is still an extraordinarily powerful tool. It is also extremely intuitive for the basic user and insanely powerful for the super user who really knows how to take advantage of the more advanced features. And with VBA integrated, it gets even more powerful. I've seen some pretty amazing sheets/templates put together in a few days that would take programmers weeks or months to put together. And since you don't have to know how to program at all to use it, it's easy to see why it remains around.
endorphone 9y ago
The same excel workbook that let's the traders do their job would take months / years to convert to a web app Simply taking an Excel workbook -- even data connected -- and using that as the requirements that drive an implementation would be extremely quick and speedy, the result having a litany of advantages. But that's never how such projects go. Instead everyone has to greenfield some idealized pinnacle product.
- deleted 9y ago
  [deleted]
- chadgeidel 9y ago
  I don't know your background but "taking an Excel workbook and using that as requirements to drive an implementation" is not speedy. I've done this several times in my career and it's very, very difficult. Not because there is any "idealization", but because the reason the spreadsheet works is because a human is doing half of the work. These spreadsheets are not some "hello world" easy to understand product - they are the result of months/years of effort by a team of individuals which has grown beyond Excel's ability to handle. Translating that into code is not "quick and speedy" even if one is doing a like-for-like implementation.
  - simooooo 9y ago
    Currently web-ifying a huge pivot table (30+ columns) into a dynamic library SQL report for access via the web. It's a nightmare of horribly complicated aggregate grouping/totalling across many columns. Additionally the original excel sheet made incorrect assumptions, resulting in a re-specification/design of what all the totals actually count. NOT speedy.
    - endorphone 9y ago
      "Additionally the original excel sheet made incorrect assumptions ... NOT speedy." So...not as I described then? If you are evaluating assumptions and going back to the drawing board, of course it won't be speedy. The point is that if it's simply a requirement to be implemented as is, it is a quick and speedy process. And it is. I mean, there are dozens of products that will automate the entire process it's so rote.
ig1 9y ago
Unless it's changed in recent versions Excel isn't threadsafe, so using realtime updates is something that works 99.9% of the time and blows up the other time leaving no evidence behind it when it comes to audit.
murph-almighty 9y ago
>I've seen very sophisticated OMSs (Order Management Systems) built completely using excel workbooks, with 1 sheet containing orders and formula's to pull realtime pricing from bloomberg, reuters, another sheet for pre-trade analysis, another for the actual trades, and a summary sheet with pivot tables and charts to show the trade progress and summary. Wait, people are making API calls in VBA to insert into Excel spreadsheets?

toomuchtodo 9y ago

Someone on HN said this once, and I found it profound and have stolen it: Excel is the most powerful REPL in the finance industry, and it's ubiquitous.

deleted 9y ago
[deleted]

maxxxxx 9y ago

I can see this in my company. IT has for some reason decided to put some data into Hadoop. Now the users need to write spec documents for each little thing and every trivial report takes months to get done. Before they had Excel and could create whatever they needed in minutes. Us programmers don't always make life easier for end users. Often we create a level of bureaucracy. EDIT: I like a lot of the responses here. They clearly illustrate the disconnect between end users and IT/devs in many companies.

adambard 9y ago
If you're putting Excel-sized data in Hadoop, something has gone seriously wrong in your IT department's decision-making process. It does make for a nice parable about the dangers of shiny-new-thing-syndrome though.
- maxxxxx 9y ago
  Yep. this is going on everywhere. How many solid but boring Java applications have been replaced by a microservice, cloud, NoSQL, NoieJS monster?
  - sogen 9y ago
    http://www.zdnet.com/article/disgruntled-developer-breaks-thousands-of-javascript-node-js-apps/ http://www.zdnet.com/article/disgruntled-developer-breaks-th...
- jordanb 9y ago
  Not really. They could be trying to build a company-wide data lake. Some files in the lake will be very small, others will be very large. Because it's in HDFS it won't matter if the very small ones suddenly become very large. End users groan because people groan every time anything changes and they have to learn something new.
  - ghostly_s 9y ago
    If users were previously empowered to do a task themselves, and now need to request another team do that task on their behalf, that's a very valid reason to be groaning IMO.
    - wil421 9y ago
      You have to understand that enterprise class companies don’t want Access databases under people desks. It might be valid reason for the sales or ordering guy who loses control to complain. For the IT professionals at the company it allows them to create a scalable solution that can be used at an enterprise level.

alecco 9y ago

> The same excel workbook that let's the traders do their job would take months / years to convert to a web app, and even then, the speed may not match up. Sorry but I have to call nonsense on that part. I've seen recently a whole trading operation converted to a system very close to a web app within 6 months. The hard part of the job was untangling the mess they created. And knowledgeable quants who actually program told us it's typical. The financial pros have horrible, horrible stuff in their spreadsheets. They are used to it and they want (perhaps unconsciously) to keep their dirty hacks out of sight. Lest they fall from heaven.

EpicEng 9y ago
Except excel is a generic solution. If you have to build a specific solution for each processes your costs skyrocket and people lose the ability to quickly implement something on their own. Everything takes forever to get done because you're constantly haggling with development and writing requirements instead of just opening excel and getting work done.
- osrec 9y ago
  Not really. In the long term, reg fines and dev costs balance out.
  - philipov 9y ago
    Citation please.
    - osrec 9y ago
      Pretty much every rogue trading scandal in the last few years is caused by some guy coming to the front office from the back and gaming the system with knowledge of its weaknesses. In more or less every case, poor front office controls, systems or data practices were identified as the root cause. Example https://www.google.co.uk/amp/s/amp.theguardian.com/business/nils-pratley-on-finance/2012/nov/26/ubs-failure-control-rogue-trader-kweku-adoboli https://www.google.co.uk/amp/s/amp.theguardian.com/business/...

sgt101 9y ago

Errm, I've seen spreadsheets that take an hour to open and "overnight" to update.

threatofrain 9y ago

What I don't understand is when shops start stressing the scalability of Excel, such as accountancy shops that share their spreadsheets around through primitive systems of locking (even some of the big 10 accountancies do this), why they never considered anything like Access. The jump is straight from Excel to full-on DB backend, with no transition in the middle.

fny 9y ago
The transition isn't free. Additional software means more technical overhead. Now you need to worry about training your teams to understand and maintain an Access DB instead of pure spreadsheets. You'll now also have to worry about hiring people who'll be able to adapt to the new system. You'll probably need to hire a full time DBA too. Also, if it ain't broke, why fix it? I feel like a transition to a new system at times is akin to a software rewrite.
- deleted 9y ago
  [deleted]
Nullabillity 9y ago
If there's something the world needs even less than more Excel, it's more Access.

turc1656 9y ago

Yes. I use Excel every day and while there are some serious cons (like a large data sets eating up tons of memory, crashing as a result, etc.) it is still an extraordinarily powerful tool. It is also extremely intuitive for the basic user and insanely powerful for the super user who really knows how to take advantage of the more advanced features. And with VBA integrated, it gets even more powerful. I've seen some pretty amazing sheets/templates put together in a few days that would take programmers weeks or months to put together. And since you don't have to know how to program at all to use it, it's easy to see why it remains around.

endorphone 9y ago

The same excel workbook that let's the traders do their job would take months / years to convert to a web app Simply taking an Excel workbook -- even data connected -- and using that as the requirements that drive an implementation would be extremely quick and speedy, the result having a litany of advantages. But that's never how such projects go. Instead everyone has to greenfield some idealized pinnacle product.

deleted 9y ago
[deleted]
chadgeidel 9y ago
I don't know your background but "taking an Excel workbook and using that as requirements to drive an implementation" is not speedy. I've done this several times in my career and it's very, very difficult. Not because there is any "idealization", but because the reason the spreadsheet works is because a human is doing half of the work. These spreadsheets are not some "hello world" easy to understand product - they are the result of months/years of effort by a team of individuals which has grown beyond Excel's ability to handle. Translating that into code is not "quick and speedy" even if one is doing a like-for-like implementation.
- simooooo 9y ago
  Currently web-ifying a huge pivot table (30+ columns) into a dynamic library SQL report for access via the web. It's a nightmare of horribly complicated aggregate grouping/totalling across many columns. Additionally the original excel sheet made incorrect assumptions, resulting in a re-specification/design of what all the totals actually count. NOT speedy.
  - endorphone 9y ago
    "Additionally the original excel sheet made incorrect assumptions ... NOT speedy." So...not as I described then? If you are evaluating assumptions and going back to the drawing board, of course it won't be speedy. The point is that if it's simply a requirement to be implemented as is, it is a quick and speedy process. And it is. I mean, there are dozens of products that will automate the entire process it's so rote.

ig1 9y ago

Unless it's changed in recent versions Excel isn't threadsafe, so using realtime updates is something that works 99.9% of the time and blows up the other time leaving no evidence behind it when it comes to audit.

murph-almighty 9y ago

>I've seen very sophisticated OMSs (Order Management Systems) built completely using excel workbooks, with 1 sheet containing orders and formula's to pull realtime pricing from bloomberg, reuters, another sheet for pre-trade analysis, another for the actual trades, and a summary sheet with pivot tables and charts to show the trade progress and summary. Wait, people are making API calls in VBA to insert into Excel spreadsheets?