7 ms·

Adventures in financial and software engineering

118 points by blondie9x 11y ago

osullivj 11y ago
I started working in financial tech in 97, and I've seen a number of "one model to rule them all" efforts come and go since. Back in the late 90s I saw consultants touting Java coded UML models as the way for banks to build the definitive object model of their trading operations. In the early 2000s I worked on JP Morgan's Kapital, an early 90s Smalltalk risk management system using the Gemstone DB. That was supposed to be the last word in derivatives abstraction. And more recently I spent some time working on JP's Athena project, a Python NoSQL system with an implicitly functional graph programming system that enables Python code to behave like spreadsheets. I suspect the ability of traders to invent products, and more recently regulators to invent rules, will always outstrip the ability of coders to invent clean abstractions that will factor the financial domain neatly.
- mianos 11y ago
  Same here, only I started some time earlier and have implemented the central trading system side of all these instruments. I agree, the basic contracts and simple combinations are the simplest part.
  - lifeisstillgood 11y ago
    But are not the basic contracts where all the volume is - is the opportunity for new banks to come in and provide volume on a new cheap stack? Existing banks seen usually add enormous implementation costs trying to solve the last 20% (reasonably as that's where profit tends to be) but it makes implementing the first 80% almost impossible I see this elsewhere too - solving business problems with software tools not with business tools.
- drglitch 11y ago
  (exJPM here) This is one of the cleanest, simplest descriptions of Kapital and Athena I heard in a while :) one thing to also keep in mind is no matter the sophistication of models, there is always gonna be some person in front/middle/back office somewhere, running a critical process on excel for 3 years.
  - cmdkeen 11y ago
    I'm an in house dev for a financial company - if you really want to "stress test" a financial institution turn off Excel for a week and you'll find all the things you never realised people were doing. I've been bitten far too frequently to ever believe what the product owners says a process is, there's always a spreadsheet somewhere used to work around the existing system...
    - 6stringmerc 11y ago
      I'd like to affirm your view that work arounds are a fact of life in finance, and frankly, other white-collar enterprises as well. While they're stashed relatively deep in my memory banks (mostly for the purpose of forgetting), I do have dozens of experiences 'touching up' PDFs, reverse engineering or cloning things from PDFs of PowerPoint presentations, or otherwise using whatever digital hammer and chisel would get the job done. Screenshot > Paint > Fix > PDF, boom!
    - threefour 11y ago
      Love that. I'm working in group benefits insurance now and spreadsheets are how everyone communicates. The software is just there to hold the data in between spreadsheet imports and exports.
      - osullivj 11y ago
        Are the spreadsheets doing real number crunching? Do they use XLLs and VBA, or just worksheet functions?
        drglitch 11y ago
        Very often yes, to a scary degree. The amount of data normalization, adjustments, aggregation, analysis, etc that happens in some poorly documented and ill maintained macro is mind-boggling (there is a term for it - a model/process being "offline", i.e. not properly source-controlled or auditable). The real reason is because Excel is still by far the simplest, quickest way to get something done fast when the deadlines are always "yesterday!".
- lifeisstillgood 11y ago
  Forgive me, but I will be stealing that last sentence ... It's great.
- boothead 11y ago
  There is one (at least) of these systems live based on the original paper by Simon Peyton Jones in a dialect of Haskell. JPM/BAML/GS have fallen down due in large part to their choice of language in my view. A graph database written in a hurry in C++ then glued together with tens of millions of lines of hacked up python is not the way to build a bank wide platform in my book!
  - kchoudhu 11y ago
    One of these banks is definitively not like the others.
    - boothead 11y ago
      And yet GS was the progenator of both Athena and Quartz....
      - osullivj 11y ago
        More specifically, the GS alumni who lead the Athena and Quartz builds at JP and BAML are now commercialising that approach as wsq.io
        boothead 11y ago
        Correct. I hope they learned some of the lessons from both!
  - deleted 11y ago
    [deleted]
  - nicolapede 11y ago
    What is the paper's title? I'd like to understand the theory behind this approach.
    - deleted 11y ago
      [deleted]
pandatigox 11y ago
I'm digressing a bit, but what language is used here? A skim-over looks like Prolog, but recommendation links at the bottom say Scala. Are the functions (when, and) predefined and not part of the language itself?
- desdiv 11y ago
  It's Scala. They show the definitions of the "When" and "And" in the post: case class And(contract1: Contract, contract2: Contract) extends Contract case class When(date: LocalDate, contract: Contract) extends Contract It's basically a DSL on top of Scala.
- paperwork 11y ago
  Not a digression, one of the reasons for this post was to practice Scala a bit further.
jiantastic 11y ago
Looks to be a really interesting article. Would really appreciate a font change and justified text though.
- napo 11y ago
  $('p').css('text-align', 'justify');
- asw 11y ago
  wtf
  - cfarre 11y ago
    you're so rude!
- paperwork 11y ago
  Good idea. I'll justify the text. What's wrong with the font?
baq 11y ago
there was a blog post or an article by somebody at jane street about their own contract DSL a few years ago. can't find the link now though.
- cheez 11y ago
  The paper that kicked this all off is in the references section. I believe the original company was called lexifi
lucozade 11y ago
Declarative descriptions for financial contracts and markets having been actively used for a long time now. Executable descriptions not so much. The issue really is that it's solving problems we don't have with a solution that gets in the way of solving the problems we do. That's not to say people haven't tried, far from it. But successes tend to be either very rare or very niche.
cheez 11y ago
I worked on aa commercial product called F3 which, among other things, implemented this type of contract specification as a subset of its functionality. Unfortunately, I'm not sure if they ever learned how to sell it well. One of the most memorable stints in my career so far.
jackgavigan 11y ago
FpML (Financial Products Markup Language) was conceived shortly before the paper referenced in the blog post.[1] One of the paper's authors went on to create MLFi, a modelling language for finance[2] This stuff is a bit esoteric but I think it will attract more attention because of its relevance to smart contracts. 1: http://www.fpml.org/ http://www.fpml.org/ 2: https://www.lexifi.com/product/technology/contract-description-language https://www.lexifi.com/product/technology/contract-descripti...
tdees40 11y ago
I put together a Haskell proof of concept Monte Carlo pricing tool loosely based on SPJ's work: https://github.com/boundedvariation/quantfin https://github.com/boundedvariation/quantfin Still needs some love, but it was a pretty cool experience.
- zhte415 11y ago
  I did similar, but I did it Excel. It was cool, not an experiment, and it was done quickly, as Excel is the ultimate Agile tool in finance.
platz 11y ago
I'm a fan of the combinator way, but of course now you have to 'interpret' or 'evaluate' the structure to know what the contract means, so it might be hard to run a 'query' over a bazillion of them sitting in a database - you'd have to interpret each one (each one is a little black box). This is assuming that running a query or search over all the contracts is something that is useful to do, which I have no idea if that is the case (though seems like a common task in applications programming).

118 points by blondie9x 11y ago

I started working in financial tech in 97, and I've seen a number of "one model to rule them all" efforts come and go since. Back in the late 90s I saw consultants touting Java coded UML models as the way for banks to build the definitive object model of their trading operations. In the early 2000s I worked on JP Morgan's Kapital, an early 90s Smalltalk risk management system using the Gemstone DB. That was supposed to be the last word in derivatives abstraction. And more recently I spent some time working on JP's Athena project, a Python NoSQL system with an implicitly functional graph programming system that enables Python code to behave like spreadsheets. I suspect the ability of traders to invent products, and more recently regulators to invent rules, will always outstrip the ability of coders to invent clean abstractions that will factor the financial domain neatly.

mianos 11y ago
Same here, only I started some time earlier and have implemented the central trading system side of all these instruments. I agree, the basic contracts and simple combinations are the simplest part.
- lifeisstillgood 11y ago
  But are not the basic contracts where all the volume is - is the opportunity for new banks to come in and provide volume on a new cheap stack? Existing banks seen usually add enormous implementation costs trying to solve the last 20% (reasonably as that's where profit tends to be) but it makes implementing the first 80% almost impossible I see this elsewhere too - solving business problems with software tools not with business tools.
drglitch 11y ago
(exJPM here) This is one of the cleanest, simplest descriptions of Kapital and Athena I heard in a while :) one thing to also keep in mind is no matter the sophistication of models, there is always gonna be some person in front/middle/back office somewhere, running a critical process on excel for 3 years.
- cmdkeen 11y ago
  I'm an in house dev for a financial company - if you really want to "stress test" a financial institution turn off Excel for a week and you'll find all the things you never realised people were doing. I've been bitten far too frequently to ever believe what the product owners says a process is, there's always a spreadsheet somewhere used to work around the existing system...
  - 6stringmerc 11y ago
    I'd like to affirm your view that work arounds are a fact of life in finance, and frankly, other white-collar enterprises as well. While they're stashed relatively deep in my memory banks (mostly for the purpose of forgetting), I do have dozens of experiences 'touching up' PDFs, reverse engineering or cloning things from PDFs of PowerPoint presentations, or otherwise using whatever digital hammer and chisel would get the job done. Screenshot > Paint > Fix > PDF, boom!
  - threefour 11y ago
    Love that. I'm working in group benefits insurance now and spreadsheets are how everyone communicates. The software is just there to hold the data in between spreadsheet imports and exports.
    - osullivj 11y ago
      Are the spreadsheets doing real number crunching? Do they use XLLs and VBA, or just worksheet functions?
      - drglitch 11y ago
        Very often yes, to a scary degree. The amount of data normalization, adjustments, aggregation, analysis, etc that happens in some poorly documented and ill maintained macro is mind-boggling (there is a term for it - a model/process being "offline", i.e. not properly source-controlled or auditable). The real reason is because Excel is still by far the simplest, quickest way to get something done fast when the deadlines are always "yesterday!".
lifeisstillgood 11y ago
Forgive me, but I will be stealing that last sentence ... It's great.
boothead 11y ago
There is one (at least) of these systems live based on the original paper by Simon Peyton Jones in a dialect of Haskell. JPM/BAML/GS have fallen down due in large part to their choice of language in my view. A graph database written in a hurry in C++ then glued together with tens of millions of lines of hacked up python is not the way to build a bank wide platform in my book!
- kchoudhu 11y ago
  One of these banks is definitively not like the others.
  - boothead 11y ago
    And yet GS was the progenator of both Athena and Quartz....
    - osullivj 11y ago
      More specifically, the GS alumni who lead the Athena and Quartz builds at JP and BAML are now commercialising that approach as wsq.io
      - boothead 11y ago
        Correct. I hope they learned some of the lessons from both!
- deleted 11y ago
  [deleted]
- nicolapede 11y ago
  What is the paper's title? I'd like to understand the theory behind this approach.
  - deleted 11y ago
    [deleted]

pandatigox 11y ago

I'm digressing a bit, but what language is used here? A skim-over looks like Prolog, but recommendation links at the bottom say Scala. Are the functions (when, and) predefined and not part of the language itself?

desdiv 11y ago
It's Scala. They show the definitions of the "When" and "And" in the post: case class And(contract1: Contract, contract2: Contract) extends Contract case class When(date: LocalDate, contract: Contract) extends Contract It's basically a DSL on top of Scala.
paperwork 11y ago
Not a digression, one of the reasons for this post was to practice Scala a bit further.

jiantastic 11y ago

Looks to be a really interesting article. Would really appreciate a font change and justified text though.

napo 11y ago
$('p').css('text-align', 'justify');
asw 11y ago
wtf
- cfarre 11y ago
  you're so rude!
paperwork 11y ago
Good idea. I'll justify the text. What's wrong with the font?

baq 11y ago

there was a blog post or an article by somebody at jane street about their own contract DSL a few years ago. can't find the link now though.

cheez 11y ago
The paper that kicked this all off is in the references section. I believe the original company was called lexifi

lucozade 11y ago

Declarative descriptions for financial contracts and markets having been actively used for a long time now. Executable descriptions not so much. The issue really is that it's solving problems we don't have with a solution that gets in the way of solving the problems we do. That's not to say people haven't tried, far from it. But successes tend to be either very rare or very niche.

cheez 11y ago

I worked on aa commercial product called F3 which, among other things, implemented this type of contract specification as a subset of its functionality. Unfortunately, I'm not sure if they ever learned how to sell it well. One of the most memorable stints in my career so far.

jackgavigan 11y ago

FpML (Financial Products Markup Language) was conceived shortly before the paper referenced in the blog post.[1] One of the paper's authors went on to create MLFi, a modelling language for finance[2] This stuff is a bit esoteric but I think it will attract more attention because of its relevance to smart contracts. 1: http://www.fpml.org/ http://www.fpml.org/ 2: https://www.lexifi.com/product/technology/contract-description-language https://www.lexifi.com/product/technology/contract-descripti...

tdees40 11y ago

I put together a Haskell proof of concept Monte Carlo pricing tool loosely based on SPJ's work: https://github.com/boundedvariation/quantfin https://github.com/boundedvariation/quantfin Still needs some love, but it was a pretty cool experience.

zhte415 11y ago
I did similar, but I did it Excel. It was cool, not an experiment, and it was done quickly, as Excel is the ultimate Agile tool in finance.

platz 11y ago

I'm a fan of the combinator way, but of course now you have to 'interpret' or 'evaluate' the structure to know what the contract means, so it might be hard to run a 'query' over a bazillion of them sitting in a database - you'd have to interpret each one (each one is a little black box). This is assuming that running a query or search over all the contracts is something that is useful to do, which I have no idea if that is the case (though seems like a common task in applications programming).