4 ms·
So, is it data, a database, or a risk-management tool? Sounds like something that could have been hard with constraints of 1990s hardware, but these days any o
by lordnacho 10y ago
So, is it data, a database, or a risk-management tool?
Sounds like something that could have been hard with constraints of 1990s hardware, but these days any old person could rent a cloud to do this.
Also, I suspect people who aren't involved directly in the technology talk it up a lot more than would someone who was involved in building it. The execs have bet on it with budget money, and they get to open or close the gate, making it seem more important than it really is.
At the end of the day, every financial form has a bunch of code that lets them slice and dice their data. GS didn't survive the crisis because they had this system, they survived because they had a system and made some wise choices.
- jnordwick 10y agoSecDB has been in a class of its own for years now. I've seen and heard of many other banks try to replicate it and failed. It's customizability and performance are very well respected in fintech circles.
- Maven911 10y agoWhy do they fail to replicate it ? Especially if I am hearing it's the same people developing (re-developing) it over and over again
- kchoudhu 10y agoThe people who wrote SecDB are now in charge of the technology org at Goldman, and they have presented the stack as a core firm asset in need of constant development and investment to the business, which has -- surprisingly -- accepted this position. SecDB is not just another IT project with an end date for the business. Ultimately, that is what distinguishes it from all the other copycats on the Street.
- KMag 10y agoHaving some friends who were poached for one of those replication efforts, it sounds like an issue of scale, culture, and transplant rejection. A lot of the advantages of a unified cross-desk risk calculation system aren't realized until it's widely adopted, and it's difficult getting the desks to buy in to changing their thinking to an FRP mindset and adapting all of their existing code. At Goldman, SecDb was developed by the Strats, who were part of the individual desks in a hybrid tech/quant role, not part of the Tech division. To some extent, SecDb was a bottom-up system driven by the desks who needed it, but in the other firms, it has been the technology divisions trying to push the new system on the desks. It's partly a political problem. In order to jumpstart their systems, the other firms poached experienced talent from Goldman. Goldman's culture is generally receptive to suggestions and constructive criticism, at least comparatively. These new upstarts showing up and making suggestions to the old guard has not always gone over well at other firms. I'm told one ex-Strat who wrote discount curve calculation code at Goldman showing up at the new firm's technology division, noticing a discontinuity in his new firm's curve, poking around the code in the quant svn repository, and gently pointing out to the author that the same mistake had been made at Goldman several years prior. The response was not so much the expected "oh, hey thanks", but instead the very next day the entire technology division's read access to the quants' svn repository got revoked.
- danpalmer 10y agoAs I understand it, the closest thing would be something like writing your application for MSSQL. You have a cluster that is storing your data, and with a complex set of triggers, views, queries, etc, you're re-calculating things live. Unfortunately you program SecDB mostly in proprietary internal languages (not SQL), which while I understand they are moderately capable, they lag behind the 'state of the art' in programming by quite a long way - I wouldn't image it's even on a par with Python, I think somewhere like Lua is probably more accurate. Caveat - this is based on chatting to people from GS, I don't have experience working with any of this first-hand.
- kchoudhu 10y agoIt's all of the above. The secret sauce isn't the technology -- it's the decades of quant models that have been programmed using the stack.
- smallnamespace 10y agoIt's also a very slick integrated development environment for financial use cases. The securities/price data are exposed as first-class entities in an ORM layer and you can manipulate them easily with scoping rules. For example, if you want to get the current USD/JPY rate, it's just: Rate(Security("USD/JPY")) If you want to know what the price of an FX contract is under different rate regimes, you can actually set that price and then reevaluate the contract, e.g. SetDiddlescope() { Foreach(X, [90, 100, 110]) { Rate(Security("JPY/USD")) = X // Only takes effect within this scope Print(X + ": " + Price(MyFXSwap)) } } How this actually works is that most object's fields are dynamically recalculated using an early form of functional reactive programming (note that SecDB was mostly built in the 90s). Fields specify their dependencies, and when their dependencies change, the field's value gets invalidated. This is very similar to how ReactJS or Angular work. Unlike React or Angular though, you can force intermediate calculations to take on certain values as well; for example, in the scope above, you can also force Price(MyFXSwap) to take on a particular value. This is useful, for example, you had a book of securities, MyFXSwap was one of them, and you wanted to see how the price of your book would change. Even though all the securities are actually in a database somewhere, anything happening locally gets cached to an in-memory database so the performance is pretty decent. The security database itself, btw, was much closer to a pure distributed object store; there wasn't much in the way of querying ability.
- RockyMcNuts 10y agoTo get the benefit, you need the securities models…and the data to feed the models…real-time prices…covariance matrices…volatility surfaces…at which point you're living in a very Goldman world. Not that that's a bad thing but it creates a lot of lock-in. Also I'm not sure what the upside is to letting people run on-premises, vs. providing cloud APIs based on it (they make it sound like the former). Just the software, without the models, data, strategists, is kind of like providing a proprietary environment of Microsoft SQL + R.
- kchoudhu 10y ago> form of functional reactive programming I worked there for almost three years before I realized that we were playing around with FRP. > in an ORM layer ... and I didn't realize that I was using an ORM for a good six months. Life at Goldman was good as a developer. Retinal scarring from those damned blue windows aside, I kinda miss it sometimes.