8 ms·
We are moving into the quantitative finance industry
- kopter 13y agoGrowing up and moving into the finance industry? Excellent! Please fix the typo in the 'FP Haskell Center (TM) Overview' section: "... The personal edition aslo includes Emacs support ..."
- G228 13y agoThank you for pointing that out.
- KingMob 13y agoWhile I want to cheer on esoteric languages and functional programming, this seems like just more smart people wasting their brains on zero-sum detrimental finance games. At least for them, they'll probably make a lot of money.
- _delirium 13y agoEh, could produce some more generally applicable improvements, though it depends on whether they share back any code or expertise to the community. Some stuff I don't much like has produced very useful ancillary advances (the U.S. military is probably the biggest example), though some tech+finance companies are kind of black holes that never publish code/data/anything, so depends.
- confusedread 13y ago"zero-sum detrimental finance games" That depends on the market, doesn't it? They didn't say they were explicitly building systems for Forex, right? And who are you to tell people what they should "waste their brains on"? A lot of programmers I know in research labs/academia speak condescendingly towards really intelligent SV programmers who "waste their brains" on moving pixels around a screen in order to make people click more ads.
- _delirium 13y agoFwiw, that line actually came from someone in the tech industry, though no doubt some academics also like it. The oft-paraphrased quote is from Jeff Hammberbache, an early Facebook employee, who at the time of the comment was at Cloudera: "The best minds of my generation are thinking about how to make people click ads. That sucks."
- gjm11 13y agoSo far, no one seems to have come up with an economic system that's more effective in harnessing the collective skills of the human race to improve the world (on balance) for everyone than the one we have at the moment. It has the regrettable feature that it strongly incentivizes some very smart people to spend their careers encouraging people to click on advertisements, and some other very smart people to spend their careers arranging for the profits from exploiting subtle equity mispricings to flow to one random set of rich people rather than another. But the net effect is pretty good, or more precisely less depressingly wretched than that of any other system found to date. (And some of those very smart people get rich by encouraging ad-clicking or exploiting equity mispricings, and then use their money and/or freedom to do neat things from which everyone benefits. Is that such a bad way for things to work?)
- _delirium 13y agoI guess I have some doubt that the American economic system in particular is a local optimum. Something radically different carries risks, yes. But the American system is a bit of an extreme in its encouragement to do anything that makes a profit, even within the local space of variation of vaguely market-based mixed economies. In Denmark, we have a bit more "managed" economy, perhaps even "socialist" in a certain weak sense (there used to be more of a socialist ethos, alas). The country is a bit less focused on VC returns, big paydays, and "exits", and frowns a bit more on people doing mercenary nonsense for profit. You get paid to go to university, there is extensive arts funding, an industrial/scientific policy, etc., among other things that try to redirect energies elsewhere. And that historical "approximate equality for all, big profits for none, put your energy into something worthwhile" approach to an economy has really not been disastrous, quite the opposite.
- gaze 13y agoYes, I really don't like that they're dipping their toes into this crap. I like that it means we might get a better Repa or whatever but still.
- gjm11 13y ago> zero-sum detrimental finance games Except: 1. Fortunately we have taxation to turn highly profitable zero-sum professions into slightly less profitable positive-sum ones. 2. If doing stuff for the quantitative finance industry means more resources for building generally useful Haskell libraries, then everyone wins. 3. If you make a lot of money playing zero-sum detrimental finance games, then you can (a) retire and do more interesting or socially-useful things (like contributing to interesting software projects) and/or (b) give a pile of it away. On 3b, I'll remark that although I don't work in the finance industry (I've spent most of my career doing R&D for small technology companies), my salary has never been very large by present-day Silicon Valley standards, and my charitable giving isn't especially heroic, I think it very likely that I've done more net good to the world by giving away some of what I earn than I ever have by doing my actual job. If I were to move into finance (which isn't impossible -- I'm a mathematician and quite a lot of mathematicians do that) then it would probably be a net gain in global utility even if everything I did could correctly be called zero-sum detrimental finance games.
- judk 13y ago#1 is not how macroeconomics works, not without some quite controversial assumptions.
- gjm11 13y agoExplain? Joe is a hedge fund manager. His job consists of arranging for one bunch of random rich people (his fund's clients) to get a little bit richer at the expense of another bunch of random rich people (other people trading in the same markets). For doing this, his clients pay him $10M/year. The government takes $3M/year of this (note: all numbers in this comment are made-up nonsense), which it uses for building roads and paying schoolteachers and paying for some people's healthcare. End result: Joe's clients, in exchange for Joe's skill at siphoning money out of the stock market (or wherever) to them, have in effect paid $7M to Joe and paid $3M to build roads, provide medical services to poor people, provide education to everyone, etc. What controversial assumptions are necessary to make that end result (1) plausible and (2) positive-sum despite the net uselessness of Joe's work? In some cases one can argue that the useful things done with tax revenue need to be offset against the reduced incentive to work created by the existence of taxes. Not in this case, I think. Joe's work is, to a very good approximation, completely valueless to society, and it does good overall only by being taxed. And if the prospect of being paid only $7M/year makes Joe quit hedgefundery in favour of some easier and less stressful but less well-paid profession, the chances are he'll do more good there. (More precisely: If the world's Joes collectively decide to do a bit less hedge fund management and a bit more engineering or something, that's probably all to the good.) (It may sound as if I'm criticizing Joe. I'm not. He's not doing any harm to speak of, I think his earnings do a lot of good by being taxed, and of course he may be doing other useful things with some of that money.)
- tom_b 13y agoI would like to read details about the model -> production code that the IAP product uses. I've been developing an interest in statistical modeling and moving those models to production code (and of course, quantitative finance is an "obvious" place where this happens). But I find, even in the statistical genomics world, that the default process still seems to be: (1) specialized tools for data cleaning and initial crunching (maybe hadoop, maybe HPC clusters (2) consumption of data from (1) in R or Stata/SAS for model generation (3) only vague, research-y things where the model from (2) goes to production. The vague part of (3) is probably just due to my simply not knowing what people are doing with the generated models in (2). Specifically regarding quantitative finance, it would seem that trumpeting your implementation programming language would mainly be met with blank stares, Jane Street excepted . . .
- kasey_junk 13y agoHaving built systems that try to do this in the past, I encourage you to question a few assumptions that many people have about the inefficiencies of this silo'd model. (1) That quants want to program (or that the business should want them to) in something other than R, Matlap, NumPy, Julia, etc. (2) That the hard part of moving a model into production is translating the model. (3) That the model is the part of the system that will be the main driver of success. Good Luck!
- Estragon 13y agoThis is very interesting. What actually is the hard part and what drives success, in your experience?
- kasey_junk 13y agoI've found that quants do not follow software best practices. They do not produce production ready code and it isn't because they aren't using the right language. It's because they aren't production software developers (and I don't want them to be, I want them out there doing analysis). Further, I've never encountered a model that didn't over simplify operations/execution requirements. They do this so that they don't get bogged down in details when they haven't figured out the big picture yet, and that's a good thing. What drives success is having a good set of quants who can communicate with production capable software engineers. At the end of the day, the hard part of a quants job is usually finding concepts. Once they are found translating them into something actionable is usually reasonably easy.
- tdees40 13y agoMatlab, C++, C# and Java rule the roost in the finance world. Anything else is treated with intense suspicion by finance companies. (I was once at a very large financial company that wouldn't approve Python for use internally at all because it was open-source. Let that sink in. We used it anyway of course.) And yes, R and Python are making real inroads in quant finance, but they still have a small market share. Most of the guys writing code either want MS Office integration (sadly all traders love Excel), raw speed (C/C++/Java), or easy prototyping (Matlab). People don't know Haskell, audit departments don't know Haskell, and Haskell isn't as fast as C++, so this is an uphill climb, but I wish them the best. If you want to know the state of the art in quant finance, go stare at the horrible over-engineered debacle that is QuantLib and see what kind of programmers we have working in this area.
- sheetjs 13y ago> sadly all traders love Excel Why do you say 'sadly'? Excel is an incredible tool that other traders can work with, it is sufficiently expressive to run trades and backtests, and the connectors (e.g. Bloomberg) are better than the python equivalent
- kasey_junk 13y agoI don't have any problem with csv files, but models that are embedded in a mix of formula's, multiple workbooks connected over the network, & VBA are tons harder to translate to production level code than those written in other numerical systems.
- tdees40 13y agoThe problem with Excel is a problem of state. It's great for viewing immutable information (although custom web apps aren't difficult for most basic tasks), but as soon as you get into running production processes in Excel things fall apart. A single stray keystroke populates a random cell, which blows up your calculations, and you can't figure out what you did. This has happened dozens of times in my career in finance, and I've found no reasonable solution to it.
- nazka 13y ago
- dia80 13y agoIPython + Pandas[0] + (sklearn/cvxopt/etc.) are an amazing combination and as far as I can see gaining share. I would love to work in Haskell but I'm always scared the library support / interactivity won't be there. Pandas has 8600 commits at the time of writing, unassailable? [0] http://pandas.pydata.org/ http://pandas.pydata.org/
- radikalus 13y agoThis stack gets a LOT of use in HFT. (Including us) Honestly, the three things I'd most love to have are: - more wrappers, support, documentation on PyMC -- a LOT could be done here and this could replace HUGE amounts of code in all sorts of place - libraries/models on top of cvxopt - more polished ipython notebook -> pdf or d3.js html -- I do this a fair amount for auto eod reports etc And maybe a fourth...a good open source alternative to kx...preferably one that can accept .h5 files.
- hogu 13y agowhat group are you with? we're working on some relevant stuff http://continuum.io/ http://continuum.io/ I can be reached at hugo@continuum.io if you want to chat about it
- notastartup 13y agoCan someone help me understand how hedge funds, trading desks use this type of software?
- ixmatus 13y agoHaskell has been used to model financial contracts by a few firms with an eye towards type-level correctness guaranteed by the compiler (that you cannot get with "mundane" type systems found in C++ / C / Java / etc...) That, coupled with Haskell's speed, concurrency features, and other amazing features makes it an attractive platform. Alpha Heavy Industries is a small(er) SV based firm using Haskell a lot.
- noname123 13y agoHi ixmatus, Can you give a more specific example? Are you saying that instead of in Java/C++ where you have: GenericSecurity > GenericOption > EuropeanOption, BarrierOption, AmericanOption You can have more mutable types/classes at run-time and more injections instead of the concrete ones defined at compiled time? Also, can you give any concurrency examples in terms of concurrency features? Is it similar to Actor model in Scala? Or based more on concurrent data structures as in java.util.concurrent.*?
- tel 13y agoTake a look at [1] for details on the contract language they built. Instead of modeling things in a taxonomy, they build the logic of contracts up from primitives using a combinator library. Concurrency in Haskell is a big thing. At the lowest level, purity means there's a lot of opportunity for parallelization. Atop that Haskell has a really great green threads system and new GHC has a very, very performant IO manager for running them. It's GC'd so you do have thread-local pauses and they're not as nice as Erlang's actor-local GC. Atop that you've got a nice channel system, the best implementation of Software Transactional Memory around, and neat new libraries like LVish for doing eventual consistency in a very reasonable fashion. [1] http://research.microsoft.com/en-us/um/people/simonpj/Papers/financial-contracts/contracts-icfp.htm http://research.microsoft.com/en-us/um/people/simonpj/Papers...
- deleted 13y ago[deleted]
- zmanian 13y agoThe advantage Haskell has in finance is there are a number of long term success stories to point to. Standard Chartered traders use an excel to Haskell interface for risk management and trade execution. S&P Capital IQ uses a Haskell DSL called Ermine for their reporting engine. Tsuru capital is the largest hedge fund to use Haskell for trading but there many smaller shops like Alpha Heavy Industries. http://www.haskell.org/haskellwiki/Haskell_in_industry http://www.haskell.org/haskellwiki/Haskell_in_industry FP Complete repeatedly mentioned Standard Chartered's Excel to Haskell system as something that they would want to see more widely adopted.
- deleted 13y ago[deleted]
- rajacombinator 13y agothis is not a good industry to move into
- habosa 13y agoIs there a difference between quantitative finance and HFT? I imagine it's a square-rectangle relationship. I'm a big fan of new tools to make mathematical models for traditional investing, but I am fairly against the idea and practice of HFT.
- gjm11 13y agoIf you hope to make money from trading (beyond the quite respectable gains you can get just from buying index funds or whatever), you basically have two ways to do it. (1) Make better predictions (or better adjustments for risk, or better hedging of one asset against another, etc.; these are all closely related) than others. (2) Buy and sell without any attempt at predicting the future; be willing to take the other side of any trade anyone wants to make, in return for which you get to buy a little low and sell a little high. For #1 you need quantitative finance. #2 is the activity known as "market making" and HFT is what you get when you automate it and then let the people doing it compete. The algorithms in HFT are pretty simple-minded; the goal is to offer as thin a spread between buying and selling prices as you can while still making a profit, and to offer it faster (when someone comes looking to trade) than anyone else. The latter is genuinely useful when it's the difference between (say) 5 minutes and 5 seconds. Unfortunately the same incentives that make it worth while for high-frequency traders to offer sub-second response times then make it worth while for them to keep chasing faster and faster times, until the most important asset a HFT shop has is a bunch of computers positioned slightly closer to a big exchange than anyone else's. There's not much of what people mostly mean by "quantitative finance" in a typical HFT system, I think.
- tunesmith 13y agoI think quantitative finance is interesting, but for personal purposes I'd really only want to trade once a month or so, as opposed to multiple times a day. I'm not sure what kind of software libraries/languages are suited for that kind of approach, when everything seems tuned for HFT.
- BSousa 13y agoFor that kind, you don't really need any specific software. You can do it in Ruby/Python/Haskell/Whatever. As long as you can communicate with an API (Yahoo Finance to download the data for example and one broker that allows you API access to issue orders) any language will do. You can run your software once a month or week, analyse the data and issue the orders. If you are keen, you can even use Google News/NLP/Sentiment Analysis in your model, again, in any languages you want. Take a look at: quantstart.com He has some tutorials with Python to do so, and he is releasing a book (hopefully soon) that should give you a good overview of it all. Also, I recommend you read Options, Futures and Other Derivatives to understand how most of this financial instruments work. There is a lot of good information there.
- tunesmith 13y agoThank you - that is very good information. I have done some perl scripting using "Finance::Quote" and "Finance::QuoteHist" but I guess I have been looking for the next step up. Particularly historical financial statement information, like EPS and Sales data quarterly history. Historical fundamentals as a data feed. I don't know where to find that. I'll check quantstart and see what they offer.