7 ms·

>topological quantum field theory lolol. i will bet all of my 401k that they absolutely do not. i've interviewed at a couple of hedge funds and have lots of f

>topological quantum field theory lolol. i will bet all of my 401k that they absolutely do not. i've interviewed at a couple of hedge funds and have lots of friends at various prop shops/hft firms/etc. most of them are using linear models. the ones that have market maker businesses deploy those linear models to fpgas. people act like these places are spooky magic cauldrons of physics + math + computer science. they're not (but it's definitely in their interest to promote such rumors). what they do have is very very robust data collection/aggregation infrastructure and backtesting systems. the rest is just correctness of execution of strats.

lordnacho 5y ago
I'll second that. I've done options trading, trend following, and HFT. Often you need smart people to do simple things, but they are still simple things. Gotta remember there's a lot of noise in financial data, and you don't actually have as much of it as people think. Even all the ticks on all the exchanges fit in a few GB of binary per day. RenTech is especially good at acting like they're from another planet, which I'm sure helps them attract those stellar mathematicians.
- mellavora 5y ago
  Your especially right about the noise in financial data. Nice thing about linear models (vs i.e. random forest, or even worse deep learning) is that they are limited in how they can overfit the noise. They only fit a known function to known regressors.
  - xapata 5y ago
    And, even more important, they aren't susceptible to adversarial activity like a deep net could be.
- dave_sullivan 5y ago
  That's actually been surprising to me but makes sense: "the entire history of the stock market" is not that big in terms of data, and the dynamics change frequently enough to where things that happened in 2000 are almost useless from a machine learning standpoint, my models perform best when trained on a much more selective subset (I believe because the reasons for trades and market participants keep changing, so the dynamics change enough to where old rules don't apply anymore).
  - helsinki 5y ago
    Correct. You can make money using only yesterday’s data for training purposes.
- WanderPanda 5y ago
  Can you distinguish noise from non-linear correlations? I feel like you can never prove the absence of non-linear correlation, but only showing it‘s presence once modelled/detected
- njarboe 5y ago
  RenTech is also basically the only largest hedge fund that has beat the market by a wide margin for decades. An average 66% annual return in their Medallion Fund since 1988 according to wikipedia. That is pretty much insane and seems to be something only space aliens could do. If anyone is using quantum computers to help trade, it is probably them.
  - kasey_junk 5y ago
    Worth noting that their public funds don’t return anything like the rumored Medallion fund returns (though they are generally pretty good).
    - selectodude 5y ago
      Medallion is also kept very small because I would guess the tricks that they use to keep those returns so high (maybe even some sort of topological quantum field theory!) don't scale well to larger investments.
      - njarboe 5y ago
        Not sure if I would call ~$10 billion very small and making $6-7 billion per year is good chunk of money. But, yea, I'm sure that would get bigger if they thought they could keep their returns high.
  - RationPhantoms 5y ago
    The Medallion Fund has already reached their max investment amount so while it's ground-breaking, no new RenTech employees can take advantage of it.
  - wolverine876 5y ago
    > An average 66% annual return in their Medallion Fund since 1988 according to wikipedia. The claim, the secondary (tertiary?) source, and the primary source make that all hard to not smile at. 66% annualized returns are more likely to be pyramid schemes - but that couldn't happen, could it?
- pgwhalen 5y ago
  > Even all the ticks on all the exchanges fit in a few GB of binary per day. This isn’t true once you include equity options, which have several orders of magnitude more data. However your general point still stands, because most options trading strategies don’t need such extreme granularity of data. Much of it can be ignored, or close to it.
dahak27 5y ago
You're right overall that most of these places are a lot less flashy on the inside (have worked at a couple). Linear models everywhere as you said However I think you're underestimating RenTech here. They're genuinely just in another league compared to what most people consider the "elite" quant shops. You're not getting in unless you're an actually impressive academic with a track-record, so I wouldn't be surprised if they're trying some weird stuff that other shops can't even understand (although I would imagine in small size vs. more vanilla stuff). IMO places like JS/CitSec do a really good job of bombarding campuses to inculcate this idea they're the absolute apex of mathematical wizardry, but the places that are really doing some dark magic shit aren't trying to get undergrads to apply to them. Ofc maybe I'm falling for the same kind of propaganda for RenTech
- noduerme 5y ago
  All of this wizardry operates under the premise that given just the right meta-meta-meta-formulae some tiny edge over randomness can be squeezed from the mountain of historical data by sheer willpower and brute force computation. It cannot. The sooner people accept this as an iron rule, the sooner they'll stop falling for scams that promise to foretell the future.
  - carnitine 5y ago
    But it obviously can. The returns of Renaissance’s Medallion fund cannot be explained by randomness. If every company in America was a hedge fund since 1776, the returns of Medallion would not arise by chance. They clearly have an edge (and don’t even accept outsider money, so no need to falsify).
    - ackbar03 5y ago
      I think your definitely idolizing them a bit too much. Maybe a bit ahead of the curve but not doing anything super fancy (although I guess those two statements are somewhat contradictory). Just from snippets I've sort of heard/read about, I think they were one of the earlier ones to move into HFT (although maybe not the super fast infrastructure heavy type these days). In some interview Simmons said they realized returns became more predictable they shorter the time frame they looked at and they pushed it to the extreme. I think there is also reason to suspect that they may have adopted some NLP strategies early on as well since Mercer was involved in that or something, and they initially hired a bulk of their team from IBMs NLP research team. Also they did not dodge 2008 completely, in some interview they said they lost close to / more than half their portfolio value in the market crash, but because they didn't have stupid leverage or outside investors or something like that, and also because they trusted their models, they didn't sell and held on. So maybe just slightly better execution but mostly the same. Anyways, I was reading about these guys back in 09 when quant trading wasn't so blown up. Now every kid whose decent at math seems to want to be the next renaissance, which just makes me feel like the best years for that are over.
santiagobasulto 5y ago
Where are Reinforcement Learning algorithms standing today in Finance? I remember ~3 years ago they were supposed to be the next big thing.
noduerme 5y ago
I mean, every model's output is in the eye of the beholder (or modeler). Take any unpredictable wave-like chart and ask someone which direction it's going in next - and guarantee you'll throw $100M on their choice. Hey! You just successfully collapsed a wave! It's funny how after all these millennia, most people still don't understand that someone claiming to divine the future from bloody egg yolks or whatever is just manipulating the future by getting people to believe in their magical divination.
bagels 5y ago
This sounds a lot more plausible. Being able to extract signals faster or that others cannot seems a much more likely strategy.
amelius 5y ago
Linear models, so not even deep learning models?
- short_sells_poo 5y ago
  No. Most "advanced" ML methods are extremely difficult to get to work in finance. You have a limited dataset that is low signal to noise ratio, highly non-normal and heteroscedastic. Crucially, you can't easily make more data (and no, synthetizing new price data is not an option in most cases). This makes the upfront costs very high, and you then have to prove that you can beat the performance of simple models, and not just by a little. A complicated ML model that offers a 10% better risk/reward ratio (e.g. 2.2 Sharpe instead of 2) is a complete failure, because you traded a simple model that is easy to reason about intuitively for a total blackbox that is very difficult to understand. Sure, you've got the quintessential marketing induced ML overlay that many firms do, but in all cases I've seen so far it's completely defanged and really there only so that it can serve as a marketing move.
  - quantumofalpha 5y ago
    Really depends on time horizon you're talking about. They actually work pretty well for HFT, there's plenty of data around, and most of the information is just in market data - no nasty low frequency stuff to deal with like news, earnings, alternative data, insider trading, butterflies flapping wings in china etc. But the problem is by the time your GPU spits out a datapoint somebody else can go in and trade a few thousand times in the meantime. State of the art on the most heavy competed exchanges is that your fpga (or even asic) with a fiber connected directly to the exchange needs to start sending ethernet/ip headers even before it made up its mind what it wants to send in the payload. At lower frequencies when the data gets thin and noise/overfitting is a major problem, yeah it makes sense to use simpler models. Bias/variance tradeoff in action.
    - short_sells_poo 5y ago
      Fair points. I talked with a couple of HFTs and the general view was that their asymptotic backtests look promising on GPUs but that for most of the liquid markets the latency is way too high - basically confirming what you write. On niche markets with low liquidity, one doesn't have such tight latency envelopes but those markets also offer more opportunities in general so again there's no real justification to use fancy ML models or GPUs in general.
RandomLensman 5y ago
Seconded. I think generally people give way too little weight to actually doing things well. There can be some clever mathematical ideas in there here and there, but in the end it has to come together into a running organization that makes little mistakes and can run consistently (while also observing a lot regulations). For me the "spooky magic" is more the ability to pull this off in size. Edit: I do have friends on the buy side that use pretty fancy models, but those places are not in the HFT/market making game. But using those models does not take away from needing to be able to translate the edge consistently and efficiently.
- deleted 5y ago
  [deleted]

lordnacho 5y ago

I'll second that. I've done options trading, trend following, and HFT. Often you need smart people to do simple things, but they are still simple things. Gotta remember there's a lot of noise in financial data, and you don't actually have as much of it as people think. Even all the ticks on all the exchanges fit in a few GB of binary per day. RenTech is especially good at acting like they're from another planet, which I'm sure helps them attract those stellar mathematicians.

mellavora 5y ago
Your especially right about the noise in financial data. Nice thing about linear models (vs i.e. random forest, or even worse deep learning) is that they are limited in how they can overfit the noise. They only fit a known function to known regressors.
- xapata 5y ago
  And, even more important, they aren't susceptible to adversarial activity like a deep net could be.
dave_sullivan 5y ago
That's actually been surprising to me but makes sense: "the entire history of the stock market" is not that big in terms of data, and the dynamics change frequently enough to where things that happened in 2000 are almost useless from a machine learning standpoint, my models perform best when trained on a much more selective subset (I believe because the reasons for trades and market participants keep changing, so the dynamics change enough to where old rules don't apply anymore).
- helsinki 5y ago
  Correct. You can make money using only yesterday’s data for training purposes.
WanderPanda 5y ago
Can you distinguish noise from non-linear correlations? I feel like you can never prove the absence of non-linear correlation, but only showing it‘s presence once modelled/detected
njarboe 5y ago
RenTech is also basically the only largest hedge fund that has beat the market by a wide margin for decades. An average 66% annual return in their Medallion Fund since 1988 according to wikipedia. That is pretty much insane and seems to be something only space aliens could do. If anyone is using quantum computers to help trade, it is probably them.
- kasey_junk 5y ago
  Worth noting that their public funds don’t return anything like the rumored Medallion fund returns (though they are generally pretty good).
  - selectodude 5y ago
    Medallion is also kept very small because I would guess the tricks that they use to keep those returns so high (maybe even some sort of topological quantum field theory!) don't scale well to larger investments.
    - njarboe 5y ago
      Not sure if I would call ~$10 billion very small and making $6-7 billion per year is good chunk of money. But, yea, I'm sure that would get bigger if they thought they could keep their returns high.
- RationPhantoms 5y ago
  The Medallion Fund has already reached their max investment amount so while it's ground-breaking, no new RenTech employees can take advantage of it.
- wolverine876 5y ago
  > An average 66% annual return in their Medallion Fund since 1988 according to wikipedia. The claim, the secondary (tertiary?) source, and the primary source make that all hard to not smile at. 66% annualized returns are more likely to be pyramid schemes - but that couldn't happen, could it?
pgwhalen 5y ago
> Even all the ticks on all the exchanges fit in a few GB of binary per day. This isn’t true once you include equity options, which have several orders of magnitude more data. However your general point still stands, because most options trading strategies don’t need such extreme granularity of data. Much of it can be ignored, or close to it.

dahak27 5y ago

You're right overall that most of these places are a lot less flashy on the inside (have worked at a couple). Linear models everywhere as you said However I think you're underestimating RenTech here. They're genuinely just in another league compared to what most people consider the "elite" quant shops. You're not getting in unless you're an actually impressive academic with a track-record, so I wouldn't be surprised if they're trying some weird stuff that other shops can't even understand (although I would imagine in small size vs. more vanilla stuff). IMO places like JS/CitSec do a really good job of bombarding campuses to inculcate this idea they're the absolute apex of mathematical wizardry, but the places that are really doing some dark magic shit aren't trying to get undergrads to apply to them. Ofc maybe I'm falling for the same kind of propaganda for RenTech

noduerme 5y ago
All of this wizardry operates under the premise that given just the right meta-meta-meta-formulae some tiny edge over randomness can be squeezed from the mountain of historical data by sheer willpower and brute force computation. It cannot. The sooner people accept this as an iron rule, the sooner they'll stop falling for scams that promise to foretell the future.
- carnitine 5y ago
  But it obviously can. The returns of Renaissance’s Medallion fund cannot be explained by randomness. If every company in America was a hedge fund since 1776, the returns of Medallion would not arise by chance. They clearly have an edge (and don’t even accept outsider money, so no need to falsify).
  - ackbar03 5y ago
    I think your definitely idolizing them a bit too much. Maybe a bit ahead of the curve but not doing anything super fancy (although I guess those two statements are somewhat contradictory). Just from snippets I've sort of heard/read about, I think they were one of the earlier ones to move into HFT (although maybe not the super fast infrastructure heavy type these days). In some interview Simmons said they realized returns became more predictable they shorter the time frame they looked at and they pushed it to the extreme. I think there is also reason to suspect that they may have adopted some NLP strategies early on as well since Mercer was involved in that or something, and they initially hired a bulk of their team from IBMs NLP research team. Also they did not dodge 2008 completely, in some interview they said they lost close to / more than half their portfolio value in the market crash, but because they didn't have stupid leverage or outside investors or something like that, and also because they trusted their models, they didn't sell and held on. So maybe just slightly better execution but mostly the same. Anyways, I was reading about these guys back in 09 when quant trading wasn't so blown up. Now every kid whose decent at math seems to want to be the next renaissance, which just makes me feel like the best years for that are over.

santiagobasulto 5y ago

Where are Reinforcement Learning algorithms standing today in Finance? I remember ~3 years ago they were supposed to be the next big thing.

noduerme 5y ago

I mean, every model's output is in the eye of the beholder (or modeler). Take any unpredictable wave-like chart and ask someone which direction it's going in next - and guarantee you'll throw $100M on their choice. Hey! You just successfully collapsed a wave! It's funny how after all these millennia, most people still don't understand that someone claiming to divine the future from bloody egg yolks or whatever is just manipulating the future by getting people to believe in their magical divination.

bagels 5y ago

This sounds a lot more plausible. Being able to extract signals faster or that others cannot seems a much more likely strategy.

amelius 5y ago

Linear models, so not even deep learning models?

short_sells_poo 5y ago
No. Most "advanced" ML methods are extremely difficult to get to work in finance. You have a limited dataset that is low signal to noise ratio, highly non-normal and heteroscedastic. Crucially, you can't easily make more data (and no, synthetizing new price data is not an option in most cases). This makes the upfront costs very high, and you then have to prove that you can beat the performance of simple models, and not just by a little. A complicated ML model that offers a 10% better risk/reward ratio (e.g. 2.2 Sharpe instead of 2) is a complete failure, because you traded a simple model that is easy to reason about intuitively for a total blackbox that is very difficult to understand. Sure, you've got the quintessential marketing induced ML overlay that many firms do, but in all cases I've seen so far it's completely defanged and really there only so that it can serve as a marketing move.
- quantumofalpha 5y ago
  Really depends on time horizon you're talking about. They actually work pretty well for HFT, there's plenty of data around, and most of the information is just in market data - no nasty low frequency stuff to deal with like news, earnings, alternative data, insider trading, butterflies flapping wings in china etc. But the problem is by the time your GPU spits out a datapoint somebody else can go in and trade a few thousand times in the meantime. State of the art on the most heavy competed exchanges is that your fpga (or even asic) with a fiber connected directly to the exchange needs to start sending ethernet/ip headers even before it made up its mind what it wants to send in the payload. At lower frequencies when the data gets thin and noise/overfitting is a major problem, yeah it makes sense to use simpler models. Bias/variance tradeoff in action.
  - short_sells_poo 5y ago
    Fair points. I talked with a couple of HFTs and the general view was that their asymptotic backtests look promising on GPUs but that for most of the liquid markets the latency is way too high - basically confirming what you write. On niche markets with low liquidity, one doesn't have such tight latency envelopes but those markets also offer more opportunities in general so again there's no real justification to use fancy ML models or GPUs in general.

RandomLensman 5y ago

Seconded. I think generally people give way too little weight to actually doing things well. There can be some clever mathematical ideas in there here and there, but in the end it has to come together into a running organization that makes little mistakes and can run consistently (while also observing a lot regulations). For me the "spooky magic" is more the ability to pull this off in size. Edit: I do have friends on the buy side that use pretty fancy models, but those places are not in the HFT/market making game. But using those models does not take away from needing to be able to translate the edge consistently and efficiently.

deleted 5y ago
[deleted]