7 ms·

Statistical Arbitrage – An Easy Walkthrough

150 points by xgdgsc 3y ago

cesarvarela 3y ago
Imagine getting to know about this 20 years ago.
- Galanwe 3y ago
  Statarb, the pair trading form, has been known since more than 20 years
- lextuto 3y ago
  Here's a link[0] to my (non-monetized, WIP) blog about Renaissance Technologies[1] who were one of the first ones to use statistical arbitrage. Others were D.E. Shaw[2] and Pete Mueller at Morgan Stanley (later PDT Partners[3].) [0] https://sileret.com/blog/renaissance-technologies/ https://sileret.com/blog/renaissance-technologies/ [1] https://en.wikipedia.org/wiki/Renaissance_Technologies https://en.wikipedia.org/wiki/Renaissance_Technologies [2] https://en.wikipedia.org/wiki/Pete_Muller_(businessman_and_singer-songwriter) https://en.wikipedia.org/wiki/Pete_Muller_(businessman_and_s... [3] https://en.wikipedia.org/wiki/PDT_Partners https://en.wikipedia.org/wiki/PDT_Partners
- mhh__ 3y ago
  Thorpe has some interesting (unfortunately online the wayback machine has them) articles on stat-arb and his (modest) discovery of black-scholes from the 90s
mg 3y ago
The unintuitive thing about the stock market is that you are not trying to predict a system unrelated to yourself. But you are trying to predict other people's behavior. People who are just like you. So when you notice a pattern - say "Stock S trades lower on Tuesdays and higher on Fridays", what do you do? Do you buy on Tuesday to exploit the pattern? Or do you buy on Friday because you are not a unique snowflake and others will notice the same historical pattern and therefore reverse it in the future? There is no way to decide. Because even if you see long-running patterns in the history of the stock market, those patterns might have reverted just at that moment when people like you noticed them. This holds for all patterns. Even if they are as complex as the ones described in the article.
- btheshoe 3y ago
  ya so notice the pattern before anyone else and extract money by filling the inefficiency. this isn't a profound insight, the nature of alpha is that it's temporary and limited. doesn't mean it doesn't exist and you can't take advantage of it - as long as you're better than everyone else (faster, smarter, cheat-ier). tbh it's just like entrepreneurship - starting a business will attract competitors. but you can just outcompete them.
  - mg 3y ago
    You don't know if "anyone else" already noticed the pattern. That does not reflect in the historical prices you can see. It will reflect in future prices which you can't see. And waiting for those does not help either. You are always at square one. Because others are sitting somewhere, watching the data and thinking about it just like you.
    - beagle3 3y ago
      They don’t all notice a long time pattern at exactly the same time. Alpha decays over time as more people notice and exploit it, or the market slowly changes. When alpha disappears overnight, it’s almost surely for reasons unrelated to other statistical event players - e.g. a tectonic shift caused by some bankruptcy, interest change, political decision, etc. However, many traders do not know how to properly model and backtest. It works on backtest, fails in the real world, and they explain that “alpha is gone” when in reality it wasn’t there to begin with - it’s just that their backtest was bad - usually overfit or unrealistic assumptions.
      - mg 3y ago
        You describe a world where you can watch the pattern slowly fade out and stop when it is gone. But in reality, there is noise. Say you start exploiting the pattern. You buy on Tuesday and sell on Friday. After 3 weeks of doing so, you lost money every time. Is the pattern gone? Or is this just statistical noise? Should you stop or plow through? You don't know. Another way to look at it: We would have the exact same discussions if stocks prices were just random walk series. To make a point in favor of pattern arbitrage, one would have to show that stocks differ from random walk series. Enough to be worth trading against this difference. As far as I know, nobody ever came up with a good argument in favor of this assumption.
lordnacho 3y ago
As someone who does kind of thing, I think while this is a good explainer, it doesn't explain the thing that actually makes it work. It can be fascinating to read about and obviously it's an immensely deep subject. But there's a tendency to focus on the pure intellectual "how do the prices move" models that doesn't take into account the business around how you actually get the model operating, both from an engineering point of view of "how do I code up the execution and housekeeping" and in the larger business sense of "how do I arrange the shop so that I am able to trade this model".
- TheAlchemist 3y ago
  This exactly. Working in the field too - I tend to think now that for quite a lot of this strategies, it's not really the genius idea behind that is worth money, it's all the accumulated knowledge about what could kill you that's getting incorporated into the system. Usually boring, tedious stuff create the best moat for the business. How do you make sure your ticker database is up to date, how do you make sure the dividends calendars are up to date, that you are trading the ticker you want to trade, and not the same name but on a different exchange and with a different fiscality, etc. It's a bit like the saying about startups - the ideas are worthless about execution.
- tilne 3y ago
  > how do I arrange the shop so that I am able to trade this model Any chance you could explain this a bit more? Intuitively it makes sense that this would be as important if not more so than the modeling itself, but I have no context for the difficulties encountered when implementing it.
  - lordnacho 3y ago
    So for instance your model might need access to borrowing. In fact I know people whose entire trading strategy depends on being able to access shares to borrow in large size so that they can squeeze out a few bps of yield on certain situations. The guys who control the borrowing are picky about who they want to lend to. Or your model might need to be really, really fast. Not everyone can just have access to a microwave link. The guys with the microwave are also picky. Or your model needs certain kinds of data, which you will need to get certain vendors to sell you. You might have a need to trade multiple models at once, which are not profitable on their own (uncorrelated, share costs). This is a juggling act done often at the business level. For almost all trading you need some staff who know how to do things. It's not always the case that you just pay them a salary to do some job. Business decisions there as well.
nurettin 3y ago
After reading the whole thing, the main idea feels more like "statistical arbitrage, debunked!". He does the womp-womp when he doesn't find any alpha. He even mentions that the paper's author is often cited as an expert.
- Galanwe 3y ago
  What is showcased here is a dummy pair trading example on single stock vs basket. Of course it does not work anymore. There are a lot more evolved kinds of statarb though, that still work nowadays, and most likely new ones that we don't know just yet. You can change the distance function, do it fully cross sectionally, etc.
  - plouffy 3y ago
    Can you share some links or books that you would recommend for those that want to read further ?
    - Galanwe 3y ago
      Reading articles and publications is part of the job of quantitative research, and being a judge of their quality and reproducability even more so. So, in all fair honesty with you, no I will not give you recommendations.
taylorius 3y ago
A small disagreement with the article. Statistical arbitrage is a more general principle than "reliance on mean reversion" (though that is certainly a common example of statarb). It really encompasses any useful statistical prediction that can be used to trade with.
- Galanwe 3y ago
  While I agree that the article is somewhat vague and imprecise, it is not accurate to encompass all statistical predictions as statarb either. That would more be what "quantitative" is. For a strategy to be considered statistical arbitrage, it needs to be something _against_ something else. You can do purely directional statistical predictions, and that would not be statarb.
  - taylorius 3y ago
    Very true! Though the dividing line between purely directional, and something against something else is unclear. Every price is against something else, in some sense.
chollida1 3y ago
A few notes We normally use ordinary least squares when we do the linear modelling. Maybe they are and I can’t read the Julia code. Also they aren’t adjusting their closing prices do dividends as far as I can tell. Nor are they taking dividends payments into account at all as far as I can tell. They did get the shift from daily to hourly right when they went to their crypto example so good for them. They are ahead of most day traders there:) Also no work in the hurst exponent to figure out the reversion time frame Also no modelling on how each pair will fit into your portfolio composition and if you can get borrow, which you certainly can in this instance, and trading fees. Also they skip regime detection, which is determining what regime the market in general is. Lots of pairs only work in up or down or sideways markets and you’d want a different ratio for each. And you’d want Tom look at how your pairs works in each market regime. I find this to be the most important piece and. You never seen pairs tutorials take this into account. Probably because you his is the hard piece that makes you money. This is why pairs are thought to trade. This link is the star arg 101 tutorial that is in 50 different places in the internet but it doesn’t show any of the actual meaningful work you need to do to trade pairs. Other missing pieces are leg sizing, how long you let a leg hang, how many legs do you out out, portfolio composition, dynamic hedge ratio calculation,market regime detection. It shows the easy piece that you can code up in an afternoon. And now I feel like I’ve shit all over this poor kids project. It’s well done and covers the 101 issues with a few pretty large missing pieces that I’ve pointed out
- tdubhro1 3y ago
  curious what exactly you're referring to by market regime detection / classification, anything I've seen on this (several models from bulge sellside) has been backward looking and fairly useless.
  - chollida1 3y ago
    We’ll refine detection is detecting when the market refine changes. But I assume you know that. Typically you’ll run one set of strategies in a bull market for a given Security and another in a sideways market and others in a bear market. Keep in mind different securities can be in different regimes at the same time. And the market in general can be in a different return from individual securities. To detect regime change you typically model it as a hidden Markov process. For back testing we typical pre-label the data by the regime that we know the market was in because we have the benefit of hindsight:
deleted 3y ago
[deleted]
apwheele 3y ago
I have an analysis of cointegration between Bitcoin/Ethereum (in python) if folks are interested, https://andrewpwheeler.com/2022/02/09/cointegration-analysis-of-ethereum-and-bitcoin/ https://andrewpwheeler.com/2022/02/09/cointegration-analysis... This was over a year ago, so dynamics could of course have changed. But based on that analysis, BTC tended to lead ETH (Bitcoin goes up, ETH goes up). ETH up lead to BTC down though. Not a trader, so don't take it as advice. Impressed by folks who beat the market, it always appears too volatile to me in my amateur stat modelling attempts to even kid myself.
laputan_machine 3y ago
> We this that if JPM rises higher than XLF then it will soon revert and trade lower shortly. Likewise, if JPM falls lower than XLF then we think it will soon trade higher. Why?
- alpark3 3y ago
  XLF is a financial sector ETF. JPM is a huge bank. If JPM does badly, XLF should probably do badly, and vice versa. Then it stands to reason that if JPM falls below XLF, it will soon go back up to where XLF is. This is relative, so the same situation with XLF rising above JPM, and JPM catching up/XLF falling back down, and similar situations.
  - laputan_machine 3y ago
    Thanks! I know it may seem obvious to people clued up on this stuff but it's basically a dark art to me. I appreciate the comment
victorbjorklund 3y ago
"The good thing about it being a negative result means that we don’t have to start considering transaction costs or other annoying things like that." This cracked me up. Haha. That is a good way to look as a "loosing strategy".

150 points by xgdgsc 3y ago

cesarvarela 3y ago

Imagine getting to know about this 20 years ago.

Galanwe 3y ago
Statarb, the pair trading form, has been known since more than 20 years
lextuto 3y ago
Here's a link[0] to my (non-monetized, WIP) blog about Renaissance Technologies[1] who were one of the first ones to use statistical arbitrage. Others were D.E. Shaw[2] and Pete Mueller at Morgan Stanley (later PDT Partners[3].) [0] https://sileret.com/blog/renaissance-technologies/ https://sileret.com/blog/renaissance-technologies/ [1] https://en.wikipedia.org/wiki/Renaissance_Technologies https://en.wikipedia.org/wiki/Renaissance_Technologies [2] https://en.wikipedia.org/wiki/Pete_Muller_(businessman_and_singer-songwriter) https://en.wikipedia.org/wiki/Pete_Muller_(businessman_and_s... [3] https://en.wikipedia.org/wiki/PDT_Partners https://en.wikipedia.org/wiki/PDT_Partners
mhh__ 3y ago
Thorpe has some interesting (unfortunately online the wayback machine has them) articles on stat-arb and his (modest) discovery of black-scholes from the 90s

mg 3y ago

The unintuitive thing about the stock market is that you are not trying to predict a system unrelated to yourself. But you are trying to predict other people's behavior. People who are just like you. So when you notice a pattern - say "Stock S trades lower on Tuesdays and higher on Fridays", what do you do? Do you buy on Tuesday to exploit the pattern? Or do you buy on Friday because you are not a unique snowflake and others will notice the same historical pattern and therefore reverse it in the future? There is no way to decide. Because even if you see long-running patterns in the history of the stock market, those patterns might have reverted just at that moment when people like you noticed them. This holds for all patterns. Even if they are as complex as the ones described in the article.

btheshoe 3y ago
ya so notice the pattern before anyone else and extract money by filling the inefficiency. this isn't a profound insight, the nature of alpha is that it's temporary and limited. doesn't mean it doesn't exist and you can't take advantage of it - as long as you're better than everyone else (faster, smarter, cheat-ier). tbh it's just like entrepreneurship - starting a business will attract competitors. but you can just outcompete them.
- mg 3y ago
  You don't know if "anyone else" already noticed the pattern. That does not reflect in the historical prices you can see. It will reflect in future prices which you can't see. And waiting for those does not help either. You are always at square one. Because others are sitting somewhere, watching the data and thinking about it just like you.
  - beagle3 3y ago
    They don’t all notice a long time pattern at exactly the same time. Alpha decays over time as more people notice and exploit it, or the market slowly changes. When alpha disappears overnight, it’s almost surely for reasons unrelated to other statistical event players - e.g. a tectonic shift caused by some bankruptcy, interest change, political decision, etc. However, many traders do not know how to properly model and backtest. It works on backtest, fails in the real world, and they explain that “alpha is gone” when in reality it wasn’t there to begin with - it’s just that their backtest was bad - usually overfit or unrealistic assumptions.
    - mg 3y ago
      You describe a world where you can watch the pattern slowly fade out and stop when it is gone. But in reality, there is noise. Say you start exploiting the pattern. You buy on Tuesday and sell on Friday. After 3 weeks of doing so, you lost money every time. Is the pattern gone? Or is this just statistical noise? Should you stop or plow through? You don't know. Another way to look at it: We would have the exact same discussions if stocks prices were just random walk series. To make a point in favor of pattern arbitrage, one would have to show that stocks differ from random walk series. Enough to be worth trading against this difference. As far as I know, nobody ever came up with a good argument in favor of this assumption.

lordnacho 3y ago

As someone who does kind of thing, I think while this is a good explainer, it doesn't explain the thing that actually makes it work. It can be fascinating to read about and obviously it's an immensely deep subject. But there's a tendency to focus on the pure intellectual "how do the prices move" models that doesn't take into account the business around how you actually get the model operating, both from an engineering point of view of "how do I code up the execution and housekeeping" and in the larger business sense of "how do I arrange the shop so that I am able to trade this model".

TheAlchemist 3y ago
This exactly. Working in the field too - I tend to think now that for quite a lot of this strategies, it's not really the genius idea behind that is worth money, it's all the accumulated knowledge about what could kill you that's getting incorporated into the system. Usually boring, tedious stuff create the best moat for the business. How do you make sure your ticker database is up to date, how do you make sure the dividends calendars are up to date, that you are trading the ticker you want to trade, and not the same name but on a different exchange and with a different fiscality, etc. It's a bit like the saying about startups - the ideas are worthless about execution.
tilne 3y ago
> how do I arrange the shop so that I am able to trade this model Any chance you could explain this a bit more? Intuitively it makes sense that this would be as important if not more so than the modeling itself, but I have no context for the difficulties encountered when implementing it.
- lordnacho 3y ago
  So for instance your model might need access to borrowing. In fact I know people whose entire trading strategy depends on being able to access shares to borrow in large size so that they can squeeze out a few bps of yield on certain situations. The guys who control the borrowing are picky about who they want to lend to. Or your model might need to be really, really fast. Not everyone can just have access to a microwave link. The guys with the microwave are also picky. Or your model needs certain kinds of data, which you will need to get certain vendors to sell you. You might have a need to trade multiple models at once, which are not profitable on their own (uncorrelated, share costs). This is a juggling act done often at the business level. For almost all trading you need some staff who know how to do things. It's not always the case that you just pay them a salary to do some job. Business decisions there as well.

nurettin 3y ago

After reading the whole thing, the main idea feels more like "statistical arbitrage, debunked!". He does the womp-womp when he doesn't find any alpha. He even mentions that the paper's author is often cited as an expert.

Galanwe 3y ago
What is showcased here is a dummy pair trading example on single stock vs basket. Of course it does not work anymore. There are a lot more evolved kinds of statarb though, that still work nowadays, and most likely new ones that we don't know just yet. You can change the distance function, do it fully cross sectionally, etc.
- plouffy 3y ago
  Can you share some links or books that you would recommend for those that want to read further ?
  - Galanwe 3y ago
    Reading articles and publications is part of the job of quantitative research, and being a judge of their quality and reproducability even more so. So, in all fair honesty with you, no I will not give you recommendations.

taylorius 3y ago

A small disagreement with the article. Statistical arbitrage is a more general principle than "reliance on mean reversion" (though that is certainly a common example of statarb). It really encompasses any useful statistical prediction that can be used to trade with.

Galanwe 3y ago
While I agree that the article is somewhat vague and imprecise, it is not accurate to encompass all statistical predictions as statarb either. That would more be what "quantitative" is. For a strategy to be considered statistical arbitrage, it needs to be something _against_ something else. You can do purely directional statistical predictions, and that would not be statarb.
- taylorius 3y ago
  Very true! Though the dividing line between purely directional, and something against something else is unclear. Every price is against something else, in some sense.

chollida1 3y ago

A few notes We normally use ordinary least squares when we do the linear modelling. Maybe they are and I can’t read the Julia code. Also they aren’t adjusting their closing prices do dividends as far as I can tell. Nor are they taking dividends payments into account at all as far as I can tell. They did get the shift from daily to hourly right when they went to their crypto example so good for them. They are ahead of most day traders there:) Also no work in the hurst exponent to figure out the reversion time frame Also no modelling on how each pair will fit into your portfolio composition and if you can get borrow, which you certainly can in this instance, and trading fees. Also they skip regime detection, which is determining what regime the market in general is. Lots of pairs only work in up or down or sideways markets and you’d want a different ratio for each. And you’d want Tom look at how your pairs works in each market regime. I find this to be the most important piece and. You never seen pairs tutorials take this into account. Probably because you his is the hard piece that makes you money. This is why pairs are thought to trade. This link is the star arg 101 tutorial that is in 50 different places in the internet but it doesn’t show any of the actual meaningful work you need to do to trade pairs. Other missing pieces are leg sizing, how long you let a leg hang, how many legs do you out out, portfolio composition, dynamic hedge ratio calculation,market regime detection. It shows the easy piece that you can code up in an afternoon. And now I feel like I’ve shit all over this poor kids project. It’s well done and covers the 101 issues with a few pretty large missing pieces that I’ve pointed out

tdubhro1 3y ago
curious what exactly you're referring to by market regime detection / classification, anything I've seen on this (several models from bulge sellside) has been backward looking and fairly useless.
- chollida1 3y ago
  We’ll refine detection is detecting when the market refine changes. But I assume you know that. Typically you’ll run one set of strategies in a bull market for a given Security and another in a sideways market and others in a bear market. Keep in mind different securities can be in different regimes at the same time. And the market in general can be in a different return from individual securities. To detect regime change you typically model it as a hidden Markov process. For back testing we typical pre-label the data by the regime that we know the market was in because we have the benefit of hindsight:

deleted 3y ago

[deleted]

apwheele 3y ago

I have an analysis of cointegration between Bitcoin/Ethereum (in python) if folks are interested, https://andrewpwheeler.com/2022/02/09/cointegration-analysis-of-ethereum-and-bitcoin/ https://andrewpwheeler.com/2022/02/09/cointegration-analysis... This was over a year ago, so dynamics could of course have changed. But based on that analysis, BTC tended to lead ETH (Bitcoin goes up, ETH goes up). ETH up lead to BTC down though. Not a trader, so don't take it as advice. Impressed by folks who beat the market, it always appears too volatile to me in my amateur stat modelling attempts to even kid myself.

laputan_machine 3y ago

> We this that if JPM rises higher than XLF then it will soon revert and trade lower shortly. Likewise, if JPM falls lower than XLF then we think it will soon trade higher. Why?

alpark3 3y ago
XLF is a financial sector ETF. JPM is a huge bank. If JPM does badly, XLF should probably do badly, and vice versa. Then it stands to reason that if JPM falls below XLF, it will soon go back up to where XLF is. This is relative, so the same situation with XLF rising above JPM, and JPM catching up/XLF falling back down, and similar situations.
- laputan_machine 3y ago
  Thanks! I know it may seem obvious to people clued up on this stuff but it's basically a dark art to me. I appreciate the comment

victorbjorklund 3y ago

"The good thing about it being a negative result means that we don’t have to start considering transaction costs or other annoying things like that." This cracked me up. Haha. That is a good way to look as a "loosing strategy".