6 ms·

The advantage high-frequency trading firms have over everyone else

32 points by chollida1 11y ago

chollida1 11y ago
I thought I'd submit this as it lays out in great detail just how badly the direct feed that the markets supply beat the SIP that is used to price the NBBO(National Best Bid/Offer). The reason this can be a problem is that many of the 40+ trading venues in the US are notified of changes from other venues via the SIP. That means there is a latency arb opportunity when a trade happens at one venue. This is especially an issue in dark pools who perform the bulk of their trades at the mid point. If you are a retail trader, I'll state with about 99% certainty that your trade is being filled in someone's dark pool. As a counter point to Nanex, check out Cliff Asness' notes. He founded AQR inside of Goldman Sachs and it became one of the first real power house quant funds. His writings are awesome! Or just read "The Quants" http://www.amazon.ca/The-Quants-Whizzes-Conquered-Destroyed/dp/0307453383 http://www.amazon.ca/The-Quants-Whizzes-Conquered-Destroyed/... as he's featured prominently in it. https://www.aqr.com/cliffs-perspective/ https://www.aqr.com/cliffs-perspective/
- vasilipupkin 11y ago
  the direct feeds from exchanges are available to everyone not just HFT firms, so this is a misleading headline
  - beagle3 11y ago
    Indeed, but they are incredibly expensive; as in tens of thousands of dollars per exchange per month. I think that the outrage against HFTs is basically that they've raised to bar - if you want to have a chance of getting the good fills, you have to spends hundreds of thousands of dollars per month. Personally, I think that this outrage is misplaced. Playing field was never fair. The costs of playing are just clearer now. (And higher, thanks to HFT, yes - but they weren't low before HFTs; just opaque)
    - vasilipupkin 11y ago
      costs are lot lower than ever. Think about what you had to do before direct feeds and colocation. You had to have guys on the floor of the exchange. If you think paying them was cheaper than a direct feed, then you are wrong. Direct feeds cost money because....everything costs money. Exchanges are businesses, why should they give away their data for free ?
      - harmegido 11y ago
        Couldn't agree with you more. The question surrounding HFT should not be: "is this perfect?" , but instead should be: "is this better than before?". And also, I'd point out that it takes hundreds of thousands of dollars to start a bank as well. Do existing banks have an unfair advantage over prospective start-ups due to these barriers to entry? Of course, but that's pretty much how the world works everywhere. It's not specific to HFT.
      - beagle3 11y ago
        > costs are lot lower than ever. Depends on your time frame. Back in 2005 or so (until NASDAQ bought INET), you could get everything INET for ~$1000/month, and be on par with the big guys (as far as INET was concerned). And practically overnight after NASDAQ bought them, the price went up to ~$6000 (for exactly the same service). And HFTs are (at least partially) to blame - they all used INET, raising the volume there significantly, which got NASDAQ scared, rushed to buy INET, and jack up the price to pay for it (and because now, monopolizing the nasdaq trades again, they could). Also back in 2005, Eurex did NOT have a fast feed, only a slow one (updating twice a second, if memory serves me right, and fitting on a 512Kb line). And it cost just a couple of hundred dollars to get above infrastructure costs (dedicated line, data providers, trading systems, etc). Then the fast feed came, and it costs thousands of dollars per month (in addition to infrastructure). And it came because of the HFTs. Costs are lower than ever in the sense that slow data is cheaper than it was, and fast data is cheaper than it was. However, costs are not at all lower than ever in the sense that in 2000 to ~2006, you could be on par with the most sophisticated players for much cheaper than you can today. > You had to have guys on the floor of the exchange. If you think paying them was cheaper than a direct feed, then you are wrong. Although the floors still dominated in the early 2000s, you only needed to trade them if you traded huge blocks. Smaller blocks worked perfectly well (and more efficiently, and with much better price discovery) on the electronic exchanges. How far back are you thinking of? > Direct feeds cost money because....everything costs money. Exchanges are businesses, why should they give away their data for free ? They shouldn't. Some people (me not among them) think it's unfair that to get that data completely and timely is so expensive. Personally, I think the unfairness runs so much deeper that it makes no difference. And I also think that expectations of fairness are also misguided. As you mentioned, the exchanges are businesses - they are in business of making money, not of being fair. Unfortunately, most people are blind to that.
- lordnacho 11y ago
  --That means there is a latency arb opportunity when a trade happens at one venue. This is in fact the business proposition of a whole number of markets, vis-a-vis the HFT firms. If you didn't know this you might have wondered how on earth they get people to cross at midpoint.
javajosh 11y ago
It's Friday, so I'd like to float a stupid idea: can't we trade through the day in blocks of like 10 minutes? That is the market takes all information for the last 10 minutes (kept in secret) and then integrates it using a public match-making algorithm, and publishes the result. You might even get a little fancy and specify that you want your order to get into the next bin, not this one. The idea is that markets are for humans to allocate capital to other humans, usually in businesses that cater to the needs of humans. If that is the purpose of a market, then I don't see why we can't limit ourselves to 6 integrations an hour, or something like 40 per day. (Heck, I'm personally in favor of having one integration per hour - limiting the allocation decisions to like 8 per day. How can even a professional investor make more than 8 allocation decisions a day if he's investing on fundamentals?)
- animefan 11y ago
  One issue is that this will still require people to behave strategically, since the auction is a complex game when you take into account everyone's information. In theory, continuous trading should be the simplest game, and let people focus on their individual information and liquidity needs. In practice it certainly induces strategic behavior, but it might still be the simplest system. The real problem in my opinion is the active human trading that drives these markets. The only reason there is so much is that people irrationally invest with actively traded funds. In fact if people were perfectly rational there would probably be too little price discovery, as people world but market portfolios and any other trade would stand out as information based. But in practice we are way on the other side of the spectrum, with way more active trading than is socially optimal.
- wdewind 11y ago
  I think you're vastly underestimating how quickly markets change, how technically difficult it would be to provide breaking information fairly at the ten minute mark and how important liquidity is to the market. If we slowed trades down then the economy would be significantly less responsive and markets in other countries would have significant advantages. Shorter answer: If I can gain a profit now instead of in 10 minutes I can reinvest just as quickly and the entire economy benefits. These small chunks add up.
mperham 11y ago
Can someone explain what value HFT actually provides to society at large? Why not add a one cent tax to every trade and wipe out HFT completely?
- wdewind 11y ago
  Sure: basically when large institutional investors go to sell shares they sell them in blocks. There are not a ton of people who can afford to buy these entire blocks, though they may want to buy some portion of them. HFTs add value by buying these large blocks, and then breaking them up and selling them in smaller chunks. This is called "market making." This added liquidity makes it far easier for both the institutional investor to sell their shares as well as for the final buyer to purchase them. So the value they provide is increased liquidity to the market. Edit: Whoops, I'm incorrect. This is a description of market making in general, but not how HFTs do it (which have a variety of strategies). But the general premise is that the value provided is still increased liquidity. That being said, not all HFTs employ a market making strategy.
  - beagle3 11y ago
    Those who do the block trades are not generally HFTs
    - wdewind 11y ago
      Please correct me, happy to learn.
  - oojgaoj 11y ago
    This isn't what HFTs do. One common HFT trick is exactly the opposite of what you wrote: they'll attempt to identify probable sellers of large blocks who are disguising their actions by using smaller blocks (they can identify this by measuring latency orders arrive at different exchanges), and then using knowledge of the order books at different exchanges to buy the large block at 10.00 from the seller who was already committed to sell and then sell it at 10.02 to the buyer who was already committed to buying. They can do this because the institutional seller is using the NBBO based on the SIP, and they're watching 200micros ahead using direct feeds.
jkot 11y ago
At least HFT pushes technical boundaries and gives in something back.
- harmegido 11y ago
  Haha you can say that about war too.
bko 11y ago
I'm awaiting an article titled "The advantage tall people have over other basketball players" In all seriousness, the faster people learn that investing in the stock market is not a game, the better. There is a whole industry built around perpetrating the myth that individual investors can pick stocks and somehow "beat the market". Don't actively manage your portfolio. The only person who is going to win are the middle-men charging you for trades or advice. Rebalance periodically based on your risk profile. There are no secrets.
noname123 11y ago
I trade options spreads on orders of 2-week to month holding periods. So HFT isn't my specialty. But I'm trying to understand how latency arbitrage work, please let me know if this sounds about right. (assuming all quotes at top of book is 100-lot). at t = 0 ns IWM quotes with NBBO of 120.10/120.15 BATS quotes IWM at spread of 120.10/120.15 ARCA quotes IWM at spread of 120.10/120.16 at t = 1 ns Someone lowers their offer on ARCA, improving the spread on ARCA to 120.10/120.12* Someone hikes their bid on BATS, improving the spread on BATS to 120.13*/120.16 Now according RegNMS, that improved ARCA offer should cross with the improved BATS bid and make a trade at 100 lot of IWM at 120.13. However, SIP lags by X ns; so either ARCA nor BATS route their customer's order externally for best price before X ns. at t = a ns where a < X (where x is the SIP latency) HFT co-located to BATS and ARCA detects this improved in price. HFT bot takes the BATS bid, short IWM at 120.13, (position: -100 IWM; acct-cash: $12013) HFT bot takes the ARCA offer, buys back IWM at 120.12, (position: 0 IWM; acct-cash: $1) $1 revenue, and you do this a much of times a day and profit?
- harmegido 11y ago
  I'd also point out that it can be more simple than that - what you've described is 100% arbitrage. However, perhaps someone simply buys several levels (sweeps) the BATS book. Assuming they didn't also take out ARCA and IWM and everywhere else, it then becomes a race to buy at those exchanges. Sure that example won't give a risk-free profit, but getting good edge like that thousands of times a day will average out to a nearly guaranteed winning day.
- crazypyro 11y ago
  I believe they also front-run (I think this is the correct term) big orders. So if HFT bot at BATS sees someone buying all of the lower priced shares of IWM, the HFT bot tries to buy the shares of IWM at other exchanges that are less than the shares left at BATS and before the other exchanges can update prices from the BATS sale. They have this information about the large order microseconds before the non-HFT firms.
  - kasey_junk 11y ago
    This is completely incorrect on at least 2 counts. First of they did that it wouldn't be front running, which requires both seeing your order before it hits the exchange and a fiduciary duty to you. Second the hft doesn't buy up existing shares from someone else on the other exchange they change the price of (or cancel entirely) the shares they themselves are offering on the other exchange.
liamconnell 11y ago
Traders in general dont proved any service to the market except liquidity, so I never pay attention the old guard whine about HFT stealing their profit. Markets don't owe traders anything. A futures commodity market (very highly traded) really exists so that companies can hedge risk. Its why Burger King can survive beef price jumps. They dont care about HFT, they just are happy that their orders get filled when they place them. So we have to tolerate someone providing liquidity in the market. And in order to do that we have to allow leveraging and margin trading so that its worth it for those traders. Old style trading, or anything that's not high frequency is too crude to make profits without making extremely leveraged bets and thats bad for the market in general. What happens when markets start to look unsettled and everyone is leveraged? Margins (amount on money needed to hold position) go up. Firms realize that they cant put down money for margins so they try to dump their positions. But everyone is doing that so the bottom falls out of the market and margins go up even higher. And this is caused by everyday hedgefunds on the SIP, since in order to be profitable with their rather crude tools they have to leverage more. So I'm not defending HFT, I just think that everyone wants to see them as the bad guy, but its not like traders themselves are heroes. We just need to see the problem as "who can provide liquidity and stability to a market" rather than "who has an unfair advantage"
- oojgaoj 11y ago
  > We just need to see the problem as "who can provide liquidity and stability to a market" rather than "who has an unfair advantage" And essentially every single analysis of THAT problem says that HFTs consume liquidity in times of instability, and "provide" liquidity in times of stability. As such, HFT destroys the value that people actually care about. Anyway... no point in arguing about this; the sociopathic assholes who run HFT aren't going to stop, the exchanges aren't going to fire their best customers, and the regulators are all bought and paid for. The only good news about HFT, at all, is that because it's a completely commoditized playing field, their margins tend to get compressed over time, so at least they're screwing us slightly less every year.

32 points by chollida1 11y ago

I thought I'd submit this as it lays out in great detail just how badly the direct feed that the markets supply beat the SIP that is used to price the NBBO(National Best Bid/Offer). The reason this can be a problem is that many of the 40+ trading venues in the US are notified of changes from other venues via the SIP. That means there is a latency arb opportunity when a trade happens at one venue. This is especially an issue in dark pools who perform the bulk of their trades at the mid point. If you are a retail trader, I'll state with about 99% certainty that your trade is being filled in someone's dark pool. As a counter point to Nanex, check out Cliff Asness' notes. He founded AQR inside of Goldman Sachs and it became one of the first real power house quant funds. His writings are awesome! Or just read "The Quants" http://www.amazon.ca/The-Quants-Whizzes-Conquered-Destroyed/dp/0307453383 http://www.amazon.ca/The-Quants-Whizzes-Conquered-Destroyed/... as he's featured prominently in it. https://www.aqr.com/cliffs-perspective/ https://www.aqr.com/cliffs-perspective/

vasilipupkin 11y ago
the direct feeds from exchanges are available to everyone not just HFT firms, so this is a misleading headline
- beagle3 11y ago
  Indeed, but they are incredibly expensive; as in tens of thousands of dollars per exchange per month. I think that the outrage against HFTs is basically that they've raised to bar - if you want to have a chance of getting the good fills, you have to spends hundreds of thousands of dollars per month. Personally, I think that this outrage is misplaced. Playing field was never fair. The costs of playing are just clearer now. (And higher, thanks to HFT, yes - but they weren't low before HFTs; just opaque)
  - vasilipupkin 11y ago
    costs are lot lower than ever. Think about what you had to do before direct feeds and colocation. You had to have guys on the floor of the exchange. If you think paying them was cheaper than a direct feed, then you are wrong. Direct feeds cost money because....everything costs money. Exchanges are businesses, why should they give away their data for free ?
    - harmegido 11y ago
      Couldn't agree with you more. The question surrounding HFT should not be: "is this perfect?" , but instead should be: "is this better than before?". And also, I'd point out that it takes hundreds of thousands of dollars to start a bank as well. Do existing banks have an unfair advantage over prospective start-ups due to these barriers to entry? Of course, but that's pretty much how the world works everywhere. It's not specific to HFT.
    - beagle3 11y ago
      > costs are lot lower than ever. Depends on your time frame. Back in 2005 or so (until NASDAQ bought INET), you could get everything INET for ~$1000/month, and be on par with the big guys (as far as INET was concerned). And practically overnight after NASDAQ bought them, the price went up to ~$6000 (for exactly the same service). And HFTs are (at least partially) to blame - they all used INET, raising the volume there significantly, which got NASDAQ scared, rushed to buy INET, and jack up the price to pay for it (and because now, monopolizing the nasdaq trades again, they could). Also back in 2005, Eurex did NOT have a fast feed, only a slow one (updating twice a second, if memory serves me right, and fitting on a 512Kb line). And it cost just a couple of hundred dollars to get above infrastructure costs (dedicated line, data providers, trading systems, etc). Then the fast feed came, and it costs thousands of dollars per month (in addition to infrastructure). And it came because of the HFTs. Costs are lower than ever in the sense that slow data is cheaper than it was, and fast data is cheaper than it was. However, costs are not at all lower than ever in the sense that in 2000 to ~2006, you could be on par with the most sophisticated players for much cheaper than you can today. > You had to have guys on the floor of the exchange. If you think paying them was cheaper than a direct feed, then you are wrong. Although the floors still dominated in the early 2000s, you only needed to trade them if you traded huge blocks. Smaller blocks worked perfectly well (and more efficiently, and with much better price discovery) on the electronic exchanges. How far back are you thinking of? > Direct feeds cost money because....everything costs money. Exchanges are businesses, why should they give away their data for free ? They shouldn't. Some people (me not among them) think it's unfair that to get that data completely and timely is so expensive. Personally, I think the unfairness runs so much deeper that it makes no difference. And I also think that expectations of fairness are also misguided. As you mentioned, the exchanges are businesses - they are in business of making money, not of being fair. Unfortunately, most people are blind to that.
lordnacho 11y ago
--That means there is a latency arb opportunity when a trade happens at one venue. This is in fact the business proposition of a whole number of markets, vis-a-vis the HFT firms. If you didn't know this you might have wondered how on earth they get people to cross at midpoint.

javajosh 11y ago

It's Friday, so I'd like to float a stupid idea: can't we trade through the day in blocks of like 10 minutes? That is the market takes all information for the last 10 minutes (kept in secret) and then integrates it using a public match-making algorithm, and publishes the result. You might even get a little fancy and specify that you want your order to get into the next bin, not this one. The idea is that markets are for humans to allocate capital to other humans, usually in businesses that cater to the needs of humans. If that is the purpose of a market, then I don't see why we can't limit ourselves to 6 integrations an hour, or something like 40 per day. (Heck, I'm personally in favor of having one integration per hour - limiting the allocation decisions to like 8 per day. How can even a professional investor make more than 8 allocation decisions a day if he's investing on fundamentals?)

animefan 11y ago
One issue is that this will still require people to behave strategically, since the auction is a complex game when you take into account everyone's information. In theory, continuous trading should be the simplest game, and let people focus on their individual information and liquidity needs. In practice it certainly induces strategic behavior, but it might still be the simplest system. The real problem in my opinion is the active human trading that drives these markets. The only reason there is so much is that people irrationally invest with actively traded funds. In fact if people were perfectly rational there would probably be too little price discovery, as people world but market portfolios and any other trade would stand out as information based. But in practice we are way on the other side of the spectrum, with way more active trading than is socially optimal.
wdewind 11y ago
I think you're vastly underestimating how quickly markets change, how technically difficult it would be to provide breaking information fairly at the ten minute mark and how important liquidity is to the market. If we slowed trades down then the economy would be significantly less responsive and markets in other countries would have significant advantages. Shorter answer: If I can gain a profit now instead of in 10 minutes I can reinvest just as quickly and the entire economy benefits. These small chunks add up.

mperham 11y ago

Can someone explain what value HFT actually provides to society at large? Why not add a one cent tax to every trade and wipe out HFT completely?

wdewind 11y ago
Sure: basically when large institutional investors go to sell shares they sell them in blocks. There are not a ton of people who can afford to buy these entire blocks, though they may want to buy some portion of them. HFTs add value by buying these large blocks, and then breaking them up and selling them in smaller chunks. This is called "market making." This added liquidity makes it far easier for both the institutional investor to sell their shares as well as for the final buyer to purchase them. So the value they provide is increased liquidity to the market. Edit: Whoops, I'm incorrect. This is a description of market making in general, but not how HFTs do it (which have a variety of strategies). But the general premise is that the value provided is still increased liquidity. That being said, not all HFTs employ a market making strategy.
- beagle3 11y ago
  Those who do the block trades are not generally HFTs
  - wdewind 11y ago
    Please correct me, happy to learn.
- oojgaoj 11y ago
  This isn't what HFTs do. One common HFT trick is exactly the opposite of what you wrote: they'll attempt to identify probable sellers of large blocks who are disguising their actions by using smaller blocks (they can identify this by measuring latency orders arrive at different exchanges), and then using knowledge of the order books at different exchanges to buy the large block at 10.00 from the seller who was already committed to sell and then sell it at 10.02 to the buyer who was already committed to buying. They can do this because the institutional seller is using the NBBO based on the SIP, and they're watching 200micros ahead using direct feeds.

jkot 11y ago

At least HFT pushes technical boundaries and gives in something back.

harmegido 11y ago
Haha you can say that about war too.

bko 11y ago

I'm awaiting an article titled "The advantage tall people have over other basketball players" In all seriousness, the faster people learn that investing in the stock market is not a game, the better. There is a whole industry built around perpetrating the myth that individual investors can pick stocks and somehow "beat the market". Don't actively manage your portfolio. The only person who is going to win are the middle-men charging you for trades or advice. Rebalance periodically based on your risk profile. There are no secrets.

noname123 11y ago

I trade options spreads on orders of 2-week to month holding periods. So HFT isn't my specialty. But I'm trying to understand how latency arbitrage work, please let me know if this sounds about right. (assuming all quotes at top of book is 100-lot). at t = 0 ns IWM quotes with NBBO of 120.10/120.15 BATS quotes IWM at spread of 120.10/120.15 ARCA quotes IWM at spread of 120.10/120.16 at t = 1 ns Someone lowers their offer on ARCA, improving the spread on ARCA to 120.10/120.12* Someone hikes their bid on BATS, improving the spread on BATS to 120.13*/120.16 Now according RegNMS, that improved ARCA offer should cross with the improved BATS bid and make a trade at 100 lot of IWM at 120.13. However, SIP lags by X ns; so either ARCA nor BATS route their customer's order externally for best price before X ns. at t = a ns where a < X (where x is the SIP latency) HFT co-located to BATS and ARCA detects this improved in price. HFT bot takes the BATS bid, short IWM at 120.13, (position: -100 IWM; acct-cash: $12013) HFT bot takes the ARCA offer, buys back IWM at 120.12, (position: 0 IWM; acct-cash: $1) $1 revenue, and you do this a much of times a day and profit?

harmegido 11y ago
I'd also point out that it can be more simple than that - what you've described is 100% arbitrage. However, perhaps someone simply buys several levels (sweeps) the BATS book. Assuming they didn't also take out ARCA and IWM and everywhere else, it then becomes a race to buy at those exchanges. Sure that example won't give a risk-free profit, but getting good edge like that thousands of times a day will average out to a nearly guaranteed winning day.
crazypyro 11y ago
I believe they also front-run (I think this is the correct term) big orders. So if HFT bot at BATS sees someone buying all of the lower priced shares of IWM, the HFT bot tries to buy the shares of IWM at other exchanges that are less than the shares left at BATS and before the other exchanges can update prices from the BATS sale. They have this information about the large order microseconds before the non-HFT firms.
- kasey_junk 11y ago
  This is completely incorrect on at least 2 counts. First of they did that it wouldn't be front running, which requires both seeing your order before it hits the exchange and a fiduciary duty to you. Second the hft doesn't buy up existing shares from someone else on the other exchange they change the price of (or cancel entirely) the shares they themselves are offering on the other exchange.

liamconnell 11y ago

Traders in general dont proved any service to the market except liquidity, so I never pay attention the old guard whine about HFT stealing their profit. Markets don't owe traders anything. A futures commodity market (very highly traded) really exists so that companies can hedge risk. Its why Burger King can survive beef price jumps. They dont care about HFT, they just are happy that their orders get filled when they place them. So we have to tolerate someone providing liquidity in the market. And in order to do that we have to allow leveraging and margin trading so that its worth it for those traders. Old style trading, or anything that's not high frequency is too crude to make profits without making extremely leveraged bets and thats bad for the market in general. What happens when markets start to look unsettled and everyone is leveraged? Margins (amount on money needed to hold position) go up. Firms realize that they cant put down money for margins so they try to dump their positions. But everyone is doing that so the bottom falls out of the market and margins go up even higher. And this is caused by everyday hedgefunds on the SIP, since in order to be profitable with their rather crude tools they have to leverage more. So I'm not defending HFT, I just think that everyone wants to see them as the bad guy, but its not like traders themselves are heroes. We just need to see the problem as "who can provide liquidity and stability to a market" rather than "who has an unfair advantage"

oojgaoj 11y ago
> We just need to see the problem as "who can provide liquidity and stability to a market" rather than "who has an unfair advantage" And essentially every single analysis of THAT problem says that HFTs consume liquidity in times of instability, and "provide" liquidity in times of stability. As such, HFT destroys the value that people actually care about. Anyway... no point in arguing about this; the sociopathic assholes who run HFT aren't going to stop, the exchanges aren't going to fire their best customers, and the regulators are all bought and paid for. The only good news about HFT, at all, is that because it's a completely commoditized playing field, their margins tend to get compressed over time, so at least they're screwing us slightly less every year.