12 ms·

A Gazillion-Dollar Standoff over Two High-Frequency Trading Towers

173 points by jakub_g 8y ago

drexlspivey 8y ago
If you want to know how HFT works I recommend the book Flash Boys by Michael Lews or the first 15 minutes of this talk by Brad Katsuyama (the book protagonist) https://www.youtube.com/watch?v=N9hoqFpDjVs https://www.youtube.com/watch?v=N9hoqFpDjVs. In short, when you send an order from your computer to buy say 10000 stocks, this order gets split to all the multiple exchanges. When the order hits the first (closest) exchange, HF Traders will pick that up and race you to the rest of the exchanges and try to front run you by buying the stock in lower price if available and make an offer to your bid price that they know is coming.
- growse 8y ago
  > In short, when you send an order from your computer to buy say 10000 stocks, this order gets split to all the multiple exchanges. When the order hits the first (closest) exchange, HF Traders will pick that up and race you to the rest of the exchanges and try to front run you by buying the stock in lower price if available and make an offer to your bid price that they know is coming. 1. This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different. 2. Orders don't get split across multiple venues by accident. It's a conscious decision. Anyone who chooses to route multiple orders to different venues in a way that causes an information imbalance (ie, so the orders hit the venues at different times) is going to get their lunch eaten by the rest of the market. Only idiots do this - the rest of the world times their orders taking transmission latency into account.
  - MuffinFlavored 8y ago
    > the rest of the world times their orders taking transmission latency into account. Could you elaborate on this please? Say I want to place an order for 100 shares of XYZ at $100.00 per share. I know my trade is going to take 50ms to get fulfilled. What do I do with this information? Predict where the price is going to be?
    - growse 8y ago
      > > the rest of the world times their orders taking transmission latency into account. > Could you elaborate on this please? Say I want to place an order for 100 shares of XYZ at $100.00 per share. I know my trade is going to take 50ms to get fulfilled. What do I do with this information? Predict where the price is going to be? Simplistically, If you're making an order that's too big for the order book on a single venue to get a sensible price, you might chop it up and buy some in NY, some in London and some in Tokyo. Let's say you're sat in NY: you can work out that it'll take your order 1ms to hit the NYSE, 150ms to hit the LSE and 600ms to hit the TSE. So rather than dumbly hitting 'transmit' on all your orders at once, you send the Tokyo order first, then then the London order 450ms later, and then the NY order 149ms after that. That way, your orders (and the public information that your making those orders) will hit each venue at exactly the same time. This is common practice for any broker or institutional trader.
  - drexlspivey 8y ago
    > This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different. I guess that depends on what you consider public market data. > Front running, also known as tailgating, is the prohibited practice of entering into an equity (stock) trade on advance, nonpublic knowledge of a large pending transaction that will influence the price of the underlying security When an order hits an exchange, the exchange sends a flash order to the owners of the proprietary data feeds before sending the trade information out on the broader public markets, and making it available to all potential investors. It seems to me that the owners of those feeds have advanced knowledge of non public data.
    - growse 8y ago
      > > This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different. > I guess that depends on what you consider public market data. Usefully, the regulators define it for us. No consideration needed. > When an order hits an exchange, the exchange sends a flash order to the owners of the proprietary data feeds before sending the trade information out on the broader public markets, and making it available to all potential investors. > It seems to me that the owners of those feeds have advanced knowledge of non public data. Can anyone buy one of these feeds? If so, it's public data. "Non public" means something very specific (and different) in this context.
      - drexlspivey 8y ago
        Semantics aside, the gist of the issue is: should exchanges offer faster data feeds as a service? In a perfect world the goal of an exchange should be to facilitate fair trading not to make money by allowing sharks to feed on the rest of the participants.
        princeb 8y ago
        anyone who wants the "faster feeds" are welcome to pay for it. hfts being able to trade on better infrastructure is no different from walmart being able to distribute soda to you at a fraction of the price they are retailing it at.
- tptacek 8y ago
  Flash Boys is terrible (it's not even good as a book --- and I'm an avid fan of Michael Lewis's). It's embarrassingly bad on the technical details. I like "Flash Boys: Not So Fast" as a corrective. A much better book, from another HFT skeptic, is "Dark Pools". The example you provide (of basic, naive latency arbitrage) matters for trading desks at giant investment banks. They make their money in part by selling execution services to giant holders of stock. Major trades by these people (big "blocks" being "shopped") are tradable market news. The giant holders of stock would prefer it if the news didn't get out until after their orders were filled. That's a service people like Katsuyama used to be able to offer his wealthy clients with very little effort on his part. That service has since been automated away, and it bothers him. I guess I'd be bothered too, but I'm not sure why it's moral dilemma. Front running isn't a technical problem and it has little to do with how fast traders are. It's an agency problem: a broker "front-runs" their client when they receive an order from that client and make proprietary trades against and ahead of it. Random people in the markets don't have an agency relationship with other traders and are under no obligation to avoid competing with them.
  - integrate-this 8y ago
    I was an engineer for a high frequency trading desk at a firm not mentioned in this article. Flash Boys is a great book and we required our new hires to read it as it got a lot of things right. Front running is illegal now, but arbitrage opportunities still exist between exchanges. Being fast is still an easy way to make money. You're right about dark pools though. Those are the future (honestly they are already up and functioning) and they hide trading activity very well.
    - reverend_gonzo 8y ago
      Uh, being fast is not easy nor cheap. And once you are fast, you have to stay fast. Remaining fast is not easy or cheap either. Sure, if you have the part that’s really hard to get right, making money is then really easy.
    - tptacek 8y ago
      This is kind of a series of non-sequiturs. Front running has always been illegal. Arbitrage is not illegal. Latency arb certainly isn't easy. "Dark Pools" is a book. It's also a thing, but that's obviously not what I was talking about. There's nothing especially shady about dark pools. We'd call them "private exchanges" if the exchanges weren't themselves already private companies, I think.
- kasey_junk 8y ago
  Note, me & every other person I knew in the industry were extremely disappointed with Flash Boys. It was a submarine ad for IEX & got things just plain wrong. Feel free to read it but remember that IEX & Brad Katsuyama work on behalf of one set of market participants that are decidedly against the interests of other market participants such as retail traders.
lordnacho 8y ago
I was working in HFT myself, though in Europe, and I asked around about this. Specifically WTF do they do with this line from Chicago to New York that's so insanely important? The US market has this interesting phenomenon that the futures trade in Chicago, but the single stocks trade in New York. For those who are not familiar, this means you can make a bet on the S&P 500 index as a whole in Chicago by buying or selling futures. But if you wanted to replicate the basket, you would go to New York. Naturally there's a relationship between the two. If the futures are going down, the prices of all the stocks needs to be adjusted. And if you look at stock prices, it turns out quite a lot of any stock price is simply exposure to the market at large. So you really want to know as soon as possible if the market is moving. Now you might have thought that the index is the index of shares, which is in NYC, so why does it matter what Chicago thinks? Well actually the tail wags the dog in this case. The futures are how people express a view on which way the index is going, and you want to know that view immediately. How does it actually work? Variants on "if the market goes down, pull all your bid orders". Basically if things are fine you are happy to look at queue position and imbalance as an indicator of whether you want to leave orders to capture spread with. But the moment there's an indication from Chicago that the market is dipping, you forget that and pull your orders. Or if you're really fast dump a sell order on the guy behind you in the queue. There's a huge amount of stock trading in NY each day, so it's worth your while to have this line that you use to send the warnings down.
- BonesJustice 8y ago
  There’s also more straightforward reasons to want a microwave connection from NYC to Chicago. Say you want to trade off news signals: you want to relay the headlines (from Bloomberg, DJ, etc.) to your colo sites as quickly as possible.
  - pgwhalen 8y ago
    Can anyone in the industry comment on whether this kind of strategy really exists? Surely any automated, news-based strategy's bottleneck is not due to sub-microsecond latencies stemming from microwave tower placement. I've always assumed that the strategies that require microwave networks are using market data and only market data as a primary input.
openloop 8y ago
seems legit. etrade gets like 1 second right?
tomalpha 8y ago
Fun fact: New Line Networks mentioned in the article is named after the New Line Tavern in Chicago [0]. Which is actually quite good. And might just attract network engineers that work for HFT firms. Purely coincidentally. [0] https://www.newlinetavern.com https://www.newlinetavern.com
- syedkarim 8y ago
  New Line Tavern is the only bar I've seen that has drink/food specials on Friday and Saturday.
chrisco255 8y ago
Oof, solid article but Bloomberg breaks the back button if you scroll past the end of the article. It will load several new articles and push them onto the nav stack, meanwhile it doesn't seem like I can click enough times to get back to HN from there.
- PhantomGremlin 8y ago
  Are you on mobile? The article reads fine w/o JavaScript, desktop Firefox. And the back button keeps working. I assume that's thanks to the absence of JavaScript, rather than something special that Firefox is doing.
themagician 8y ago
Can someone explain to me the purpose of HFT? Not the economic incentive, I get that, but what does it accomplish? Is it a net benefit to the system? If we arbitrarily limited transactions to human speed would there be some kind of systemwide downside? Are there any good, human readable papers on the benefits of HFT?
- 18nleung 8y ago
  It’s most commonly justified in terms of “liquidity” - allowing orders to be filled at as close to “perfect” market prices as possible
  - tptacek 8y ago
    I'd think less about the "perfection" of the price and more about the spread. The faster everyone converges on a single understanding of the price, the less you'll have to pay to bridge that uncertainty when you want to buy or sell right now. I always mentally compare our expectations about buying and selling stocks immediately (pennies or less) to that of buying and selling houses "immediately".
- BonesJustice 8y ago
  The only kind of HFT I’m directly familiar with is for automated market making, so I’ll speak to that. Market making in itself provides a valuable service by keeping spreads razor thin and ensuring there’s enough liquidity to satisfy almost any trade at close to the current spot price. HFT is mostly just a way for market makers to gain an edge over each other. In general, it neither helps nor harms ‘regular’ customers. It would hardly be noteworthy if not for the insane lengths that HFT groups go to in order to gain an edge. And, of course, the possibility that a HF system with inadequate controls goes on a trading rampage that significantly moves the markets (see Knight Capital).
  - MuffinFlavored 8y ago
    > market makers How many market makers are there?
deleted 8y ago
[deleted]
deleted 8y ago
[deleted]
pkaye 8y ago
Could they not just make one tower a few feet higher than the other?
- deleted 8y ago
  [deleted]
lolc 8y ago
I'm largely ignorant of HFT but I often wonder what the benefit of it is. It sounds like a huge zero-sum game to me. In my (probably simplistic) view, exchanges could publish and process orders at long intervals (seconds, minutes, hours?) then add a jitter of several intervals to incoming orders, with cumulating jitter for cancels. That would require traders to look beyond one interval when trading.
- shitgoose 8y ago
  would you rather trade on exchange that adds jitter to your order, or one that doesn't?
  - lolc 8y ago
    Depends on my goals. If I'm buying or selling for investment, I just don't care about that jitter. Because my decision to buy or sell doesn't happen at that speed. If I'm racing other people's orders on the sub-second level then I care. But what's the point of that? Why should people spend their lifes on it? Why should we as a society bear the cost of this activity? I'd rather see people do something with external benefits.
    - shitgoose 8y ago
      What is the point of adding X ms delay to your order? What does it fix? The "problem" with HFT is that participants can quickly pull out the limit orders, so your market order will be executed at the price that is worse than you expect. Jitter doesn't fix it. If anything, it will make it worse, i.e. your order will hit the market that is quite different from what you saw when you sent the order.
      - lolc 8y ago
        Not delay, jitter. It removes the possibility of timing orders to sub-second intervals. It's supposed to prevent people from spending resources on a zero-sum game.
- 8y ago
smallnamespace 8y ago
I don't know when we'll have practical long-distance neutrino transmitters [1], but I'll bet someday the implementation will be bankrolled by HFT firms. Why go around the Earth when you can go straight through it? [1] https://phys.org/news/2012-03-wireless-message-neutrinos.html https://phys.org/news/2012-03-wireless-message-neutrinos.htm...
- deleted 8y ago
  [deleted]
- endofcapital 8y ago
  I think when we get into cool near sci fi stuff we'll have faster than light communication pretty soon with quantum entanglement effects, which should be way easier to wrangle than neutrinos.
  - kch01 8y ago
    Quantum entanglement cannot transmit information faster than light unless contemporary physics is very very wrong. Neutrinos may be hard to use but they at least known to be physically possible.
- twic 8y ago
  Fermilab is literally a couple of miles from the CME datacentre in Aurora. They must have some old accelerators they aren't using. The original Kamiokande detector is still in the ground at the Kamioka Observatory, as far as i know. Hasn't been used for science since the '80s. That's about 300 km from either Tokyo or Osaka, so you probably still need microwaves at that end. Now you just have to think of a trade with a signal in Chicago and an instrument in Japan. The CME has Nikkei futures, but they follow the ones in Osaka. Osaka has Dow Jones futures, which presumably follow the ones in Chicago, but the volume in Osaka is pretty small. Your best best is probably FX - both Chicago and Tokyo have dollar/yen futures with good volume. So, who is already doing this, and how long have they been doing it?
- titanomachy 8y ago
  The first message sent by telegraph was supposedly "what hath God wrought?" The first message sent by gravity wave will be "SELL 400 AMZN"
chungleong 8y ago
I wonder if a lot of money is going into research on "spooky action at a distance".
human20190310 8y ago
Location is everything. If trading firms are willing to spend money to position their towers slightly closer to the trading endpoint, the town where the endpoint is located should approve construction in geographic order, allowing new towers to be constructed closer to the endpoint in a way that maximizes tax revenue to the city. Ideally this would keep a lot of tower-construction people continually employed and paying taxes, building new towers ten meters in front of the old ones every month or two.
ggm 8y ago
I get that arbitrage helps set the price, but what does HFT do, beyond deciding which of the competing nano-second enabled traders get to edge a profit in an artifact of how information flows? I don't see this HFT as being information-worthy arbitrage. I have an intent to set a real-world price, I declare it, and in the increment of time it takes my packets to flow, somebody edges in the queue and inserts trades to make me have to deal with them for some increment of price. I wind up paying more, and the original source winds up getting less, or some other event. its like re-intermediation, in a world heading to dis-intermediation. What is the real-world, wider economic "good" here? If my pension fund is routinely profiting from this, I am not sure that is a wider good btw: We could probably make more money selling Heroin, if we're going down this 'whatever it takes' track...
- jjtheblunt 8y ago
  I don't think anyone asserted there was a real world wider good; to the contrary, I think it's just the actions of a system in place.
- kasey_junk 8y ago
  > it takes my packets to flow, somebody edges in the queue and inserts trades to make me have to deal with them for some increment of price This is the biggest mistake people make when talking about HFT. At no point in time does someone see your order before it hits the exchange (unless you’ve specifically agreed to trade that way & even then they are obligated by lots of regulation to execute in particular ways to protect from that agency problem). No HFT is taking pennies from your order. If you send a limit order through the book you will get filled at the best price available up to your limit.
  - ggm 8y ago
    Thanks for explaining. If they're just squeezing margin by being quicker, and not intruding into my trade (I don't trade. this is clear I think, given how ignorant I am) then I have less to worry about.
deleted 8y ago
[deleted]
snek 8y ago
Why don't the rich people just move to New York?
QuackingJimbo 8y ago
I work in HFT as a quant dev I am proud of my work and find HFT to be an interesting intersection of computer science, math, game theory, economics, and psychology As far as the societal benefit of HFT, I think that it is neutral. It doesn't hurt anybody. It might even help a little bit (liquidity and tight spreads save people a little money and maybe spur some economic growth). However, this particular aspect (the physical latency war) makes me literally sick to my stomach. I hate reading about it and I hate thinking about it. It's like -- what are we even doing here guys?
- elgenie 8y ago
  It's only neutral (at best) if you don't consider opportunity cost. Is the best way for the country/society/humanity to deploy a bunch of smart people with a decade and a half of education truly an attempt to try to cheat the speed of light? If trades took 2x as long to execute as they currently do, what exactly would be the harm?
  - scotty79 8y ago
    Also advertising....
  - integrate-this 8y ago
    Well, you would have lost out on a lot of innovation for one. I would be willing to bet that nearly half of the world's high performance computing specialists cut their teeth with a trading firm at one point or another. By your logic, there would have been no benefit to video games or going to the moon even though each of these things have helped deliver large amounts of technological advancement. IE, Facebook is a giant waste of resources but they employ many of the world's smartest engineers. Because of that they've spawned things like ReactJS which has completely changed web development for thousands of other companies.
  - QuackingJimbo 8y ago
    The "brain drain" argument is fair, but you can successfully apply the brain drain argument to anything short of curing diseases or feeding the hungry I consider HFT to be no worse of a brain drain than, say, Instagram, or any similar virus
anfractuosity 8y ago
This film seems relevant - https://www.imdb.com/title/tt6866224 https://www.imdb.com/title/tt6866224 - The Hummingbird Project (2018). 'A pair of high-frequency traders go up against their old boss in an effort to make millions in a fiber-optic cable deal.' (It also seems to feature the microwave based approach from the trailer).
- deleted 8y ago
  [deleted]
johnnycab 8y ago
The dice is fully loaded in favour of HFT pioneers and incumbents, facilitated by the ever-growing and nouveau riche (e.g. Zayo) Tier-1 ISP's, who control most of the backhaul and dark fibre estate. However, over the next few years, the advent of maturing ML frameworks, coupled with the intensifying war and ratification of standards over 5G/IoT based infrastructure, might prove to be serendipitous for nimble-footed enterprises. https://en.wikipedia.org/wiki/Zayo_Group https://en.wikipedia.org/wiki/Zayo_Group
- mattrp 8y ago
  Highly unlikely that 5g / iOS will play a role in the context of hft. Also your view of the ecosystem (eg zayo and co being the big winners) is massively outdated by five years at minimum.
tlrobinson 8y ago
The use of wireless communication in high frequency trading begs the question: how much could a malicious actor profit from momentarily jamming that CyrusOne (or similar) tower?
soheil 8y ago
I did the math to use the neutrino transmitter it'd be 0.13% faster to transmit information from Chicago to NYC. Using equations: s = r(theta) l = (2r)sin(thea/2) s: arc length (air travel distance 714 miles) r: radius of earth (3,958.8 miles) theta: arc angle l: length of cord (straight line under arc) (s-l)/l * 100 gives the % increased difference in length
- PhantomGremlin 8y ago
  You need to add a little extra time to the microwave route because microwaves are line of sight. The signal needs to be detected and re-amplified multiple times along the way. How much extra time? I don't know, I'm not an RF guy, maybe someone can chime in and tell us how much latency each amplifier adds. The neutrino route only needs a single detector at the very end, nothing along the way.
raviolo 8y ago
Another example for people to appreciate what a small latency advantage means. Consider SPX options which is a huge market. These “trade on CBOE” which is a Chicago exchange - but they physically reside in NJ. Now consider SPX futures - ES - on CME, physically in Aurora. Now imagine a 20-tick / 5-handle “sweep” in the ES — “sweep” is instantaneous price move (either up or down). Not “very fast” but exactly that - instantaneous. These happen as a result of large incoming aggressor order taking out many book levels at once. Sweeps can be of different magnitude (from just few ticks all the way to max allowed per CME’s “velocity logic”) - but they happen all the time. If you know enough about e.g. ES futures but have never seen sweeps, chances are you didn’t have access to the right tools/platforms. Even some supposedly-professional-grade platforms like TT or CQG do not show trade data with enough detailization to see these. Anyway... why is this important? Because immediately following the sweep in e.g. ES, option prices in e.g. SPX options are temporarily way off. Dutch market makers have not yet moved their options quotes, because they do not yet know about the sweep - at least not in NJ. If you can somehow get orders to NJ faster than market makers can cancel their option quotes, you can take out hundreds of mispriced options. Multiple expires, dozens of strike, puts and calls. Very juicy bunch of stale quotes. PnL... well that gets tricky. On one hand, just this one example is in theory worth hundreds of million per year if market did not respond. But the market will respond. That’s inevitable. No one enjoys being ripped off for too long, so if they cannot defend they will either leave the market or set quotes wide enough that they are never mispriced. The same goes for any other “latency arbitrage” game. I always love how people throw numbers like “this is worth X”, almost always assuming status quo. HFTs are very responsive to market conditions. If I run a strategy that’s been making money every single day for 10 years and all of a sudden it starts losing - I press the pause button and go do some serious analysis. So maybe I will let you have it for few hours - but after that if I can’t compete I’m out. So it’s not that simple.
mattrp 8y ago
Follow @sniperinmahwah if he is still active... everything else is stuff that’s been known for quite some time.
freshcatch_ 8y ago
Capitalism is just so absurd.

173 points by jakub_g 8y ago

If you want to know how HFT works I recommend the book Flash Boys by Michael Lews or the first 15 minutes of this talk by Brad Katsuyama (the book protagonist) https://www.youtube.com/watch?v=N9hoqFpDjVs https://www.youtube.com/watch?v=N9hoqFpDjVs. In short, when you send an order from your computer to buy say 10000 stocks, this order gets split to all the multiple exchanges. When the order hits the first (closest) exchange, HF Traders will pick that up and race you to the rest of the exchanges and try to front run you by buying the stock in lower price if available and make an offer to your bid price that they know is coming.

growse 8y ago
> In short, when you send an order from your computer to buy say 10000 stocks, this order gets split to all the multiple exchanges. When the order hits the first (closest) exchange, HF Traders will pick that up and race you to the rest of the exchanges and try to front run you by buying the stock in lower price if available and make an offer to your bid price that they know is coming. 1. This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different. 2. Orders don't get split across multiple venues by accident. It's a conscious decision. Anyone who chooses to route multiple orders to different venues in a way that causes an information imbalance (ie, so the orders hit the venues at different times) is going to get their lunch eaten by the rest of the market. Only idiots do this - the rest of the world times their orders taking transmission latency into account.
- MuffinFlavored 8y ago
  > the rest of the world times their orders taking transmission latency into account. Could you elaborate on this please? Say I want to place an order for 100 shares of XYZ at $100.00 per share. I know my trade is going to take 50ms to get fulfilled. What do I do with this information? Predict where the price is going to be?
  - growse 8y ago
    > > the rest of the world times their orders taking transmission latency into account. > Could you elaborate on this please? Say I want to place an order for 100 shares of XYZ at $100.00 per share. I know my trade is going to take 50ms to get fulfilled. What do I do with this information? Predict where the price is going to be? Simplistically, If you're making an order that's too big for the order book on a single venue to get a sensible price, you might chop it up and buy some in NY, some in London and some in Tokyo. Let's say you're sat in NY: you can work out that it'll take your order 1ms to hit the NYSE, 150ms to hit the LSE and 600ms to hit the TSE. So rather than dumbly hitting 'transmit' on all your orders at once, you send the Tokyo order first, then then the London order 450ms later, and then the NY order 149ms after that. That way, your orders (and the public information that your making those orders) will hit each venue at exactly the same time. This is common practice for any broker or institutional trader.
- drexlspivey 8y ago
  > This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different. I guess that depends on what you consider public market data. > Front running, also known as tailgating, is the prohibited practice of entering into an equity (stock) trade on advance, nonpublic knowledge of a large pending transaction that will influence the price of the underlying security When an order hits an exchange, the exchange sends a flash order to the owners of the proprietary data feeds before sending the trade information out on the broader public markets, and making it available to all potential investors. It seems to me that the owners of those feeds have advanced knowledge of non public data.
  - growse 8y ago
    > > This is not 'front running', this is just other parties reacting to and executing on public market data before you. The term 'front running' is understood to mean something specific, and different. > I guess that depends on what you consider public market data. Usefully, the regulators define it for us. No consideration needed. > When an order hits an exchange, the exchange sends a flash order to the owners of the proprietary data feeds before sending the trade information out on the broader public markets, and making it available to all potential investors. > It seems to me that the owners of those feeds have advanced knowledge of non public data. Can anyone buy one of these feeds? If so, it's public data. "Non public" means something very specific (and different) in this context.
    - drexlspivey 8y ago
      Semantics aside, the gist of the issue is: should exchanges offer faster data feeds as a service? In a perfect world the goal of an exchange should be to facilitate fair trading not to make money by allowing sharks to feed on the rest of the participants.
      - princeb 8y ago
        anyone who wants the "faster feeds" are welcome to pay for it. hfts being able to trade on better infrastructure is no different from walmart being able to distribute soda to you at a fraction of the price they are retailing it at.
tptacek 8y ago
Flash Boys is terrible (it's not even good as a book --- and I'm an avid fan of Michael Lewis's). It's embarrassingly bad on the technical details. I like "Flash Boys: Not So Fast" as a corrective. A much better book, from another HFT skeptic, is "Dark Pools". The example you provide (of basic, naive latency arbitrage) matters for trading desks at giant investment banks. They make their money in part by selling execution services to giant holders of stock. Major trades by these people (big "blocks" being "shopped") are tradable market news. The giant holders of stock would prefer it if the news didn't get out until after their orders were filled. That's a service people like Katsuyama used to be able to offer his wealthy clients with very little effort on his part. That service has since been automated away, and it bothers him. I guess I'd be bothered too, but I'm not sure why it's moral dilemma. Front running isn't a technical problem and it has little to do with how fast traders are. It's an agency problem: a broker "front-runs" their client when they receive an order from that client and make proprietary trades against and ahead of it. Random people in the markets don't have an agency relationship with other traders and are under no obligation to avoid competing with them.
- integrate-this 8y ago
  I was an engineer for a high frequency trading desk at a firm not mentioned in this article. Flash Boys is a great book and we required our new hires to read it as it got a lot of things right. Front running is illegal now, but arbitrage opportunities still exist between exchanges. Being fast is still an easy way to make money. You're right about dark pools though. Those are the future (honestly they are already up and functioning) and they hide trading activity very well.
  - reverend_gonzo 8y ago
    Uh, being fast is not easy nor cheap. And once you are fast, you have to stay fast. Remaining fast is not easy or cheap either. Sure, if you have the part that’s really hard to get right, making money is then really easy.
  - tptacek 8y ago
    This is kind of a series of non-sequiturs. Front running has always been illegal. Arbitrage is not illegal. Latency arb certainly isn't easy. "Dark Pools" is a book. It's also a thing, but that's obviously not what I was talking about. There's nothing especially shady about dark pools. We'd call them "private exchanges" if the exchanges weren't themselves already private companies, I think.
kasey_junk 8y ago
Note, me & every other person I knew in the industry were extremely disappointed with Flash Boys. It was a submarine ad for IEX & got things just plain wrong. Feel free to read it but remember that IEX & Brad Katsuyama work on behalf of one set of market participants that are decidedly against the interests of other market participants such as retail traders.

lordnacho 8y ago

I was working in HFT myself, though in Europe, and I asked around about this. Specifically WTF do they do with this line from Chicago to New York that's so insanely important? The US market has this interesting phenomenon that the futures trade in Chicago, but the single stocks trade in New York. For those who are not familiar, this means you can make a bet on the S&P 500 index as a whole in Chicago by buying or selling futures. But if you wanted to replicate the basket, you would go to New York. Naturally there's a relationship between the two. If the futures are going down, the prices of all the stocks needs to be adjusted. And if you look at stock prices, it turns out quite a lot of any stock price is simply exposure to the market at large. So you really want to know as soon as possible if the market is moving. Now you might have thought that the index is the index of shares, which is in NYC, so why does it matter what Chicago thinks? Well actually the tail wags the dog in this case. The futures are how people express a view on which way the index is going, and you want to know that view immediately. How does it actually work? Variants on "if the market goes down, pull all your bid orders". Basically if things are fine you are happy to look at queue position and imbalance as an indicator of whether you want to leave orders to capture spread with. But the moment there's an indication from Chicago that the market is dipping, you forget that and pull your orders. Or if you're really fast dump a sell order on the guy behind you in the queue. There's a huge amount of stock trading in NY each day, so it's worth your while to have this line that you use to send the warnings down.

BonesJustice 8y ago
There’s also more straightforward reasons to want a microwave connection from NYC to Chicago. Say you want to trade off news signals: you want to relay the headlines (from Bloomberg, DJ, etc.) to your colo sites as quickly as possible.
- pgwhalen 8y ago
  Can anyone in the industry comment on whether this kind of strategy really exists? Surely any automated, news-based strategy's bottleneck is not due to sub-microsecond latencies stemming from microwave tower placement. I've always assumed that the strategies that require microwave networks are using market data and only market data as a primary input.

openloop 8y ago

seems legit. etrade gets like 1 second right?

tomalpha 8y ago

Fun fact: New Line Networks mentioned in the article is named after the New Line Tavern in Chicago [0]. Which is actually quite good. And might just attract network engineers that work for HFT firms. Purely coincidentally. [0] https://www.newlinetavern.com https://www.newlinetavern.com

syedkarim 8y ago
New Line Tavern is the only bar I've seen that has drink/food specials on Friday and Saturday.

chrisco255 8y ago

Oof, solid article but Bloomberg breaks the back button if you scroll past the end of the article. It will load several new articles and push them onto the nav stack, meanwhile it doesn't seem like I can click enough times to get back to HN from there.

PhantomGremlin 8y ago
Are you on mobile? The article reads fine w/o JavaScript, desktop Firefox. And the back button keeps working. I assume that's thanks to the absence of JavaScript, rather than something special that Firefox is doing.

themagician 8y ago

Can someone explain to me the purpose of HFT? Not the economic incentive, I get that, but what does it accomplish? Is it a net benefit to the system? If we arbitrarily limited transactions to human speed would there be some kind of systemwide downside? Are there any good, human readable papers on the benefits of HFT?

18nleung 8y ago
It’s most commonly justified in terms of “liquidity” - allowing orders to be filled at as close to “perfect” market prices as possible
- tptacek 8y ago
  I'd think less about the "perfection" of the price and more about the spread. The faster everyone converges on a single understanding of the price, the less you'll have to pay to bridge that uncertainty when you want to buy or sell right now. I always mentally compare our expectations about buying and selling stocks immediately (pennies or less) to that of buying and selling houses "immediately".
BonesJustice 8y ago
The only kind of HFT I’m directly familiar with is for automated market making, so I’ll speak to that. Market making in itself provides a valuable service by keeping spreads razor thin and ensuring there’s enough liquidity to satisfy almost any trade at close to the current spot price. HFT is mostly just a way for market makers to gain an edge over each other. In general, it neither helps nor harms ‘regular’ customers. It would hardly be noteworthy if not for the insane lengths that HFT groups go to in order to gain an edge. And, of course, the possibility that a HF system with inadequate controls goes on a trading rampage that significantly moves the markets (see Knight Capital).
- MuffinFlavored 8y ago
  > market makers How many market makers are there?

deleted 8y ago

[deleted]

deleted 8y ago

[deleted]

pkaye 8y ago

Could they not just make one tower a few feet higher than the other?

deleted 8y ago
[deleted]

lolc 8y ago

I'm largely ignorant of HFT but I often wonder what the benefit of it is. It sounds like a huge zero-sum game to me. In my (probably simplistic) view, exchanges could publish and process orders at long intervals (seconds, minutes, hours?) then add a jitter of several intervals to incoming orders, with cumulating jitter for cancels. That would require traders to look beyond one interval when trading.

shitgoose 8y ago
would you rather trade on exchange that adds jitter to your order, or one that doesn't?
- lolc 8y ago
  Depends on my goals. If I'm buying or selling for investment, I just don't care about that jitter. Because my decision to buy or sell doesn't happen at that speed. If I'm racing other people's orders on the sub-second level then I care. But what's the point of that? Why should people spend their lifes on it? Why should we as a society bear the cost of this activity? I'd rather see people do something with external benefits.
  - shitgoose 8y ago
    What is the point of adding X ms delay to your order? What does it fix? The "problem" with HFT is that participants can quickly pull out the limit orders, so your market order will be executed at the price that is worse than you expect. Jitter doesn't fix it. If anything, it will make it worse, i.e. your order will hit the market that is quite different from what you saw when you sent the order.
    - lolc 8y ago
      Not delay, jitter. It removes the possibility of timing orders to sub-second intervals. It's supposed to prevent people from spending resources on a zero-sum game.
8y ago

smallnamespace 8y ago

I don't know when we'll have practical long-distance neutrino transmitters [1], but I'll bet someday the implementation will be bankrolled by HFT firms. Why go around the Earth when you can go straight through it? [1] https://phys.org/news/2012-03-wireless-message-neutrinos.html https://phys.org/news/2012-03-wireless-message-neutrinos.htm...

deleted 8y ago
[deleted]
endofcapital 8y ago
I think when we get into cool near sci fi stuff we'll have faster than light communication pretty soon with quantum entanglement effects, which should be way easier to wrangle than neutrinos.
- kch01 8y ago
  Quantum entanglement cannot transmit information faster than light unless contemporary physics is very very wrong. Neutrinos may be hard to use but they at least known to be physically possible.
twic 8y ago
Fermilab is literally a couple of miles from the CME datacentre in Aurora. They must have some old accelerators they aren't using. The original Kamiokande detector is still in the ground at the Kamioka Observatory, as far as i know. Hasn't been used for science since the '80s. That's about 300 km from either Tokyo or Osaka, so you probably still need microwaves at that end. Now you just have to think of a trade with a signal in Chicago and an instrument in Japan. The CME has Nikkei futures, but they follow the ones in Osaka. Osaka has Dow Jones futures, which presumably follow the ones in Chicago, but the volume in Osaka is pretty small. Your best best is probably FX - both Chicago and Tokyo have dollar/yen futures with good volume. So, who is already doing this, and how long have they been doing it?
titanomachy 8y ago
The first message sent by telegraph was supposedly "what hath God wrought?" The first message sent by gravity wave will be "SELL 400 AMZN"

chungleong 8y ago

I wonder if a lot of money is going into research on "spooky action at a distance".

human20190310 8y ago

Location is everything. If trading firms are willing to spend money to position their towers slightly closer to the trading endpoint, the town where the endpoint is located should approve construction in geographic order, allowing new towers to be constructed closer to the endpoint in a way that maximizes tax revenue to the city. Ideally this would keep a lot of tower-construction people continually employed and paying taxes, building new towers ten meters in front of the old ones every month or two.

ggm 8y ago

I get that arbitrage helps set the price, but what does HFT do, beyond deciding which of the competing nano-second enabled traders get to edge a profit in an artifact of how information flows? I don't see this HFT as being information-worthy arbitrage. I have an intent to set a real-world price, I declare it, and in the increment of time it takes my packets to flow, somebody edges in the queue and inserts trades to make me have to deal with them for some increment of price. I wind up paying more, and the original source winds up getting less, or some other event. its like re-intermediation, in a world heading to dis-intermediation. What is the real-world, wider economic "good" here? If my pension fund is routinely profiting from this, I am not sure that is a wider good btw: We could probably make more money selling Heroin, if we're going down this 'whatever it takes' track...

jjtheblunt 8y ago
I don't think anyone asserted there was a real world wider good; to the contrary, I think it's just the actions of a system in place.
kasey_junk 8y ago
> it takes my packets to flow, somebody edges in the queue and inserts trades to make me have to deal with them for some increment of price This is the biggest mistake people make when talking about HFT. At no point in time does someone see your order before it hits the exchange (unless you’ve specifically agreed to trade that way & even then they are obligated by lots of regulation to execute in particular ways to protect from that agency problem). No HFT is taking pennies from your order. If you send a limit order through the book you will get filled at the best price available up to your limit.
- ggm 8y ago
  Thanks for explaining. If they're just squeezing margin by being quicker, and not intruding into my trade (I don't trade. this is clear I think, given how ignorant I am) then I have less to worry about.

deleted 8y ago

[deleted]

snek 8y ago

Why don't the rich people just move to New York?

QuackingJimbo 8y ago

I work in HFT as a quant dev I am proud of my work and find HFT to be an interesting intersection of computer science, math, game theory, economics, and psychology As far as the societal benefit of HFT, I think that it is neutral. It doesn't hurt anybody. It might even help a little bit (liquidity and tight spreads save people a little money and maybe spur some economic growth). However, this particular aspect (the physical latency war) makes me literally sick to my stomach. I hate reading about it and I hate thinking about it. It's like -- what are we even doing here guys?

elgenie 8y ago
It's only neutral (at best) if you don't consider opportunity cost. Is the best way for the country/society/humanity to deploy a bunch of smart people with a decade and a half of education truly an attempt to try to cheat the speed of light? If trades took 2x as long to execute as they currently do, what exactly would be the harm?
- scotty79 8y ago
  Also advertising....
- integrate-this 8y ago
  Well, you would have lost out on a lot of innovation for one. I would be willing to bet that nearly half of the world's high performance computing specialists cut their teeth with a trading firm at one point or another. By your logic, there would have been no benefit to video games or going to the moon even though each of these things have helped deliver large amounts of technological advancement. IE, Facebook is a giant waste of resources but they employ many of the world's smartest engineers. Because of that they've spawned things like ReactJS which has completely changed web development for thousands of other companies.
- QuackingJimbo 8y ago
  The "brain drain" argument is fair, but you can successfully apply the brain drain argument to anything short of curing diseases or feeding the hungry I consider HFT to be no worse of a brain drain than, say, Instagram, or any similar virus

anfractuosity 8y ago

This film seems relevant - https://www.imdb.com/title/tt6866224 https://www.imdb.com/title/tt6866224 - The Hummingbird Project (2018). 'A pair of high-frequency traders go up against their old boss in an effort to make millions in a fiber-optic cable deal.' (It also seems to feature the microwave based approach from the trailer).

deleted 8y ago
[deleted]

johnnycab 8y ago

The dice is fully loaded in favour of HFT pioneers and incumbents, facilitated by the ever-growing and nouveau riche (e.g. Zayo) Tier-1 ISP's, who control most of the backhaul and dark fibre estate. However, over the next few years, the advent of maturing ML frameworks, coupled with the intensifying war and ratification of standards over 5G/IoT based infrastructure, might prove to be serendipitous for nimble-footed enterprises. https://en.wikipedia.org/wiki/Zayo_Group https://en.wikipedia.org/wiki/Zayo_Group

mattrp 8y ago
Highly unlikely that 5g / iOS will play a role in the context of hft. Also your view of the ecosystem (eg zayo and co being the big winners) is massively outdated by five years at minimum.

tlrobinson 8y ago

The use of wireless communication in high frequency trading begs the question: how much could a malicious actor profit from momentarily jamming that CyrusOne (or similar) tower?

soheil 8y ago

I did the math to use the neutrino transmitter it'd be 0.13% faster to transmit information from Chicago to NYC. Using equations: s = r(theta) l = (2r)sin(thea/2) s: arc length (air travel distance 714 miles) r: radius of earth (3,958.8 miles) theta: arc angle l: length of cord (straight line under arc) (s-l)/l * 100 gives the % increased difference in length

PhantomGremlin 8y ago
You need to add a little extra time to the microwave route because microwaves are line of sight. The signal needs to be detected and re-amplified multiple times along the way. How much extra time? I don't know, I'm not an RF guy, maybe someone can chime in and tell us how much latency each amplifier adds. The neutrino route only needs a single detector at the very end, nothing along the way.

raviolo 8y ago

Another example for people to appreciate what a small latency advantage means. Consider SPX options which is a huge market. These “trade on CBOE” which is a Chicago exchange - but they physically reside in NJ. Now consider SPX futures - ES - on CME, physically in Aurora. Now imagine a 20-tick / 5-handle “sweep” in the ES — “sweep” is instantaneous price move (either up or down). Not “very fast” but exactly that - instantaneous. These happen as a result of large incoming aggressor order taking out many book levels at once. Sweeps can be of different magnitude (from just few ticks all the way to max allowed per CME’s “velocity logic”) - but they happen all the time. If you know enough about e.g. ES futures but have never seen sweeps, chances are you didn’t have access to the right tools/platforms. Even some supposedly-professional-grade platforms like TT or CQG do not show trade data with enough detailization to see these. Anyway... why is this important? Because immediately following the sweep in e.g. ES, option prices in e.g. SPX options are temporarily way off. Dutch market makers have not yet moved their options quotes, because they do not yet know about the sweep - at least not in NJ. If you can somehow get orders to NJ faster than market makers can cancel their option quotes, you can take out hundreds of mispriced options. Multiple expires, dozens of strike, puts and calls. Very juicy bunch of stale quotes. PnL... well that gets tricky. On one hand, just this one example is in theory worth hundreds of million per year if market did not respond. But the market will respond. That’s inevitable. No one enjoys being ripped off for too long, so if they cannot defend they will either leave the market or set quotes wide enough that they are never mispriced. The same goes for any other “latency arbitrage” game. I always love how people throw numbers like “this is worth X”, almost always assuming status quo. HFTs are very responsive to market conditions. If I run a strategy that’s been making money every single day for 10 years and all of a sudden it starts losing - I press the pause button and go do some serious analysis. So maybe I will let you have it for few hours - but after that if I can’t compete I’m out. So it’s not that simple.

mattrp 8y ago

Follow @sniperinmahwah if he is still active... everything else is stuff that’s been known for quite some time.

freshcatch_ 8y ago

Capitalism is just so absurd.