14 ms·
The high-frequency trading arms race: frequent batch auctions (2015)
- deleted 5y ago[deleted]
- bob1029 5y agoThere are severe technological consequences for pushing for synthetic discrete time. Exchanges that currently execute in a serialized fashion may no longer be able to support the trading volume if the underlying platform is unable to develop batch sizes that naturally align with hardware capabilities and timings. Put differently, I think what is going to happen is you will start stacking way more orders at each interval than you can process before the next because the wonderful CPU pipelining effects get wrecked each time you hit an arbitrary time slice boundary. I suppose you could intentionally spin the CPU instead of yielding back to the OS during these delays, but that means you are not able to process any orders that are currently arriving, so your tail ends up growing longer and longer.
- zamadatix 5y agoWhy would a premade queue be worse for pipelining than a random queue which you also have to modify as you process on it? Seems like a single pause per batch rather than constantly checking if the work queue has something new put in it.
- andylynch 5y agoThis hasn’t materialised as a problem (batch auctions are one on the MiFID II venue models) - some eu venues have run this model for around four years now, its definitely less widely used than other models but has a niche.
- twic 5y agoIt might not have materialised as a problem because those venues are not handling as much traffic as the busiest CLOBs are.
- atq2119 5y agoI would be very worried if the machines actually executing orders today were anywhere close to 100% load on average, because of the well-known issue where tail latency explodes as you get closer to 100% load. So I doubt the relevance of everything you wrote there. Batched auctions require different algorithms, sure. They may even be more expensive to execute. I suppose you have to sort the batch once instead of sorting as you go. Maybe that makes it O(n log n) instead of O(n)? Can you keep a traditional order book up-to-date in O(1) per transaction? Either way, seems like this should be a non-issue. Even if exchanges need to add more shards for order processing, that's just not a big deal.
- kamaitachi 5y agoI was working in HFT as a dev team lead around the time of this article (2105). I remember this was being seriously considered by one of our target exchanges (can't remember if it was Eurex or Globex). Our main HFT trader didn't seem worried - he said that the race would just change from a race to pick off an opportunity into a race to align with any auction timeframe. Back then, our strategies were implemented in FGPA so our response to events could be timed very accurately. Even randomly-timed rolling auctions wouldn't have posed any challenges. Probably explains why this idea never ended up being implemented by any of the major exchanges.
- amelius 5y agoIf your strategies were implemented in FPGA, they were probably not very complicated (considering the things you can do on a regular CPU). Wouldn't markets function better if every participant had a reasonable amount of time to make decisions?
- deleted 5y ago[deleted]
- tedunangst 5y agoAs a retro computing enthusiast who does algo trading from my Commodore 64, I would very much like more time to compute my trades.
- hansor 5y agoThis is interesting. May I ask how do you do it from software poinr of view? As for CPU and memory for sure it is possible, but I'm more interested about how do you connect into exechange/broker? Contiki, SLIP, PPP, or maybe some ethernet expansion ?
- twic 5y agoI'd love to read about you trying to persuade the exchange to let you put that C64 in the colo.
- 5y ago
- carterschonwald 5y agoWhile I was at jpmorgan I actually spent some time thinking about alternative auction structures (vs the order book model). The current trading model is ultimately a mechanization of the rules from trading happened in a literal trading floor room, and a lot of the structural issues stem from those rules treating time as infinite resolution and the speed of information propagation/light being instaneous. There’s some interesting details about how large trades are done today that could perhaps be better reflected into some element of auction design.
- wolverine876 5y agoSo if we started with a clean slate, no trading room floor history framing our perspectives, how could we do it?
- carterschonwald 5y agothats that billion/trillion dollar question :) so I think theres a huge design space, and I think it partially turns into a "mechanism design" challenge to articulate a landscape of transaction / market auction mechanisms that 1) incentivize maximizing market liquidity 2) recognize the speed of light is finite, and have that inform the minimal time scale matching can happen on. 3) obviate/remove the need to obscure large trades as a large number of smaller trades (which is half the value of so called algorithmic trading strategies to institutional investors). This could be via having one design constraint on auctions be that the market impact of the sum of the small trades should be equivalent to the single large trade. (ignoring the issue of the exogenous information of there was a large trade ). some interesting knock on consequences of these ideas are the following 1) the larger the time scale you're willing wait for the trade to be matched to "the other side", the cheaper it should be to trade! (creating liquidity is valuable!) 2) if you're willing to allow your trade to be "partially matched" instead of all or nothing, that too creates liquidity. the point being, you start with "what are all the complications of how people do large/complicated trades today that should just be trivial with the right auction" is sortah my perspective. thats glossing over a lot of complexity and other concerns, but those are some high notes. that said,this is just the tip of the iceberg, and these sort of market design questions are genuinely under studied in my mind, and i could easily spend hours talking about this in greater depth over coffee or such.
- hcarlens 5y agoThis type of order book is actually quite common in Europe now! It provides an interesting alternative to central limit order books and dark pools. Around the introduction of MiFID II regulation in 2018, several exchange operators added these frequent auction books. Cboe's period auctions book is the biggest of these by volume: https://www.cboe.com/europe/equities/trading/periodic_auctions_book/ https://www.cboe.com/europe/equities/trading/periodic_auctio... In addition to Cboe, Turquoise, Goldman Sachs, UBS, Virtu and Aquis also run frequent batch auction venues: https://www.cboe.com/europe/equities/market_share/market/venue/2021-10-16/#ct=02vrCC&sc=03NrCC&dm=tbpcan&dr=5day&mt=1&ms=0&hc=1&f=0&ID=8d4b5bedc69239a55eb4&V=33bd3ce3c54b531565bd https://www.cboe.com/europe/equities/market_share/market/ven... I actually co-wrote a paper about this at the time, and it's very rare I get a chance to talk more about it! https://jot.pm-research.com/content/13/3/5 https://jot.pm-research.com/content/13/3/5 (sadly it's paywalled)
- kasey_junk 5y agoPeriodic auctions still need tie breakers. CBOE for instance falls back to size then time. This is the same tie breaker that some CME futures contracts have used in a continuous order book. Those contracts always had more gamesmanship than standard price/time contracts when I was trading. Has that become true in the auction space at well?
- hcarlens 5y agoWhen they were first introduced, each of the fba books had slightly different mechanics (matching priority, timing, price determination), so each book needs a slightly different approach. I guess you could see it as gamesmanship, but in equity markets dealing with market mechanics properly is just part of the job.
- deleted 5y ago[deleted]
- ur-whale 5y ago> sadly it's paywalled Which mean most people here won't be able to read it. What aren't uploading it to sci-hub or some other free access venue?
- deleted 5y ago[deleted]
- akvadrako 5y agoIs there any mathematical proof that it's harder to game batch auctions than what we have now? For example, while other markets and the real world moves on, you gain info. So the later in the batch you can submit a trade, the greater your advantage.
- aserafini 5y agoThat was my initial thought as well: this would just become an arms race to submit your trade last? But maybe if the trades were priority queued it would negate that.
- fighterpilot 5y agoCorrect. I believe you need to randomize the auction time. This massively reduces the speed advantage. Without that speed is as important as ever. How does a priority queue work?
- akvadrako 5y agoThat just leads to the same question: is there proof its harder to game random batches?
- fighterpilot 5y agoGameability is distinct to whether there is an advantage to low latency. A random batch auction will be gameable but not so much via speed. Not sure how a math proof would work for that though.
- harryh 5y agoBut maybe if the trades were priority queued it would negate that. If the trades are priority queued, then you have just recreated the speed "arms race" that this idea hopes to eliminate.
- aserafini 5y ago
- harryh 5y agoProbably worth appending 2015 to the title on HN.
- joshu 5y agoalternatively you could have one or more specific crossings once a day and be done with the whole problem.
- robocat 5y agoI thought part of the problem is that exchanges charge more for faster access (e.g. physical collocation), so the exchanges profitability depends in part on creating a bidding war between HFT companies?
- elzbardico 5y agoWhat is the benefit for society of allowing HFT? This is what we should be asking ourselves.
- kortilla 5y agoBetter prices when you want to sell your stock
- H8crilA 5y agoETFs only exist thanks to HFT Have you ever wondered how is it possible that when you buy some shares of SPY someone out there is somehow able to collect 500 securities to fulfil your order? Even if it's not a literal action-reaction, that is what must be happening at the margin. Also, if you still don't believe me, then try to find out how to unpack X amount of shares of SPY (for some non-small amount of X) into individual securities. Can you do it yourself, for example? Whom to call, where's the button for that, who can do it?
- bob229 5y agoImagine all of the great minds that are wasting their time in the financial services sector and understand why we are doomed as a species
- twic 5y agoIf you're a market maker, you really, really want to be able to do low-latency trading in order to hedge fills before the market moves against you. If market makers can't do this, they will make worse markets - show less size and wider prices, or just get out of the game. How do you do this under continuous batch auctions? I have an underdeveloped idea that what we really need is limit order types with built-in hedging. "Bid to buy 100 gizmos at 30c each, and for every five gizmos bought, immediately offer to sell 1 widget at $1.20; cancel this order if the best offer for widgets moves below that price" sort of thing. Basically, you're moving the simple reasoning that has to be executed at low latency from the market maker's FPGA to the exchange's matching engine. Sometimes, you can do this by putting orders in spreads, but only where a spread exists (or can be defined) for the two legs you care about, in the right ratio. You might also want to do more complicated things, like pulling an order in one product if another product moves a lot, because you think that presages a move in the product you're quoting. The idea would be, firstly, to make it much easier to make markets without having to invest in low-latency infrastructure, broadening the base of participants who can do it, and secondly, to reduce the negative impact of speed-blunting interventions like continuous batch auctions or speed bumps.
- Nevermark 5y agoSo generalize simple market offers toward time-limited smart contracts? And everyone having the ability to do so at the same level. Seems like a good idea to me, assuming contract constraints that guarantee market resolution system will resolve quickly and behave predictably. And some nano-fees for contract execution to make DNS attacks unprofitable (for the attacker, profitable for the market).
- sjbase 5y agoThe hedging scenario you describe is one of the hallmarks of combinatorial auctions[0], which let participants enter bids on packages. (Disclaimer, I'm a founder at OneChronos which is applying these auctions to US equities.) So a market maker can express something like: "fill me for any package that includes `x` gizmos AND `k * x` anti-gizmos simultaneously". The more powerful and general version of this is: "Buy and sell any mix of products, subject to the total package being neutral across these 10 risk factors I care about." > You might also want to do more complicated things, like pulling an order in one product if another product moves a lot This is a key problem in US equities or any market with similar fragmentation. The way we're approaching that is to allow those package bids to also include constraints on "current" market conditions at the moment of the auction. A simple one would be "if the momentary spread between asset A and B is greater than X, don't trade." [0]: https://www.forbes.com/sites/forbestechcouncil/2021/12/30/the-pivotal-role-of-mathematical-optimization-in-the-2020-nobel-prize-in-economics/ https://www.forbes.com/sites/forbestechcouncil/2021/12/30/th...
- secondcoming 5y agoThe ad-tech world worked on a Vickery Auction for quite some time. I've often wondered what things would look like if the financial world worked that way instead. (Vickery Auctions are pretty much dead now because websites saw that bidders were bidding $X and automatically assumed that because they weren't getting $X, but rather $(X - Y), they were being ripped off)
- chaps 5y agoMaybe someone can explain how it would work, but wouldn't this need a wide price spread to cover risk?
- dang 5y agoOne small past thread: The High-Frequency Trading Arms Race: Frequent Batch Auctions as a Solution [pdf] - https://news.ycombinator.com/item?id=20003222 https://news.ycombinator.com/item?id=20003222 - May 2019 (4 comments)
- lpage 5y agoBudish et al. start with the statement that "The high-frequency trading arms race is a symptom of flawed market design" and present a mechanism that mostly addresses a specific feature of the current market structure. In 2021, most folks on all sides of the table (liquidity providers, executing brokers, financial institutions) agree that the arms race is individually and collectively value-destroying. They don't claim that it's the "best mechanism" for capital markets. And every mechanism is a set of tradeoffs, so no one should ever really make that claim. At OneChronos (YC S16) [1], we view the arms race as something that's very much worth solving for, but a tiny piece of a much bigger opportunity to make markets function better for all players. [1] https://www.onechronos.com https://www.onechronos.com
- tails4e 5y agoHFT seems like it should be illegal. The 'idea' of the markets is that its fair access to all, at least in theory. I understand there is always assymetry, but things like insider knowledge has been made illegal to try keep the fairness (or at least attempt). HFT does what no ordinary person can, its an unfair advantage. So should it not be banned? I recall there was a company setting up a market with a minimum latency, ensured by all trades going through a spool of miles of fibre optic cable in their office. The idea was to prevent HFT at source, which was cool, but a shame wider markets just let HFT slide, and the fairness assymetry widen.
- amsully 5y agoThe exchange you are referring to is IEX [1]. Michael Lewis (author of The Big Short) wrote Flash Boys about the intricacies of low latency trading and this exchange [2]. One of the primary "unfair" aspect of HFT is Front Running and has been illegal even before electronic trading (the name comes from traders racing ahead of big buyers in the trading pit). It was a big problem during the advent of electronic trading but has since been tamed. Exchanges arbitraging off their clients order books is another matter. Trading fast in reaction to real time (public) events is a market efficiency. Not accessible to the masses, but neither was traveling to wall street to place a trade. [1] https://en.m.wikipedia.org/wiki/IEX https://en.m.wikipedia.org/wiki/IEX [2] https://en.m.wikipedia.org/wiki/Flash_Boys https://en.m.wikipedia.org/wiki/Flash_Boys