7 ms·

Flash Boys 2: Frontrunning, transaction reordering, consensus instability (2019)

146 points by lawrenceyan 4y ago

ironSkillet 4y ago
HFT firms are re-applying a lot of techniques that worked a decade or more ago in regular markets, except now there's way less competition and way less regulation. They are making a killing right now in crypto.
- bcj7393nsk 4y ago
  > They are making a killing right now in crypto. HFT bonuses (many HFT firms trade crypto now) have been crazy since 2020. Even infra SWEs are making 7 figures of TC.
- neatze 4y ago
  In past few years, I got calls with offer to build artificial market making models (aka pump and dump) for crypto exchanges, with 7 figures payout per model + commission, I refuse to do such work, this is just disgusting, don't care how legal or illegal this is, it did not seem they have infrastructure for HFT in terms of Order Routing like it is on stock exchanges like BATS etc ...
  - throwaway_1928 4y ago
    That makes you better than most people TBH.
  - dehrmann 4y ago
    Is it any more disgusting than people speculating in crypto, but lying to themselves and calling it an "investing?"
    - kibwen 4y ago
      Heh, you're falling into the same trap: "speculation" is just a five-dollar word for "gambling".
    - unethical_ban 4y ago
      Yes.
  - mgaunard 4y ago
    It's because the markets have such bad technology that there is such opportunity to do "funny" stuff.
- kasey_junk 4y ago
  It’s not just HFT firms. Every professional trader in the world who got put out of business in the regular markets for being too inefficient is running old fashioned games on the crypto exchanges and making money. Watching the full depth order book on any crypto exchange is like “Back To the Future”.
  - skippyboxedhero 4y ago
    Just imo, I think some of the "games" are fair play. Spoofing only became illegal when computerised trading started. When it was just humans trading electronically, everyone knew spoofing happened, the evidence was in the order book, and people traded accordingly (and, ofc, spoofing involved taking on monumental levels of risk). The rules we have are pretty bent. Everything is geared towards transparency but if algos work out your trades are actually moving the price, the liquidity is gone. So vanishing liquidity is okay, but spoofing isn't? Hiding your intention is an innate part of markets, and is a way to defend yourself against other traders. I actually support more consumer protection, the pump and dumps indicate that some consumers shouldn't be in these markets. But trading has always been the Wild West (and still is but the banditos run the town instead of the Sheriff now).
- mgaunard 4y ago
  To my knowledge the most money people make in crypto are from token services. Traditional HFT market-making is still somewhat competitive in crypto, and it's not helped by the fact the exchanges aren't sufficiently technologically literate to provide a fair playing ground for all participants.
deleted 4y ago
[deleted]
finexplained 4y ago
Couple things. (1) The word "frontrunning" applies only in a very specific context, where you trade in front of an entity with which you have a client relationship. That is different from capturing a trading opportunity because you are faster than another party. Frontrunning is illegal and law-abiding HFT firms don't do it. They often don't even have clients to which this applies! Being faster is not illegal. Can we please stop using this term to describe behavior we don't like. (2) What do they mean they introduce the term MEV (miner extractable value)? Everyone calls it that, and it's part of the incentive that miners have to validate transactions. (3) Might want to add (2019) to the title, this paper is pretty old. Perhaps this answers (2).
- zydex 4y ago
  Everyone calls it MEV because of this very paper. They coined the term. You did answer your own question in (3).
  - finexplained 4y ago
    Yep, that one is my bad. That's why I added the edit.
    - zydex 4y ago
      I think (1) is a bit harsh relating to the terminology. Technically front running is based on plainly “non-public” information. The term has been generalised and expanded to include crypto transactions which are slightly more obfuscated from the average user who doesn’t sit and watch the mempool. I think the term is completely appropriate.
      - finexplained 4y ago
        I respectfully don't think it is too harsh. Every legal definition I know of frontrunning requires a client relationship. Even in the case of "non-public" information, frontrunning refers to trading ahead of your client based on non-public information related to the clients securities. Trading on non-public information when a client is not involved is referred to as insider trading. Also in this context, the reason that these sandwich attacks can happen is because all the information is public. The adversary can see your transaction waiting to be validated, and pay the miner a higher gas fee to be executed first.
seibelj 4y ago
Anyone who doesn’t like crypto doesn’t have to play with it. It’s a tiny blip of nothing in size when compared to actual markets, and the “crime” that exists in crypto is nothing compared to what large banks facilitate for drug kingpins and oligarchs every single day. Let this weird parallel financial system develop for another decade and see what comes out of it. Might actually solve some problems.
- mgaunard 4y ago
  It's not a tiny blip, it's already bigger than European equities...
  - BozeWolf 4y ago
    Not sure what numbers you compared, but quick research on market cap [1,2]: European market cap 2019: 8,078,748,800,000 (8 trn) Coinmarketcap today: 1,583,350,504,806 (1 trn) Not a tiny blip. Compared to world equity 2021 [3]: 117 trn I would call it a small blip. 1: https://tradingeconomics.com/european-union/market-capitalization-of-listed-companies-current-us$-wb-data.html https://tradingeconomics.com/european-union/market-capitaliz... 2: https://coinmarketcap.com/ https://coinmarketcap.com/ 3: https://www.sifma.org/resources/research/research-quarterly-equities/ https://www.sifma.org/resources/research/research-quarterly-...
    - righttoolforjob 4y ago
      Except that is mostly printed money. You can't take the crypto market cap seriously. If a significant portion of it tries to get converted from "magic internet money" to real money the price will plummet.
      - kuished 4y ago
        You realise that applies to “real money” too right?
        4y ago
aphroz 4y ago
Interesting coincidence, I was reading this paper after seeing it being quoted here: https://www.paradigm.xyz/2020/08/ethereum-is-a-dark-forest https://www.paradigm.xyz/2020/08/ethereum-is-a-dark-forest
ruuda 4y ago
This was the paper that got it started, but now a few years later, strategies have gotten much more advanced. There was an entire conference day devoted to it a few weeks ago, with some interesting talks, if you are interested in the state of the art: https://flashbots.notion.site/MEV-DAY-836f88806995412dabc1c7bb7ce4e830 https://flashbots.notion.site/MEV-DAY-836f88806995412dabc1c7...
- t_mann 4y ago
  Interesting, thanks!
anonu 4y ago
I tell people who are interested in the crypto market: just look at the structures and concepts that exist in "centralized finance" and replicate them for the crypto world. (Eg: risk systems, custodial systems, index calculators, portfolio management, the list goes on, etc... ) This 2019 paper simply re-confirms that thought. Crypto today is no different than the 1900 turn-of-the-century bucket-shop pump-and-dump scams of previous Wall Street iterations.
- deleted 4y ago
  [deleted]
- CryptoPunk 4y ago
  Crypto has unique qualities that enable new applications. For example, you can issue loans to yourself using MakerDAO: https://oasis.app/borrow https://oasis.app/borrow The process is permissionless, meaning, among other things, that it can be automated, and it carries almost zero fees (only the Ethereum gas fee for EVM instructions). To take another example, the world's leading decentralized exchange, Uniswap, has now overtaken centralized exchanges in market depth for some major trading pairs: https://uniswap.org/blog/uniswap-v3-dominance https://uniswap.org/blog/uniswap-v3-dominance This is a credibly neutral value exchange protocol that major centralized exchanges are coalescing around as a source of liquidity. The market depth is truly impressive. For example, you can sell hundreds of thousands of dollars worth of ETH in one trade with almost zero slippage. So crypto is not just another iteration of Wall Street. And to the point of past iterations of Wall Street: the 19th century banking system was not rife with scams as the caricature suggests: https://www.cato.org/blog/fable-cats https://www.cato.org/blog/fable-cats
- syzygyhack 4y ago
  Please tell me where in the world of centralized finance I can go to take out a zero-collateral multi-million dollar loan, transact with it, and immediately pay it back?

146 points by lawrenceyan 4y ago

HFT firms are re-applying a lot of techniques that worked a decade or more ago in regular markets, except now there's way less competition and way less regulation. They are making a killing right now in crypto.

bcj7393nsk 4y ago
> They are making a killing right now in crypto. HFT bonuses (many HFT firms trade crypto now) have been crazy since 2020. Even infra SWEs are making 7 figures of TC.
neatze 4y ago
In past few years, I got calls with offer to build artificial market making models (aka pump and dump) for crypto exchanges, with 7 figures payout per model + commission, I refuse to do such work, this is just disgusting, don't care how legal or illegal this is, it did not seem they have infrastructure for HFT in terms of Order Routing like it is on stock exchanges like BATS etc ...
- throwaway_1928 4y ago
  That makes you better than most people TBH.
- dehrmann 4y ago
  Is it any more disgusting than people speculating in crypto, but lying to themselves and calling it an "investing?"
  - kibwen 4y ago
    Heh, you're falling into the same trap: "speculation" is just a five-dollar word for "gambling".
  - unethical_ban 4y ago
    Yes.
- mgaunard 4y ago
  It's because the markets have such bad technology that there is such opportunity to do "funny" stuff.
kasey_junk 4y ago
It’s not just HFT firms. Every professional trader in the world who got put out of business in the regular markets for being too inefficient is running old fashioned games on the crypto exchanges and making money. Watching the full depth order book on any crypto exchange is like “Back To the Future”.
- skippyboxedhero 4y ago
  Just imo, I think some of the "games" are fair play. Spoofing only became illegal when computerised trading started. When it was just humans trading electronically, everyone knew spoofing happened, the evidence was in the order book, and people traded accordingly (and, ofc, spoofing involved taking on monumental levels of risk). The rules we have are pretty bent. Everything is geared towards transparency but if algos work out your trades are actually moving the price, the liquidity is gone. So vanishing liquidity is okay, but spoofing isn't? Hiding your intention is an innate part of markets, and is a way to defend yourself against other traders. I actually support more consumer protection, the pump and dumps indicate that some consumers shouldn't be in these markets. But trading has always been the Wild West (and still is but the banditos run the town instead of the Sheriff now).
mgaunard 4y ago
To my knowledge the most money people make in crypto are from token services. Traditional HFT market-making is still somewhat competitive in crypto, and it's not helped by the fact the exchanges aren't sufficiently technologically literate to provide a fair playing ground for all participants.

deleted 4y ago

[deleted]

finexplained 4y ago

Couple things. (1) The word "frontrunning" applies only in a very specific context, where you trade in front of an entity with which you have a client relationship. That is different from capturing a trading opportunity because you are faster than another party. Frontrunning is illegal and law-abiding HFT firms don't do it. They often don't even have clients to which this applies! Being faster is not illegal. Can we please stop using this term to describe behavior we don't like. (2) What do they mean they introduce the term MEV (miner extractable value)? Everyone calls it that, and it's part of the incentive that miners have to validate transactions. (3) Might want to add (2019) to the title, this paper is pretty old. Perhaps this answers (2).

zydex 4y ago
Everyone calls it MEV because of this very paper. They coined the term. You did answer your own question in (3).
- finexplained 4y ago
  Yep, that one is my bad. That's why I added the edit.
  - zydex 4y ago
    I think (1) is a bit harsh relating to the terminology. Technically front running is based on plainly “non-public” information. The term has been generalised and expanded to include crypto transactions which are slightly more obfuscated from the average user who doesn’t sit and watch the mempool. I think the term is completely appropriate.
    - finexplained 4y ago
      I respectfully don't think it is too harsh. Every legal definition I know of frontrunning requires a client relationship. Even in the case of "non-public" information, frontrunning refers to trading ahead of your client based on non-public information related to the clients securities. Trading on non-public information when a client is not involved is referred to as insider trading. Also in this context, the reason that these sandwich attacks can happen is because all the information is public. The adversary can see your transaction waiting to be validated, and pay the miner a higher gas fee to be executed first.

seibelj 4y ago

Anyone who doesn’t like crypto doesn’t have to play with it. It’s a tiny blip of nothing in size when compared to actual markets, and the “crime” that exists in crypto is nothing compared to what large banks facilitate for drug kingpins and oligarchs every single day. Let this weird parallel financial system develop for another decade and see what comes out of it. Might actually solve some problems.

mgaunard 4y ago
It's not a tiny blip, it's already bigger than European equities...
- BozeWolf 4y ago
  Not sure what numbers you compared, but quick research on market cap [1,2]: European market cap 2019: 8,078,748,800,000 (8 trn) Coinmarketcap today: 1,583,350,504,806 (1 trn) Not a tiny blip. Compared to world equity 2021 [3]: 117 trn I would call it a small blip. 1: https://tradingeconomics.com/european-union/market-capitalization-of-listed-companies-current-us$-wb-data.html https://tradingeconomics.com/european-union/market-capitaliz... 2: https://coinmarketcap.com/ https://coinmarketcap.com/ 3: https://www.sifma.org/resources/research/research-quarterly-equities/ https://www.sifma.org/resources/research/research-quarterly-...
  - righttoolforjob 4y ago
    Except that is mostly printed money. You can't take the crypto market cap seriously. If a significant portion of it tries to get converted from "magic internet money" to real money the price will plummet.
    - kuished 4y ago
      You realise that applies to “real money” too right?
      - 4y ago

aphroz 4y ago

Interesting coincidence, I was reading this paper after seeing it being quoted here: https://www.paradigm.xyz/2020/08/ethereum-is-a-dark-forest https://www.paradigm.xyz/2020/08/ethereum-is-a-dark-forest

ruuda 4y ago

This was the paper that got it started, but now a few years later, strategies have gotten much more advanced. There was an entire conference day devoted to it a few weeks ago, with some interesting talks, if you are interested in the state of the art: https://flashbots.notion.site/MEV-DAY-836f88806995412dabc1c7bb7ce4e830 https://flashbots.notion.site/MEV-DAY-836f88806995412dabc1c7...

t_mann 4y ago
Interesting, thanks!

anonu 4y ago

I tell people who are interested in the crypto market: just look at the structures and concepts that exist in "centralized finance" and replicate them for the crypto world. (Eg: risk systems, custodial systems, index calculators, portfolio management, the list goes on, etc... ) This 2019 paper simply re-confirms that thought. Crypto today is no different than the 1900 turn-of-the-century bucket-shop pump-and-dump scams of previous Wall Street iterations.

deleted 4y ago
[deleted]
CryptoPunk 4y ago
Crypto has unique qualities that enable new applications. For example, you can issue loans to yourself using MakerDAO: https://oasis.app/borrow https://oasis.app/borrow The process is permissionless, meaning, among other things, that it can be automated, and it carries almost zero fees (only the Ethereum gas fee for EVM instructions). To take another example, the world's leading decentralized exchange, Uniswap, has now overtaken centralized exchanges in market depth for some major trading pairs: https://uniswap.org/blog/uniswap-v3-dominance https://uniswap.org/blog/uniswap-v3-dominance This is a credibly neutral value exchange protocol that major centralized exchanges are coalescing around as a source of liquidity. The market depth is truly impressive. For example, you can sell hundreds of thousands of dollars worth of ETH in one trade with almost zero slippage. So crypto is not just another iteration of Wall Street. And to the point of past iterations of Wall Street: the 19th century banking system was not rife with scams as the caricature suggests: https://www.cato.org/blog/fable-cats https://www.cato.org/blog/fable-cats
syzygyhack 4y ago
Please tell me where in the world of centralized finance I can go to take out a zero-collateral multi-million dollar loan, transact with it, and immediately pay it back?