13 ms·
Trading bots are running wild on cryptocurrency exchanges
- mjfl 8y agoNote: they can really only "frontrun" you if you are making a massive trade. Even then, from the other side it's not really perceived as frontrunning, rather as "getting out of the way" of a huge order that's going to change the price. You don't want to be the one that sold them some stock for $1 that they will immediately push to $10, which then you will have to buy back for $10, as you are the market maker...
- BubRoss 8y agoIt isn't just getting out of the way. HFTs see the big order and realize it is too big for one exchange, so they buy from the other exchanges before the rest of the order gets there.
- momentmaker 8y agoHow is this any different from algo trading on the stock market? I read or watched somewhere where the algo traders would skim a tiny percentage of every transaction of mutual funds because they know exactly when to strike. There are lots of sharks in the muddy water...
- Scoundreller 8y agoAnyone can watch crypto going into the exchange. Usually people don’t (shouldn’t?) hold it on the exchange for excessive periods of time, so once it confirms, they try to sell and withdraw. If you can see this, you can front the sell (before the deposit even confirms) and then be the buyer shortly after it confirms. But I don’t see this as being much different than providing liquidity and taking the risk that someone else doesn’t want to take.
- dfox 8y agoThat depends on the underlying cryptocurrency, for example I think that there is no way to design exchange such that BTC deposit would be readily apparent on blockchain before it is credited to the exchange account. So it is simply another argument why "smart contracts" does not solve any useful problem.
- raiyu 8y agoAlgorithmic trading is about building an algo that determines how a market or stock will move based on news, economy, other factors. In flash boys the front-running that these quant funds are running isn't about an algorithm, but literally all about having high capacity pipes that can see an order come in and then fill it between different exchanges and based on the minute differences in stock prices pocket the difference. So an algo fund can lose money if the algorithm is incorrect. While front-running you never lose money. It's basically like being a tax collector.
- phildaian 8y agoGreat explanation! For an in depth and relatively formal (and mathematical) look at this from an economist I recommend https://faculty.chicagobooth.edu/eric.budish/research/HFT-FrequentBatchAuctions.pdf https://faculty.chicagobooth.edu/eric.budish/research/HFT-Fr...
- afarah 8y ago>all about having high capacity pipes that can see an order come in I don't understand how they can see the order come in. Is this something that only happens in decentralized exchanges?
- aianus 8y agoYou can pay the NASDAQ (or any exchange) to colocate your server beside their server with a low-latency network link and see a live feed of the orders as they hit the book.
- chii 8y agoSo isn't the problem with the exchange revealing data before it is "committed"? How does one frontrun if the exchange doesn't allow it?
- afarah 8y agoI know about colocation, but I was under the impression that once you see an order in the book it is in the book. I.e. your order will be placed after it.
- ForHackernews 8y agoNothing to see here, just a bunch of rational actors pursuing their self-interest in a market free from nanny-state big government interference.
- natalyarostova 8y agoWord.
- pbhjpbhj 8y agoSome people value other things above rationality. What a crazy world it would be if people acted morally even when not being watched by "nanny-state big government"; such government might be entirely unnecessary then.
- momentmaker 8y agoCoincidentally, there is a paper and talk about this called: Flash Boys 2.0: Frontrunning, Transaction Reordering, and Consensus Instability in Decentralized Exchanges https://arxiv.org/abs/1904.05234v1 https://arxiv.org/abs/1904.05234v1 https://www.cbspringconference.org/ https://www.cbspringconference.org/
- LawnboyMax 8y agoHere's a talk about this research that Philip Daian gave at Devcon 4: https://slideslive.com/38911605/smart-contract-security-incentives-beyond-the-launch https://slideslive.com/38911605/smart-contract-security-ince... Probably my favourite talk from that conference.
- kbody 8y agoHowever, that's not quite front-running in the established sense. That paper is about a side-effect based on the design of "Decentralized Exchanges" on Ethereum and similar systems. That's why people should always dive in the details of those blockchain solutions; not only you have implementation issues, but even design issues where the "solution" falls apart when game theory is accounted for all actors and interactions with the network. Unfortunately, the complexity of such "decentralized solutions" makes it easy to miss a lot of issues. Even with audits that cost thousands of dollars per case, you still have several cases where serious vulnerabilities go unnoticed.
- alexpetralia 8y agofrontrunning (n): the practice by market makers of dealing on advance information provided by their brokers and investment analysts, before their clients have been given the information The prototypical example is a large investment bank hearing about a large client buy order, then putting in a buy order first on their own book, knowing it will go up due to the client order. It is a violation of your fiduciary duty to the client.
- Misdicorl 8y agoMarket makers aren't agents/brokers. They do not have clients.
- tptacek 8y agoRight, and races between mutually disinterested market participants aren't frontrunning, which is what I assume the subtext of that comment was.
- BubRoss 8y agoExcept one party doesn't realize it is a race.
- tptacek 8y agoThat doesn't matter. The actual offense of frontrunning involves an agent-principle relationship between the parties.
- phildaian 8y agoAuthor of the paper here. I justified our use of the word "frontrunning" on a Twitter thread: https://twitter.com/phildaian/status/1116698738343460864 https://twitter.com/phildaian/status/1116698738343460864 There are a few different points here. Firstly, there are plenty of unambiguous definitions of frontrunning that are accepted by the scientific community that this blatantly does fall under; see the Clark SoK we cite in our paper for this in the blockchain space, or the definition Wikipedia uses, which was from an economics article by Khan & Lu (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1140694 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1140694): "trading by some parties in advance of large trades by other parties, in anticipation of profiting from the price movement that follows the large trade." The second point here is even if you want to restrict the definition to require privileged information, privilege in p2p networks is not binary: there are some nodes that are better peered on the topology, have better mining equipment, etc. who can use these privileges to extract market rent, harming the security of a blockchain. This is a complex and dense paper that took 8 people 18 months to produce, and I recommend you read it with an open mind, rather than having a gut reaction to whether we use your preferred definition of "frontrunning". I also recommend reading the primary source material directly, or at least my summary on Twitter - https://twitter.com/phildaian/status/1116155236433956865 https://twitter.com/phildaian/status/1116155236433956865 Happy to answer any questions!
- RobLach 8y agoBy design.
- village-idiot 8y agoFinancial system setup by those ideologically opposed to current regulatory regime suffers from the same problems the current regulatory regime was designed to mitigate. Report at 11.
- Kenji 8y agoSoo... the problem is what exactly?
- brentis 8y agoNon issue. Of course. Just put in a limit order and be done with it. If you are really scrapping pennies in front of steam rollers you should find another gig because algos will eat you up.
- Tomminn 8y agoThere is no steamroller in this approach. Perfectly exploited, you act as an infinitesimal holding-time middleman that has zero probability of not being able to sell, since the future queue is already known. You buy enough so that the real buyer has to pay the upper limit for their limit order every time they buy. Imperfectly exploited, you still make a very reliable, very low risk profit.
- prewett 8y agoThe OP was referring to the person affected by the front-running, not the front-runners. "Picking up pennies in front of a steam-rollers" is from Taleb, and refers to short-term traders who buy/sell a large amount on margin (or backed by loans), and hold it for a very short period. Since the market tends to go up on more days than it goes down, on average these guys make money (before taxes and trading fees). The steam-roller is the black swan event, the large, unpredictable event that will drop the price faster than you can get out, causing you to lose more money than in the years leading up, and quite possibly bankrupting you due to margin calls or the loans. The OP is saying that the only way someone front-running you causes you problems is if you are doing many short-term trades, because otherwise the percentage you lose is too small to worry about. If the front-running loss (which is small) is a large percentage of your gain, then you must be picking up pennies, and if you are doing that, you are risking being flattened by Taleb's black swan steamrollers.
- Tomminn 8y agoThe only way someone front running causes you actual profit margin problems is if your profit margins rely on trading a large volume in a given time period, yes. This essentially means you're profiting off an understanding of volatility. As Taleb showed, this requires care, since the standard deviation of a security cannot be reliably quantified from historical data, yes. But there are almost always ways to trade large volumes on a prediction of volatility without running into Taleb's steamroller. The easiest first step is to never write a call option on a stock you don't already own (or short sell stocks in any other fashion). Then your losses are bounded since you can't go below zero dollars. Then you've just got to make sure you don't bust when you're left holding the busted security. This can be done with ordinary bankroll management like a poker player might use. But it's also what hedging is for. With hedging, you control the risk distribution of a given trade. All you do is simultaneously make other trades that boom in the case that the first trade busts.
- deleted 8y ago[deleted]
- infecto 8y agoI am not as familiar with the crypto trading industry but I did work in equity markets for a number of years and for a while on short term, high volume strategies. I still have a hard time trusting any article that uses Flash Boys. I still find that book to be one of the worst representations of the electronic markets. It paints this evil picture similar to a chemtrails kind of conspiracy where a single entity is causing harm. In reality it is a complex organism where each entity is try to gain an advantage. Since a lot of comments are talking about the equity markets I figured I would add my thoughts. There is definitely a need for regulation. No we don't need to get rid of electronic trading, frontrunners or algs but we do need to pay attention to whats happening. I remember that window of time where the NASDAQ was selling order information subseconds before it happened. This is bad. Or the exchange (forgot which one) that was creating special order types for just big clients that were unpublished, this is also illegal and was stopped. I love frontrunning. People seem to forget the era pre computer market making where there was a guy that made the market for a specific stock. Or how slowly information traveled. People on the floor would know the moment something happened and be able to trade on it right away compared to everyone else. We had dollar size spreads. Today we have penny or even sub-penny level spreads. Sure someone is making money off of me when I make a tread, but their margin is A LOT less than what it once was. Is someone manipulating my order...probably not, I am too small of a fry. Is someone manipulating Vanguards orders, people are probably trying but I suspect based on literature that firms like Vanguard have put out...they overall like it, cost of trades are drastically lower than historically. TLDR if someone has a way to trade across markets faster than the rest...thats great, they are serving a useful job in making prices across markets even and if they are doing it well they are earning their money for the risk and cost associated with it. Edit: I highly recommend the book Dark Pools. Its a much more unbiased book about electronic markets and how they came to be. I feel that it paints a much more realistic picture compared to Lewis' hype.
- rb808 8y agoI'm worried about articles like this are because people want to try to regulate crypto currencies. I really like the idea of crypto being the wild west. If you use a cheap/crappy broker or exchange you're going to get a worse price than if you use a good (expensive?) one. Deal with it. If you want to deal with a heavily regulated currency use EUR/USD/JPY etc.
- verdverm 8y agoCryptocurrency will never achieve any meaningful status without regulation. We have anti money laundering and other financial laws for a reason.
- trophycase 8y agoUnfortunately this is the sacrifice that needs to be made if you want permissionless money. Regulation is not necessary.
- malcolmgreaves 8y agoPermissionless money isn't useful to the large majority of the world. It's a nice to have for rich people -- or "temporarily poor millionaires." More useful is to ensure that governments have the rules that their populace wants. And even most useful is to ensure that the human beings who don't have money (for the multitude of unfair reasons that may cause such a situation) receive enough to survive.
- verdverm 8y agoFortunately the populace wants regulated money.
- pavel_lishin 8y ago> I really like the idea of crypto being the wild west. Some people liked the idea of the wild west being the wild west. But it turned out that a lawless frontier favored a few people over a lot of others, and just because something is beneficial to you doesn't mean it's equally beneficial to others, or to society at large.
- harryh 8y agoSince we're close to the topic, let's take a moment to all remember that Flash Boys is a crappy book that gets a lot of stuff really really wrong. Read the rebuttal! https://www.amazon.com/Flash-Boys-Insiders-Perspective-High-Frequency-ebook/dp/B00P0QI2M2 https://www.amazon.com/Flash-Boys-Insiders-Perspective-High-...
- Hasz 8y agoDark Pools is an excellent alternative, on a slightly different topic, although similar material is covered.
- tptacek 8y agoStrongly recommend Dark Pools for people interested in this topic from just a sort of history-of-technology perspective; it also covers a much longer span of history. Dark Pools also takes kind of a gimlet eye to algorithmic trading, in case anyone thinks the recommendation is because the author thinks the opposite thing to Lewis.
- SEJeff 8y agoAdditionally, I can personally vouch for Peter Kovac, the author of the rebuttal to Flash Boys: https://www.amazon.com/Flash-Boys-Insiders-Perspective-High-Frequency-ebook/dp/B00P0QI2M2 https://www.amazon.com/Flash-Boys-Insiders-Perspective-High-... I can vouch for Peter because I worked with him for close to 5 years at Madison Tyler Trading / EWT, LLC (which later merged with Virtu Financial). He’s a very talented technologist who build the real-time clearing and risk system along with an encyclopedic memory of financial regulations and nuance. He always stuck out to me as an “honest to a fault” kind of guy, and that is one of the reasons we got on so well, as I was never much for nuance.
- kristianp 8y agoMichael Lewis is a former trader himself. I find it hard to believe he could get things that wrong.
- 8y ago
- integrate-this 8y agoWhere are the people that told me that front-running is only used to describe your actions in response to your own client's orders? Just because you were taught not to use it in email doesn't mean that certain forms of arbitrage are not front running.
- rajacombinator 8y agoAnyone who is a serious participant in the crypto markets knows they are nothing but a pure Wild West free for all. That is, like all other markets except without the pretense of regulations that purportedly protect the little guy. (But usually achieve the opposite.)
- docker_up 8y agoThe point is that it's not a free-for-all. The biggest players completely manipulate the markets so that all the small fish end up losing to them.
- village-idiot 8y agoThat’s exactly what I would expect if a system was described as a “free for all”, the strong preying on the weak.
- nodesocket 8y agoAm I wrong, but don't all the crypto "whales" know each other, and can't they synchronize buying and selling? Essentially their bots all start buying together, wait for retail and other bots to join the uptick and then again all sell at the same time. Profit. Repeat.
- swiley 8y agoI'm pretty sure they don't all know each other unless you're referring to fake ones used to scam users of centralized exchanges (in which case they're the same entity.)
- throwaway14213 8y agoI run several of the bots listed in the covered paper (Figure 6) and can prove ownership of these addresses to a mod. I can tell you right away that there are not hundreds of bots "running wild" here, but a handful of entities that operate multiple, sometimes seemingly independent bots, with almost all profit captured by the top few entities. As others have pointed out, the vast majority of bot activity on decentralized exchanges is not in fact front-running, but simply filling limit orders that cross order books of other exchanges, or that move the price of an automated market reserve. It's just arbitrage. To compare this type of activity to centralized exchange manipulation, as this article does, or to suggest that this is somehow worse than the structural deficiencies in traditional financial exchanges is just disingenuous and untrue. The paper that Philip and others authored is great in highlighting how miners have an advantage in capturing profits over non-mining arbitrageurs in the latest version of Ethereum DEXs, and there are many designs in the pipeline that aim to rectify these issues (zero-knowledge proofs, proposed changes to the 0x Protocol, etc). Several of the other problems can and will be fixed by moving to proof of stake and implementing finality. Again, we're talking about problems with rewarding the arbitrageur's efforts, not at all with the security of the user's funds. The $20,000/day is not far off the mark, but it has since declined considerably given the downturn in price and decreased interest in ICOs. The real story is what is going on with the unregulated centralized exchanges, but we can only guess.
- deleted 8y ago[deleted]
- codingslave 8y agoAny reference to flash boys is commonly an article looking for click bait. If you look at modern HFT, most of the profits have been eaten away among the competition, its no longer some hugely profitable industry. It is contracting. Sure a few firms are making millions, but its really not a big deal. Its really getting tired, most people love to get all up in arms about this, at this point high speed trading is run of the mill. The same will happen to cryptos. There is always early profitability because theres no competition, but the profitability will erode.
- fouc 8y agoWhy would DEXes (Decentralized Exchanges) allow priority ordering by paying higher fees? Wouldn't they be aware that enables front-running?
- craze3 8y agoI'm pretty sure the author was referring to "gas" on the Ethereum blockchain. The more you pay, the faster your transaction completes. More info here: https://ethereum.stackexchange.com/questions/3/what-is-meant-by-the-term-gas https://ethereum.stackexchange.com/questions/3/what-is-meant...
- infinii 8y agoSurprised that nobody has taken the effort to explain "how" this is happening. The article talks about flaws in DEX design, opening the risk of front running. What's happening is that DEXs work on chain, therefore orders are submitted on chain and thus visible in the mempool while waiting for miners to include it in a block. Front runners view these pending orders and then submit a similar order (but with a higher gas fee), thus incentivizing miners to take their Ethereum transaction instead of the earlier order (for higher fees). While the article in question is correct with regards to what I explained above, I hate how generalizing it is. Not all DEXs are designed the same way. The first generation of DEXs did everything on chain which also has the downside of limited throughput (limited by blockchain transactions per second). The next generation DEXs are working to solve this with off chain solutions which would also solve the front running problem.
- veryworried 8y agoHow would one go about becoming a flash boy, if it were still feasible?