14 ms·

A High Frequency Trader's Apology, Pt 2

• harryh 14y ago
  Additional evidence that HFTers are not stealing from speculative investors: Anyone can start a market. If HFTers were stealing pennies someone would have come along and started a market that banned HFTers (or changed the rules to otherwise get rid of them). Then all of the speculative investors would use this market instead since they could be assured that no one was siphoning off pennies on every trade. Eventually markets with HFTers would shut down due to a lack of customers. The fact that this hasn't happened has to be considered evidence that the HFTers are, in fact, providing value to speculative investors not the other way around.
  • sailfrog 14y ago
    I'm not sure reg NMS allows this. Registered exchanges publish protected quotes that other venues are not allowed to trade through.
    • boxy_brown 14y ago
      > I'm not sure reg NMS allows this. This is correct. It would be extremely, unequivocally forbidden under Reg NMS to create a quote-disseminating market center with the proposed property.
  • boxy_brown 14y ago
    Your comment is conjecture, not evidence. It's an invalid conjecture, because it makes sweeping assumptions about some of the most complicated, chaotic properties of securities markets. And it's incorrect. There are market centers that prohibit high frequency traders. Reg NMS requires any center that disseminates quotes to prohibit trading through the NBBO (best price at any point in time across all exchanges). So these market centers must be dark pools, not exchanges nor ECNs. These centers represent a small and shrinking amount of trading volumes. Nonetheless, they exist.
• tedunangst 14y ago
  At the very end, Farnsworth buys from, not sells to, Bender. I think it's clear, but had to read it twice.
  • yummyfajitas 14y ago
    Thanks. Fixed it.
    • daltonlp 14y ago
      One more: "Fry - all he knows is that he bought 100 shares @ $10.00 from anonymous counterparty." Didn't Fry sell 100 shares to an anonymous counterparty?
      • yummyfajitas 14y ago
        Yes. Thanks. Arrggh, so many people reading my bad writing.
• AznHisoka 14y ago
  Why apologize? Noone's robbing anyone here. If it's legal, it's legal.
  • burgerbrain 14y ago
    Are you suggesting that people should only apologize for things that are illegal?
  • kaonashi 14y ago
    So it's not possible, in your opinion, for something to be both legal and unethical?
  • haberman 14y ago
    You have just admitted to not even reading the first paragraph of the article, in which he explains his meaning of the word "apology."
  • sophacles 14y ago
    Legal and Ethical are two different things. Confounding both, in the eyes of people is Understanding. It may be legal to something, but unethical. It may be ethical to do something, but illegal. There are many examples of both. The reason for this article is apparently to spread understanding amongst people who think it's unethical to do HFT, but may not understand what HFT does and how it works. By writing this article, he is hoping to convince - via understanding - people that HFT is also ethical.
  • jreeve 14y ago
    The first sentence in the article: "About the title Many people were misled by it, and this wasn’t my intention. I was merely alluding to the book by Hardy, A Mathematician’s Apology, which spent time explaining and justifying his life as a mathematician. (Hardy, in turn, was alluding to the Apology of Socrates. It’s an apology in the classical sense (an intellectual justification), not an expression of remorse. My mistake - as a math geek, it’s easy for me to forget that most people have never heard of Hardy." "Apology" doesn't necessarily imply remorse, just explication.
• arjunnarayan 14y ago
  One point that is bothering me is this: how much of the existence of HFTs is an artifact of the rules of the exchange? In particular, the rule that the first bid gets priority in executing the trade. It strikes me that the entire existence of HFTs seems to be taking advantage of this failure of mechanism design.
  • fr0sty 14y ago
    There was an extensive discussion on this point in the previous HN thread: http://news.ycombinator.com/item?id=3855610 http://news.ycombinator.com/item?id=3855610 The short summary is no one can come up with something better than price-time priority for matching orders.
    • btilly 14y ago
      I still like my suggestion at http://news.ycombinator.com/item?id=3855846 http://news.ycombinator.com/item?id=3855846. True, the HFT folks would still try to provide liquidity by maintaining a bid/ask spread. But every trade that executes because someone wanted to trade while the price was somewhere between the bid and the ask would cut the HFT folks out of the loop to the benefit of everyone else. And scary anomalies like the flash crash would be impossible. After careful thought I am sure that it would not eliminate HFT. But it would reduce their size and impact on the market.
      • alphaBetaGamma 14y ago
        As far as I understand your suggestion, it exactly describes the way current markets work.
        • btilly 14y ago
          Then you did not understand the suggestion. :-( The key point of what I suggested is that the price drifts slowly, and orders can only execute at the current price. In today's market the price can move instantaneously. That's a pretty big difference.
• mykolasmith 14y ago
  Thanks for following up on this. Nothing irks me more than "Pt. 1" posts, never to hear from the OP again. As an undergraduate scheduled to be doing HFT systems development at a major bank this summer, I find these posts very useful!
• haberman 14y ago
  Loving this series of articles! Does any of this low-latency work make it back upstream into Linux or other parts of the software stack? As a latency-obsessed person I would love to know that the fruits of all this labor were available to me for my own low-latency systems. To me, that alone would be enough to feel that the latency-race is providing value to the world. Also, one thing that was not clear to me is how electronic market makers make decisions. The article clearly states that they "have no opinion or information on whether Apple is a valuable company." So what do they have information about? The price history of the stock? The current contents of the book? And on what basis do they make decisions? I guess answering that question in too much detail would be giving up the "secret sauce," but I'd love to know even in broad strokes how an agent that knows so little can so consistently make money.
  • anonDataUser 14y ago
    Here's the secret sauce: http://appli8.hec.fr/rosu/research/news.pdf http://appli8.hec.fr/rosu/research/news.pdf
    • haberman 14y ago
      Awesome, thanks for the reference!
  • deleted 14y ago
    [deleted]
  • yummyfajitas 14y ago
    Usually the low latency stuff is kept pretty secret. However, I got good at writing performing code, and as a result styloot.com is pretty fast. The article clearly states that they "have no opinion or information on whether Apple is a valuable company." This is an exaggeration. Some HFT's incorporate speculative mechanisms into their strategy, supposedly a few people buy the twitter firehose and market news and feed that into their strategy. But in general, HFT's run purely technical strategies. So what do they have information about? The price history of the stock? The current contents of the book? And on what basis do they make decisions? All this sort of stuff. For example, if the book has 20 buy orders at $10.00, and only one sell order at $10.03, that's an indication the price might be going up shortly. But really, it's not all that vital to predict things. The HFT is passively waiting with orders at $10.00 and $10.03 for someone else to fill them - they make money when they sell to someone at $10.03, then close their position by buying at $10.00. The trick is to avoid adverse selection - if the market is likely to move up, don't get caught with a sell order.
• scotty79 14y ago
  What's the social and economical value of precisely discovering the price of casino tokens? Stock market is economically important only when company issues new stock. Apart from that it's just a huge casino where people like to play with their or other peoples money.
  • Confusion 14y ago
    The stock market is economically important every time someone wants to invest in a company, while someone else wants to end his investment in that company. The bid and sell prices of outstanding stock of companies differ per day, per person.
  • chrisaycock 14y ago
    The secondary market provides liquidity to investors who participated in the initial company stock issuance. Without a robust means for selling their shares at some point in the future, investors would be loathe to provide capital in the first place.
    • soup10 14y ago
      Most markets have far more liquidity than is really needed.
      • tptacek 14y ago
        What does it mean to have "too much liquidity"? Liquidity is the cost of buying or selling. You're saying that it would be better if it cost more to sell out of a position, or to buy into a new one?
        • soup10 14y ago
          Well the question is what's the point of the stock market and financial sector if the only time investment is really happening is when shares are offered. The answer the parent suggested is that the stock market provides liquidity that is necessary for people to be willing to invest. I believe that most stocks have far far more liquidity than is necessary for people to feel comfortable investing in them. More liquidity has very little value at this point. It's a problem because thousands of bright minds are piped into the financial industry when they could be contributing much more to society.
• rdlowrey 14y ago
  As a programmer with an economics degree, I'm thrilled to see articles like these on HN ... popular media outlets do a terrible job of explaining how these systems work and usually devolve into generalizations like "HFT is evil and you should hate it" because they don't understand basic economic principles.
  • MarkPNeyer 14y ago
    maybe they think basic economic principles such as "everyone is an equally informed sociopath acting at all times to consume as much as possible without regard to relationships" are absurd. economics is a bullshit field.
    • rdlowrey 14y ago
      But math isn't. Everyone is entitled to their own opinion; mine just happened to result from the mathematical proofs I was forced to study and write for the postulations of that "bullshit field." I would also submit that your argument ignores the law of leaky abstractions (http://www.joelonsoftware.com/articles/LeakyAbstractions.html http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...) ... All non-trivial abstractions, to some degree, are leaky. Finally, economics doesn't assume people "[act] at all times to consume as much as possible without regard to relationships." It assumes that individual actors make decisions that maximize their utility. Utility can result from completely selfless acts -- it's whatever makes you happy. "Selfishness" does not rule out actions that benefit others or society at large. This argument exemplifies the misunderstanding of "basic economic principles" referenced in my original comment.
      • nasmorn 14y ago
        That is funny because the proofs had the opposite effect on me. It dawned on me that the math of e.g walrasian price setting is surely correct but the story that this is supposedly how our economy works was much less believable after each supposedly logical assumption (free disposal, no money pump) could be traced back not to psychology or physical realities but conditions for the equations to remain solve able. Looks to me like a cryptologist predicting what passwords people would choose based on the needed computing power to crack them. I wonder if 'password' made the list.
• deleted 14y ago
  [deleted]
• jmn 14y ago
  One market structure alternative that is gaining some traction in US equities is PDQ: http://www.pdqenterprises.com/faq.html http://www.pdqenterprises.com/faq.html Basically, liquidity providers upload their market making algo to the matching engine. Every time an order comes in, there is a 20ms 'mini auction' in which market making algos compete on intelligence and not on speed.
• SkyMarshal 14y ago
  One of the biggest justifications for HFT that I see is that it increases liquidity. However, did major markets ever really have a huge problem with lack of liquidity, 20 or 30 years ago before HFT? I can't help but wonder if this level of liquidity is only really useful to HFT, if it is something that HFT is both the primary provider and beneficiary of, and if they're sort of using their existence to justify their existence, so to speak. Chris uses mom and pop trader examples to explain the concepts here, but the fact is that Algo trading now accounts for much of the volume on major exchanges, and HFT is probably a decent portion of that. I'd love to see Chris or any other HFT trader's take on that. He sort of does near the end, but would be interesting to see a more in depth discussion: >Although the latency competition provides little to no value to the end consumers, all HFTs must play the game. If they don’t, a faster HFT will beat them to market and their order flow will be reduced. This is a very real social cost of HFT - many very smart people spend a lot of time and effort reducing the latency of trading systems. I’m no fool, but when I worked as an HFT I often felt like one. The field contains a lot of very smart people, and the world would probably be better off if the stock market had a little more latency and those smart people were building products for the world. For example, after leaving HFT I built a useful consumer product. But with current market mechanics, the global optimum where trading is marginally slower and smart people build useful products is impossible to reach. The latency arms race has us stuck in a suboptimal Nash Equilibrium which is similar in character to signalling competitions (e.g., the education bubble).
  • tptacek 14y ago
    Yes. The reason you didn't read about it is because the markets didn't really consider it a "problem" that they were fleecing people who needed to cross the spreads to make trades; it was just part of the business model.
  • harryh 14y ago
    > However, did major markets ever really have a huge problem with lack of liquidity, 20 or 30 years ago before HFT? Yes. It used to be that humans were the market makers. Humans are (generally) a lot more expensive than computers so providing liquidity came at a higher cost. Bid/ask spreads used to be much higher. HFT it just an example of computers replacing humans at a lower cost. A phenomenon that can be observed in many industries.
• ChristianMarks 14y ago
  Liquidity has gone down with the advent of HFT: http://opusminimax.wordpress.com/2012/04/20/ardent-support-for-hft-on-hacker-news/ http://opusminimax.wordpress.com/2012/04/20/ardent-support-f...
  • anonymoushn 14y ago
    You don't seem to provide any evidence of this claim.
    • yummyfajitas 14y ago
      Indeed - the author seems to confuse trade volume with liquidity.
      • ChristianMarks 14y ago
        The author gave you the benefit of the doubt by not spelling out the essential relation between liquidity and volume over time. Just what exactly do you think liquidity is, if it has nothing to do with volume over time?
        • anonymoushn 14y ago
          It is the ability to transact, rather than the fact of having transacted.
          • ChristianMarks 14y ago
            This is imprecise. OK, let's say that liquidity has gone up in the long run, thanks to automation (and to some extent, regulation). This reference http://www.tau.ac.il/~azibenr/Liquidity_BKW.pdf http://www.tau.ac.il/~azibenr/Liquidity_BKW.pdf shows that more liquidity has not helped: "...the effect of each unit of liquidity on returns has declined over the years." So one cannot flatly assert that HFT has led to improved liquidity and that this benefits everyone. And still one cannot discount the effect of decreased volume: this indicates that retail investors have left. Here's what that incontrovertible reference, the Wikipedia, states: Another elegant definition of liquidity is the probability that the next trade is executed at a price equal to the last one. The Wikipedia also "confuses" liquidity with volume: The liquidity of a product can be measured as how often it is bought and sold; this is known as volume. Source: http://en.wikipedia.org/wiki/Market_liquidity http://en.wikipedia.org/wiki/Market_liquidity
• DannoHung 14y ago
  Why not explain what's going on here and how it's not defrauding markets: http://www.zerohedge.com/news/step-right-its-hft-whack-mole-time http://www.zerohedge.com/news/step-right-its-hft-whack-mole-...
  • sailfrog 14y ago
    Zerohedge can be entertaining but the objectivity of it' conclusions is dubious at best. As already stated in the OP, HFT firms place and cancel orders quickly. These charts show this. How exactly is that defrauding anyone? Aside from stating that it does, the article does not explain how. Let's also keep in mind that pretty charts insinuating wrongdoing with a catchy title on a highly trafficked website like ZH is pretty sweet marketing for a company who sells market data products.
  • paperwork 14y ago
    Can you explain what is going on and how it is defrauding markets? There isn't a lot of explanation. Is the problem that the nbbo is changing too quickly?
  • yummyfajitas 14y ago
    I discussed this phenomenon before - it's basically just poorly written algorithms behaving oddly. http://news.ycombinator.com/item?id=1564445 http://news.ycombinator.com/item?id=1564445 Note that your article provides no explanation as to who is being defrauded ("markets" isn't a person) or how, so I don't know what you want me to explain.
    • tptacek 14y ago
      What is your take on Zero Hedge? It drives me sort of bananas, but I'm not a pro.
      • yummyfajitas 14y ago
        It's like a conspiracy theory site but without the theories.
    • 14y ago
• rscale 14y ago
  I have no objection to the provision of liquidity. That said, the flash crash seems to me to be a perfect example of a danger created when liquidity is provided largely by algorithms. We ran into a situation where the market was already volatile, and a bad trade exacerbated the issue by causing a number of HFTs to take unexpected losses and withdraw from their markets, consuming further liquidity while driving prices down, which created more losses for the remaining market-makers, who had to close their positions, consuming further liquidity, driving prices further down; all of those also negatively affecting long-term investors. This also seemed to me an example of the opportunism of HFT, where the HFT shaves the spread by a penny or two during calm markets, but withdraws (and exacerbates issues) during volatile and troubled markets, which seems to me the point in time at which liquidity provision is most valuable. I'm not suggesting that HFT should be outlawed, nor that HFT firms should be forced to register, act, and be regulated as official market-makers, with the associated duties. But I do note the benefits seem to come with costs. Long-term, I doubt it matters. It seems inevitable that the provision of liquidity will become commoditized, and that the days of concerns about flash crashes will eventually disappear into the past along with $50 retail trades, and $0.50 bid/ask spreads.
  • rdlowrey 14y ago
    I can appreciate your reasoning. I know you aren't suggesting this, but the popular blanket conclusion that HFT is bad because anomalies like "flash-crashes" sometimes occur is misguided. It's equivalent to saying we should outlaw highways because of an occasional traffic-jam or 12-car-pileup. On the whole, the benefits to the economy (and therefore, individuals) resulting from HFT far outweigh the costs. It's never a good idea to throw out the baby with the bath water and I've yet to encounter a nuanced mathematical argument (backed by concrete data) to demonstrate how the economy is harmed by more efficient markets.
  • yummyfajitas 14y ago
    Flash Crashes are not a phenomenon caused by algorithms. We have actually had two flash crashes - the first was in 1962. http://online.wsj.com/article/SB10001424052748703957604575272791511469272.html http://online.wsj.com/article/SB1000142405274870395760457527... Also, the main reason many HFTs pulled out of the market is the risk of broken trades (regulatory risk [1]). Staying in the market would have been a big moneymaker absent that risk - spreads were often huge. But broken trades were dangerous. If you buy accenture at $1.00, and sell at $30.00, you've helped fix the flash crash. You also just lost $10.00 - your $1.00 trade was broken, and you now have a short position you bought at $30.00 (Accenture recovered to $40.00, so you lost $10.00/share). [1] Regulation by the exchanges, not the SEC. Maybe there is an SEC regulation mandating they do this, but I have no knowledge on that point.
• Anm 14y ago
  After reading both articles, it seems to me much of the race for faster trading speed is fueled by the first come, first serve order matching. However, I don't see how this is actually important to either price discovery or liquidity. In other words, this artificial restriction does not actually contribute to greater good of the stock market. I presume it was designed to ensure fairness, but is it any more 'fair' than random selection (true random, from a physical source that cannot be influenced) from all available matching bids. Would anyone care to present an argument on how FIFO matching, after best price, is beneficial in the broader sense?
  • harryh 14y ago
    Let's say I want to sell 100 shares of MomCorp at $10. Bob also wants to sell 100 shares of MomCorp at $10. If, instead of employing first come first serve you use a random selection then both of us have an incentive to tell the market that we actually want to sell more than 100 shares to increase the odds that we'll actually get to sell what we want. Various bad/unpredictable/unstable things start to happen in this sort of situation. FIFO matching leads to a stable market.
  • sailfrog 14y ago
    Let's imagine you head over to the DMV over lunch to get your license renewed. Rather than have a line of people being served via FIFO they randomly pick who gets serviced next. Your lunch hour expires before you get your license renewed, meanwhile other customers barely have to wait thanks to the random selection. Rushing back to work you get pulled over ... Silly example I know but FIFO matching is consistent with this idea of fairness. Without it it's possible that market participants could keep getting jumped by the random selection such that they never get an order execution.
• kylemaxwell 14y ago
  I have little to add on the substance other than to note that I find these posts (and the discussions here) incredibly informative. They shed light on an area that turns out to be far more interesting than I would have imagined. However, the last example may be more realistic than you suppose. DEF CON last year contained a presentation[1] on security considerations in HFT networks, which I attended. It's thought-provoking and something I hope gets a little more attention. [1]: https://www.youtube.com/watch?v=kjIdzBtTBnI https://www.youtube.com/watch?v=kjIdzBtTBnI
  • tptacek 14y ago
    What did you learn from this? I found the slides and read them, and it seems like he's just saying "the people who build these systems believe that firewalls and TLS add too much latency to be deployed". (I have a lot of opinions on this subject but I am mercifully restricted from sharing most of them owing to professional obligation; we do a lot of work in this field).
    • kylemaxwell 14y ago
      Mostly, insight into the technical underpinnings. I had absolutely no concept of the existence of this world before that talk. In terms of take-home value, not much more than I got from your talk at BH last year, though largely because of the fire-hose effect there. :)
      • tptacek 14y ago
        If you're looking for good talks on the security or technical implications of trading markets, look for microstructure details. Any talk that has a diagram of "orders" going to a "trading engine" is addressing itself at a higher level than you're interested in. Just the order entry side of a real firm OMS is too complicated to get one bubble in a diagram. Again: strongly recommend _Trading & Exchanges_ by Harris. The TCP/IP Illustrated of markets. Supremely readable.
        • RickHull 14y ago
• wtvanhest 14y ago
  In the Hacker News comments on part 1 of this series, there were several comments suggesting that HFT’s somehow steal pennies from ordinary investors. I don’t agree with this claim, but I’m very interested in hearing well reasoned disagreement. I would go past this and say that anyone who did come up with a "logical" argument would be ignoring time value of money.
• tolos 14y ago
  What makes me uneasy about HFT is the speed in which things happen. For example, I believe the BATS IPO that happened last month began trading around $15, and was below $1 in about 900ms before trading was halted. (Nanex.net has news postings about these kinds of things, and I wouldn't mind hearing other people's opinion about the site) I guess my question is, is it possible for traders to make money in the short term (hours/minutes) or is that forever in the land of robots now?
  • sailfrog 14y ago
    Insofar as Nanex is concerned remember they are attempting to sell you a product, so be aware of their motivations before swallowing any unsubstantiated assumptions. With that said from what I have seen their charts are accurate but the interpretation of what they mean is sometimes a bit dramatic :) I can't speak directly to the BATS IPO debacle but I can say that when the NBBO in a symbol is locked or crossed (market speak for the bid and ask being the same or being inverted) matching engines can ignore the NBBO. This results in trades being executing at prices all over the map and can explain how a stock can go from $15 to $1 almost immediately.
• leoh 14y ago
  How is this an apology? Have I missed something? It sounds like this guy is giving tips for how to do this shit yourself.
  • tedunangst 14y ago
    Apparently you missed the entire second paragraph. Or you suck at reading.
    • leoh 14y ago
      No, you're a moron. I'm not getting into a debate with someone online, but I think that a good apology would actually apologize. Then, if so inclined, the apologizer might go to lengths to prevent the egregious act from being committed again. Instead, this writer enables others by giving insider knowledge of how to commit unethical acts. Thanks for reading between the lines, though, asshole.
• SomeCallMeTim 14y ago
  OK, answer me this: How does an HFT, in practice, add liquidity to a market? None of the examples given showed liquidity being added, because the people involved would have traded with each other directly without an HFT being there. Further, I submit that any HFT will only place a buy/sell spread in the case where the volume is high enough that they can complete their purchases within seconds or at most minutes. At best an HFT will cut a few minutes (more often seconds) off of the time of a trade. Anyone who is worried about waiting a few minutes for a trade to complete is Doing It Wrong. Therefore, the "liquidity" HFTs provide is only in the cases where it's not truly needed because the volume is high enough to provide it anyway. I've traded in a lot of stocks where there was very little liquidity, and fewer than a few dozen trades would happen per day. I'd sometimes wait hours for a trade to complete. Where are all the HFTs providing me "liquidity"? They weren't anywhere to be found, because in the cases where the volume is too low HFTs can't make a sure bet. And when the volume is high, they aren't needed. So what good are they again? At any point that an HFT is willing to buy stock, it's because there's a high likelihood that someone will show up to pay more after a few seconds or minutes. IMO the popular media actually has it right. HFTs really are just stealing pennies on every transaction, because they only enter a stock and "add liquidity" if and when the stock doesn't need it. The onus isn't on me to prove they aren't providing liquidity, but on the HFTs to prove they are, and the OA doesn't even come close. In every example OP gives, the people actually buying and/or selling stock would have been better off without the HFTs in the mix (at best their trades were a few minutes faster). If we just taxed or put a fee on a "short term hold" of a stock, say less than one day, then high frequency trading would vanish. I see HFTs as a parasite on a broken system, nothing more, and I'm shocked that so many HNers idolize them.
  • harryh 14y ago
    > How does an HFT, in practice, add liquidity to a market? Bid/ask spreads used to be 10 cents or more. Now they're generally a penny. That is evidence of a more liquid market. In practice, this is because computers are now market makers instead of humans so they can do this job at a lower cost.
    • tazzy531 14y ago
• albertsun 14y ago
  Going off the last example, what difference does it make if there is a trojan or not? Bender might very well have chosen to buy at $10.10 and sell at $10.15 without any evil foreknowledge to narrow the spread and it would have caused the some effect on Prof. Farnsworth. Yes, it does mean that whoever had a sell order at $10.10 gets their order filled faster. But is that trade off worth it?
  • yummyfajitas 14y ago
    ...what difference does it make if there is a trojan or not? Absent the Trojan, Bender has no idea if Farnsworth will show up or not. If Farnsworth doesn't show up, this might happen: BUY(bender, $10.10, 100, 12:00:00.000) SELL(bender, $10.15, 100, 12.00:00.100) ...crickets chirping... SELL(hermes, $10.05, 100, 12.00.10.000) SELL(zapp, $10.00, 100, 12.00.15.000) BUY(amy, $10.00, 100, 12.00.16.000) (Amy trades with Zapp) ...MomCorp continues it's downward slide. I.e., without the trojan Bender is just a guy who thinks MomCorp is going up.
• jellicle 14y ago
  > Of course, it’s highly illegal to use trojans to rob retail investors and game the stock market, so this story is not particularly realistic. But in fact that entire scenario he just described is perfectly legal. The industry calls them "flash orders", and due to industry pushback against an SEC initiative to ban them in 2009, they are still legal. Nutshell description: - you put in an order to buy AAPL - your trusted exchange "flashes" the order to a company that has paid for the information, so they know you want to buy AAPL. They now have exactly .5 second (SEC rule) to act on that information before your order hits the wider market. - said company goes and buys up the AAPL on the market, ahead of you - your order to buy AAPL hits, executing at a slightly higher price than you expected - said company sells AAPL at the new higher price (perhaps to you) You've acquired AAPL at a slightly inflated price, and another company has pocketed the extra money you paid. They could buy low and sell high with a GUARANTEE of success because they knew your order was coming to the market. We used to call this front-running and call it illegal, but not in the modern U.S. stock market. This is completely legal in the United States. It's happening now. It happened hundreds of times while you read this sentence.
  • grandalf 14y ago
    This is done in order to avoid intra-ETN transaction costs. There is nothing malicious or underhanded going on. This has always happened. Suppose you're in a room with your investor club and you want to buy 500 shares of AAPL. The evil "flash order" is akin to mentioning to your local group "hey guys I want to buy 500 shares of AAPL at $x, in case any of you want the other side of the transaction". If none of your local group (with whom a trade would incur no transaction cost) wants it, then it goes through to the wider market where it may or may not find a counterparty. The 0.5 second time limit is arbitrary, chosen to accomodate a range of latencies for the various local would-be-counterparties. Nobody is stealing anything. It's just a way around some of the fees that one willingly pays for harder to fill orders. If an entity on your local ETN has a market making strategy where he/she is willing to fill some orders, then a flash order rule can improve efficiency by removing transaction cost. If you think this is a bad thing then you have a profound misunderstanding of its mechanics. Ironically, like most of the anti-HFT claims, your position benefits the old school establishment exchanges. ETNs are the little guys trying to compete against the big guys by offering better technology and lower transaction costs. Flash orders are part of their service to fight against the competition-stifling fees charged by the big guys to route electronic orders.
• ynniv 14y ago
  My understanding is that HFT has been used for frontrunning (which is undeniably stealing pennies), and associated with unintended market instability (flash crashes) due to unforeseen trader interactions. This post primarily defends algo trading, which does not have the negative connotations of HFT. The author claims speed is mostly important for order precedence, but I find this to be a naive view. The potential upside for a trader coming to a conclusion and executing a trade first is minute compared to the potential for frontrunning trades that have been executed but not yet communicated to the market. Yes, this kind of frontrunning requires (sometimes[1]) automated illegal eavesdropping on secure financial transactions, and is highly immoral and illegal. But you don't make $100M a day without compromising some morals[2]. Algo I'm okay with, but HFT is a fancy toy for messing with the markets. [1 | http://zerohedge.blogspot.com/2009/07/is-goldman-legally-frontrunning-its.html http://zerohedge.blogspot.com/2009/07/is-goldman-legally-fro...] [2 | http://www.bloomberg.com/apps/news?pid=newsarchive&sid=a7HGVAn8w73Y http://www.bloomberg.com/apps/news?pid=newsarchive&sid=a... ]
  • scott_s 14y ago
    HFT is, essentially, automated market making, and only market makers have the opportunity to do front running.
  • azmenthe 14y ago
    Your association of a dollar amount to morality is unnerving. If Happy Unicorn Widget Inc. upgrades it's production line speed, it experiences some increase in revenue and can thus even make a $100M a day gasp. I work at a HFT firm and any serious suggestion of frontrunning would probably get you fired. The truth is the majority of HFT is liquidity providing or some form of arbitrage.
    • ynniv 14y ago
      Your association of a dollar amount to morality is unnerving That's a silly assertion. What if it were $100B a day? That's $35T/year, or half the GDP of the world. Do you think that a company could receive half of the productivity of the entire world morally? I work at a HFT firm and any serious suggestion of frontrunning would probably get you fired. Another silly statement. Of course people don't talk about it openly - who plans illicit activities openly? I would expect that anyone involved in such a system would have no knowledge of it being used immorally/illegally - there is too much risk in that. HFT is an assault rifle. People who make them talk about how they keep the peace, but that isn't the use that caused them to be outlawed.
• benthumb 14y ago
  My 2 cents: 1. Productive is Destructive ... literally (that's why we're in the position now of having figure out a way to mine asteroids -- industrial productivity under the current model is environmentally unsustainable) 2. Markets are (viewed in aggregate) evolving to be something akin to a global autonomic nervous system ... money like transduction makes the world go 'round.
• zopf 14y ago
  Some concrete examples of ways that HFTs inhibit, disrupt, or defraud market participants: http://www.chrisstucchio.com/blog/2012/hft_apology2.html#comment-510202854 http://www.chrisstucchio.com/blog/2012/hft_apology2.html#com...
• briholt 14y ago
  Fantastic series, please keep them coming.
• Schwolop 14y ago
  To illustrate with arbitrary numbers, if HFT’s currently focus on 50% strategy (i.e., price competition and liquidity improvements) and 50% latency, a more optimal scenario would be 50% fewer HFT’s focusing 100% of their efforts on strategy. Nitpicking - "more optimal" is non-sensical. Optimality is binary. You're either optimal or you're not. Of the sub-optimal cases there are better and worse options, but neither is "more optimal" than any other.
• hartror 14y ago
  I don't have any more or less dislike for HFT over more traditional trading. My biggest problem with finance is the brain drain it creates, in HFT's case a lot of great maths and computer science people get sucked into the finance black hole which saddens me greatly. Certainly having liquidity in the market is important but what level of liquidity is enough and beyond that no more value is provided to society and so the brain drain in fact holds back society?
• jfager 14y ago
  This is a great series so far, but I think some people are taking it too seriously. This is roughly the equivalent of giving you a good tutorial about how a short map-reduce implementation works and then asking you to agree that Hadoop is awesome. It may or may not be, but the toy example isn't sufficient to make the determination. It's just background information for the uninitiated. Some relevant questions that are completely glossed over: 1. What compels an HFT to actually trade? Is there anything forcing them to keep supplying liquidity even if the market's moving against them? How is an HFT different than an actual market maker? 2. How does an HFT decide that it has a better-than-even shot at turning a profit on a trade? Most of the objections to HFTs revolve around the answers to this question (i.e. pseudo-front-running by trying to detect large buys/sells that get split over lots of orders) and their implications (i.e. 'real' investors leaving the exchanges). 3. The "market-maker strategy" HFTs you describe are indisputably compensated for providing liquidity and taking on risk, but is the return on HFTs actually equivalent to the return on other investments with equivalent risk? If not, and they earn a premium, why isn't that evidence that something's broken?
  • yummyfajitas 14y ago
    It's just background information for the uninitiated. That's all it's intended to be. Is there anything forcing them to keep supplying liquidity even if the market's moving against them? No. If not, and they earn a premium, why isn't that evidence that something's broken? Wait for part 3. I do believe HFTs are capturing fractions of a penny in rent and I have an idea of how to fix this.