10 ms·

The Idiot's Guide to High Frequency Trading

149 points by jerryhuang100 13y ago

tptacek 13y ago
If HFT GUARANTEES profits, why are the profits to HFTs declining so sharply, and why do HFT firms make such a small fraction of what the buy-side firms make? Much more discussion here: https://news.ycombinator.com/item?id=7531429 https://news.ycombinator.com/item?id=7531429
- ajtulloch 13y ago
  I think the somewhat sparse academic research on this topic is quite interesting - "The High-Frequency Trading Arms Race: Frequent Batch: Auctions as a Market Design Response" by Budish et. al at Chicago (http://faculty.chicagobooth.edu/eric.budish/research/HFT-FrequentBatchAuctions.pdf http://faculty.chicagobooth.edu/eric.budish/research/HFT-Fre...) is one such paper. The key results in this paper are that: - market correlations 'predictably' break down at high-frequency horizons - these correlation breakdowns create a winner-takes-all speed arms race to exploit these opportunities - a theoretical model can be built to explain the investment in speed as a function of these opportunities - this model reduces to the prisoner's dilemma from game theory - the equilibrium state of this model is an environment higher spreads and thinner markets than is socially optimal (without the arms race). It's a very interesting and readable paper, peppered with interesting market observations and theoretical insights.
  - 001sky 13y ago
    Along these lines, a note on so-called latency arbitrage: http://web.eecs.umich.edu/srg/wp-content/uploads/2013/02/ec38-wah.pdf http://web.eecs.umich.edu/srg/wp-content/uploads/2013/02/ec3... But in a study last year, University of Michigan professor Michael Wellman and I developed a model across two exchanges. We examined a strategy called latency arbitrage, which utilizes advantages in access and response time to exploit price disparities caused by fragmentation across those dozens of competing venues. HFTs employing latency arbitrage examine current market information to predict immediate price movements, essentially computing the best prices available before the exchange has even had a chance to update its price quote. What we found may be even scarier than Lewis's book-selling punchline: it's not simply a matter of the HFT crowd taking profits away from regular investors. Predatory strategies like latency arbitrage have the potential to reduce trading gains for all market participants – high-frequency traders and Average Joes alike. Their solution was to go to periodic call structure.
    - aet 13y ago
      Lets say we go to a call structure. Will there be competitive pressure to go back to a continuous market? For example, lets say you demand all U.S. exchanges go to a call structure. Would there be reason for me to go to London and open up a new continuous market?
      - 001sky 13y ago
        You're illustrating regulatory arbitrage. The answer to that question is always: it depends. Not helpful, I know. But IMHO the issue is not so much here understanding all of the -N- ways to regulate the problem. It's helpful in the first instance to have clear articulations of the problem you need to consider. Market micro-structure is like taxes. It's almost impossible to generalize an answer. An "actual answer" will entail a level of detail and laywers fees that is prohibitive to discuss "for fun" =D. We can apply this critique to all of these academic papers too: they are abstractions and model-markets. The actual particulars of any given order routing...etc. So what is the purpose of this even being posted? It is simply to illustrate how sub-optimal outcomes are possible on a whiteboard. The general takeaway is there is a problem (worsening equilibrium) and a solution (improving one) that relate to tweeking the variables (timing/trade sequencing) discussed upthread. Those are not purely neutral or unrelated to welfare outcomes. It helps to have someone else spell it out sometimes. Both to save the brain damage of spelling it out in HN comments, and also to provide some fresh voices to the dialogue.
    - kasey_junk 13y ago
      This is actually a very good paper and I heartily recommend it if you are into wonky market arbitrage. You don't mention it, but they actually recommend a centralized discrete call market. How is that to be implemented? They don't say and they don't count the additional added cost/risk of adding a centralized monopoly player into the market (in fact they completely ignore transaction fees). We moved away from centralized markets in the first place because those costs were non-trivial.
- mayukh 13y ago
  Declining profitabilities caused by increased competition. What market risk are the scalp-style of hft strategies (not all hft's are scalping) taking ? If they can cancel orders at abandon and make pennies if they win the race (against other hft's), but can simply x out of their order if the price doesnt go their way is as close to a riskless profit as it gets. virtu's prospectus as a case in point. https://www.sec.gov/Archives/edgar/data/1592386/000104746914002070/a2218589zs-1.htm https://www.sec.gov/Archives/edgar/data/1592386/000104746914...
  - lrm242 13y ago
    Manoj Narang of Tradeworx has stated their average holding time is up to 10 minutes. Is he taking risk? http://washpost.bloomberg.com/Story?docId=1376-N2CB0F6TTDTQ01-3SAG95I8F8SD907LL6IA4CLK0P http://washpost.bloomberg.com/Story?docId=1376-N2CB0F6TTDTQ0...
    - mayukh 13y ago
      'Average' holding times don't help understand the issue. You could have one position that was a long-term bet edit: I did not imply that none of the hft strategies were taking market risk. The ones that scalp certainly seem to.
      - tptacek 13y ago
        This is the second time you've used the word "scalp", as if all liquidity on the public markets for the last century weren't funded by "scalping". In the absence of "scalping", trading in stocks works like trading in houses. There are lots of buyers. There are lots of sellers. In the majority of cases, they disagree materially on the correct price. Therefore, it (a) takes forever to enter or exit a position, and (b) often forces people to accept terribly unfavorable pricing. The "scalp" market makers take is the market price for always having a counterparty willing to trade with you at a price near the true market value of the trading instrument. If you want the markets to work more like the real estate market, you can do that: place limit orders. The fact that market orders carry a premium price isn't a subtle detail of the market; it's trading 101. All things being equal, you want the "scalp" to be as thin as possible. The wider the spread, the closer the scalping blade comes to the skull. Liquidity has a price, and investors want that price to be as low as possible. So now, an exercise for you: at the height of HFT profit-taking, was the price of liquidity (a) lower or (b) higher than it was during the 1990s?
- wglb 13y ago
  So let's try an experiment to see what happens: https://news.ycombinator.com/item?id=7532584 https://news.ycombinator.com/item?id=7532584
- foobarqux 13y ago
  You can "guarantee" profits but still be revenue constrained.
deleted 13y ago
[deleted]
- ForHackernews 13y ago
  http://www.amazon.com/Trillionaire-Next-Door-Investors-Trading/dp/0066620767 http://www.amazon.com/Trillionaire-Next-Door-Investors-Tradi...
nbouscal 13y ago
If you want to know what the actual effects of HFT are on individual investors rather than hypotheticals, a study was done in Canada last year on that exact question. Spoiler alert: HFT is good for the little guys. http://qed.econ.queensu.ca/pub/faculty/milne/322/IIROC_FeeChange_submission_KM_AP3.pdf http://qed.econ.queensu.ca/pub/faculty/milne/322/IIROC_FeeCh...
- dangerlibrary 13y ago
  1. That paper hasn't been published or peer reviewed. 2. One of the authors used to work for HSBC as a derivatives trader. 3. That study analyzes the impact of a fee-per-order regulatory regime. It is not a study, as you claim, of a world with high frequency traders vs a counter factual world without them.
a8da6b0c91d 13y ago
Some aspects of HFT seem highly analogous to insider trading to me, and insider trading is currently illegal. Abusing the ability to cancel orders fast in order to ping for non-public data about other orders sure seems like an insider advantage. As I understand it this "pinging" is central to HFT strategies. Am I missing something?
- lrm242 13y ago
  There is nothing about HFT that is analgous to insider trading. Unfortunately the people describing it in public like Cuban simply don't understand what's going on. There is no non-public information at work here.
  - guelo 13y ago
    The information that the HFT traders pay millions of dollars to acquire before the rest of the market does is not public at the time that they acquire it by any definition of public that does not include having to pay millions of dollars to acquire the information.
    - lrm242 13y ago
      That's fundamentally wrong. Direct feeds are available to anyone who wishes to purchase them. Do you have a source for a piece of data purchasable by a firm that isn't on the public side of the information pipe?
      - erichocean 13y ago
        > Direct feeds are available to anyone who wishes to purchase them. Out of curiosity, how much does a direct feed cost, and is there a practical limit to the number of simultaneous buyers (for example, physical server space in a data center)?
        lrm242 13y ago
        Price lists are all public. Look at nasdaqtrader.com or batstrading.com.
theorique 13y ago
Complete idiot's guide is right. Exchanges offer price-time priority. A 'better' price (higher bid, lower offer) gives you priority over a 'worse' price. Given two orders at the same price, an earlier order gives you priority over a later order. If you spend millions of dollars on computers, data feeds, and salaries to skilled personnel, to predict the motion of markets correctly and work within the system to make money, then you've earned that money fair and square. Companies that don't operate within the rules get shut down fast and hard by the SEC and/or FINRA. This is a highly regulated and policed industry. HFT is hardly different from Warren Buffett identifying a company with a 10-100 year time horizon. Should we penalize Buffett because he's smarter and better at that game than everyone else in the world? Is he scamming the person from whom he bought shares in 1984? As Knight Capital showed us two years ago, you can spend those millions, and still screw up and lose $10M per minute for the better part of an hour until your firm is bankrupt. No one is guaranteed to make money. Posting an order and canceling it is fair game. When the order is in the market, it's a live intent to trade. When it's canceled, it's no longer a live intent to trade. Why is this "moral" when it's a person doing this on a time scale of seconds or minutes, but "fraud on the market" when it's a computer doing it on a time scale of micro or milliseconds? If someone hits that HFT computer's bid, there's an obligation to fill that order - nothing is different about such an order just because it's placed and canceled quickly. The only objection is fear of the unknown. And the only difference here is that it's being done based on an automated strategy, faster than people can react. Your broker has access to HFT tools. Your 401(k) manager has access to HFT tools. If you're a day trader, your order is probably being routed through HFT tools.
- kylebrown 13y ago
  > Why is this "moral" when it's a person doing this on a time scale of seconds or minutes, but "fraud on the market" when it's a computer doing it on a time scale of micro or milliseconds? Its not always fraud, but some the of tactics get pretty scammy. One highlighted in Flash Boys is a classic bait-and-switch, perpetuated with perverse fee incentives of a "taker rebate" on BATS BYX. I said more here: https://news.ycombinator.com/item?id=7532902 https://news.ycombinator.com/item?id=7532902
  - 13y ago
notastartup 13y ago
If HFT adds no value, rigs the game, adds cost to everyday investors and traders looking to make an honest fair buck from the markets, will it be allowed in the future? I almost feel like SEC will come down on them hard. Basically, I'm bummed at the idea that as individuals, we can't innovate in this field without ponying up several million dollars in startup cost.
- lrm242 13y ago
  It doesn't cost several million dollars. It does require some capital, likely less than your average valley angel deal. It is not easy and you'll probably fail, but you can certainly try. Phone up Nasdaq, they'll happily give you all you need to know along with pricing information on their website.
- jerryhuang100 13y ago
  SEC might even be part of the ecosystem. According to the book: 1) When the Canadian guy in the book, Brad Katsuyama, went to the SEC for their concern the SEC staffers just shrugged; 2) Since 2007 more than 200 SEC staffers left to work for HFT firms. (Source: Michael Lewis, Flash Boys)
stygiansonic 13y ago
My (non-expert) take: HFT is a tool and it can be used for good or bad. Potential good uses may include: reduction of bid/ask spreads, improved liquidity and faster arbitrage of things like ETFs. However, it can also be used for bad: http://www.sec.gov/News/PressRelease/Detail/PressRelease/1365171484972#.Uz75MfldXm4 http://www.sec.gov/News/PressRelease/Detail/PressRelease/136... In that link, the SEC charges that a certain firm employed a trading strategy using non-bona fide orders to attract interest on another (bona fide) order. This involved submitting orders on the other side[1] to simulate interest. HFT makes such strategies viable, or at least more viable. 1. http://www.nanex.net/aqck2/3598.html http://www.nanex.net/aqck2/3598.html EDIT: Corrected to remove inaccurate information.
- lrm242 13y ago
  The action you link to was not perpetuated by HFT. The vast majority of Lightspeed's business is good ole fashioned screen traders, and that is likely what this was. Previous actions for spoofing and layering were also the result of manual traders using regular day trading tools.
  - stygiansonic 13y ago
    My mistake. I assumed HFT was involved because the timescale of the "layering" orders and subsequent cancellation (following the bona fide order execution) was on the order of several hundred milliseconds.
herbig 13y ago
John Stewart had a really interesting interview with Michael Lewis this week about HFT: http://thedailyshow.cc.com/guests/michael-lewis http://thedailyshow.cc.com/guests/michael-lewis
jseliger 13y ago
Note that this: https://news.ycombinator.com/item?id=7524315 https://news.ycombinator.com/item?id=7524315 is a better place to start than the linked post.
cgmoore120 13y ago
read a great article on this the other night http://www.nytimes.com/2014/04/06/magazine/flash-boys-michael-lewis.html?_r=1 http://www.nytimes.com/2014/04/06/magazine/flash-boys-michae...
JumpCrisscross 13y ago
The NYSE has always required market makers continuously buy and sell the stocks they specialise in. They must do this in all market conditions, even if it means running a loss. In exchange, they get privileged access to order flow information. This is why those seats are valuable. NASDAQ challenged that model by removing the physical trading floor. Instead of humans standing in a pit there were humans sitting behind screens. But in both cases there is an institution guarding access to the market's nerve centres. Barriers to entry were raised and the incumbents protected. HFT takes the abstraction one step further. Democratised is that access to privileged order flow - anyone who can pay can get it. Lost is the role of a designated market maker - HFTs can pull out when markets get rough. This is a valid debate. But don't confuse yourself. The order flow privileges being criticised have always accrued to institutional market makers. It is no surprise that those incumbents are the ones driving the present lobby.
- AndrewBissell 13y ago
  > The NYSE requires designated market makers to buy and sell the securities they specialise in. They must do this even in violent market conditions This "mandate" tended to break down when tested by actual market panics. Bids dried up in 1987, and histories of the 1929 crash often mention clerks and floor runners who entered stink bids at $0.01 for stocks that had recently traded at $30 or $40, and got filled because the specialists had abandoned their posts. Whether human or computer, market makers know better than to stand in front of a freight train.
  - JumpCrisscross 13y ago
    Market makers can declare "self help" in the event of technical difficulty. Exchange rules prohibit declaring self help to avoid adverse market conditions. In practice, self help declarations have a peculiar habit of cropping up in rough markets. Nevertheless, "guarantors of market liquidity" remains a rallying cry for traditional market makers.
arjn 13y ago
Correct me if I'm wrong but couldn't this problem be solved by some simple rules or act of legislation ? For example, make it a rule that when you buy a stock, you have to hold it for at least some time - say a minimum of 5 minutes. Or you could just impose a fee/penalty on traders that buy sell the same stock within one day of trading. (note - I'm merely speculating here and have no specialized knowledge)
deleted 13y ago
[deleted]
tlrobinson 13y ago
Is "registering algorithms" really a good solution? It seems incredibly hard to analyze or simulate such a complex system.
njharman 13y ago
"With these changes the fastest players were now able to make money simply because they were the fastest traders." Directly contradicts "speed is not a problem". Speed directly translates into being able to "front of trades from slower market participants". If they did not have the speed, they could not algorithmically predict the actions of (or react to activity on one exchange before it reaches other exchanges) and jump in front of slower participants. There's other issues on top of this, such as paying for early or privileged access. But, if you are faster, you can successfully arbitrage across time or distance(really same thing as time)
transfire 13y ago
The solution is very simple: a Tobin Tax.
Havoc 13y ago
Step 1) Have deep pockets, big balls and the right skills
deleted 13y ago
[deleted]
thikonom 13y ago
The title of the article sounds oxymoronic.
arasmussen 13y ago
I really wish the "Idiot's guide to high frequency trading" would start out by explaining what high frequency trading is to me like I'm an idiot...
mianos 13y ago
Just jump to the second comment for the actual details. Welcome to the internet, where if you are popular you can write any shit you like and people will read it, even if you don't actually know what you are talking about. (I wrote the software that runs some of the major stock exchanges. I did that for 13 years).
asbestoshft 13y ago
I have a lot of experience with HFT. Lets clear all of this up. How exchanges work for dummies. All of the modern US stock exchanges work as a limit order book. Orders are filled on a price-time basis. In other words, you put in an order, if it isn't filled immediately it gets put in the limit order book with your order going to the end of the line ( lowest priority ) at that price. That's it, that is how it works. Lets look at an example. I want to buy 250 shares of ABC. Right now there are orders in the book. 5 orders to buy at 9.95, each one for 100 shares. There are also 5 orders to sell at 9.96 also each order for 100 shares. I have several options. (1) I can put in a market order to buy 250, when this order gets to the order book sell orders will be filled starting with the oldest ones, the ones that have been in the book longest, first. My order will be matched, this is why they call it a matching engine, with two of the sell orders and then a partial on the third one. As orders are filled in the order book messages are sent out to all subscribers of the data that those orders were filled so everyone can keep track of the state of the book. It is important to note that when my order gets to the matching engine/order book there is no chance for the orders in the order book to get cancelled. Those orders aren't told "you are about to get filled do you want to cancel". (2) I can put in a limit order to buy at 9.96 in which case the order will be filled as described in 1 unless the sell orders at 9.96 are cancelled before my order gets to the matching engine. (3) I can put in a limit order to buy at 9.95 in which case my order goes into the order book at 9.95 and I have to wait for the orders ahead of me to either get filled or cancelled and then I have to wait for someone to send in a sell order, either market or limit, that will get matched with my buy order. That's it! Repeat, all day, every day. OK, that is a lie, there are special order types and that is one way exchanges try to differentiate themselves but that is a simplified way of how this works. So where is the problem that Brad Katsuyama saw? I don't know him all I know is what I've seen and read. One thing to note is that he talks about buying large blocks of stock. Very large. From http://www.nytimes.com/2014/04/06/magazine/flash-boys-michael-lewis.html?_r=0 http://www.nytimes.com/2014/04/06/magazine/flash-boys-michae... "It used to be that when his trading screens showed 10,000 shares of Intel offered at $22 a share, it meant that he could buy 10,000 shares of Intel for $22 a share." 10k shares at $22k/share, that is $220k in value. So he isn't talking about some small retail order, this is, as he points out, institutional flow. So lets think about what happens in this kind of case. There are 13 exchanges, all running limit order books. They are geographically disperse and internally they are implemented differently and have different latencies and different message protocols etc. At some instance in time all 13 exchanges have some orders to sell at $22.00. Think about that. In some sense that is awesome. Look how efficient the market is. If you went to 13 gas stations what are the chances that a gallon of regular gas will cost exactly the same at every station? Anyway, back to our problem. You want to buy 10,000 shares. You fire off your order to exchange A to buy 10,000 shares. This hits the matching engine and you get filled on 10,000 shares ( THERE IS NO WAY FOR THOSE SELL ORDERS THAT ARE IN THE BOOK TO BE CANCELLED!!!! The orders in the book aren't sent a message "hey, a 10,000 share order is coming in and about to hit your order do you want to cancel your order first?". THAT DOES NOT HAPPEN!!! ). The thing is that those 10,000 shares that got matched, lets say each order in the book was 200 shares. Your 10,000 share order just took out 50 orders. The fact that these orders were executed is sent out to every data subscriber who is listening for that data. All messages are timestamped, sometimes down to the nanosecond. From that the recipients can tell "wow, all of these 50 orders for 10,000 shares were executed at the same time so that was one order." Now, if you had orders in at other exchanges to sell at $22 what would your response be? For a second lets think about something more mundane, like TVs. Suppose you work at Target and you have a friend who works at Walmart. You're working one night and a guy shows up driving a semi-trailer. He walks in and says "How many 50 inch flat screens do you have? I'll buy them all, as advertised at $400." You sell all 20 that you have to him. As he's walking out the door he says "Now I'm going to Walmart." Just then your buddy at Walmart texts you and says "what's up? slow night right." and you go "Hell no, just sold 20 50 inch TVs to a guy, he bought everything I had and says he is headed for your Walmart to buy more." Now what does your friend do? In a world where prices are set by supply and demand Walmart could look at that and go "wow, there is a big buyer of 50 inch TVs out there, lets increase our price from $400 to $410." And so that is what Walmart does and sure enough the guy comes in and buys all the TVs from Walmart for $410. Does the TV buyer say the market is "rigged"? In any case that is very similar to what happens in this case. All of the HFT and market makers see this big order rip through a lot of sell orders at exchange A and they go "wow, there is a big buyer, I better change my sell order from selling at $22.00 to selling at $22.01 or $22.10 or $25 or whatever". Brad Katsuyama looks at that and goes "I've been front run, this market is rigged." And then he has a piece of software written so that all orders arrive at the different markets at the same time, as opposed to leaving his trading workstation at the same time, and suddenly there isn't any front running. There isn't any front running at all, what he is seeing is the reaction to supply and demand. HFT and market makers live in fear of something called "adverse selection". Market makers are required to post a two sided market, they are supposed to put up a price and quantity that they are willing to both buy and sell at right now. They publish this and they can't back away from it, they have to live up to it if another order shows up to buy or sell at a price that matches they have to execute the trade. The risk to them is that they are trading with someone who has information that they don't have. Just like playing poker or any game, you're going to lose if you play with people who are better than you. In this game though sometimes just having a big order is enough to move the market even if the person behind the order doesn't actually have any new information. Market makers and HFT either want to buy on the bid and sell on the offer and make the spread or they want to do arbitrage. That is how they make money. They lose money if they trade with people who are smarter or faster than they are. As soon as they see big orders come in they get really skittish and so they immediately move their orders to reduce their risk. I've written a lot and bored almost everyone. Ask away and I'll do my best to answer your questions. A couple other comments: 1. I'm not saying HFT is good or bad, I don't know. 2. Lots of people mention costs. Just to ballpark the whole thing. For the technology you need to Arista switches, that is $500/port, get a 48 port switch, that is $24k. For servers you need something decent, not crazy, dual CPU, 64 GB, running linux, maybe solarflare for kernel bypass networking, custom software. The hardware will cost you $6k/server. Then you need a connection to an exchange, varies but lets say $10k/month, then you need market data, $5k/month, then you need an order entry port, that is maybe $1k/month. That's it. Well, that is at one exchange, then you need low latency 10 Gbps cross connects between exchanges and you need to do that at each exchange. Then you need to find a clearing firm and you need to become a broker dealer ( exchanges and clearing firms only talk to BDs ). This is a highly regulated environment. You must have a compliance person, you will get audited yearly by SEC, your exchange, FINRA, all kinds of stuff. You will need a serious accountant. You have to have written procedures. 3. And how in the world do you avoid the Knight type of meltdown? Well, when you send in an order the order goes to the exchange. But the exchange also sends the order out via a "drop copy". This might go to your clearing firm so they know you aren't losing tons of money. But you can also get the drop copies and then you can constantly do your own risk checks that your orders you think are open are actually open but the ones you cancelled are actually cancelled. Most exchanges also have "cancel on disconnect" so when you really don't know what is going on you shut off your connection to the exchange and any open orders in the order book are cancelled for you.

149 points by jerryhuang100 13y ago

If HFT GUARANTEES profits, why are the profits to HFTs declining so sharply, and why do HFT firms make such a small fraction of what the buy-side firms make? Much more discussion here: https://news.ycombinator.com/item?id=7531429 https://news.ycombinator.com/item?id=7531429

ajtulloch 13y ago
I think the somewhat sparse academic research on this topic is quite interesting - "The High-Frequency Trading Arms Race: Frequent Batch: Auctions as a Market Design Response" by Budish et. al at Chicago (http://faculty.chicagobooth.edu/eric.budish/research/HFT-FrequentBatchAuctions.pdf http://faculty.chicagobooth.edu/eric.budish/research/HFT-Fre...) is one such paper. The key results in this paper are that: - market correlations 'predictably' break down at high-frequency horizons - these correlation breakdowns create a winner-takes-all speed arms race to exploit these opportunities - a theoretical model can be built to explain the investment in speed as a function of these opportunities - this model reduces to the prisoner's dilemma from game theory - the equilibrium state of this model is an environment higher spreads and thinner markets than is socially optimal (without the arms race). It's a very interesting and readable paper, peppered with interesting market observations and theoretical insights.
- 001sky 13y ago
  Along these lines, a note on so-called latency arbitrage: http://web.eecs.umich.edu/srg/wp-content/uploads/2013/02/ec38-wah.pdf http://web.eecs.umich.edu/srg/wp-content/uploads/2013/02/ec3... But in a study last year, University of Michigan professor Michael Wellman and I developed a model across two exchanges. We examined a strategy called latency arbitrage, which utilizes advantages in access and response time to exploit price disparities caused by fragmentation across those dozens of competing venues. HFTs employing latency arbitrage examine current market information to predict immediate price movements, essentially computing the best prices available before the exchange has even had a chance to update its price quote. What we found may be even scarier than Lewis's book-selling punchline: it's not simply a matter of the HFT crowd taking profits away from regular investors. Predatory strategies like latency arbitrage have the potential to reduce trading gains for all market participants – high-frequency traders and Average Joes alike. Their solution was to go to periodic call structure.
  - aet 13y ago
    Lets say we go to a call structure. Will there be competitive pressure to go back to a continuous market? For example, lets say you demand all U.S. exchanges go to a call structure. Would there be reason for me to go to London and open up a new continuous market?
    - 001sky 13y ago
      You're illustrating regulatory arbitrage. The answer to that question is always: it depends. Not helpful, I know. But IMHO the issue is not so much here understanding all of the -N- ways to regulate the problem. It's helpful in the first instance to have clear articulations of the problem you need to consider. Market micro-structure is like taxes. It's almost impossible to generalize an answer. An "actual answer" will entail a level of detail and laywers fees that is prohibitive to discuss "for fun" =D. We can apply this critique to all of these academic papers too: they are abstractions and model-markets. The actual particulars of any given order routing...etc. So what is the purpose of this even being posted? It is simply to illustrate how sub-optimal outcomes are possible on a whiteboard. The general takeaway is there is a problem (worsening equilibrium) and a solution (improving one) that relate to tweeking the variables (timing/trade sequencing) discussed upthread. Those are not purely neutral or unrelated to welfare outcomes. It helps to have someone else spell it out sometimes. Both to save the brain damage of spelling it out in HN comments, and also to provide some fresh voices to the dialogue.
  - kasey_junk 13y ago
    This is actually a very good paper and I heartily recommend it if you are into wonky market arbitrage. You don't mention it, but they actually recommend a centralized discrete call market. How is that to be implemented? They don't say and they don't count the additional added cost/risk of adding a centralized monopoly player into the market (in fact they completely ignore transaction fees). We moved away from centralized markets in the first place because those costs were non-trivial.
mayukh 13y ago
Declining profitabilities caused by increased competition. What market risk are the scalp-style of hft strategies (not all hft's are scalping) taking ? If they can cancel orders at abandon and make pennies if they win the race (against other hft's), but can simply x out of their order if the price doesnt go their way is as close to a riskless profit as it gets. virtu's prospectus as a case in point. https://www.sec.gov/Archives/edgar/data/1592386/000104746914002070/a2218589zs-1.htm https://www.sec.gov/Archives/edgar/data/1592386/000104746914...
- lrm242 13y ago
  Manoj Narang of Tradeworx has stated their average holding time is up to 10 minutes. Is he taking risk? http://washpost.bloomberg.com/Story?docId=1376-N2CB0F6TTDTQ01-3SAG95I8F8SD907LL6IA4CLK0P http://washpost.bloomberg.com/Story?docId=1376-N2CB0F6TTDTQ0...
  - mayukh 13y ago
    'Average' holding times don't help understand the issue. You could have one position that was a long-term bet edit: I did not imply that none of the hft strategies were taking market risk. The ones that scalp certainly seem to.
    - tptacek 13y ago
      This is the second time you've used the word "scalp", as if all liquidity on the public markets for the last century weren't funded by "scalping". In the absence of "scalping", trading in stocks works like trading in houses. There are lots of buyers. There are lots of sellers. In the majority of cases, they disagree materially on the correct price. Therefore, it (a) takes forever to enter or exit a position, and (b) often forces people to accept terribly unfavorable pricing. The "scalp" market makers take is the market price for always having a counterparty willing to trade with you at a price near the true market value of the trading instrument. If you want the markets to work more like the real estate market, you can do that: place limit orders. The fact that market orders carry a premium price isn't a subtle detail of the market; it's trading 101. All things being equal, you want the "scalp" to be as thin as possible. The wider the spread, the closer the scalping blade comes to the skull. Liquidity has a price, and investors want that price to be as low as possible. So now, an exercise for you: at the height of HFT profit-taking, was the price of liquidity (a) lower or (b) higher than it was during the 1990s?
wglb 13y ago
So let's try an experiment to see what happens: https://news.ycombinator.com/item?id=7532584 https://news.ycombinator.com/item?id=7532584
foobarqux 13y ago
You can "guarantee" profits but still be revenue constrained.

deleted 13y ago

[deleted]

ForHackernews 13y ago
http://www.amazon.com/Trillionaire-Next-Door-Investors-Trading/dp/0066620767 http://www.amazon.com/Trillionaire-Next-Door-Investors-Tradi...

nbouscal 13y ago

If you want to know what the actual effects of HFT are on individual investors rather than hypotheticals, a study was done in Canada last year on that exact question. Spoiler alert: HFT is good for the little guys. http://qed.econ.queensu.ca/pub/faculty/milne/322/IIROC_FeeChange_submission_KM_AP3.pdf http://qed.econ.queensu.ca/pub/faculty/milne/322/IIROC_FeeCh...

dangerlibrary 13y ago
1. That paper hasn't been published or peer reviewed. 2. One of the authors used to work for HSBC as a derivatives trader. 3. That study analyzes the impact of a fee-per-order regulatory regime. It is not a study, as you claim, of a world with high frequency traders vs a counter factual world without them.

a8da6b0c91d 13y ago

Some aspects of HFT seem highly analogous to insider trading to me, and insider trading is currently illegal. Abusing the ability to cancel orders fast in order to ping for non-public data about other orders sure seems like an insider advantage. As I understand it this "pinging" is central to HFT strategies. Am I missing something?

lrm242 13y ago
There is nothing about HFT that is analgous to insider trading. Unfortunately the people describing it in public like Cuban simply don't understand what's going on. There is no non-public information at work here.
- guelo 13y ago
  The information that the HFT traders pay millions of dollars to acquire before the rest of the market does is not public at the time that they acquire it by any definition of public that does not include having to pay millions of dollars to acquire the information.
  - lrm242 13y ago
    That's fundamentally wrong. Direct feeds are available to anyone who wishes to purchase them. Do you have a source for a piece of data purchasable by a firm that isn't on the public side of the information pipe?
    - erichocean 13y ago
      > Direct feeds are available to anyone who wishes to purchase them. Out of curiosity, how much does a direct feed cost, and is there a practical limit to the number of simultaneous buyers (for example, physical server space in a data center)?
      - lrm242 13y ago
        Price lists are all public. Look at nasdaqtrader.com or batstrading.com.

theorique 13y ago

Complete idiot's guide is right. Exchanges offer price-time priority. A 'better' price (higher bid, lower offer) gives you priority over a 'worse' price. Given two orders at the same price, an earlier order gives you priority over a later order. If you spend millions of dollars on computers, data feeds, and salaries to skilled personnel, to predict the motion of markets correctly and work within the system to make money, then you've earned that money fair and square. Companies that don't operate within the rules get shut down fast and hard by the SEC and/or FINRA. This is a highly regulated and policed industry. HFT is hardly different from Warren Buffett identifying a company with a 10-100 year time horizon. Should we penalize Buffett because he's smarter and better at that game than everyone else in the world? Is he scamming the person from whom he bought shares in 1984? As Knight Capital showed us two years ago, you can spend those millions, and still screw up and lose $10M per minute for the better part of an hour until your firm is bankrupt. No one is guaranteed to make money. Posting an order and canceling it is fair game. When the order is in the market, it's a live intent to trade. When it's canceled, it's no longer a live intent to trade. Why is this "moral" when it's a person doing this on a time scale of seconds or minutes, but "fraud on the market" when it's a computer doing it on a time scale of micro or milliseconds? If someone hits that HFT computer's bid, there's an obligation to fill that order - nothing is different about such an order just because it's placed and canceled quickly. The only objection is fear of the unknown. And the only difference here is that it's being done based on an automated strategy, faster than people can react. Your broker has access to HFT tools. Your 401(k) manager has access to HFT tools. If you're a day trader, your order is probably being routed through HFT tools.

kylebrown 13y ago
> Why is this "moral" when it's a person doing this on a time scale of seconds or minutes, but "fraud on the market" when it's a computer doing it on a time scale of micro or milliseconds? Its not always fraud, but some the of tactics get pretty scammy. One highlighted in Flash Boys is a classic bait-and-switch, perpetuated with perverse fee incentives of a "taker rebate" on BATS BYX. I said more here: https://news.ycombinator.com/item?id=7532902 https://news.ycombinator.com/item?id=7532902
- 13y ago

notastartup 13y ago

If HFT adds no value, rigs the game, adds cost to everyday investors and traders looking to make an honest fair buck from the markets, will it be allowed in the future? I almost feel like SEC will come down on them hard. Basically, I'm bummed at the idea that as individuals, we can't innovate in this field without ponying up several million dollars in startup cost.

lrm242 13y ago
It doesn't cost several million dollars. It does require some capital, likely less than your average valley angel deal. It is not easy and you'll probably fail, but you can certainly try. Phone up Nasdaq, they'll happily give you all you need to know along with pricing information on their website.
jerryhuang100 13y ago
SEC might even be part of the ecosystem. According to the book: 1) When the Canadian guy in the book, Brad Katsuyama, went to the SEC for their concern the SEC staffers just shrugged; 2) Since 2007 more than 200 SEC staffers left to work for HFT firms. (Source: Michael Lewis, Flash Boys)

stygiansonic 13y ago

My (non-expert) take: HFT is a tool and it can be used for good or bad. Potential good uses may include: reduction of bid/ask spreads, improved liquidity and faster arbitrage of things like ETFs. However, it can also be used for bad: http://www.sec.gov/News/PressRelease/Detail/PressRelease/1365171484972#.Uz75MfldXm4 http://www.sec.gov/News/PressRelease/Detail/PressRelease/136... In that link, the SEC charges that a certain firm employed a trading strategy using non-bona fide orders to attract interest on another (bona fide) order. This involved submitting orders on the other side[1] to simulate interest. HFT makes such strategies viable, or at least more viable. 1. http://www.nanex.net/aqck2/3598.html http://www.nanex.net/aqck2/3598.html EDIT: Corrected to remove inaccurate information.

lrm242 13y ago
The action you link to was not perpetuated by HFT. The vast majority of Lightspeed's business is good ole fashioned screen traders, and that is likely what this was. Previous actions for spoofing and layering were also the result of manual traders using regular day trading tools.
- stygiansonic 13y ago
  My mistake. I assumed HFT was involved because the timescale of the "layering" orders and subsequent cancellation (following the bona fide order execution) was on the order of several hundred milliseconds.

herbig 13y ago

John Stewart had a really interesting interview with Michael Lewis this week about HFT: http://thedailyshow.cc.com/guests/michael-lewis http://thedailyshow.cc.com/guests/michael-lewis

jseliger 13y ago

Note that this: https://news.ycombinator.com/item?id=7524315 https://news.ycombinator.com/item?id=7524315 is a better place to start than the linked post.

cgmoore120 13y ago

read a great article on this the other night http://www.nytimes.com/2014/04/06/magazine/flash-boys-michael-lewis.html?_r=1 http://www.nytimes.com/2014/04/06/magazine/flash-boys-michae...

JumpCrisscross 13y ago

The NYSE has always required market makers continuously buy and sell the stocks they specialise in. They must do this in all market conditions, even if it means running a loss. In exchange, they get privileged access to order flow information. This is why those seats are valuable. NASDAQ challenged that model by removing the physical trading floor. Instead of humans standing in a pit there were humans sitting behind screens. But in both cases there is an institution guarding access to the market's nerve centres. Barriers to entry were raised and the incumbents protected. HFT takes the abstraction one step further. Democratised is that access to privileged order flow - anyone who can pay can get it. Lost is the role of a designated market maker - HFTs can pull out when markets get rough. This is a valid debate. But don't confuse yourself. The order flow privileges being criticised have always accrued to institutional market makers. It is no surprise that those incumbents are the ones driving the present lobby.

AndrewBissell 13y ago
> The NYSE requires designated market makers to buy and sell the securities they specialise in. They must do this even in violent market conditions This "mandate" tended to break down when tested by actual market panics. Bids dried up in 1987, and histories of the 1929 crash often mention clerks and floor runners who entered stink bids at $0.01 for stocks that had recently traded at $30 or $40, and got filled because the specialists had abandoned their posts. Whether human or computer, market makers know better than to stand in front of a freight train.
- JumpCrisscross 13y ago
  Market makers can declare "self help" in the event of technical difficulty. Exchange rules prohibit declaring self help to avoid adverse market conditions. In practice, self help declarations have a peculiar habit of cropping up in rough markets. Nevertheless, "guarantors of market liquidity" remains a rallying cry for traditional market makers.

arjn 13y ago

Correct me if I'm wrong but couldn't this problem be solved by some simple rules or act of legislation ? For example, make it a rule that when you buy a stock, you have to hold it for at least some time - say a minimum of 5 minutes. Or you could just impose a fee/penalty on traders that buy sell the same stock within one day of trading. (note - I'm merely speculating here and have no specialized knowledge)

deleted 13y ago

[deleted]

tlrobinson 13y ago

Is "registering algorithms" really a good solution? It seems incredibly hard to analyze or simulate such a complex system.

njharman 13y ago

"With these changes the fastest players were now able to make money simply because they were the fastest traders." Directly contradicts "speed is not a problem". Speed directly translates into being able to "front of trades from slower market participants". If they did not have the speed, they could not algorithmically predict the actions of (or react to activity on one exchange before it reaches other exchanges) and jump in front of slower participants. There's other issues on top of this, such as paying for early or privileged access. But, if you are faster, you can successfully arbitrage across time or distance(really same thing as time)

transfire 13y ago

The solution is very simple: a Tobin Tax.

Havoc 13y ago

Step 1) Have deep pockets, big balls and the right skills

deleted 13y ago

[deleted]

thikonom 13y ago

The title of the article sounds oxymoronic.

arasmussen 13y ago

I really wish the "Idiot's guide to high frequency trading" would start out by explaining what high frequency trading is to me like I'm an idiot...

mianos 13y ago

Just jump to the second comment for the actual details. Welcome to the internet, where if you are popular you can write any shit you like and people will read it, even if you don't actually know what you are talking about. (I wrote the software that runs some of the major stock exchanges. I did that for 13 years).

asbestoshft 13y ago

I have a lot of experience with HFT. Lets clear all of this up. How exchanges work for dummies. All of the modern US stock exchanges work as a limit order book. Orders are filled on a price-time basis. In other words, you put in an order, if it isn't filled immediately it gets put in the limit order book with your order going to the end of the line ( lowest priority ) at that price. That's it, that is how it works. Lets look at an example. I want to buy 250 shares of ABC. Right now there are orders in the book. 5 orders to buy at 9.95, each one for 100 shares. There are also 5 orders to sell at 9.96 also each order for 100 shares. I have several options. (1) I can put in a market order to buy 250, when this order gets to the order book sell orders will be filled starting with the oldest ones, the ones that have been in the book longest, first. My order will be matched, this is why they call it a matching engine, with two of the sell orders and then a partial on the third one. As orders are filled in the order book messages are sent out to all subscribers of the data that those orders were filled so everyone can keep track of the state of the book. It is important to note that when my order gets to the matching engine/order book there is no chance for the orders in the order book to get cancelled. Those orders aren't told "you are about to get filled do you want to cancel". (2) I can put in a limit order to buy at 9.96 in which case the order will be filled as described in 1 unless the sell orders at 9.96 are cancelled before my order gets to the matching engine. (3) I can put in a limit order to buy at 9.95 in which case my order goes into the order book at 9.95 and I have to wait for the orders ahead of me to either get filled or cancelled and then I have to wait for someone to send in a sell order, either market or limit, that will get matched with my buy order. That's it! Repeat, all day, every day. OK, that is a lie, there are special order types and that is one way exchanges try to differentiate themselves but that is a simplified way of how this works. So where is the problem that Brad Katsuyama saw? I don't know him all I know is what I've seen and read. One thing to note is that he talks about buying large blocks of stock. Very large. From http://www.nytimes.com/2014/04/06/magazine/flash-boys-michael-lewis.html?_r=0 http://www.nytimes.com/2014/04/06/magazine/flash-boys-michae... "It used to be that when his trading screens showed 10,000 shares of Intel offered at $22 a share, it meant that he could buy 10,000 shares of Intel for $22 a share." 10k shares at $22k/share, that is $220k in value. So he isn't talking about some small retail order, this is, as he points out, institutional flow. So lets think about what happens in this kind of case. There are 13 exchanges, all running limit order books. They are geographically disperse and internally they are implemented differently and have different latencies and different message protocols etc. At some instance in time all 13 exchanges have some orders to sell at $22.00. Think about that. In some sense that is awesome. Look how efficient the market is. If you went to 13 gas stations what are the chances that a gallon of regular gas will cost exactly the same at every station? Anyway, back to our problem. You want to buy 10,000 shares. You fire off your order to exchange A to buy 10,000 shares. This hits the matching engine and you get filled on 10,000 shares ( THERE IS NO WAY FOR THOSE SELL ORDERS THAT ARE IN THE BOOK TO BE CANCELLED!!!! The orders in the book aren't sent a message "hey, a 10,000 share order is coming in and about to hit your order do you want to cancel your order first?". THAT DOES NOT HAPPEN!!! ). The thing is that those 10,000 shares that got matched, lets say each order in the book was 200 shares. Your 10,000 share order just took out 50 orders. The fact that these orders were executed is sent out to every data subscriber who is listening for that data. All messages are timestamped, sometimes down to the nanosecond. From that the recipients can tell "wow, all of these 50 orders for 10,000 shares were executed at the same time so that was one order." Now, if you had orders in at other exchanges to sell at $22 what would your response be? For a second lets think about something more mundane, like TVs. Suppose you work at Target and you have a friend who works at Walmart. You're working one night and a guy shows up driving a semi-trailer. He walks in and says "How many 50 inch flat screens do you have? I'll buy them all, as advertised at $400." You sell all 20 that you have to him. As he's walking out the door he says "Now I'm going to Walmart." Just then your buddy at Walmart texts you and says "what's up? slow night right." and you go "Hell no, just sold 20 50 inch TVs to a guy, he bought everything I had and says he is headed for your Walmart to buy more." Now what does your friend do? In a world where prices are set by supply and demand Walmart could look at that and go "wow, there is a big buyer of 50 inch TVs out there, lets increase our price from $400 to $410." And so that is what Walmart does and sure enough the guy comes in and buys all the TVs from Walmart for $410. Does the TV buyer say the market is "rigged"? In any case that is very similar to what happens in this case. All of the HFT and market makers see this big order rip through a lot of sell orders at exchange A and they go "wow, there is a big buyer, I better change my sell order from selling at $22.00 to selling at $22.01 or $22.10 or $25 or whatever". Brad Katsuyama looks at that and goes "I've been front run, this market is rigged." And then he has a piece of software written so that all orders arrive at the different markets at the same time, as opposed to leaving his trading workstation at the same time, and suddenly there isn't any front running. There isn't any front running at all, what he is seeing is the reaction to supply and demand. HFT and market makers live in fear of something called "adverse selection". Market makers are required to post a two sided market, they are supposed to put up a price and quantity that they are willing to both buy and sell at right now. They publish this and they can't back away from it, they have to live up to it if another order shows up to buy or sell at a price that matches they have to execute the trade. The risk to them is that they are trading with someone who has information that they don't have. Just like playing poker or any game, you're going to lose if you play with people who are better than you. In this game though sometimes just having a big order is enough to move the market even if the person behind the order doesn't actually have any new information. Market makers and HFT either want to buy on the bid and sell on the offer and make the spread or they want to do arbitrage. That is how they make money. They lose money if they trade with people who are smarter or faster than they are. As soon as they see big orders come in they get really skittish and so they immediately move their orders to reduce their risk. I've written a lot and bored almost everyone. Ask away and I'll do my best to answer your questions. A couple other comments: 1. I'm not saying HFT is good or bad, I don't know. 2. Lots of people mention costs. Just to ballpark the whole thing. For the technology you need to Arista switches, that is $500/port, get a 48 port switch, that is $24k. For servers you need something decent, not crazy, dual CPU, 64 GB, running linux, maybe solarflare for kernel bypass networking, custom software. The hardware will cost you $6k/server. Then you need a connection to an exchange, varies but lets say $10k/month, then you need market data, $5k/month, then you need an order entry port, that is maybe $1k/month. That's it. Well, that is at one exchange, then you need low latency 10 Gbps cross connects between exchanges and you need to do that at each exchange. Then you need to find a clearing firm and you need to become a broker dealer ( exchanges and clearing firms only talk to BDs ). This is a highly regulated environment. You must have a compliance person, you will get audited yearly by SEC, your exchange, FINRA, all kinds of stuff. You will need a serious accountant. You have to have written procedures. 3. And how in the world do you avoid the Knight type of meltdown? Well, when you send in an order the order goes to the exchange. But the exchange also sends the order out via a "drop copy". This might go to your clearing firm so they know you aren't losing tons of money. But you can also get the drop copies and then you can constantly do your own risk checks that your orders you think are open are actually open but the ones you cancelled are actually cancelled. Most exchanges also have "cancel on disconnect" so when you really don't know what is going on you shut off your connection to the exchange and any open orders in the order book are cancelled for you.