22 ms·
A High Frequency Trader's Apologia, Part 1
- javert 15y agoI would be really interested to know the author's motivation for leaving HFT. For example, did you feel that you could make more money elsewhere? Eventually I’ll even put forward a policy prescription which I believe could cause HFT to focus primarily on socially valuable activities I'm really against the government making rules ot try to force people to serve society in some fashion. That's totally contrary to the principles of individual rights and pursuit of happiness upon which the country was founded.
- yummyfajitas 15y agoI would be really interested to know the author's motivation for leaving HFT. I left to do a startup. I am hoping to make considerably more money than I would have in HFT. I'm really against the government making rules ot try to force people to serve society in some fashion. I'd suggest you should stay tuned to the next post, since you will likely not object to my proposal. It's not a punitive measure designed to express disapproval, it's a minor tweak to market mechanics.
- javert 15y agoI'd suggest you should stay tuned to the next post Definitely looking forward to it.
- deathflute 15y agoIt is interesting that you believe that E(compensation in a start up) > E(compensation running a HFT strategy). The popular media is awash with reports of billions of dollars being made by HFT traders. Are they completely off? What about HFTs that are running market taking strategies?
- javert 15y agoE(compensation in a start up) > E(compensation running a HFT strategy) I also would like to hear more about this calculation. More the right side than the left, since I know far less about the right side than the left. But I imagine this might be getting into the author's personal details quite a bit.
- bearmf 15y agoWell, that is what ends up in the news: guys making billions. What articles are you referring to? There are also plenty of people making not so much money in HFT and plenty of people losing money. This is a competitive market and it has become more so.
- jrockway 15y agoIt's a myth that the financial industry pays amazingly high salaries. The maximum programmer salary at a bank is probably bigger than the maximum programmer salary anywhere else, but the average salary is about the same. I worked on electronic trading systems at BofA, but Amazon and Google both offered me more money to work for them. (Some people get good bonuses, but the best way to guarantee that is to threaten to leave for somewhere that will give you XX bonus on paper. I did that but realized I actually wanted to leave :)
- sid6376 15y agoWhat I would love to hear in the follow up to this post is what are the typical technical challenges which a typical HFT encounters? If someone wants to be an HFT what are the core skills one must have? (Strong in algorithms, good c++ skills?). How much math should one know and what kind of math?
- netvarun 15y agoThere have been a couple of HN discussion threads regarding your last two questions. 1. http://news.ycombinator.com/item?id=3177815 http://news.ycombinator.com/item?id=3177815 2. http://news.ycombinator.com/item?id=966716 http://news.ycombinator.com/item?id=966716
- radikalus 15y agoI think to be successful your team needs to be strong in four key areas: - Trading (Strategy) - Software - IT - Research There isn't necessarily one of the four that trumps the rest, and different players have different relative strengths across those four disciplines. (Which are as you would imagine, not terribly specific to HFT) I imagine the question is developer-focused, so on the software side, I think I value most highly: - Don't fuck up. This is obvious, and the trading adage of "Don't lose money" is overdone...but essential. Assume you will fuck up and have nets in place to catch you. Doing safeties well isn't sexy and doesn't add any edge, but I've witnessed "unlikely" safeties save trading systems. - Understand the hardware. Undestand the strategy. You cannot write fast "generic" code. Have a sense for how fast you could optimally be on the hardware and network stack you're using. This requires interfacing with IT and, sometimes, trading, as some "strategies" traders come up with aren't easy to do FAST; you might be able to suggest something similar that is potentially much quicker on the critical path. - If you really want to compete in low-latency, C++. If you just want to be pretty fast and your strategies aren't competing for specific orders with the other market makers, I don't think it matters. You can probably do a lot of strategies in 100us in any language. GUI can be done in anything, but it should be able to scale. Most of the GUI pieces are pre-existing. Sortable tables for fills. Ladders for displaying markets, etc. I've used Java/C#/C++ GUIs -- C++ has always worked best for me, but I wouldn't be opposed to doing the front-end in JS/HTML honestly. - Understand the critical path. This and, to some degree, only this, has to be FAST. Obsessing about how to optimally build a theoretical value cache more quickly etc etc is often not that big of a deal. (It's not on the path) - Linux kernel and TCP/UDP stack DETAILED understanding. (Or verilog I guess...) - Locks. When does a mutex really need to be locked? Obviously depends on system architecture. And can often dictate architecture. I think almost anything else I'd say would be either vague or generic. I think there's a lot of niches one can carve out in HFT and, while I personally value generalists who can holistically understand the goal (make $), there's a ton of people doing really well by just being the most badass at X/Y/Z.
- anonDataUser 15y agoThis offers no insight into HFT and is not an apology. Why is it here? For people interested in HFT, start your exploring with a google search for "latency arbitrage".
- joshu 15y ago"an intro to limit order books" Without looking, as it has been years since I did this professionally: is latency arbitrage looking at price moves out of sync with different ECNs and liquidity providers?
- anonDataUser 15y agoSorry for the lack of guidance in my original response. Latency arb was born out of the fragmentation that was introduced when the SEC passed Reg ATS. For those unfamiliar with trading, there are many exchanges, not just one. All of those exchanges are obligated to trade at the same prices. To ensure this, Reg ATS establishes the NBBO: National Best Bid/Offer. If you're buying, and one exchange has an offer at $10.00 and another has an offer at $9.00, $9.00 is the best offer. If you send an order to the exchange that has a $10.00 offer, they are legally obligated to send that order to the exchange with the best price. Latency arb is the process of keeping quotes/prices in sync by trading when a specific exchange's prices don't match NBBO. Many of the largest HFT players work on a very simple principle: they subscribe to all of the exchange feeds and construct their own NBBO that is faster than the best commercially available NBBO feed. With this, you know what direction stocks are moving as it happens. Keep in mind that a "price movement" rarely occurs at a specific instance in time. It occurs over the horizon that it takes for the market to synchronize -- which happens as fast as possible and is based on the infrastructure investments of high frequency traders. If you can do it faster and you build the infrastructure the world is yours. Like Vanderbilt with railroads.
- firebones 15y agoIt's an apology in the second sense of the word apology: "2. a defense, excuse, or justification in speech or writing, as for a cause or doctrine." Basically, apology as "explanation and defense of an idea".
- RobertKohr 15y agoSo if you remove the restriction on subpenny increments, you remove the need for HFT?
- RickHull 15y agoHFT would still serve a role with subpenny increments. Nonetheless, the subpenny rule is a very important mechanic in the construction of HFT.
- AndrewHampton 15y agoI agree. I think there is currently a balance to be struck between accuracy of the price determined by the algorithm and the speed at which it obtains that price. I think removing the subpenny rule would just reduce the importance of speed and increase the importance of accuracy. I'd imagine trades would still happen extremely fast though.
- deathflute 15y agoThanks, this is a very clear article on the most important ingredients of HFT. I am curious to hear the apology but I am not sure there is anything to apologize for. The market marker is taking on substantial risk of adverse selection and is being compensated for it. As long as legal means are used to achieve that end, this is not really different from any other business strategy. Causal implied benefits to society from any business are subjective and the most important consequences are usually latent. Anyways, I am curious to know if elaborate prediction models are used here or it is mostly a game of adjusting prices based on correlated instruments? Also, how does one build a simulator for such strategies?
- wglb 15y agoApology in the sense explained in comment in another thread: http://news.ycombinator.com/item?id=3852595 http://news.ycombinator.com/item?id=3852595
- ChristianMarks 15y agoI'll believe that HFT adds liquidity to the market when the typical start of employment to retirement period is 15 milliseconds.
- yummyfajitas 15y agoI'll believe that Amazon's network engineers (and whoever else is involved in making their page fast) add value when it takes 200ms to ship products to my home.
- lrm242 15y agoAn expected and uneducated statement. It's simple really: any participant who rests an order on the book is adding liquidity. Period. Many HFT strategies are largely passive (market making, for example) and therefore entirely rely on resting orders (as Chris' article points out). Therefore, they add liquidity. Now, if you want to get into a discussion as to the quality of that liquidity then you have to go educate yourself a bit more. Once you do you'll find that the liquidity being added is not toxic nor fleeting. However, what you and most others don't understand is that it is also significantly more informed passive liquidity than it used to be. This may lead folks to believe that the liquidity is fleeting because they can cancel away before you can execute. However, that simply means they are better and faster than you, not that they are doing anything wrong. Why, then, don't we simply require them to rest their orders for a longer period of time? Simple: you want tight spreads. As spreads shrink market making becomes less profitable. When market making is less profitable market makers have to be more risk averse to avoid getting adversely selected and losing money. More intelligent models (they become more informed) and less latency (they can leverage that informedness to avoid being selected) is how market makers stay profitable in a market where the average spread is 1 penny.
- _delirium 15y agoHow about rather than adding regulations, just discretize the market's clock? Would a market that performed a trade-resolution tick once every 100ms lose anything important? I believe arbitrage improves efficiency, but I'm less clear on the benefits of extremely high temporal resolution. Currently that's what the incentives encourage optimizing for, among other things. If you predict a market movement 10ms ahead of time, you can profit from that brief 10ms temporal-arbitrage window. And like any arbitrage, that does indeed improve the pricing signal in an absolute sense, in this case by taking a price change that was 10ms later than it "should've been", and moving it up in time via your trading. But it's not clear that 10ms-level pricing inefficiencies are actually something particularly important to smooth out via arbitrage, when you can just define them out of existence by going to a discrete-time market (which are fairly well-studied in the mathematical literature).
- lrm242 15y agoA few notes: (a) Orders for stocks priced below $1.00 are permitted to display in sub-penny increments. Not that this is really relevant to HFT, but it is often a point of confusion. (b) Orders may rest on the book and display at a price but execute at a separate price. For example, if a market is 20.01 / 20.02 and I believe true value is 20.017 I can bid 20.01 but tell the matching engine that I'm willing to accept 20.015. That you are willing to trade at the midpoint also places you at a higher priority in the queue, above those only willing to buy at 20.01.
- yummyfajitas 15y agoRegarding b), I believe it's the MPL order you are referring to. http://usequities.nyx.com/markets/nyse-arca-equities/order-types http://usequities.nyx.com/markets/nyse-arca-equities/order-t... Some dark pools also allow trades at subpenny increments, although of course (being a darkpool) the orders are not displayed.
- lrm242 15y agoMidpoint orders exist on all the venues. All but EDGA/EDGX provide some rebate for resting a midpoint order. DirectEdge charges both sides for what they refer to as "midpoint match".
- georgek 15y agoTransactions that occur in dark pools (over half the liquidity on NYSE listed stocks these days) typically get posted to the exchange after the fact. This used to cause anonymous hedge fund I used to work for all kinds of problems in the leas liquid markets as half the daily volume would suddenly post dive minutes
- anonDataUser 15y agoThis seems inaccurate. Dark pools make up 15% of market volume.
- deathflute 15y agoTo add to (b) Midpoint orders have to be hidden though because lit markets cannot allow displayable sub-penny orders.
- jboggan 15y agoWill the following installations address the role of high frequency trading in flash crashes? Although high frequency trading seems like a very good strategy when arrayed against human opponents I have to wonder what happens when all the inputs and outputs are controlled by HFT algorithms trading against each other. How do you debug a HFT algorithm?
- vasco 15y agoI am also interested in the debugging part.
- wglb 15y agoThe rapidity of recovery from the flash crash can be credited to HFT.
- toomuchtodo 15y agoLets all rejoice in how fast those who created the disaster fixed it. rolls eyes
- yummyfajitas 15y agoIt was a fat fingered mutual fund who created the flash crash, not HFT's. It was HFT's who prevented it from being a disaster. http://www.sec.gov/news/studies/2010/marketevents-report.pdf http://www.sec.gov/news/studies/2010/marketevents-report.pdf
- runeks 15y agoHow is the price of a stock market plummeting a "disaster"? For those who own the stock it's of course not advantageous, to those who wish to buy it, it is. I'd like the HFT traders to get into milk and apples. Maybe they can crash the price of these too so I can get some cheap groceries.
- GFischer 15y agoGroceries do have huge variations (at least here in Uruguay), and there is a futures market in them. There's definitely talk of fruit futures :) http://www.goodfruit.com/Good-Fruit-Grower/October-2011/Futures-market-for-concentrate-coming-soon/ http://www.goodfruit.com/Good-Fruit-Grower/October-2011/Futu... "The status of the long-awaited futures trading contract in apple-juice concentrate hasn’t changed. It’s still coming “soon.” " Milk doesn't only because it's regulated (here at least).
- dbecker 15y agoI've seen so many news articles full of hyperbole about HFT, and this is the first legitimate explanation I've seen. Thank you.
- joejohnson 15y agoLast week, Bloomberg published this op-ed defending HFT: http://www.bloomberg.com/news/2012-04-10/high-speed-trading-is-progress-not-piracy.html http://www.bloomberg.com/news/2012-04-10/high-speed-trading-... The piece is largely propaganda. A nice rebuttal is found here: http://www.ritholtz.com/blog/2012/04/hft-pirates-and-their-academic-friends/ http://www.ritholtz.com/blog/2012/04/hft-pirates-and-their-a... Can anyone explain why multiple exchanges exist (besides for historical or political reasons)? It seems anti-competitive that exchanges can be private at all, let alone that exchanges can be allowed to operate as monopolies in a market or country.
- HeXetic 15y agoThat's not a "nice rebuttal", that's some bullet points spat back. Whereas the original linked article in this discussion provides some context and explanation, the bullet points in your second link are meaningless to the layperson (e.g, me).
- joejohnson 15y ago...the original linked article in this discussion... Do you mean the Chris Stucchio piece? Because, despite it being an apology, it doesn't justify how HFT is socially beneficial. This piece makes hand-wavy claims about liquidity, and says that it will better justify HFT in a coming blog post. As for the Bloomberg piece defending HFT, it fails to address a few issues that the SEC and critics have with HFT. These issues are enumerated in the Ritholtz blog post. Pointing out certain facts that an argument ignores is a valid rebuttal.
- tptacek 15y agoJiminy! Chris Stucchio is 'yummyfajitas, one of the oldest contributors on HN. He was a dev for an actual HFT firm for a bunch of years before doing a totally unrelated startup. Could he just maybe write a post and spark a discussion about how his field actually works so we can learn something about it? I DO NOT UNDERSTAND this HN mentality that equates "learning more about X" with "joining the death cult of X". And it comes up all the time: no, we can't discuss how SEO works, because that would make us morally black hat SEOs; no, we can't discuss how HFT works, because that would make us morally Goldman Sachs executives; no, we can't discuss how a law is written, but that would mean we want to give the Internet to the MPAA. The particular mode of argument in your comment and many like it on this thread is directly counter to the whole idea behind this site and every other worthwhile message board on the Internet. Yes, we get it, you don't know who G.H. Hardy is or didn't pick up on the joke in the title --- but it's obvious from the rest of the post that it's trying to explain how things work, not make a case for why it works that way.
- someben 15y agoKinda an obvious introduction to order books. Later I hope he gets into the rebate mechanism: http://www.interactivebrokers.com/en/accounts/fees/NYSEstkfee.php?ib_entity=llc http://www.interactivebrokers.com/en/accounts/fees/NYSEstkfe... Note that you get PAID for placing certain types of orders.
- caladri 15y agoMen buy, women sell, and also if women are sad you can feed them strawberries and get off on it. Got it. Now I understand HFT perfectly.
- francoisdevlin 15y agoIt's a firefly reference.
- caladri 15y agoThat much was obvious. When you build your examples around a sexist experience of sexist media, your examples will themselves end up sexist. I mean, why isn't Zoe buying, too? What is it that Inara's selling? I think we know, and from the heternormative misogyny rises the situation in which a cheeky description of market making has exclusively men buying and exclusively women selling.
- yummyfajitas 15y agoWhat is it that Inara's selling? Shares of Blue Sun. http://firefly.wikia.com/wiki/Blue_Sun_Corporation http://firefly.wikia.com/wiki/Blue_Sun_Corporation I'll be sure to have Zoe and Book make an appearance in part 2. Sorry that my fanfic missed your favorite character.
- scott_s 15y agoFor the record, I made no connection between Inara's occupation on the show, and her "selling" in this example. I doubt Chris intended such a connection.
- johnohara 15y agoThe astute reader will probably ask this straightforward question: -- "Jubal, does that seem right to you?"
- tptacek 15y agoThe concept of a market maker? Market makers are not an HFT "innovation"; they're kind of central to the whole concept of a trading exchange. He hasn't yet explained anything more than what a market maker is or where HFTs fit into them. This post is the first in a series. It's background material. It would be a little weird if anything in this particular post got your hackles up.
- johnohara 15y agoMal, Inara, Jayne, Kaylee, River, Simon and Wash are all characters from the television series "FireFly." Jubal Early (Richard Brooks) is the bounty hunter in the last episode who ultimately ends up as an object in space. Throughout the episode, his hypothetical situations always end with the rhetorical question, "Does that seem right to you?" My comment was a tip of the hat back to Chris Stucchio. I enjoyed the article and also look forward to more.
- tptacek 15y agoD'oh!
- lifeformed 15y agoNot making any value judgements here, I just find it fascinating how much we have abstracted away the original concepts of the stock market.
- grandalf 15y agoWhy do you say that? How would a basic marketplace work without the same rules?
- lifeformed 15y agoI'm not saying different rules are necessary, just that it's interesting how abstract the whole concept is. I think about stock like how it was explained to me a long time ago, in very simple terms: Bob runs a restaurant worth $9000, and he needs $1000 more to buy a new oven. 10 of his friends lend him $100 each, in exchange for 1% ownership of the restaurant. Now with the new oven, Bob can bring in more money and he shares 10% of that with his ten friends. Now we have this gigantic hurricane of integers that we call the stock market, and people make millions by running complex programs that find numbers that are slightly bigger than other numbers, and then run some code. An electrical signal is sent across the country, and in a database somewhere, transistors that represent somebody's bank account are changed into a different configuration. Everyone is sitting behind a desk, or standing at a terminal, pushing buttons. Based on what pattern of colors the screen flashes, people's lives are ruined or made.
- grandalf 15y agoWell articulated comment. Indeed it has become much more abstract. I guess one way to think of it is: If someone is trying to raise money for an oven it may be fairly difficult to find people willing to invest. If investors are found, it may be difficult to agree on a fair arrangement. The stock market has grown precisely because it offers price discovery and liquidity. Putting shares on an open market lets the fair price be discovered, and when investments are priced fairly they become more liquid, which means there is less risk to the investor b/c he/she can change his/her mind about the investment decision and not be stuck with an investment that no longer meets his/her objectives. This lowered risk leads to more investment, which in turn leads to even fairer prices, and so on. I think the key point about HFT is that to the extent that HFT provides liquidity (which it does by adding orders to the book at competitive prices) it helps with price discovery. Since the "fair" price is a function of the risk various parties are willing to take by their willingness to risk trading at the wrong price, HFT not only makes prices fairer but its participants risk their own capital to vouch for the prices. Clearly if things turn out unexpectedly, the HFT firm is footing the bill b/c of its devalued inventory.
- xaa 15y agoDo HFTers pay any sort of commission or transaction fee (to the exchange)?
- zrail 15y ago(not a trader but I've researched it a bit) Yes, often per share traded. A lot of traders use Interactive Brokers which has a few different pricing structures[1]. [1]: http://www.interactivebrokers.com/en/p.php?f=commission http://www.interactivebrokers.com/en/p.php?f=commission
- anonDataUser 15y agoSub-pennies, negotiated directly with exchanges. They do the same negotiation that Interactive brokers does, except unlike interactive brokers, HFT players can trade 10% of market volume which give them significant leverage.
- sailfrog 15y agoThe common way transaction fees are handled in US equity markets is called the "maker/taker" model. When a participant adds liquidity to the exchange order book, AND that order is executed, they are given a very small rebate per share (they are the "maker"). The other side of the order, the firm who "removed" liquidity from the order book, is charged a very small fee. The difference between the rebate and the fee is the spread the exchange earns for executing the trade. Note that not all orders that add liquidity qualify for rebates, the rebate may depend on particular order properties as well as the way in which the trade executes.
- radarsat1 15y agoWhat I don't understand about HFT is why its efficiency does not seem to be limited by transaction fees. I've tried a bit in the past to play with some algorithms that buy and sell (on paper) and it seems like the faster you go, the more you need the market to move in order to scrape a few pennies of profit, because the movement has to be sufficient to cover the spread, the transaction fees, _and_ provide some profit. Of course with a simple example like "buy at $20, sell at $20.10", it seems simple enough to extract some profit, but in fact it would need to be enough profit that it covers the transaction fees, which seem to be a kind "friction" that limits the smallest change in price that is useful. From what I can tell this kind of movement just doesn't generally occur in the sub-millisecond time frame, so how does it work? Or is it just that they are trading so much $$ in one shot that a very tiny change in price covers the transaction fee without issue? (Meaning they are taking very large risks..)
- yummyfajitas 15y agoThe sub-millisecond timeframe is part of a race to be at the top of the book or a race to pull orders. Very few HFT's literally trade a single security repeatedly on a sub-millisecond timeframe. The general strategy is "our signal changed - move fast, get to the top of the book in milliseconds". Then you wait, and seconds to minutes later someone will fill your order sitting at the top of the book. Then you play the same game on the opposite side to unload your position. The game of shaving milliseconds is solely about beating other HFT's to the punch, it's not about actual market movements.
- minus1 15y agoGreat article. I'm curious what your thoughts are on the Sub-Penny Rule.
- gbhn 15y agoThanks for the article. It seems to me like a missing piece is a discussion of volatility. That is, I get it that HFT drives spreads lower, but spreads are the wrong direction to make money, even in volume. That is, if pre-HFT spread is that the best buy price is $10.00 and best sell price is $10.05, and post-HFT best buy = $10.03 and best sell = $10.04, you still lose a penny on every trade instead of a nickel -- you can't make that up on volume! :-) It seems to me that volatility is required for any market maker to make money, and that's a fundamental conflict, isn't it? That is, "outside" traders would prefer the market to be smooth, whereas "inside" traders want it to fluctuate. I can see the argument that, in actuality, HFT on the whole needs less fluctuation to extract enough profit to provide liquidity, so in theory, it would be expected to be a more stable market maker than human operators. Is that basically equivalent to what you're saying in the article? (And is there any data on that hypothesis?)
- achy 15y agoFrom my understanding of this, HFT basically makes it impossible for a non-HFT trader to buy at anything but his maximum buying price and to sell at anything but his lowest selling price. This seems like it would significantly undermine the profitability of the market for non-HFT traders.
- tptacek 15y agoHFT makes things very bad for day traders. It makes things marginally better for people who want to take or sell a long-term position. It doesn't change limit orders.
- toddh 15y agoAccepting the earliest bid seems quite arbitrary. How would the system change if a random participant was selected? Or perhaps you could buy preferential treatment or use an algorithm weighted by some good actor metric.
- 3pt14159 15y agoIt is in the market's best interest to reward liquidity, so by giving it to the earliest bidder they reward those that have maximized the market's liquidity over time.
- mayoff 15y agoYou can buy preferential treatment: If you set a higher price for your buy order, then you are preferred over every lower-priced buy order. If you select a participant at random, you are encouraging each trader to place many orders (at the same price level), to increase the probability that one of his orders is chosen randomly. If you use a pro-rata algorithm based on order size, you are encouraging each trader to inflate his order quantity, and cancel very rapidly after all (or nearly all) of his desired quantity is filled.
- Fando 15y agovery interesting
- losvedir 15y agoHuh, I'm glad this was posted! 1) Didn't realize when I read this it was by yummyfajitas, whose comments I always respect. 2) Do I understand correctly that the order book is public (albeit anonymized)? That surprised me -- it seems like this would lead to meta-games and jockeying for position and such, as opposed to different parties just submitting into a black box what some security is worth to them. 3) Are there any good papers/articles modeling different market set-ups? For instance my black box one above. Or one where orders are matched randomly rather than chronologically. Or one where trades happen in one batch once per day. I can't say I'm opposed to HFT, it just seems to sap a very large amount of engineering brainpower for not that much societal marginal benefit anymore. If I could snap my fingers and give up a bit of the liquidity and get all those engineers back I probably would. 4) I once worked in a sell-side equity research shop which traded stocks the old fashioned way (based on fundamentals) and had a non high-frequency trading desk, etc. That type of company is getting hurt by the lower spreads offered by HFT. But I never did quite comprehend why it made sense to pay for our product (research) with trading commissions. Seemed to cross two unrelated services, although that kind of business model seemed deeply entrenched in the market.
- Chirono 15y agoRegarding 2: You're correct, the main order book is public and usually anonymous. There are also non-public books called dark pools where you cannot see what other people are bidding. You are also correct in supposing that a lot of meta-games go on, but dark pools don't remove this entirely; they just change the rules. For instance you could, if it is allowed, submit a tiny orders in increasing price until it gets filled; then you know something about the price in the pool.
- losvedir 15y ago> For instance you could, if it is allowed, submit a tiny orders in increasing price until it gets filled. Ha, that's clever. I suppose any system that people come up with will always have ways to game it. Reminds me of Arrow's Impossibility Theorem[1] which basically proved that there's no possible perfect voting system. Ah well. [1] https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem https://en.wikipedia.org/wiki/Arrow%27s_impossibility_theore...
- MikeCapone 15y agoQuestion: Are all HFTs only market makers like this, or do some of them use other strategies (front-running?) that actually make transactions more expensive for others, or move markets a lot (especially in small caps, I would guess)?
- Nrsolis 15y agoHFT is a badly abused term that gets applied to any firm that uses computers to trade. In reality, there are many other strategies that use automated techniques to model the values of securities using internal metrics and then trade when the price is above/below the internally calculated value. Such conditions usually indicate that the market has "mispriced" a security and will quickly realize it's mistake and return to fair-value pricing. Many "algorithmic traders" use statistical techniques to determine optimum times to trade so they capture some profit margin with a high degree of certainty. Those statistical techniques can involve anything from automated balance sheet analysis and natural language processing of news items, to pattern-matching techniques designed to notice when a large buyer/seller has entered the market and needs to move a large block of shares.
- photon137 15y agoCan you say how feasible it is to build an HFT startup - the operating costs of execution/connectivity/colocation(!) aside - do you think regulatory authorities make the barrier-to-entry too high? SEC/FINRA have Series 7/56 requirements for employees, UK has FSA-related principal investment regulations - both require a company to sponsor a person to trade. Also, the MiFiD regulations in EU require an "adequate capital" of 700k euros - is it really possible to get venture/seed capital covering these bare costs so that you just remain "legal", let alone operate?
- veyron 15y agoThanks to sponsored access and the large trader rule, you really just need the blessing of a clearing firm and pay some colocation fees. As a one time cost anyone can get in with 200K deposit (on a 6:1 levered setup) and about 50K in startup expenses, with a 25k/mo recurring charge. P.S.: that is what I am doing now :)
- photon137 15y agoThat's useful to know - thanks!
- djhworld 15y agoNot sure where the apology is...or if there is even one needed in the first place. I really enjoyed your article, it explained (in laymans terms) something that's often been bewildering to me as an outsider to the financial sector. Maybe this is why people outside of the sector are hostile to the idea of HFT - because they haven't got a clue how it works?
- adamsmith 15y agoThe way the system is set up makes it a tax on global efficiency. All of these resources (crazy smart people, money, equipment) are needlessly wasted on getting lower latency when really it adds no systematic value. Instead, just as the system disallows fractional penny bids, it could discretize order times to the nearest e.g. 5 seconds. Boom. Suddenly you have more MIT/etc grads solving better problems for the world.
- market_hacker 15y agono. that would only work if every traded product on the planet was in a single matching engine.
- noname123 15y agoSome exchanges already do that. But the issue is also liquidity not latency. When you disallow fractional penny bids and match trades at set time-interval, this causes market-makers to widen their bid/ask spread because of the inventory risk. Ideally, as a dealer you'd like to buy low and sell high and move the merchandise on your lot before the prevailing market condition moves against you. The spread even at highly liquid stocks used to be 0.10+/cents before the introduction of penny pricing and HFT; this saves money for retail investors. Also, introduction of set-time matching also means that algo's will just be tweaked to new market conditions. New arbitrage algo's will be introduced to exploit how some exchanges will be slower to react to price-actions. Predatory algo's will exploit how some institutional VWAP orders cannot be canceled quickly against volatile conditions. It's anti-virus vs. the virus writers. Oh not to mention the exchanges are in on this game, more trades mean more revenue for them. Exchanges make money by volume of trades, not by P&L of their customers; and like affiliate marketers, they actually offer liquidity rebates to any traders that offer trades that get taken at market price to boost their volume.
- bradleyjg 15y agoThe spread even at highly liquid stocks used to be 0.10+/cents before the introduction of penny pricing and HFT; this saves money for retail investors. ---- It saves money for retail traders (i.e. fools). It has negligible effects on investors in well diversified passively managed funds.
- deathflute 15y agoI am also confused about how latency arbs work. If you are co-located at one exchange, wouldn't you be further away from other exchanges?
- wglb 15y agoBig players are co-located at every exchange.
- deleted 15y ago[deleted]
- tolitius 15y agomarket makers work (have deals) with exchanges. e.g. a market maker M guarantees to provide liquidity (e.g. orders) for a certain set of securities to an exchange E, and gets certain benefits in return. This market maker M makes sure that he is as close as possible to the exchange metal, so he has an upper hand. The importance of colocation is directly proportional to the liquidity (amount of orders) you provide. e.g. if you send orders manually every minute or so, it does not really matter where you are in comparison to a large bank that sends orders every several microseconds..
- redwood 15y agoThe author describes changing bid/sell prices based on new information (like a new press release putting pressure on value, say). He doesn't mention whether people are feeding the algos up/down pressure based on these press releases or if they're automatic. I assume they're at least partially automated. In other words for every article that comes out about a given company, some algo (or human) gives it a plus/minus ranking and magnitude based on what it means for the company. Assuming this is partially done by algos that crawl press releases looking for signals, or similar (in order to be first to market with latest news) one can imagine a new type of hacking. Hack the press releases and watch the algo trades go haywire....
- polynomial 15y ago> The rise of algorithmic trading is merely a special case of machines replacing humans. One of the article's more salient points.
- ptolosav 15y agoVery interesting. As an undergraduate looking to learn more and hopefully get some knowledge so i can get that "edge" in the field when i graduate, what books do you guys recomend on trading? I know HFT is very specific and probably there isn't lots of literature on it, but something more general will do. Thanks a lot.
- surement 15y agoAs I understand it, it's better to learn assembly, algorithms and parallel computing. Some firms even look for PhDs in machine learning. The finance side you can learn when you find a job.
- LinXitoW 15y agoWhile the article itself is very interesting to someone like me that knows squat about diddledy, i must commend the author for including FireFly references.
- tjmania2003 15y agoReally enjoyed reading this, great stuff. Even better Chris spent some time at Snevets :)