9 ms·
Mysterious Algorithm Was 4% of Trading Activity Last Week
- ahi 14y ago"Translation: the ultimate goal of many of these programs is to gum up the system so it slows down the quote feed to others and allows the computer traders (with their co-located servers at the exchanges) to gain a money-making arbitrage opportunity." I thought the justification for HFT was liquidity. The market is clearly broken.
- jacobwil 14y agoI believe that they were talking specifically about the programs which create orders which are never fulfulled. That way their programs creating real trades have an advantage (since they are running on servers located closer to the exchanges).
- skorgu 14y agoBear in mind the exchanges aren't stupid, Nasdaq at least has instituted a penalty system for sending a bajillion orders and not actually trading [0]. One may certainly quibble with the exact rates and penalties but this is a thing that exists. [0] http://www.nasdaqtrader.com/TraderNews.aspx?id=ETA2012-13 http://www.nasdaqtrader.com/TraderNews.aspx?id=ETA2012-13
- greenyoda 14y agoIt makes a lot of sense to penalize that kind of behavior. Deliberately slowing down the quote feed or cancelling all your trades seems to be intended as a denial of service attack on your competitors.
- infinii 14y agoTake a look at the example and how small the penalty is. Not much of a deterrent for traders making multi-million $ bonuses is it?
- davidtyleryork 14y agoThis makes Mark Cuban's article on High Frequency Trading (HFT) even more relevant than it was before (link: http://blogmaverick.com/2012/09/21/what-business-is-wall-street-in-3/ http://blogmaverick.com/2012/09/21/what-business-is-wall-str...). WHY is it not regulated or at the very least taxed, when it creates such outsized risk for the public and doesn't accomplish anything for the market besides increased volatility
- twoodfin 14y agoIf you're so worried about it, just don't invest in markets that allow it. The "public" is only capable of risking their own money.
- marshallp 14y agoMost people can't do that because their money will be put there by their 401K (which if they don't put money in willl lose out as well).
- Steko 14y agoThis sounds like "don't go outside if you're worried about being mugged". Sorry, no we almost all pay taxes used for bailouts and have our retirements invested there by our employers. We get to criticize and advocate for change.
- Evbn 14y agoPlease look at who got hailed out and what they were doing. Issuing a sham mortgage is not a high frequency trade.
- kaykurokawa 14y agoYou mean the Mark Cuban who rode out the tech bubble to sell his company making 13.5 million in revenue for 5.9 billion in stock is arguing that the stock market is now broken? How wonderful.
- confluence 14y agoWhat people don't seem to get about HFTs is that retail investors cannot and should not attempt to compete with them or trade actively at all - HFT or prop trading is a big no-no for the unsophisticated. Historically buying into an index fund during a crisis is a perfectly reasonable way to achieve relatively easy alpha above the mutual fund/bond rates (~12% per year). This type of investing (aka diversified value) is completely separate from HFT. As far as many retail investors should be concerned HFTs can duke it all out between themselves as much as they want - it really doesn't matter and is irrelevant to your long term investment decision. Indeed HFTs provide the benefit that during times of crisis they provide plenty of buy side liquidity for you to get in at a lower spread and a lower spot price relatively easily.
- smoyer 14y agoI agree completely ... let the HFTs eek out a few pennies per trade (or even fractions of cents). But I also wanted to see how much bandwidth my 4G phone could really use. Sorry for the perturbations I might have caused and I promise I'm only doing what's good for man-kind. Now if I could just get my phone to make a decent call!
- confluence 14y agoI don't know what you mean. HFTs don't do what's good for mankind - they do what's good for themselves. Just like everyone else. No one does what's good for mankind - they do what's best for themselves. We allow those who don't harm others to continue doing what benefits them - until such a time as it is shown to be harmful - then we stop it. HFTs have not been shown to be harmful.
- marshallp 14y agoIt's more nuanced than that. There are things like patents and copyrights. When it suits them, big C capitalists demand longer trading terms, then turn around and say the opposite when it comes to HFT (shorter trading terms). (They also do this with their accredited investor requirements). HFT harms society because it cuts off the number of investors that can do active trading. It's net effect is the same as regulations the keep out entrepreneurs from innovating in things like health care and drones and car manufacturing.
- marshallp 14y agoAnother reason to limit trading to daily or weekly or montly. The trading arms race is focusing on speed when it should be focusing on predicting longer term outcomes. The easiest way to do so would be to limit all trades to once per day in an auction system. Then traders would pour their resources into building sophisticated systems that make long range predictions and so also identify new innovations/companies that should exist in the market. This is in literally almost everyone's best interest - investors, public corp's, governments, consumer - only a small group of high-speed traders would disagree. Why this hasn't occurred yet is mind-boggling.
- dclusin 14y agoTrading has always been about speed. This dates back to the creation of the earliest financial markets (http://en.wikipedia.org/wiki/Rothschild_family http://en.wikipedia.org/wiki/Rothschild_family).
- marshallp 14y agoThe evil Rothschild's are still costing today. A redistribution of wealth that was accumulated prior to 1900 should be instituted (or maybe for even before 1980).
- darknoon 14y agoEven if trades were only executed daily, wouldn't a futures market develop that was traded at the millisecond level anyway?
- marshallp 14y agoReducing the trading increment is what a futures market does anyway (you can buy contracts to remove the volatility of second/minute/daily activity).
- nancyhua 14y agoHigh frequency traders talk about this and I heard about one being developed. Regulators and basically everyone you listed don't know what's in their best interest so it's up to the traders who decide they want to stop the arms race. It's hard to switch over bc traders go where the liquidity is so we'll have to see if they can overcome the chicken and egg problem.
- yk 14y agoTranslation of "Translation: the ultimate goal of many of these programs is to gum up the system so it slows down the quote feed to others and allows the computer traders (with their co-located servers at the exchanges) to gain a money-making arbitrage opportunity." Annomyous is (D)Dosing the market. :D
- lojack 14y agoSo, the video made it sound like someone was holding a large amount of stocks and very quickly sold all of them (like the flash crash). It didn't lead me to believe that they were placing orders and pulling them before they could be executed. Was this type of order discovered after the video recording, or am I misunderstanding what they said? Also, in the video they say: "What you're saying is the individual investor has no shot. Mary and Joe sitting out there have no quants on their staff" My assumption was always that the average trader holds their shares over a longer period of time, and thus aren't really affected by these HFT algorithms. Is that assumption incorrect?
- tptacek 14y agoNo, it is not incorrect. The impact of HFT on buy & hold investors is probably a wash. Automated trading has cut the bid/ask spread but may contribute to volatility (it is tougher to say than it seems). Either way, Mary and Joe were never competing with market makers and active trading strategies, and still aren't now.
- marshallp 14y agoThat's not entirely true. I'm an amateur and I want to invest actively using statistics. But getting tick data (covering years/decades) is costly and dealing with it as well. I could handle it if it were daily (if all trades were limited to 9am auction every day. This would also allow room for much more extensive modeling of other data sources and longer term predictions).
- vecter 14y agoThat's not a very convincing argument. I'd like to participate in F1 racing, but the research and engineering costs are too high for me as an individual to partake in. If they limited racing to only 4-cylinder non-modded Hondas, it would allow me to compete in their game.
- marshallp 14y ago
- mey 14y agoHow does the exchange charge for access? Per trade, per bandwidth, per colo distance? It would be simple for the exchanges to reign in trading by adjusting their pricing model. Make it more expensive to make each type of request. Adding a few cents to the request would squash HFT while not drastically effecting other trading systems/people.
- cube13 14y agoAll of the above, actually. The exchanges have a per-trade charge, and for the data, they charge fees on a per data stream basis, and charge for colocation. And they already do make each tier more expensive than the previous.
- skorgu 14y agoHere's Nasdaq's price list: http://www.nasdaqtrader.com/trader.aspx?id=pricelisttrading2 http://www.nasdaqtrader.com/trader.aspx?id=pricelisttrading2
- mileswu 14y ago>Each 56kb line with single hub and router (for remote disaster recovery sites only) Mm good old days of dialup. All yours for $900/month.
- deleted 14y ago[deleted]
- johnmacintyre 14y ago"abruptly ended at about 10:30 a.m. Friday" ... am I the only one who sees this as a probable programming bug? Mysterious algorithm? I suspect if(condition = true){...} // notice the single '=' :-D
- dfc 14y agoThe NANEX research page[1] has a lot more information about the different MOs that they spot. It looks like NANEX first spotted it on October 1st: "On October 1, 2012, we detected a new form of quote stuffing that tries to hide under the radar. It involves hundreds of stocks and millions of bogus quotes during the trading day. The algo that generates them is very careful to spread out the stocks targeted and limits each test to a blast of exactly 200 quotes in 25 milliseconds or less. The quotes all come from Nasdaq, and sometimes affect the Best Bid or Offer: the only change appears to be a fluttering of the bid or offer size."[2] Doing my best tptacek impression "the nut graf is": "We believe that this algo will continue to grow and if left unchecked, could very well contribute to the next flash crash because it removes precious network capacity and provides zero economic value such as price discovery." They posted an update on the 5th with a neat visualization of the trading activity[3]. If you just want to see a neat animated gif the direct link is: http://www.nxcoreapi.com/aqck2/Algo200.gif http://www.nxcoreapi.com/aqck2/Algo200.gif They have a list of the activity from October 1st[4], but it is not clear to me what the second and third fields are: Example: 09:01:03|P|9|UMDD 09:01:03|P|6|MVV 09:01:03|P|6|MIDU 09:01:03|P|4|IVOO 09:26:20|P|9|UMDD 09:28:44|P|7|SGOL The second and third fields have the following possible values: 13 4 N P Q 0 1 2 3 4 5 6 7 8 9 a A b B C D E F I was looking at the nxcore api docs [5] and I could not tell what these fields could be. The field values do not seem to match up to the listed values for exchanges or order types. [1] http://www.nanex.net/aqck/aqckIndex.html http://www.nanex.net/aqck/aqckIndex.html [2] New Quote Stuffing Algo http://www.nanex.net/aqck2/3610.html http://www.nanex.net/aqck2/3610.html [3] Quote Spammer Spotted http://www.nanex.net/aqck2/3614.html http://www.nanex.net/aqck2/3614.html [4] http://www.nanex.net/aqck2/3610%5C20121001.200.txt http://www.nanex.net/aqck2/3610%5C20121001.200.txt [5] http://www.nanex.net/apidocs.html http://www.nanex.net/apidocs.html EDIT: I am delighted and rather surprised to say that someone from NANEX responded to my inquiry about the second and third fields. "P = ARCA (we've since determined that the algo only ran on NYSE and Nasdaq listed issues) N = NYSE Q = Nasdaq The 3rd column is the SIP multicast line that would carry the quote. For Nasdaq listed, those are carried on UQDF which uses 6 lines and we label as A, B, C, D, E, F. For NYSE/ARCA listed those are carried on CQS which uses 12 lines, we label as 1-9,a,b."
- mcphilip 14y ago
- delinka 14y ago"Mysterious algorithm..." I suppose that's technically correct-- we don't know how the algorithm works. But they make it sound like they have no idea to whom the algorithm belongs. If your logging can't get you this information (the bot's owner), I have no hope for the security of the exchanges. If this kind of thing is a problem, then read your logs, visit the 'offending' party, and Do Something About It.
- dfc 14y agoRead your logs? The people that identified the algo are just looking at quotes coming in from different exchanges. What logs do you think they can look at?
- Jupe 14y agoSo, requests come into the trading system and there's no tracability? No credentials? No security? I'm not very religious, but heaven help us if this is the state of the Wall Street automation!
- yummyfajitas 14y agoThe SEC and individual exchanges have access to the full non-anonymized feed. Other market participants, such as nanex, only get the anonymized feed.
- theorique 14y agoEach participant is identified to the exchange (or ECN or ATS) by a MPID (market participant ID), but individual quotes on an exchange's limit order book are not tagged.
- rorrr 14y agoTrades are pretty much anonymous, only the exchange itself knows who the participants are.
- delinka 14y ago
- podperson 14y agoFor many years George Soros has been suggesting that a tiny tax on transactions would eliminate a huge amount of high frequency bullshit (he was suggesting it for currency markets, but the same applies here). I think he proposed that the revenue generated could fund the UN... It's about the same Bill Gates's suggestion of charging a penny to send an email -- no impact on legitimate use and a serious impediment to spammers.
- xenophanes 14y agoHow does charging me somewhere vaguely in the ballpark of $36.50 per year qualify as "no impact"? And I'm just an individual. What about when I send to an email list and it goes to 100 people? Should Google Groups pay a dollar to send out those 100 emails? What about MailChimp, do they have pay $200 when they send out a newsletter to 20,000 people? I'm also concerned about how this money will be collected and how, from a technical perspective, all emails will be monitored. Every SMTP server will now have to be registered with the central oversight organization? VPNs and rogue email installations will be the enemy or blocked from communicating with anyone "on the grid"?
- jon6 14y agoI suppose bitcoin could be used as the technical solution. Maybe receivers of money can white-list certain senders so they don't have to pay anything to send to them. Then only spammers and unknown people would have to pay.
- Evbn 14y agoYou may not be aware now, but postal mail is on fact charged per unit, no central registry needed, just a stamp.
- xenophanes 14y agoI don't think you understand how email works. Or the post office for that matter. There's actually one single organization that all those stamped envelopes get sent through, it's not decentralized.
- tantalor 14y agoMisleading headline; trades are not the same as quotes.
- mratzloff 14y agoSomeone help me out. Why can't they just throttle connections?