7 ms·
Algorithms take control of Wall Street
- chrisaycock 16y agoThis article is very poorly researched. For example, it refers to high-frequency traders as "flash traders". I have never heard that term before. The cited Wikipedia article discusses flash orders, which was a quoting mechanism that DirectEdge experimented with in 2009. Flash orders are not some most automated market markers even care about. Also, the article mentions Harold Bradley as "among the first traders to explore the power of algorithms in the late '90s", completely omitting much earlier pioneers such as Peter Muller and David E. Shaw. And of course it goes into depth about the Flash Crash without once mentioning Waddell & Reed, a discretionary (non-quant) fund whose errant trade started the crash to begin with. Reading this was like watching the 11-o'clock news discuss a medical discovery.
- w1ntermute 16y agoNothing unusual then. Pretty much every tech article in the WSJ is of this caliber.
- atuladhar 16y agoExcept that I see no indication this is a Wall Street Journal article. The site is Ars Technica; the by-line says "By Felix Salmon (wired.com), Jon Stokes" and the end of the article says "Felix Salmon (felix@felix salmon.com) is a blogger for Reuters."
- chrisaycock 16y agoI think what @w1ntermute means is that we can expect Ars to screw-up finance stories just as the WSJ screws up tech stories. Kind of an admonition for when journalists choose to venture beyond their expertise.
- w1ntermute 16y agoYeah, sorry I didn't make that clearer. I made the comment while this[0] poorly researched/written WSJ tech article was still on my mind. 0: http://news.ycombinator.com/item?id=2086738 http://news.ycombinator.com/item?id=2086738
- mediaman 16y agoThat's odd; Felix Salmon writes almost exclusively for the financial industry, and used to run his own blog. Reuters hired him for his finance industry audience. It's surprising that he would make elementary errors regarding his industry of primary coverage.
- ggruschow 16y agoIt's easy to be wrong when it's so hard to make out what's right. Algorithmic traders are unusually secretive about their modes of operation. The primary knowledge dissemination mechanism is via employees moving.
- jrockway 16y agoUh, but the Wikipedia article on the topic has the relevant information in this case.
- Dn_Ab 16y ago>Also, the article mentions Harold Bradley as "among the first traders to explore the power of algorithms in the late '90s", completely omitting much earlier pioneers such as Peter Muller and David E. Shaw Don't forget the super pioneering work that Claude Shannon and Ed Thorp were doing way back in the 60's and 70's too!
- jfager 16y agowhich was a quoting mechanism that DirectEdge experimented with in 2009 Could you go into more detail about what you mean by that? Unless you're being pedantic about something specific, I don't get how you could say that. Flash trading (in several variations) is basically the only thing Direct Edge offers, and they currently carry roughly 12% of the market's daily trading volume.
- beagle3 16y agoEverything else you said was spot on, but this is wrong: > And of course it goes into depth about the Flash Crash without once mentioning Waddell & Reed, a discretionary (non-quant) fund whose errant trade started the crash to begin with. Waddell & Reed had nothing to do with the big flash crash, nor with the 10 slightly smaller crashes that has happened since -- at least, not any other non-HFT trader had that day. If you want something which isn't some ridiculous CYA, read Nanex's analysis, not the SEC.
- ct 16y agoThe neural network that Bradley was working on sounds interesting.
- aothman 16y agoNB: This article (and several more like it) was in January's wired. As an AI researcher, I get suspicious when I see anyone talking about Genetic Algorithms and Neural Nets. These are techniques that current researchers simply do not use (Neural Nets are used very sparingly, GAs should never be used at all). They make up for their technical failings by being approachable, particularly for journalists. In short, these methods intuitively sound like they should work much better than they actually do.
- Eliezer 16y agoI was wondering if anyone else had that reaction. I'm looking at a story about how Bradley was using elaborate GAs to choose the weightings on a small handful of variables that, from the description, sounds like they should've yielded to a regularized linear regression producing a fast cheap optimal answer; and I'm thinking to myself, "Either the journalist completely failed to understand and/or convey the problem that was being solved, or nobody designing this system understood anything about statistical learning and were just throwing cool-sounding tools at the problem, or they were just having fun at the reporter's expense." For those unlearned in AI, an analogy: the original article reads like someone talking about how they used Haskell monads to add 2 + 2.
- istjohn 16y agoWhat techniques are now used in place of GAs?
- aothman 16y agoGAs are just a really complex version of a local search algorithm. The problem with them is that they're just too complicated - you're trying to solve some non-linear problem, and your first step is to introduce several more non-linear problems that also need to be solved (picking chromosomes, mixing, population size, etc.)?
- leogau 16y ago"Even if each individual algorithm makes perfect sense, collectively they obey an emergent logic—artificial intelligence, but not artificial human intelligence. It is, simply, alien, operating at the natural scale of silicon, not neurons and synapses. We may be able to slow it down, but we can never contain, control, or comprehend it" I can see algorithms being used to automate decision making in almost every other industry, not just Wall St. It's both scary and exciting to see what happens with the resulting "emergent intelligence".
- jacques_chester 16y agoThe systems dynamics folks point out that systems with more rapid feedback often have more variability. The classic example is a car dealership deciding to order cars from the factory. When should they be ordered, and how many should be ordered? Suppose they decide that every day they place orders based on the 7 day moving average. Then they get a busy day and over-order for a week. That's an expensive mistake. Next they decide to shrink the window to 3 days (more rapid feedback). Now after another busy day they find stocks dropping precipitously and order big to make up, follow by drops, followed by big orders. In an attempt to stabilise the system, they've made it worse. The modern stock market has extremely short feedback times, on the order of microseconds. When a feedback loop forms on the market it can spiral out of control within seconds and minutes -- hence the 'flash crash'. Two ways to deal with this might be to develop some kind of balancing feedback loop (for example, higher prices for more trades-per-second, or a progressive price for trade based on delta with the last trade) or to reduce the feedback rate. An economics blogger I host, Nicholas Gruen (you may know of him as Australia's Gov 2.0 inquiry leader), has suggested just that: http://clubtroppo.com.au/2010/08/07/a-modest-proposal-to-remove-the-more-ridiculous-waste-and-some-corruption-from-our-financial-markets/ http://clubtroppo.com.au/2010/08/07/a-modest-proposal-to-rem... http://clubtroppo.com.au/2010/10/08/a-self-denying-ordinance-for-exchanges/ http://clubtroppo.com.au/2010/10/08/a-self-denying-ordinance...
- joshu 16y agoOr to do more orderly crossings at regular intervals. The market makers (and shadow market makers) on the ECNs get paid for providing liquidity, rather than having to pay. This kind of stuff isn't supposed to happen on the NYSE, as all the trading goes through a single specialist.
- Sniffnoy 16y ago> "Automated trading systems will follow their coded logic regardless of outcome," Of course, so will humans. They won't make the absurd trades referred to, but a system of trading humans can still, say, get stuck in a positive feedback loop. Of course, they aren't pure trading algorithm, so they may notice the problem, but actually putting a stop to it requires coordination and is hard. Mostly the loop just occurs more slowly.
- chrisaycock 16y agoYep, which is precisely why economic bubbles happen. Algo traders weren't responsible for housing prices in the last decade.
- extension 16y agoComforting to know that the economy is being gambled on a roshambo programming competition. http://webdocs.cs.ualberta.ca/~darse/rsbpc.html http://webdocs.cs.ualberta.ca/~darse/rsbpc.html
- jrockway 16y agoWhy do you think this is true? For every trade, there is someone on each side. The market as a whole does not care if you make bad trading decisions. Someone else makes up for it.
- extension 16y agoI think our money needs to be working towards things that we actually care about: advancing science and technology, creating art, socializing, and so on. Putting it all into a big game of craps is not going to improve the human condition in the long run. I mean, it's an interesting game, but not interesting enough to justify the entire economy revolving around it. And people don't play because they find it interesting.