8 ms·
Quantopian Brings Algorithmic Trading To The Masses
- undrcvr 14y agobeat me to it, working on something similar... with a twist. btw they're on github : https://github.com/quantopian https://github.com/quantopian
- jbredeche 14y agoHi undrcvr, I work at Quantopian. We'd love to hear your thoughts on our open-sourced Zipline backtester: https://github.com/quantopian/zipline https://github.com/quantopian/zipline and learn more about what you're building - there are lots of interesting challenges in this space.
- monksy 14y agoHow do you guy handle situations where there may not be a trade that day. For example: with OTCBBs. Also, how would you handle unexpected market closures [such as the flash crash?] I've been developing my own back tester for quite a while [in Java] and I must say: You guys are doing some interesting stuff and its a challenging area.
- fawce 14y agoOur backtester is event driven. During any closure, there are no events, so we simply "fast forward" through the closure and send your algorithm the next available event. We provide a facility for working with trailing windows as pandas dataframes, which are updated by the events. You can control whether those trailing windows have NaN values for missing bars, or if values are filled forward. You'd keep the NaNs if you want your algo to be aware of empty bars (stock is held, thinly traded, etc). You keep the fill forward if you want to avoid coding guards on NaNs :).
- undrcvr 14y agoIndeed there is lots that can still be done. Not going to share anything here, but don't worry, I don't believe in backtesting so I'm not implementing it =)
- steve8918 14y agoIt's an interesting idea, but most successful retail algo traders probably won't trust their bread-and-butter algos to a cloud-type solution. I've been algorithmically trading for a few years now, and I've invested money into Ninjatrader, where I program my algos and run things from my own computer. If anyone had their own algos, they would probably be too paranoid that Quantopian would run backtesting on every single algo, and cherry pick the best ones for themselves. Whether or not it's true doesn't matter, it's most likely the common thought process that any successful algo trader would have. That would leave only the inexperienced and beginning traders that would be more apt to fail, since algo trading is very, very, very hard.
- fawce 14y agoTrust is essential (http://blog.quantopian.com/on-trust/ http://blog.quantopian.com/on-trust/). We see trust as something we earn over time with transparency and openness. We even open sourced our core intellectual property, the backtester itself: https://github.com/quantopian/zipline https://github.com/quantopian/zipline We also believe setting our own incentives to match our members' needs is key to sustainable trust. That's why Quantopian doesn't invest its own capital - we want to be purely focused on and motivated by serving quants. Being a successful quant takes many things - talent, mentorship, access to data, great systems, discipline. We started with backtesting because many aspiring quants end up skipping rigorous backtesting because of the time necessary to develop the test harness. We focused on community from the start to help connect new talent with mentors. If you're not ready to trust us with your algorithms, I hope you'll trust us with a bit of your time. Come share some of what you've learned over the years with our community.
- brooksbp 14y agoDo you have any book recommendations? Technical analysis? Quantitative analysis? What makes it so hard--trying to come up with a good model?
- dunster 14y agoI'd argue that a lot of the hard parts of algo writing are solved by Quantopian. Hard: * Data. You need to test your idea. Most historical stock data (like Yahoo) excludes companies that went bankrupt or were bought or otherwise disappeared. That's called survivorship bias. If you run a backtest on the finance industry and you don't include things like Lehman, you're going to get the wrong answer. Add in things like Hurricane Sandy, MLK Day, 9/11, mergers, acquisitions, stock splits, etc. and data is very painful to put together. * A backtester. Once you have you data, what do you put it into? How do you calculate commissions? How do you calculate slippage (your order affects the price, remember)? How do you avoid look-ahead-bias and other bugs that plague backtesters? Coming up with an idea to trade is hard, but it's only a part of the problem. I'd say it's the most fun part of the problem, but it's only a part. Quantopian is trying to remove all of the hard parts and let you do the easy parts. We have tens of thousands of lines of code (backtester, IDE, etc.) and we're leaving the most exciting 100 lines of code to our members. On the other question about books. I'd recommend a couple: * Ernie Chan's book is a great place to start http://www.amazon.com/Quantitative-Trading-Build-Algorithmic-Business/ http://www.amazon.com/Quantitative-Trading-Build-Algorithmic... * More advanced: http://www.amazon.com/gp/product/0470128011/ http://www.amazon.com/gp/product/0470128011/ I work at Quantopian.
- hnwh 14y agogreat work guys..
- niggler 14y agoQuantopian suffers from a few problems (not unlike Access or the other 4GLs): 1) platforms like thinkorswim get you 90% there (and have an associated brokerage so you can also run it). You can write powerful trading scripts using a wide variety of signals, but for the last 10% you will need something more powerful like C or matlab or excel. 2) terms of service are always shaky -- if you have alpha, you want to guard that like a first born child. 3) other people (http://tradingphysics.com/ http://tradingphysics.com/) offer market data at very low prices, which is far better than trusting a third party with your code 4) oftentimes strategies don't directly translate to production profits, and to a great extent they depend on simulator assumptions (let's say that there are 10K shares offered at the best selling price what happens when you try to buy all of them? What happens if you are stopped due to RegNMS or some other oddity? What happens if the quote is fake or will be canceled by the time your order reaches the exchange -- a common tactic of Chicago firms like GETCO?) for which you have no control. But now, given that they don't have a BD license, you need to go through someone else (adding another layer and process that potentially could cause problems later on -- I've seen a similar situation happen where a person leaves one company to join another, only to find out his trading strategies don't work in the new place ) Tl;dr: there are better platforms for getting your feet wet, and if they aren't good enough you are better off going with a real solution.
- ryanslade 14y agoBoth the sites you list seem to be US based. Are there any UK based solutions?
- specialdragon 14y agowww.timetotrade.eu
- undrcvr 14y agoIf you have alpha your broker will reverse engineer it until their last breath...
- yummyfajitas 14y agoWhat happens if the quote is fake or will be canceled by the time your order reaches the exchange -- a common tactic of Chicago firms like GETCO?) I'm curious - which FIX command is used to send a fake quote? And which exchanges support this feature? Also, how does GETCO cancel an order before your order reaches the exchange? I.e., how does GETCO know to send it the exact instant before you send yours?
- SeanDav 14y agoCall me cynical but this is a bad idea, and not for Quantopian either. It is well known that only a small fraction of traders ever end up making money consistently. Effectively Quantopian get a vast and relatively cheap set of researchers/traders to develop strategies and they can simply cherry pick and front run the most successful. This means that the actual developers of the successful strategy will quickly be muscled out. Of course Quantopian would deny they would ever do this but they have full access to all your code and results and can do what they like with it. You really think a "Wall Street" company is going to be happy with your $nn a month subscription fee when they can see you are running a strategy that when scaled up a bit could make so much more?
- yummyfajitas 14y agoThis argument applies to any brokerage, and is why brokerages are highly regulated and subject to extreme SEC scrutiny. Once Quantopian offers you a way to trade (either by becoming or partnering with a brokerage), they will certainly be subject to the same rules. Incidentally, "cherry pick...the most successful" is not that easy. In a universe of many traders, a few will succeed solely by random chance.
- SeanDav 14y agoRegulated by the SEC does not mean much at all. All the major trading scandals have occurred in institutions regulated by the SEC. Trading institutions can do pretty much what they want - even today there is a huge amount of insider trading and market manipulation despite all the regulation. For example the latest LIBOR scandal. Not sure I agree with your second point either. It is the same point made by Nassim Nicholas Taleb in his book Fooled by Randomness. That theory works fine if you think markets are random. They are are not, they are just hard to understand. Having said that I agree that in the very short term it is impossible to separate a strategy making money randomly from one that is fundamentally sound.
- NateDad 14y agoIt's 8 guys hacking in a small office in Boston. Hardly a "Wall Street" company. (disclosure: I've worked with half the team before... and they're all the best devs I've ever worked with. I do not work for Quantopian.)
- Shenglong 14y agoThere's a lot of criticism here, but as someone who has never done algorithmic trading before - it's very exciting. I don't believe Quantopian's intention was to target veterans, since I'm sure you vets have your own system for testing and trading already worked out. As a newbies with no experience in the field at all, it is unlikely that I would've even tried without a platform such as this. If Quantopian's intention is to bring more people into the field and get them exposure, even now, it's quite an impressive stepping stone.
- confluence 14y ago> As a newbies with no experience in the field at all, it is unlikely that I would've even tried without a platform such as this. If Quantopian's intention is to bring more people into the field and get them exposure, even now, it's quite an impressive stepping stone. This assumes that people with an inability to get experience should get into a field where one can lose one's shirt in less than 24 hours whilst competing against some of the smartest people on Wall Street. There are barriers to entry to various things for a reason - e.g. driving, steel milling etc. It's to stop newbies getting shot.
- Shenglong 14y agoYou can lose all your money in so many ways; a reasonable expectation is that players will use caution, and hedge their risks. Applications aren't meant to teach common sense.
- confluence 14y agoThis is not a good idea. People can barely invest in regulated securities doing a dipshit buy and hold strategy. For professionals - brokerage houses already do most of the hard work and provide decent APIs - so they don't need it either. Algo trading is cool - but it's most definitely NOT for the masses.
- gozmike 14y agoI'm happy for Quantoplan making steps to improve the user experience around algorithmic trading, however armchair traders and many hedge funds have been using platforms like Tradestation (and dozens of others) to back-test and develop algorithmic strategy for well over a decade. I spent a year working as a researcher for a now-failed hedge fund (failed due to regulatory issues, not performance, we were doing 20% year over year on commodity futures). As an engineer/math guy, it was an incredibly interesting experience because it opened my mind to all sorts of theoretical possibilities and explorations of pattern matching, noise filtering, exotic concepts like wavelets and more. However, I quickly learned a few things from experienced traders and from seeing my work move from testing to prod. What I learned makes me extremely hesitant to employ automated trading systems on my own money: 1. Historical back-testing is a great way to curve fit. You can hyper optimize your algorithm looking for arbitrage opportunities, trends, whatever. You'll get performance reports that make it seem like you're ready to print money. Then you get out and trade and discover that your system can't keep up with market movements because the indicators that you relied upon may have exhibited correlation but not causation. 2. The boon of algorithmic trading is that it attempts to remove emotion from the trading process, not that it is a better predictor. Listening to a machine should help alleviate the symptoms of "fear and greed" that lead to abrupt, incorrect decision making. Think about that for a minute, some hedge funds advocate algorithms not because of predictive power but as guarantees of rational decision making. 3. Conversely, while developing and testing a system, a smart person will almost inevitably try to bring in exotic concepts into price prediction, order sizing and trend following functions. Given enough time, complexity will increase until it becomes challenging to understand the rationale behind a system's output. Trading this way is scary because real money is being moved without an understanding of fundamental and macro-factors. 4. You will very likely lose money. Even at the size of our fund (1B under management) we were sometimes at the mercy of market makers who gave us crap prices on trades or seemingly manipulated prices to hit our stop orders and cause us to exit positions too early. I love seeing the ideas behind algorithmic trading popularized, however I want to make sure that anyone embarking on it understands the market as a system and not just as a time series to be modelled. It's composed of real human beings, with emotions running wild. If you decide to play, then play, but do so wisely and carefully and remember to keep it simple.
- gfodor 14y agoThis will be cool if/when it has: - options - futures - fundamental company data Until then, it's matlab. edit: Ok, this is a bit harsh. It's already pretty cool.
- hitechnomad 14y agoI will be releasing my own algo trading system open source in Feb, if anyone is interested. I'll do a post on HN once it's released.
- undrcvr 14y agowould you share a twitter link so we can be alerted when you release?
- hitechnomad 14y agoI'm @hitechnomad on twitter. I'll tweet once it's on github. The system I'm open sourcing is my own system for trading on my Interactive Brokers account. It should be able to handle any FIX-based exchange/brokerages. Currently finishing a contract at a bank, and I can't trade on my personal account while I am doing this contract, which is why I am waiting until Feb to post it.
- crucialfelix 14y agoI use IB too. What language is the framework in ? Java ?
- hitechnomad 14y agoc++
- n3rdy 14y agoWouldn't you need a broker account with minimal spread and very low broker fees for this to work? The banks who are already doing high frequency trading don't have to worry about this, they can profit off a trade after its gone up a fraction of a point or so and close the trade, your average trader has to make up for the transaction fees or spread which are sometimes 10 to 100 times what a bigger firm has to cover.
- randomsearch 14y agoIs algorithmic trading a force for good in the world? If not, is it a good idea to increase the number of people using it?
- jik 14y agoHFT isn't. We're not doing HFT. We're doing algorithmic investing. There's a big difference. (I work for Quantopian.)
- fawce 14y agoAlgorithmic trading is a force in the world; whether it is good or not depends on the people wielding it. I think the danger isn't the technology itself, but the small number of people who understand, develop, and control it. I believe having more quants will give algo trading/quant investing a better chance of being a force for good. (I work for Quantopian)
- jhales 14y agoGreat stuff.
- muyuu 14y agoI'm a bit surprised about the amount of people afraid of trading in HN. Or rather afraid that others would trade en-masse.
- dear 14y agoWhich broker-dealer do you use to route your orders to the exchanges? Do your orders flow through Getco's infrastructre at all?
- racecar789 14y agoGreat job. Surprised at the amount of negativity for an app that is meant to help people. This is one of those disruptor-style apps that pierces the veil of an industry and brings professional level tools to the masses. I sat up from my chair after seeing this and said "wow". Have not done that for an app in a long time.