8 ms·
A new sort of hedge fund relies on crowd-sourcing
- deleted 9y ago[deleted]
- deleted 9y ago[deleted]
- nether 9y agoOne of Quantopian's studies shows that backtesting poorly predicts live trading results: https://blog.quantopian.com/using-machine-learning-to-predict-out-of-sample-performance-of-trading-algorithms/ https://blog.quantopian.com/using-machine-learning-to-predic.... From the forums, it looks like 90% of the effort is devoted to getting a good backtest, a yardstick which might not have any bearing on reality. How do real quant traders deal with this discrepancy?
- hodder 9y ago-Out of sample testing. Across markets, across time. -Logical economic explanations.
- akrymski 9y agoReal firms use tick data to backtest, not the 1 minute bars that Quantopian uses, and spend a lot of time simulating network characteristics such as data and order latency. They also use machine learning, which isn't possible on Quantopian, to build models, which requires downloading lots of data (and not just equities). No serious quant will ever use such a tool. You might as well go to the casino.
- akrymski 9y agoEdit: tick data is usually collected over time, because data sources have different characteristics, and you should be testing over the same realtime data that you'll be using to trade live. You need to know the bid/ask prices and volumes, in order to know where to place limit orders. Otherwise you are just paying commissions and spreads to brokers and market makers.
- deleted 9y ago[deleted]
- mabbo 9y ago> How do real quant traders deal with this discrepancy? A lot of it comes down to access to information and speed. All the major traders and quants have access to what effectively boil down to mailing lists (that their companies pay top dollar for) that have the best analysts in the world producing in-depth analysis on various topics[0]. Also, many/most have access to a Bloomberg terminal[1] to get the Bloomberg new feeds and the AIM clone, which costs $25,000/year/person. This is why amateur day-trading is silly. You're not competing in a level playing field- they know more than you do. [0]http://www.investopedia.com/articles/analyst/03/031803.asp http://www.investopedia.com/articles/analyst/03/031803.asp [1]https://en.wikipedia.org/wiki/Bloomberg_Terminal https://en.wikipedia.org/wiki/Bloomberg_Terminal
- arkis22 9y agoThere was a great article in the WSJ about quant "factors" >A former student of Mr. Fama, Cliff Asness, founder of quantitative hedge-fund manager AQR Capital Management, said he tries to avoid being caught out by false findings by trading on anomalies he can explain, economically or through investor behavior. To assess whether the market anomalies will continue, he looks for ones which carried on after being identified, can be seen in other markets or asset classes, and where minor changes to how they are defined don’t much affect the result. These include most famously value, momentum and corporate quality, among others. https://www.wsj.com/articles/an-algorithm-an-etf-and-an-academic-study-walk-into-a-bar-1494528113 https://www.wsj.com/articles/an-algorithm-an-etf-and-an-acad...
- thablackbull 9y agoHe also blogs for those interested - obviously geared more to the finance types: https://www.aqr.com/cliffs-perspective https://www.aqr.com/cliffs-perspective
- dia80 9y agoA Bayesian mindset. When investigating a signal you have to ask: Is it too good to be true? Does it make economic sense? Then simulate rolling optimisation and trading to see what returns could have been achieved with knowledge only available at the time.
- dnadler 9y agoWe are very mindful of overfitting our models, and very skeptical of backtests on their own. Successful strategies often have some fundamental theory about why they should work. If not, the backtest needs to be very robust, and demonstrate that it is not overfit by cross-validation, and other methods.
- deleted 9y ago[deleted]
- mmargenot 9y agoYou're right, backtest overfitting is a problem. At Quantopian, we spend a lot of time on education helping people avoid overfitting like this (https://www.quantopian.com/lectures#The-Dangers-of-Overfitting https://www.quantopian.com/lectures#The-Dangers-of-Overfitti...). It is easy to make a backtest look good by tweaking things here or there and that's just textbook overfitting. If you follow good statistical principles for model development based on a sound economic basis, the backtest is more of a sanity check to make sure that whatever signal you have found survives the introduction of market impact. If you have consistency between the statistical aspects of in-sample performance (your backtest) and true out-of-sample performance (paper trading on entirely new data), there as an increased probability that you have found a viable signal. Disclaimer, I work at Quantopian.
- chollida1 9y agoWe've been seeing these quantopian fund stores for years now. Where are the numbers? I mean at some point you have to produce right? https://news.ycombinator.com/item?id=12171843#12173142 https://news.ycombinator.com/item?id=12171843#12173142 https://news.ycombinator.com/item?id=12950276#12952666 https://news.ycombinator.com/item?id=12950276#12952666 https://news.ycombinator.com/item?id=12335272#12336086 https://news.ycombinator.com/item?id=12335272#12336086 THe biggest issue I see for a quantopian fund is the "skin in the game" rule. All of the successful fund managers I know have a very unhealthy portion of their net worth in their own funds. It's not clear to me that the quantopian algo designers will be able to, or be forced to, put their own money into their strategies. I find "skin in the game" to be one of the most promising alpha signals that still persists to date.
- dnadler 9y ago> THe biggest issue I see for a quantopian fund is the "skin in the game" rule. All of the successful fund managers I know have a very unhealthy portion of their net worth in their own funds. I feel like there's a very large survivor bias here. How many managers with unhealthy amounts of money in their funds lost it all? Having some money in your own funds certainly aligns manager interests with investors. It can, however, also lead to biased decision making by the manager. Not having skin in the game can actually lead to more rational decision making, and long-term success. I don't have any numbers to back this up (though I think studies have been done), but on the other hand, neither did you :P.
- deleted 9y ago[deleted]
- laxatives 9y agoI have a very simple algo (like 30 LoC) that is currently performed 9x S&P returns on >100k in initial capital. Granted, this is over a relatively short span of time and it doesn't observe the requirements to be part of Quantopian's contest. However trading live, it is doing significantly better than any back-testing I've done. There's nothing preventing designers from putting their own money in. Contest winners get profits in addition to whatever they invest themselves, on the condition they share the algorithm (I think closed source?) with Quantopian. That being said, Quantopian probably has enough information to reverse-engineer at least the main ideas of anything you share, given they can observe the orders made and probably also your network requests.
- creeble 9y agoUh, isn't "the market" just the ultimate crowd-sourced fund?
- jonbarker 9y agoThe market is like a giant voting machine but not really a fund, since nobody is managing it.
- wamsachel 9y agohaha I had the same thought when I saw the title, 'wait, that's just regular commerce right?'
- Houshalter 9y agoThe nice thing about companies like numer.ai, is they give you a clean nicely formatted dataset to work with. With data that isn't public and that cost lots of money to obtain. There are a lot of people who are good at machine learning, but wouldn't be able to participate in the market without a service like this.
- pishpash 9y agoIt might be an interesting idea to sample the market using this (e.g. to obtain behavioral economics signals), even though I don't believe this is their original intention.
- DennisP 9y agoNumerai is also doing this: https://numer.ai/ https://numer.ai/
- stcredzero 9y agoHere's my "Tom Clancy movie plot" evil fund. Someone starts a fund that's actually based on figuring out the portfolios of Senators, moderately high net worth congresspersons and other Washington insiders. However, instead of duplicating the entire portfolios of the whole populace, you first filter for IQ, an age range where people are starting to be very well connected but still making their fortunes, and develop another filter based on a quantitative proxy for riskiness of each investment. This is what the fund will really be based on, though there will be a fig leaf fictional method advertised. The purpose of this is to piggyback on the (technically-not) insider trading such people will manage to do while still staying within the letter of the law. (Somehow, the collective portfolios of US representatives manage to greatly outperform the stock market indexes.)
- RangerScience 9y ago> Somehow, the collective portfolios of US representatives manage to greatly outperform the stock market indexes. I would totally believe this. Do you have a source?
- echion 9y agoFor example: "Trading on Private Information: Evidence from Members of Congress" (2016) https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2712297 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2712297 EDIT: Also "Abnormal Returns From the Common Stock Investments of Members of the U.S. House of Representatives" (2011) https://www.cambridge.org/core/journals/business-and-politics/article/div-classtitleabnormal-returns-from-the-common-stock-investments-of-members-of-the-us-house-of-representativesdiv/BC6C6A524BBE96738BB94D37EF0FD1A5 https://www.cambridge.org/core/journals/business-and-politic...
- ithought 9y agoThe investor mentioned in the article is the basis for Axe Axelrod in Billions. Which is a wildly outrageous portrayal of him but probably has a lot of elements of truth (eg extreme paranoia) since he was clearly guilty of insider trading (his subordinates indicted, his firm charged but not him).
- inthewoods 9y agoI'll repeat what I've said before about these platforms - I don't see why any institutional money would ever flow to them. Most startup funds require at least 5 years of audited history before any capital allocator will touch them. They might get some small quick money from fund-of-funds, but I think they'll be hard-pressed to keep it. Given the relative short history on these platforms and the likely churn in algos, I can't see anyone putting serious money into them. One of the other problems I see with these platforms is that there is nothing that stops me, as far as I can tell, from pulling my algo from that platform once it is successful. Now there are lots of reasons why I could see people not doing that (running your own fund is hard, raising capital is difficult), but I don't see why anyone that is successful wouldn't immediately exit the platform in order to maximize their return.
- jzwinck 9y agoAll automated trading technology providers suffer this paradox: failing customers will fail, excellent ones will profit and move on to develop their own platforms. It's unavoidable. You must continually attract new customers and hope some are successful for your own reputation.
- Houshalter 9y agoFor numer.ai you can't leave. The data you get is private and encrypted. So even if you come up with a great algo, you depend on them to do anything with it.
- klipt 9y agoUnless some other fund buys the same expensive data?
- deleted 9y ago[deleted]
- quantgenius 9y agoThe problem with Quantopian, many current robo-advisors (including some with large valuations) and other market-related fintechs is that by and large they don't seem to have anyone who has had real success actually trading automated quantitative strategies at a serious hedge fund or tier-1 proprietary trading group on their founding teams. I've seen successful VCs, well-known academics, market gurus, people with a background in some aspect of running a mutual fund and all manner of other people who seem like they should be good, but nobody with an actual track record. I've seen people who worked on technology at hedge funds but the technology group at a hedge fund builds what amounts to plumbing like clearing, reporting etc, not the actual trading technology, certainly nothing that can actually impact PNL. Jonathan Larkin at Quantopian comes closest to what is needed and since he joined Quantopian has certainly had good success, but even his experience was more along the lines of recruiting and risk-managing portfolio managers, not actually running a large book. He certainly helps Quantopian but Quantopian is coming from a place where when it was founded, I personally had to explain what selection bias meant to John Fawcett and despite Jonathan Larkin being there, they still seem to be making some pretty basic mistakes in how the platform is setup. Spending a few years working on a tier-1 automated trading desk is absolutely essential because, what is deployed at those firms (and what you are competing with) is years if not decades ahead of academia and the rest of the industry and you learn more in a week of working on a successful trading desk (which only happens if you demonstrate a lot of not just academic aptitude), with people sharing knowledge available nowhere else than in a decade in academia or anywhere else, even other groups at the same firm, potentially sitting 10 feet away from the trading group. I'm not suggesting people steal IP or anything like that but you do have to have sone sense of what the state-of-the-art actually is if you are going to claim to have developed something state of the art. I suspect it's going to end up like the search space, where the space will be taken over by the second generation of firms that nobody has heard of who decide to do things differently from how investment management is currently run offline taking advantage of their knowledge of how mutual funds including index funds are picked off by sophisticated traders. Interestingly, Igor Tulchinsky at Worldquant and his team who are a tier-1 trading shop have basically been running a very successful version of what Quantopian hopes to become without a lot of hoopla or publicity for years, decades if you include the time they were doing this as an independent team at Millenium.
- theprop 9y agoI think I found a new interview question: write an algorithm that outperformed the market for the past ten years...you have 35 minutes...
- dx034 9y agoThat's easy and doesn't take 35 minutes. Here's the algorithm: If year < 2009: short stocks else: buy stocks Add some leverage to this and you made a lot of money. Doesn't mean it'll work in the future, though.