6 ms·
Unsupervised machine learning with basket clusters
- rushabh 9y agoSo the author was able to beat S&P500 by 10% over a period from June-2016 to June-2017 using this solver. The most important question is whether the same underlying relationships will hold true for 2017-18. This seems like a classic example of hindsight. There are many things one can tell on hindsight when the results are out. The key is if they will hold for the future too. Am I missing something?
- justadeveloper2 9y agoIt doesn't matter if you can do it once. You have to do it consistently over years, which is not possible. I know about Renaissance Technologies, but I suspect they have insider information or some other angle and it's not all about their algorithms.
- eximius 9y agoThat is a rather large accusation. Can you restate that as something constructive or helpful to the conversation?
- justadeveloper2 9y agoDo you understand investing at all? People have been attempting to "beat the market" for decades and nobody has been successful. There is always the "reversion to the mean" issue in every case. People with algorithms think they are beating the market then something changes and their algorithm no longer works. Except for Renaissance--they have consistently been beating the market, or so it appears. Does that help?
- alextheparrot 9y agoHe simply asked you to backup your claim, which you don't seem to be able to otherwise you would have. Throwing around a "yeah they can do it, but they must be cheating" isn't exactly constructive.
- justadeveloper2 9y agoDo you know the story of LTCM? If not, do not lecture me on this subject. I'm sure in time I'll be proved correct. I'm sure you've heard of Bernie Madoff and how he "beat the market" for so long, yes? I don't have to "back up my claim" the math does it for me already! You're asking me to prove a negative--instead, show me one example of a firm that has beaten the market without resorting to funny business and I'm all ears.
- srean 9y agoI have no dog in the fight, but generally it is the extraordinary claims that need backing up. Consistently beating the market would indeed be an extraordinary claim, because we havent seen that a lot (if at all).
- justadeveloper2 9y agoAlexTheParrot--then home come they are the only one?
- alextheparrot 9y agoI'm unsure what to make of this.
- dajohnson89 9y agoNo, it doesn't help all. You have yet to provide any evidence for your accusation of insider trading (or "something else"). You can't really accuse RT of wrongdoing, just because you don't understand their methodology. Also -- if I were a quant that managed to beat the market consistently, I would shut up and go straight to RT, for two reasons. To avoid toxic dubious comments like yours, and because RT can pay me better than your standard hedge fund.
- sidlls 9y agoI didn't interpret it as an accusation. I interpreted it as he wrote: a suspicion that something other than RT's algorithms is a non trivial contributor to their success. He gave an example of insider trading, but it could be something else ranging from the equally sinister lying about outcomes (e.g. by creative accounting of what counts as a return) to the less sinister better access or timing to information anybody else has that makes the algorithms more effective and far less relevant, and many other things. His other point about the rarity of regularly beating the market certainly supports some amount of skepticism of RT, in my view. It's extraordinary, and it's an extraordinary claim that suddenly the particular talent at computational trading at RT is better than the legion of super genius other quants and techies who have been doing this for years. Disclaimer: I haven't a dog in the fight with respect to supporting or taking RT down. As I see it, being a small time player, my trading strategies are necessarily constrained to the point where RT's methods, legit or not, aren't applicable. So whether they win or lose doesn't make a difference to me.
- omgwtfbyobbq 9y agoThey, and other hedge funds, apparently classify their earnings in a way the IRS objects to. http://www.reuters.com/article/usa-irs-hedgefunds-idUSL3N0ZP3D320150709 http://www.reuters.com/article/usa-irs-hedgefunds-idUSL3N0ZP...
- 11thEarlOfMar 9y agoIt needs to be tested in both bull and bear markets. His time period was all bull (S&P +22%). I've been beating the S&P on a dividend play, but it's all in an up market as well, and the upward movement may be amplifying my choices on the positive side. A down market may have the opposite effect, and amplify on the negative. Also, I may have just hit a market rotation at the right time, which is not repeatable for me (since I don't know how to predict it). So I don't trust my results unless/until it also outperforms in a down market.
- amrrs 9y agoThis is very much of how NNT taunts some reporter trying to explain the causality of Intra-day fluctuations of a Share. Humans obsession with establish wrong attribution in an attempt to express their domain expertise never expires! Now Data Science fuels it!
- IshKebab 9y agoYeah an the fact that he starts the post with "I finally beat the S&P 500 by 10%." suggests he has basically just tried a lot of techniques on historical data until he found one that worked. Almost certainly luck.
- PLenz 9y agoI can't help but feel that this is noise multiplied by noise. Regression to the mean ought to he coming up real soon.
- leereeves 9y agoIf this approach works, is it wise to be sharing it publicly? Is there a danger than whatever gains you were able to capture with a unique insight will end up being shared with competitors?
- lightbyte 9y agoSharing it publicly is the author saying it does not actually work as he claims it does. You don't write a blog post on a method you created to make guaranteed money, you use it.
- scottlegrand2 9y agoA couple years back there were some traders talking about trying to implement a really fast version of K-means for clustering stocks that are transiently correlated on some sub-reddit. I wonder if this guy is the one who started that thread.
- wonderwonder 9y agoI would argue that this scenario would actually be a positive for the author. If others start using the same methodology as him on a large enough scale and trading on the same data sets they would buy when he buys driving prices up. On the other hand when he starts selling and others sell as well speed is a critical factor as the downward pressure will make the later movers the biggest losers.
- mynewtb 9y agoIt's just SEO.
- justadeveloper2 9y agoHere's how this always goes: Someone thinks they have a killer algorithm and it works and makes them money...for a time. Then it suddenly doesn't work anymore and they lose more than they gained. It is not possible to beat the market over the long term unless you are trading on insider information, somehow manipulating markets, front-running orders, or have some other edge that is not universally available. There is no Python script that will simply print money for you year in and year out it doesn't matter what data you feed it or how smart your AI is.
- ikeboy 9y agoI don't get the strategy here. What's the investment thesis? Are they collecting news and trading on it? It doesn't seem to mention that.
- square90 9y agoI'm skeptical of the starmine dataset[0] being used: "The dataset is based on relationships between elements in the periodic table and public companies." This description is a bit vague. [0] http://starmine.ai/datasets/dataset_builder.html http://starmine.ai/datasets/dataset_builder.html
- zenkat 9y agoI'm having trouble deciding if this whole post is a sly commentary on how easy it is to get machine learning wrong. Beat the market by 10% with k-means clustering and a feature set derived from companies and chemical elements! Hahahaha no.
- scottlegrand2 9y agoI know, right? Now mapping it to the sequences coming from the numbers stations? Totally different story!
- alextheparrot 9y agoHere's my understanding of what's going on: He has a dataset that links companies to each element of the periodic table (Intel would likely be highly linked to silicon, while not as linked to nitrogen as a fertilizer company). He then uses some clustering and whatnot to produce groups of stocks he is trading as a unit. How he picks what to trade is kinda hand wavy (Is he trading all baskets and only reporting the good one?). He also mentions using Google trends data, which might be some signal for his clusters. Given the lack of detail, my gut reaction is to assume he made tens of these strategies and has kindly reported the best one to us. At the end, you see he is a financial consultant hoping to build his resume up through posts like these.
- landtuna 9y agoIs this not a parody, making a joke about how sensitive to supervision "unsupervised" learning methods are?
- scottlocklin 9y agoIf it's not a parody, it's one of the most unintentionally hilarious things involving data I have ever seen presented. To anyone who takes this seriously: I have a price of butter in Bangladesh indicator which works REALLY WELL on the S&P500.
- drtillberg 9y agoThis is systematic semi-blind guessing at what other people will guess in the future are good investments. It sprays investment capital equally at companies that are not in any way equivalent. Hypothetically, just because two companies both have names that begin with "Z" and sell product on Amazon, that is not an investment thesis for treating them alike. Treating them alike is a recipe for misallocation of capital, in the long run, which is what it seems sometimes we have in this economy in spades.