10 ms·
Hedge funds embrace machine learning up to a point
- deleted 9y ago[deleted]
- matchagaucho 9y ago"Share-price time series going back decades still contain far less information than, say, the image data used to train Facebook’s facial-recognition algorithms." What? Clearly these ML models aren't trying hard enough. Correlating stock movements with the weather, geo populations, ... there are potentially infinite patterns buried in the data that a hedge fund could uncover.
- deepnotderp 9y agoThis is a fluff article and they know it, plenty of major quant firms analyze satellite images with neural nets to make trades.
- nsebban 9y agoDo they do that ? And why exactly do they do that ?
- mathperson 9y agoI can pay for high resolution images of every walMart in the US- Use math- estimate foot traffic- estimate average sales per person- I can estimate profitability of Walmart. Let's say it's a good quarter. I buy a bunch of Walmart stock - they then disclose quarterly earnings- investors see the good numbers- the share price goes up. My position is long-I sell. Profit. edit: obviously many steps of this process can use machine learning in general and deep learning in particular
- junkscience2017 9y agono you can't. take the absolute highest resolution images you can buy, and take 100 of the best mathematicians and using only maps you won't even be able to guess within 20%...analysts can already estimate within 1% of expected earnings
- forapurpose 9y ago> take the absolute highest resolution images you can buy, and take 100 of the best mathematicians and using only maps you won't even be able to guess within 20%... Why is that?
- mathperson 9y agothat example was more hypothetical of possible uses of satellite images. idk whether or not its too coarse to be a profitable signal https://www.cnbc.com/id/38722872 https://www.cnbc.com/id/38722872
- cocoablazing 9y agoWalmart’s last quarterly EPS beat analyst consensus by >3%, according to an aggregation by the WSJ.
- QML 9y agoWouldn’t it be easier to create an incentive for customers to directly give you data about where they shop? Like make an app that pulls up coupons specifically for Walmart..
- hu3 9y agoFor starters satelitte images can help infer crop farm size and health and use it as one of the many signals to predict corelated stocks.
- aaavl2821 9y agoi think some firms buy analyses of imagery to predict oil market movements by looking at whether oil storage containers are empty or full. could probably imagine doing similar things in other commodities markets (corn etc)
- tigerBL00D 9y agoYes, but not necessarily to trade stocks. There is a more direct use of this data - to predict demand.
- aaavl2821 9y agonot to trade stocks necessarily but to trade commodities
- bfrink 9y agoThere are about 30,000 institutionally-investable stocks in the world. They have ~252 closing prices per year. This is the thing you're trying to predict, so it forms the outputs of your training set (be sure to hold some back for out-of-sample testing and there are probably distinct regimes you need to switch between). Yes, there may be lots of features (weather, geo populations, etc., as you point out), but you're often hard up against the curse of dimensionality here. Sure, intraday data is more voluminous, but there are fewer economically reasonable influences on that.
- mathperson 9y agoAlso commodities, forex...
- lordnacho 9y agoThat only adds a few dozen items. There's only so many currencies and commods, bonds. Even if you include really illiquid stuff maybe you get 150 things.
- JumpCrisscross 9y ago> There's only so many currencies and commods, bonds Currencies are traded in pairs. Bonds are numerous, if illiquid off the run. Not to mention the bursting menagerie of derivatives our species tends to.
- lordnacho 9y agoHighly correlated. I mean you can add option surfaces, but to count each option as separate is a stretch.
- snowAbstraction 9y agoSome people are already trying and succeeding. And since a lot of these signals already incorporated into the market prices of various financial instruments, you can apply deep learning techniques to these available prices (and time series) to get similar results, to make buy and sell decisions/algorithms.
- IAmGraydon 9y agoYes, but the quote is talking about “share-price time series.”
- Cacti 9y agoJust 32x32 image space is, like, larger the the atoms in the universe. Even per-tick financial data is quite small in comparison. Or, from another angle, say you've got 10 ticks of a stock per second for 12 hours = 432,000 data points. This is the same as a single (relatively small) 380x380x3 image. If you assume there are roughly 30,000 instruments like this in the world of this volume, that's probably, I dunno, the number of images facebook sees in a second. Those are all guesses but you can see the scale of differences involved here.
- erikpukinskis 9y agoHello Frame Problem.
- sgt101 9y agoSome good answers in the thread below, but... 1. This is all observational data - even if you have all the features in the domain it's pretty clear that you won't have seen all of the generative theory. 2. You don't have all the features because the features change, renewable power suddenly makes average wind conditions relevant, and so on. 3. This domain is clearly chaotic; you can induce the various attractors and theories, but you cannot measure the conditions well enough to predict the shift between attractors. There are many people who are using ML to predict closed parts of the financial world where they have an deep and well founded understanding and excellent data. But there are lots and lots of people who are feeding data into models and getting good results and then going on good runs who are going to get a shock.
- revmoo 9y agoTheres... no article here. Like it literally doesn't render
- TuringNYC 9y agoI worked at a hedge fund for 9 years. Regression models are used for EVERYTHING. Regression is machine learning. This article has a narrow focus and doesnt understand what machine learning is.
- kgwgk 9y agoUsually « machine learning » is used to talk about things beyond basic statistical methods that predate computers. On the other hand, basic statistics are often good enough...
- TuringNYC 9y agoOther machine learning techniques widely used on wall street for over a decade - Dimensionality Reduction - Clustering - Classification techniques using trees, SVM, bayesian models
- megaman22 9y agoI feel like machine learning is one of those terms that has lost all defined meaning, under a bombardment of hype, corporate buzzwordism, and other horseshit.
- projectramo 9y agoThe article distinguishes between using a regression to generate trade ideas for a human (say, data science), or actually having the machine trade on the idea without intervention (that would be AI). The article claims the former happens but not the latter. This is entirely compatible with your statements.
- senatorobama 9y agoWhat was your ultimate salary?
- bllguo 9y agoI don't think technically sophisticated funds are going to be sharing any details at all for pieces like these. There is bound to be fancy ML in play at hedge funds, though - do not underestimate the lengths to which people will go to get an edge in the markets.
- TuringNYC 9y agoFunds would not share what they are doing because then others would do the same, narrow spreads and reduce profits for all participants. But you can get a good idea of what they might be working on by some of their workers' expertise. Notice how places like RenTech (https://en.wikipedia.org/wiki/Renaissance_Technologies https://en.wikipedia.org/wiki/Renaissance_Technologies and https://www.bloomberg.com/news/articles/2016-11-21/how-renaissance-s-medallion-fund-became-finance-s-blackest-box https://www.bloomberg.com/news/articles/2016-11-21/how-renai...), Two Sigma, and others hire loads of NLP and machine learning academics.
- QML 9y agoPerhaps these newer hedge funds just want to give off the perception that they know what they’re doing — I’m more sure about a company like RenTech since they have been through a recession and market crash, and has still come up net positive. So how much of this is simply the market rising; how much of this is due to chance? If we can remove these confounding factors then we could maybe actually evaluate the efficiency of ML-based hedge funds (or regular ones).
- ben_jones 9y ago"That's my quant". [1]: https://www.youtube.com/watch?v=FoYC_8cutb0 https://www.youtube.com/watch?v=FoYC_8cutb0
- hendzen 9y agoMany quant funds are market neutral. For every dollar long they have a dollar short. Such funds aren’t necessarily benefiting from an upward rise in the market.
- swyx 9y agoI like the shoutout to Numerai. I think they are so left field that traditional hedge funds don't even have the ability to understand how much better Numerai can be.
- dboreham 9y agoHuh? I did my commute with a guy who worked for an automated trading shop when I worked in London in 1994 and he talked constantly about what sounded just like machine learning.
- vadimberman 9y agoAn anecdote from 1990s continental Europe. A friend of mine, upon reaching a status of a "high-net worth individual", was invited to invest in a hedge fund associated with a bank of regional importance (major for that small country). They demonstrated the best of the tech of the time, and then shifted the tone to, "but we also use more traditional approaches". Unusually large portion of their "consulting research staff" was female and spread Europe-wide. Not because they were progressive or pursuing diversity agenda; these were call girls. The call girls were paid for timely tips. Even if they only report that a particular top executive stayed overnight far away from home, was in a foul mood, and a couple of top-ranking colleagues were present, this can be used to detect an significant internal event not reported publicly. If the girl was crafty enough to extract more information, even better.