7 ms·

As a frequent Kaggler (perhaps too frequent... it's a bit addicting, in a way I'm sure others on HN will understand), I was fairly intrigued to see this one pop

As a frequent Kaggler (perhaps too frequent... it's a bit addicting, in a way I'm sure others on HN will understand), I was fairly intrigued to see this one pop up in the competition list a few days ago. Finance shops have tried their hand at Kaggle before, but I think they've normally been out of their domain. e.g. Two Sigma recently did a reinforcement learning game competition. I'd caution the HN crowd not to expect production-level quant models out of this, like I'm seeing some doing in the comments already. Kagglers are excellent machine learning practitioners and the models that come out of many competitions are top-notch stuff, often making their way into research papers. But this is a short competition on limited data in a non-real-world scenario. The winning models will be very interesting educational exercises and probably wonderful recruiting material for Jane Street, but won't be the underpinnings of a new fund. That said, I can't wait to see what comes out of this one. It ticks all of my competitive boxes :)

fractionalhare 6y ago
Yes, it's overwhelmingly unlikely that the winning model will actually be a competitive trading strategy. Kaggle encourages a domain agnostic approach to modeling, in the sense that participants use sophisticated machine learning and statistical methods but typically have no domain expertise in the underlying data. This kind of approach to finance has historically performed poorly. [1] Good quantitative trading is usually backed by a strong fundamental thesis and an interpretable model, which is obtained by cross-pollinating sophisticated math and statistics with domain expertise in some part of finance. That domain expertise might be in different kinds of assets, liquidity or market microstructure, but it's there. $100k is cheap for Jane Street. If nothing else they have a new recruiting pipeline of people with demonstrable machine learning skills. ______________ 1. I would also say this is a poor way to approach statistical analysis in most domains, and usually leads to spurious or overfit results. But the idea that you can just run a model and find patterns in pricing data is especially attractive and insidious.
- usmannk 6y ago
  > Kaggle encourages a domain agnostic approach to modeling, in the sense that participants use sophisticated machine learning and statistical methods but typically have no domain expertise in the underlying data. Yes this is accurate and put very well. This is so much the case that if you have a strong background understanding of the field, the ML part can actually be picked up quite quickly or contributed by someone else. There are a few notable users who are both domain and ML experts and they tend to absolutely clean up in their field. I'm thinking of a couple of med students in particular who are formidable in every medical imaging competition.
- rahimnathwani 6y ago
  "have no domain expertise in the underlying data. This kind of approach to finance has historically performed poorly" I recently read 'The man who solved the market', about Jim Simons and Renaissance Capital. The way the book tells it, looking for patterns without seeking domain expertise (e.g. ignoring fundamental valuation of equities) is exactly what Renaissance did, and it worked out very well.
  - fractionalhare 6y ago
    I can see why someone would characterize RenTech that way but it's not really fair to do so. There is a lot of mythos about how Simons hired computer scientists, mathematicians, signal processing and NLP experts, etc. When Mercer came over from IBM, he definitely contributed a significant amount of analytical expertise that was probably nonexistent in financial trading at the time (with the possible exception of the Ed Thorp diaspora). The astrophysicists RenTech hires every year bring new insights in ways to model and understand vast amounts of data with absurd dimensionality. But all of this has to be utilized in the context of the data. The reality is that you're not going to develop a sophisticated options trading strategy without a strong understanding of what an option (and more generally, a derivative) is. You can't develop a viable statistical arbitrage strategy just by treating market microstructure as a blackbox signal to be solved with e.g. Fourier analysis. You can certainly find an edge in using fundamentally superior methods of analysis, but you still need to know what that data represents in the context of the market. Don't be fooled: people working at firms like RenTech have a strong understanding of the underlying finance. It's just that they learned it on the job, because the ethos at these firms is that learning fundamental theory in math and statistics is harder than learning fundamental theory in finance. You don't have to take my word for it though. Read about one of the few strategies of RenTech's which has been publicized: https://www.bloomberg.com/opinion/articles/2014-07-22/senate-literary-critics-don-t-like-fictional-derivatives https://www.bloomberg.com/opinion/articles/2014-07-22/senate.... Deutsche and RenTech didn't team up on this strategy (to fantastic success) by treating basket options as some kind of blackbox abstraction devoid of delta, gamma, theta and vega.
    - nojito 6y ago
riazrizvi 6y ago
Mathematical analysis of financial markets is more celebrated when applied to relative valuation of different assets, rather than prediction of the market. Black-scholes, for example, applied calculus with an underlying no-arbitrage assumption to create a thriving market in option pricing, by giving traders a mechanism to reduce risk and thereby reduce bid offer spreads. Same in fixed income, mortgage, and credit market assets over the years. The problem with predicting absolute levels, is that there is a game theoretic aspect which undermines any mathematical trading strategy as soon as it is public. optimal game theory trading strategies don’t produce great results, and they are relatively trivial to identify. Instead strong profits in market long/short macro positions are mostly created by information advantages, which don’t really make for interesting Kaggle competitions. For example, big profits in macro trading have historically been consistently achieved by front running customer orders, by building timing advantages on top of trading infrastructure, by funding research analysts that inspect operations on the ground, by lobbying for regulations that change market directions and so on. It’s very hard to tell if a best performing hedge funds that doesn’t have an unfair advantage, that declares its only using quantitative strategies, is in fact just a statistical anomaly with a hollow narrative.
- georgeecollins 6y ago
  This! >> The problem with predicting absolute levels, is that there is a game theoretic aspect which undermines any mathematical trading strategy as soon as it is public. I took finance in Business school, coming from doing a lot of statistical analysis in a research lab. I hated my finance professors and there pseudo science. Pricing formulas work great until they don't. The problem is when they don't, they really don't, in a catastrophic way. Read "When Genius Failed." Real traders know this. But some economists and finance professors act like these mathematical models are describing a predictable physical phenomena.
  - milesvp 6y ago
    To add to this, unless your model is situated, and can purturb the market, it has no way of knowing what happens when you flex your muscle. I have a friend who did algorithmic trading professionally for a few years, and he said it was amazing to watch the data. Said he could see other bots come along and poke him, trying to look for weaknesses in his algorithm to exploit. I would expect a purely formulaic trader to underperform other traders who can take advantage of others. It’s no different than how you have to win a rhoshambo turnament.
x87678r 6y ago
They even say in the instructions: Admittedly, this challenge far oversimplifies the depth of the quantitative problems Jane Streeters work on daily, and Jane Street is happy with the performance of its existing trading model for this particular question.
Spinnaker_ 6y ago
We should also remember that Jane Street is primarily an ETF market maker. Their main business isn't betting on prices of stocks or managing a portfolio. I've only taken a quick look at the data, but the problem doesn't seem to be focused on their core competencies, but instead is much more general.
- optimalsolver 6y ago
  >I've only taken a quick look at the data, but the problem doesn't seem to be focused on their core competencies, but instead is much more general How can you tell? All the features are completely anonymized.
anonu 6y ago
Can you comment on the specific setup of this kaggle competition? Versus other finance/trading related challenges?

fractionalhare 6y ago

Yes, it's overwhelmingly unlikely that the winning model will actually be a competitive trading strategy. Kaggle encourages a domain agnostic approach to modeling, in the sense that participants use sophisticated machine learning and statistical methods but typically have no domain expertise in the underlying data. This kind of approach to finance has historically performed poorly. [1] Good quantitative trading is usually backed by a strong fundamental thesis and an interpretable model, which is obtained by cross-pollinating sophisticated math and statistics with domain expertise in some part of finance. That domain expertise might be in different kinds of assets, liquidity or market microstructure, but it's there. $100k is cheap for Jane Street. If nothing else they have a new recruiting pipeline of people with demonstrable machine learning skills. ______________ 1. I would also say this is a poor way to approach statistical analysis in most domains, and usually leads to spurious or overfit results. But the idea that you can just run a model and find patterns in pricing data is especially attractive and insidious.

usmannk 6y ago
> Kaggle encourages a domain agnostic approach to modeling, in the sense that participants use sophisticated machine learning and statistical methods but typically have no domain expertise in the underlying data. Yes this is accurate and put very well. This is so much the case that if you have a strong background understanding of the field, the ML part can actually be picked up quite quickly or contributed by someone else. There are a few notable users who are both domain and ML experts and they tend to absolutely clean up in their field. I'm thinking of a couple of med students in particular who are formidable in every medical imaging competition.
rahimnathwani 6y ago
"have no domain expertise in the underlying data. This kind of approach to finance has historically performed poorly" I recently read 'The man who solved the market', about Jim Simons and Renaissance Capital. The way the book tells it, looking for patterns without seeking domain expertise (e.g. ignoring fundamental valuation of equities) is exactly what Renaissance did, and it worked out very well.
- fractionalhare 6y ago
  I can see why someone would characterize RenTech that way but it's not really fair to do so. There is a lot of mythos about how Simons hired computer scientists, mathematicians, signal processing and NLP experts, etc. When Mercer came over from IBM, he definitely contributed a significant amount of analytical expertise that was probably nonexistent in financial trading at the time (with the possible exception of the Ed Thorp diaspora). The astrophysicists RenTech hires every year bring new insights in ways to model and understand vast amounts of data with absurd dimensionality. But all of this has to be utilized in the context of the data. The reality is that you're not going to develop a sophisticated options trading strategy without a strong understanding of what an option (and more generally, a derivative) is. You can't develop a viable statistical arbitrage strategy just by treating market microstructure as a blackbox signal to be solved with e.g. Fourier analysis. You can certainly find an edge in using fundamentally superior methods of analysis, but you still need to know what that data represents in the context of the market. Don't be fooled: people working at firms like RenTech have a strong understanding of the underlying finance. It's just that they learned it on the job, because the ethos at these firms is that learning fundamental theory in math and statistics is harder than learning fundamental theory in finance. You don't have to take my word for it though. Read about one of the few strategies of RenTech's which has been publicized: https://www.bloomberg.com/opinion/articles/2014-07-22/senate-literary-critics-don-t-like-fictional-derivatives https://www.bloomberg.com/opinion/articles/2014-07-22/senate.... Deutsche and RenTech didn't team up on this strategy (to fantastic success) by treating basket options as some kind of blackbox abstraction devoid of delta, gamma, theta and vega.
  - nojito 6y ago

riazrizvi 6y ago

Mathematical analysis of financial markets is more celebrated when applied to relative valuation of different assets, rather than prediction of the market. Black-scholes, for example, applied calculus with an underlying no-arbitrage assumption to create a thriving market in option pricing, by giving traders a mechanism to reduce risk and thereby reduce bid offer spreads. Same in fixed income, mortgage, and credit market assets over the years. The problem with predicting absolute levels, is that there is a game theoretic aspect which undermines any mathematical trading strategy as soon as it is public. optimal game theory trading strategies don’t produce great results, and they are relatively trivial to identify. Instead strong profits in market long/short macro positions are mostly created by information advantages, which don’t really make for interesting Kaggle competitions. For example, big profits in macro trading have historically been consistently achieved by front running customer orders, by building timing advantages on top of trading infrastructure, by funding research analysts that inspect operations on the ground, by lobbying for regulations that change market directions and so on. It’s very hard to tell if a best performing hedge funds that doesn’t have an unfair advantage, that declares its only using quantitative strategies, is in fact just a statistical anomaly with a hollow narrative.

georgeecollins 6y ago
This! >> The problem with predicting absolute levels, is that there is a game theoretic aspect which undermines any mathematical trading strategy as soon as it is public. I took finance in Business school, coming from doing a lot of statistical analysis in a research lab. I hated my finance professors and there pseudo science. Pricing formulas work great until they don't. The problem is when they don't, they really don't, in a catastrophic way. Read "When Genius Failed." Real traders know this. But some economists and finance professors act like these mathematical models are describing a predictable physical phenomena.
- milesvp 6y ago
  To add to this, unless your model is situated, and can purturb the market, it has no way of knowing what happens when you flex your muscle. I have a friend who did algorithmic trading professionally for a few years, and he said it was amazing to watch the data. Said he could see other bots come along and poke him, trying to look for weaknesses in his algorithm to exploit. I would expect a purely formulaic trader to underperform other traders who can take advantage of others. It’s no different than how you have to win a rhoshambo turnament.

x87678r 6y ago

They even say in the instructions: Admittedly, this challenge far oversimplifies the depth of the quantitative problems Jane Streeters work on daily, and Jane Street is happy with the performance of its existing trading model for this particular question.

Spinnaker_ 6y ago

We should also remember that Jane Street is primarily an ETF market maker. Their main business isn't betting on prices of stocks or managing a portfolio. I've only taken a quick look at the data, but the problem doesn't seem to be focused on their core competencies, but instead is much more general.

optimalsolver 6y ago
>I've only taken a quick look at the data, but the problem doesn't seem to be focused on their core competencies, but instead is much more general How can you tell? All the features are completely anonymized.

anonu 6y ago

Can you comment on the specific setup of this kaggle competition? Versus other finance/trading related challenges?