10 ms·

Renaissance Technologies is rumoured to use methods from topological quantum field theory. They are fiercely and aggressively secretive, so no one really know

Renaissance Technologies is rumoured to use methods from topological quantum field theory. They are fiercely and aggressively secretive, so no one really know for sure. However, their founder worked in that field before moving to business: https://en.wikipedia.org/wiki/Chern%E2%80%93Simons_theory https://en.wikipedia.org/wiki/Chern%E2%80%93Simons_theory Famously, he was sacked by the NSA for giving an interview opposing US involvement in the Vietnam war.

matheweis 5y ago
Nick Patterson, formerly a senior statistician at Rentech, was on a podcast saying that their most important tool was a simple regression. I suppose he could’ve been lying, but the believable accounts suggest that the majority of their magic is simply having smart people use simple tools very, very well.
- dahak27 5y ago
  I think this is right - my experience as JS and similar places is that really understanding your data-generating process and the nitty-gritty assumptions of your data/simple model makes a huge difference E.g knowing what's causing missing values in your data and what implications various fixes on that might have on bias in your linear regressor is probably way way more valuable than fitting some shiny non-linear toy
- abzug 5y ago
  Do you have a link to this podcast? Or just the name.
  - matheweis 5y ago
    Talking Machines: AI Safety and The Legacy of Bletchley Park http://www.thetalkingmachines.com/episodes/ai-safety-and-legacy-bletchley-park http://www.thetalkingmachines.com/episodes/ai-safety-and-leg... Transcript with time stamps by @clausok: https://news.ycombinator.com/item?id=19065226 https://news.ycombinator.com/item?id=19065226
tubby12345 5y ago
>topological quantum field theory lolol. i will bet all of my 401k that they absolutely do not. i've interviewed at a couple of hedge funds and have lots of friends at various prop shops/hft firms/etc. most of them are using linear models. the ones that have market maker businesses deploy those linear models to fpgas. people act like these places are spooky magic cauldrons of physics + math + computer science. they're not (but it's definitely in their interest to promote such rumors). what they do have is very very robust data collection/aggregation infrastructure and backtesting systems. the rest is just correctness of execution of strats.
- lordnacho 5y ago
  I'll second that. I've done options trading, trend following, and HFT. Often you need smart people to do simple things, but they are still simple things. Gotta remember there's a lot of noise in financial data, and you don't actually have as much of it as people think. Even all the ticks on all the exchanges fit in a few GB of binary per day. RenTech is especially good at acting like they're from another planet, which I'm sure helps them attract those stellar mathematicians.
  - mellavora 5y ago
    Your especially right about the noise in financial data. Nice thing about linear models (vs i.e. random forest, or even worse deep learning) is that they are limited in how they can overfit the noise. They only fit a known function to known regressors.
    - xapata 5y ago
      And, even more important, they aren't susceptible to adversarial activity like a deep net could be.
  - dave_sullivan 5y ago
    That's actually been surprising to me but makes sense: "the entire history of the stock market" is not that big in terms of data, and the dynamics change frequently enough to where things that happened in 2000 are almost useless from a machine learning standpoint, my models perform best when trained on a much more selective subset (I believe because the reasons for trades and market participants keep changing, so the dynamics change enough to where old rules don't apply anymore).
jacquesm 5y ago
> Renaissance Technologies is rumoured to use methods from topological quantum field theory. Rumor is they use magic.
- HenryKissinger 5y ago
  Maybe they use Maybelline.
Kranar 5y ago
I have never worked at Renaissance Technologies, but I do run an HFT firm and am quite familiar with my own techniques as well as techniques of other competing firms, and none of us use anything that could be described as remotely sophisticated. One thing I tell new quants that I hire is that your job is kind of like a magic trick. To an outside observer the trick looks almost supernatural, but once you understand the trick you realize it's so unbelievably simple and straight forward you wonder how it is you were ever fooled by it. That said, even if someone tells you how a magic trick is performed and then hands you everything you need to perform it, you are almost certainly going to mess it up. It takes a lot of practice, skill and discipline to pull off even a simple magic trick even after you fully understand it. All of the difficulty of a magic trick is in the execution, not in the idea. It's the same with quantitative finance, the techniques do not involve anything remotely complex like quantum mechanics and in fact I am almost certain to reject strategies that are overly complex... but even if I revealed how our simplest of trading algorithms work, it's still incredibly difficult to actually execute them. Taking an idea and translating it into a high performance algorithm that is bug free, dealing with networking, collecting and dealing with petabytes of data, having a tight iteration loop, risk management, and most of all, having the creativity to identify something simple among a sea of complexity, those are what make my firm and other firms successful. It's unbelievably difficult work and most quants do end up failing, but it's not difficult in the way that many people think it is. If anything, most quants I've worked with fail by overcomplicating things and not being able to work from the ground up, off of first principles.
- jedimind 5y ago
  I would love to hear about some recommended reading and other sources which could help hobby traders like me to develop their own algorithms.
  - Kranar 5y ago
    Good sources are very hard to find because most of it is absolute trash intended to appeal to a certain audience who think there's money to be made off of reverse Fibonacci patterns or other silly sounding technical indicators that once again, sound technical and fancy but are completely useless. If you want to know what it's really like to be a quant, review stuff from the ARPM; everyone I hire goes through their 6 day bootcamp but they have other materials as well: https://www.arpm.co/ https://www.arpm.co/ And as for books, Algorithmic and High Frequency Trading covers the foundations: https://www.amazon.com/Algorithmic-High-Frequency-Trading-%C3%81lvaro-Cartea/dp/1107091144 https://www.amazon.com/Algorithmic-High-Frequency-Trading-%C... Those are pretty good sources to get an overview of what actual quants at successful firms know. Quantitative trading is technical, I don't want to give the impression that it's not technical... but it's not "fancy" technical. It's more along the lines of rigorous and iron clad instead of flashy and sophisticated. Every strategy is built up step by meticulous step in precise detail and every step needs to be rigorously justified and experimentally verified. Generally the thought process starts from the assumption that there is no money to be made on the stock market, either due to perfect efficiency or things like fees eating up any potential profits... when we talk about models, the models we construct describe how the stock market would behave if it were perfectly efficient, ie. free of any arbitrage opportunity. Then given our model of a perfectly efficient stock market, we simulate what we should expect to observe in such a perfectly efficient market... we then investigate empirically whether these observations happen in reality. Is the market genuinely efficient all day every day across every security. For some phenomenon it really is, but sometimes the market deviates from the model, so our model is either incorrect or an arbitrage opportunity has presented itself. If an arbitrage opportunity presents itself, we investigate how feasible it is to capture it, things like engineering effort, risk factors, profitability etc... If all of that works out, then we get to work constructing a state machine for an algorithm to capture that opportunity. We implement the state machine, write tests for it, run it through our backtester, then run it through our live simulator, and after everything checks out we deploy it live. Every algorithm is treated like a person, it's given its own human-like name, has its own account, its own set of permissions, capital allocated to it, risk profile, and algorithms are evaluated on a daily basis to reallocate capital to them and modify their risk profile.
paulpauper 5y ago
> topological quantum field theory. doubt it.
wolverine876 5y ago
> their founder worked in that field [topological quantum field theory] before moving to business: https://en.wikipedia.org/wiki/Chern%E2%80%93Simons_theory https://en.wikipedia.org/wiki/Chern%E2%80%93Simons_theory > Famously, he was sacked by the NSA for giving an interview opposing US involvement in the Vietnam war. Another cult of personality in business. Bill Gates, Jeff Bezos, Henry Ford, JP Morgan, etc. managed to do a lot without cults of personality. The more BS I see, the more I question what they are hiding, and the more I question their judgment in distracting their organizations from practical business, and distorting reality.
- bidirectional 5y ago
  The famously outperforming fund at Renaissance (Medallion) does not take outside money, and Renaissance itself is a privately held company, so there's not much need to cultivate such a cult. Honestly, I don't really get your perspective. Simons legitimately was one of the best mathematicians in the world and was sacked from the NSA. This isn't some Musk-esque 'I sleep on the factory floor and sign off on everything despite lacking the bona-fides' nonsense, he was 40 before entering the investment industry and before that won top research prizes, published quality research and ran a math department.
  - wolverine876 5y ago
    Yes, but who cares? Why is that important? > ran a math department OT, based on what university faculty have told me, so maybe not true everywhere: chairing a department is a thankless political nightmare. It's foisted on people who can't say no; it isn't an honor.
    - carnitine 5y ago
      Because people are trying to work out what his incredibly secretive hedge fund could be up to? His past is one of the few ways of getting any kind of hint.
      - wolverine876 5y ago
oarabbus_ 5y ago
>Renaissance Technologies is rumoured to use methods from topological quantum field theory. I am struggling to convey the magnitude of how skeptical I am of this claim.

matheweis 5y ago

Nick Patterson, formerly a senior statistician at Rentech, was on a podcast saying that their most important tool was a simple regression. I suppose he could’ve been lying, but the believable accounts suggest that the majority of their magic is simply having smart people use simple tools very, very well.

dahak27 5y ago
I think this is right - my experience as JS and similar places is that really understanding your data-generating process and the nitty-gritty assumptions of your data/simple model makes a huge difference E.g knowing what's causing missing values in your data and what implications various fixes on that might have on bias in your linear regressor is probably way way more valuable than fitting some shiny non-linear toy
abzug 5y ago
Do you have a link to this podcast? Or just the name.
- matheweis 5y ago
  Talking Machines: AI Safety and The Legacy of Bletchley Park http://www.thetalkingmachines.com/episodes/ai-safety-and-legacy-bletchley-park http://www.thetalkingmachines.com/episodes/ai-safety-and-leg... Transcript with time stamps by @clausok: https://news.ycombinator.com/item?id=19065226 https://news.ycombinator.com/item?id=19065226

tubby12345 5y ago

>topological quantum field theory lolol. i will bet all of my 401k that they absolutely do not. i've interviewed at a couple of hedge funds and have lots of friends at various prop shops/hft firms/etc. most of them are using linear models. the ones that have market maker businesses deploy those linear models to fpgas. people act like these places are spooky magic cauldrons of physics + math + computer science. they're not (but it's definitely in their interest to promote such rumors). what they do have is very very robust data collection/aggregation infrastructure and backtesting systems. the rest is just correctness of execution of strats.

lordnacho 5y ago
I'll second that. I've done options trading, trend following, and HFT. Often you need smart people to do simple things, but they are still simple things. Gotta remember there's a lot of noise in financial data, and you don't actually have as much of it as people think. Even all the ticks on all the exchanges fit in a few GB of binary per day. RenTech is especially good at acting like they're from another planet, which I'm sure helps them attract those stellar mathematicians.
- mellavora 5y ago
  Your especially right about the noise in financial data. Nice thing about linear models (vs i.e. random forest, or even worse deep learning) is that they are limited in how they can overfit the noise. They only fit a known function to known regressors.
  - xapata 5y ago
    And, even more important, they aren't susceptible to adversarial activity like a deep net could be.
- dave_sullivan 5y ago
  That's actually been surprising to me but makes sense: "the entire history of the stock market" is not that big in terms of data, and the dynamics change frequently enough to where things that happened in 2000 are almost useless from a machine learning standpoint, my models perform best when trained on a much more selective subset (I believe because the reasons for trades and market participants keep changing, so the dynamics change enough to where old rules don't apply anymore).

jacquesm 5y ago

> Renaissance Technologies is rumoured to use methods from topological quantum field theory. Rumor is they use magic.

HenryKissinger 5y ago
Maybe they use Maybelline.

Kranar 5y ago

I have never worked at Renaissance Technologies, but I do run an HFT firm and am quite familiar with my own techniques as well as techniques of other competing firms, and none of us use anything that could be described as remotely sophisticated. One thing I tell new quants that I hire is that your job is kind of like a magic trick. To an outside observer the trick looks almost supernatural, but once you understand the trick you realize it's so unbelievably simple and straight forward you wonder how it is you were ever fooled by it. That said, even if someone tells you how a magic trick is performed and then hands you everything you need to perform it, you are almost certainly going to mess it up. It takes a lot of practice, skill and discipline to pull off even a simple magic trick even after you fully understand it. All of the difficulty of a magic trick is in the execution, not in the idea. It's the same with quantitative finance, the techniques do not involve anything remotely complex like quantum mechanics and in fact I am almost certain to reject strategies that are overly complex... but even if I revealed how our simplest of trading algorithms work, it's still incredibly difficult to actually execute them. Taking an idea and translating it into a high performance algorithm that is bug free, dealing with networking, collecting and dealing with petabytes of data, having a tight iteration loop, risk management, and most of all, having the creativity to identify something simple among a sea of complexity, those are what make my firm and other firms successful. It's unbelievably difficult work and most quants do end up failing, but it's not difficult in the way that many people think it is. If anything, most quants I've worked with fail by overcomplicating things and not being able to work from the ground up, off of first principles.

jedimind 5y ago
I would love to hear about some recommended reading and other sources which could help hobby traders like me to develop their own algorithms.
- Kranar 5y ago
  Good sources are very hard to find because most of it is absolute trash intended to appeal to a certain audience who think there's money to be made off of reverse Fibonacci patterns or other silly sounding technical indicators that once again, sound technical and fancy but are completely useless. If you want to know what it's really like to be a quant, review stuff from the ARPM; everyone I hire goes through their 6 day bootcamp but they have other materials as well: https://www.arpm.co/ https://www.arpm.co/ And as for books, Algorithmic and High Frequency Trading covers the foundations: https://www.amazon.com/Algorithmic-High-Frequency-Trading-%C3%81lvaro-Cartea/dp/1107091144 https://www.amazon.com/Algorithmic-High-Frequency-Trading-%C... Those are pretty good sources to get an overview of what actual quants at successful firms know. Quantitative trading is technical, I don't want to give the impression that it's not technical... but it's not "fancy" technical. It's more along the lines of rigorous and iron clad instead of flashy and sophisticated. Every strategy is built up step by meticulous step in precise detail and every step needs to be rigorously justified and experimentally verified. Generally the thought process starts from the assumption that there is no money to be made on the stock market, either due to perfect efficiency or things like fees eating up any potential profits... when we talk about models, the models we construct describe how the stock market would behave if it were perfectly efficient, ie. free of any arbitrage opportunity. Then given our model of a perfectly efficient stock market, we simulate what we should expect to observe in such a perfectly efficient market... we then investigate empirically whether these observations happen in reality. Is the market genuinely efficient all day every day across every security. For some phenomenon it really is, but sometimes the market deviates from the model, so our model is either incorrect or an arbitrage opportunity has presented itself. If an arbitrage opportunity presents itself, we investigate how feasible it is to capture it, things like engineering effort, risk factors, profitability etc... If all of that works out, then we get to work constructing a state machine for an algorithm to capture that opportunity. We implement the state machine, write tests for it, run it through our backtester, then run it through our live simulator, and after everything checks out we deploy it live. Every algorithm is treated like a person, it's given its own human-like name, has its own account, its own set of permissions, capital allocated to it, risk profile, and algorithms are evaluated on a daily basis to reallocate capital to them and modify their risk profile.

paulpauper 5y ago

> topological quantum field theory. doubt it.

wolverine876 5y ago

> their founder worked in that field [topological quantum field theory] before moving to business: https://en.wikipedia.org/wiki/Chern%E2%80%93Simons_theory https://en.wikipedia.org/wiki/Chern%E2%80%93Simons_theory > Famously, he was sacked by the NSA for giving an interview opposing US involvement in the Vietnam war. Another cult of personality in business. Bill Gates, Jeff Bezos, Henry Ford, JP Morgan, etc. managed to do a lot without cults of personality. The more BS I see, the more I question what they are hiding, and the more I question their judgment in distracting their organizations from practical business, and distorting reality.

bidirectional 5y ago
The famously outperforming fund at Renaissance (Medallion) does not take outside money, and Renaissance itself is a privately held company, so there's not much need to cultivate such a cult. Honestly, I don't really get your perspective. Simons legitimately was one of the best mathematicians in the world and was sacked from the NSA. This isn't some Musk-esque 'I sleep on the factory floor and sign off on everything despite lacking the bona-fides' nonsense, he was 40 before entering the investment industry and before that won top research prizes, published quality research and ran a math department.
- wolverine876 5y ago
  Yes, but who cares? Why is that important? > ran a math department OT, based on what university faculty have told me, so maybe not true everywhere: chairing a department is a thankless political nightmare. It's foisted on people who can't say no; it isn't an honor.
  - carnitine 5y ago
    Because people are trying to work out what his incredibly secretive hedge fund could be up to? His past is one of the few ways of getting any kind of hint.
    - wolverine876 5y ago

oarabbus_ 5y ago

>Renaissance Technologies is rumoured to use methods from topological quantum field theory. I am struggling to convey the magnitude of how skeptical I am of this claim.