10 ms·

A Review of “The Man Who Solved the Market”

95 points by finite_jest 5y ago

deleted 5y ago
[deleted]
j9461701 5y ago
> People who raise hell with OSHA because the CEO is a heavy smoker: jerks. That doesn't seem jerkish. Second hand smoke in the work place is bad for everyone.
- treeman79 5y ago
  Ugh. I recall the thick yellow sludge in the computers at a new job in 90s where everyone chain smoked. I asked how many years old it was. “Six months” Promptly gave 2 weeks notice in.
- hardwaregeek 5y ago
  Yeah the author has a very uncharitable view towards Magerman. You can disagree with his political actions but I don't think he came off as a jerk, as much as someone who was aware of the old boys club nature of RenTech and wanted to change it.
kranke155 5y ago
This book is phenomenal and it also charts why Bob Mercer has become so dangerous to American democracy, so quickly - because after acquiring SCL group (the umbrella for Cambridge Analytica and AggregateIQ, who worked with Trump/Bannon/Brexit), he has a cognitive warfare firm that’s essentially funded ad infinitum by Renaissance Tech. constant money machine. The Ai solved markets from Renaissance have given one of their top members effectively infinite influence over the political landscape, because he doesn’t need to do anything anymore to keep the AÍ printing machine going.
- mikevm 5y ago
  I guess we need someone to counteract Soros, don't we?
  - pixelgeek 5y ago
    So the opposite of using your money to promote civil society and fund education is to help steal elections?
- sflicht 5y ago
  > he doesn’t need to do anything anymore to keep the AÍ [money?] printing machine going This betrays a fundamental misunderstanding of how systematic trading strategies differ from some other ML problems. It's an adversarial, nonstationary environment. The same strategies don't just keep working indefinitely. (Not to mention the fact that maintaining any complex production software system requires an enormous amount of work, even completely ignoring the need to keep developing new alpha. For instance, consider the case when certain oil prices went briefly negative in 2020 -- even if your software was so perfect that it was unaffected by that event [hah!] remember that many of the counterparties you actually trade with may not have been as clever. This is just one small example. as markets evolve, they will find ways to turn up new bugs in your code.)
  - kranke155 5y ago
    I am aware of it being an adversarial environment, but right now they are on top and they can afford the “maintenance” fees on keeping the machine going, probably with a lot of the top AÍ talent that’s available to recruit (considering their earnings). Mercer himself can just live off the earnings.
balsam 5y ago
They should absolutely get Tom Hardy to play Jim Simons. Who to play De Kapitein, then?
huitzitziltzin 5y ago
Nice review of what sounds like an interesting book. One small objection: "Cleaning data is the fundamental task that defines data science as a role: statisticians and economists buy clean data from somebody." Man I wish. I spent more than a year of my PhD (cumulatively and not all at once) cleaning various kinds of data. I joked w/ an adviser once that I thought I had signed up for an economics PhD but it turned out I was getting a PhD in data cleaning. Finding pre-cleaned, nice data for a project is like finding a $100 bill on the street. Doesn't happen.
- netizen-936824 5y ago
  I started doing some work with ML models and immediately realised that around 80-90% of my code was dedicated entirely to data cleaning and preparation. Glad I got some practice before diving into a PhD honetly
- Mc91 5y ago
  I met a Renaissance guy at a party once. His description of data cleaning was "it's like you have a pile of gold bars buried beneath a couple of tons of shit".
- bko 5y ago
  I really liked the book. I wrote my own review trying to dissect their secret sauce and I believe it to be their data pipeline. The focused on it from the very beginning and wasn't sexy at all. From the book: > Piecing together a custom-built database, Straus purchased historic commodity-price data on magnetic tape from an Indiana-based firm called Dunn & Hargitt, then merged it with the historic information others in the firm already had amassed. ... Using an Apple II computer, Straus and others wrote a program to collect and store their growing data trove. > Over time, Straus and his colleagues created and discovered additional historical pricing data, helping Ax develop new predictive models relying on Carmona’s suggestions. Some of the weekly stocktrading data they’d later find went back as far as the 1800s, reliable information almost no one else had access to. At the time, the team couldn’t do much with the data, but the ability to search history to see how markets reacted to unusual events would later help Simons’s team build models to profit from market collapses and other unexpected events, helping the firm trounce markets during those periods. They also took market prices as nearly devoid of real life meaning. I was originally very skeptical of this approach as it requires immense discipline and rigor to avoid data mining. You see economists do that where they create a signal for something like the 23 month lag of GDP. Why 23 month lag? Because 1-22 were insignificant. But they seem to get away with it due to their strong academic background. > Here’s what was really unique: The paper didn’t try to identify or predict these states using economic theory or other conventional methods, nor did the researchers seek to address why the market entered certain states. Simons and his colleagues used mathematics to determine the set of states best fitting the observed pricing data; their model then made its bets accordingly. The whys didn’t matter, Simons and his colleagues seemed to suggest, just the strategies to take advantage of the inferred states. https://mleverything.substack.com/p/how-renaissance-technologies-solved-the-market-part-1-2814eb271dc3 https://mleverything.substack.com/p/how-renaissance-technolo...
mgaunard 5y ago
So the article basically says that programmers are perfectionist jerks and you shouldn't hire them. Too bad programmers are actually the ones you need the most. Trading is mostly a technical problem, which requires a technical organization to be solved well. You're not going to be successful if you only hire MBA types and cannot build a culture of technological excellence.
- professor_v 5y ago
  The article was talking about the first computer science guy in a team of mathematicians. I don't know if mathematicians should be called technical, but they're hardly MBA types.
melling 5y ago
“Ed Thorpe’s Princeton Newport and its successor …” He’s been at it for decades. What did Ed figure out half a century ago? Btw, Ed’s a math guy also: https://en.m.wikipedia.org/wiki/Edward_O._Thorp https://en.m.wikipedia.org/wiki/Edward_O._Thorp
- AS37 5y ago
  Bet sizing, maybe? It's certainly destroyed some big funds. https://en.wikipedia.org/wiki/Edward_O._Thorp#Bibliography https://en.wikipedia.org/wiki/Edward_O._Thorp#Bibliography
- auntienomen 5y ago
  Thorpe didn't have much success in stat arb after Princeton Newport was broken up. TGS did. I'm not sure I'd credit Thorpe with whatever they were doing.
  - cloudbonsai 5y ago
    Thorp operated a small stat arb shop for a decade after P/NP, and IIRC it was quite successful (avg ~20% return with sharpe ratio > 1.5).
- deleted 5y ago
  [deleted]
pixelgeek 5y ago
I wandered through the rest of the blog. What an amazing asshat. He should be on display somewhere. His rant about US health issues[1] is breathtaking. What sort of person begins a description of autism like this? > Autism. I’m not sure this doesn’t fall into the category of “insanity.” It may also have something to do with hormonal disruption. Either way, the fact that some huge fraction of the population who thinks they don’t have to be kind and thoughtful to others because “muh disability.” I don't want to read the biography he is referring to but I do want to read a psych profile of Scott Locklin. He is the platonic ideal of a cranky, covid-denying asshat. [1]https://scottlocklin.wordpress.com/2021/11/20/us-public-health-problems-which-are-more-serious-than-corona-chan/ https://scottlocklin.wordpress.com/2021/11/20/us-public-heal...
- slibhb 5y ago
  I believe the author was banned from HN a while back. I find his bluntness funny and somewhat admirable, where I disagree with him and not. Might be a regional thing, we're both New Englanders.
  - tptacek 5y ago
    Is calling Mississippi "Amerikwaanza" a New England thing?
    - slibhb 5y ago
      No but not getting offended by it or laughing at the absurdity of it might be.
- temporaryi3 5y ago
  Politeness > accuracy & honestly is a very common preference, but not mine.
  - dahak27 5y ago
    You can be honest and accurate without being needlessly nasty/mocking as the quote seems to be Your sentiment is a really common and insidious one that way too many people use to feel smug and "rational" about being mean
drewcoo 5y ago
It lost me with the first line. How does this make sense? Did word definitions shift on me again? > Most systematic hedge funds are a racket; they either got lucky, or have a strategy that only works in one market regime.
- pixelgeek 5y ago
  I believe he is trying to say that most multi-market hedge funds (systemic hedge funds) are a fraud because the models that get generated typically only work in the market that the data modelling was done for.
- Kranar 5y ago
  Are you objecting to the word racket? Most hedge funds are not legally fraudulent, but they are fraudulent in that they sell an expensive product that is by pretty much all measure worse than the cheap off-the-shelf product. Selling ion bracelets to give you increased energy is not legal fraud either, but most people would say it's a fraud nonetheless. The same goes for most hedge funds.
  - FredPret 5y ago
    It’s not really that simple. Some funds outperform the market consistently (BRK isn’t a hedge fund but is a good example of outperformance) Others stick to their name and hedge against certain things. It’s so hard to truly evaluate performance by all metrics, it might as well be impossible. For the average person, the best way is probably to stick their money in the S&P 500 and forget about it. But what about outliers, like founders with 95% of their wealth in company x? They might want to hedge their bets as it were.
cryptica 5y ago
I could never get into serious trading because current markets are irrational. Being a good trader these days is not about knowing what's going on; it's about being irrational in the same way as the most irrational rich fools... But it's difficult to tell which group of rich fools you should imitate. Which completely insane narrative will become the new reality? There seems to be no method to the madness. It's just plain madness. I'm kind of diversified right now. Some of my money is in value-adding businesses and some is in speculative crypto ranging from "This project has potential" to "This project is 100% hype-based and has no viable technological future but could be big financially because insanity is becoming a reliable trend."
- mccorrinall 5y ago
  Engineers can determine the cause of a bridge collapse because there’s agreement that if a certain amount of force is applied to a certain area, that area will break. Physics isn’t controversial. It’s guided by laws. Finance is different. It’s guided by people’s behaviors. And how I behave might make sense to me but look crazy to you. Morgan Housel. The Psychology of Money.
  - cryptica 5y ago
    I guess that hits the nail on the head. Most people, especially rich people, seem crazy to me.
- Mikeb85 5y ago
  Trading has little to do with financial or economic models. And it definitely can be irrational. But it can be predictable if you can predict the behaviour of other traders, because price is determined by supply and demand for the asset. The underlying value is a small part of determining how other traders will value it.
  - EricE 5y ago
    lol - you left out emotion. If traders were only motivated by pure supply/demand. Ha!
bensouthworth 5y ago
You need an editor. It was impossible to read, it felt a like a bunch of notes written as you read it. That's fine, but do not call it a review. The RE, the after, the post,the after, this is poor. Sorry.
deleted 5y ago
[deleted]

95 points by finite_jest 5y ago

deleted 5y ago

[deleted]

> People who raise hell with OSHA because the CEO is a heavy smoker: jerks. That doesn't seem jerkish. Second hand smoke in the work place is bad for everyone.

treeman79 5y ago
Ugh. I recall the thick yellow sludge in the computers at a new job in 90s where everyone chain smoked. I asked how many years old it was. “Six months” Promptly gave 2 weeks notice in.
hardwaregeek 5y ago
Yeah the author has a very uncharitable view towards Magerman. You can disagree with his political actions but I don't think he came off as a jerk, as much as someone who was aware of the old boys club nature of RenTech and wanted to change it.

kranke155 5y ago

This book is phenomenal and it also charts why Bob Mercer has become so dangerous to American democracy, so quickly - because after acquiring SCL group (the umbrella for Cambridge Analytica and AggregateIQ, who worked with Trump/Bannon/Brexit), he has a cognitive warfare firm that’s essentially funded ad infinitum by Renaissance Tech. constant money machine. The Ai solved markets from Renaissance have given one of their top members effectively infinite influence over the political landscape, because he doesn’t need to do anything anymore to keep the AÍ printing machine going.

mikevm 5y ago
I guess we need someone to counteract Soros, don't we?
- pixelgeek 5y ago
  So the opposite of using your money to promote civil society and fund education is to help steal elections?
sflicht 5y ago
> he doesn’t need to do anything anymore to keep the AÍ [money?] printing machine going This betrays a fundamental misunderstanding of how systematic trading strategies differ from some other ML problems. It's an adversarial, nonstationary environment. The same strategies don't just keep working indefinitely. (Not to mention the fact that maintaining any complex production software system requires an enormous amount of work, even completely ignoring the need to keep developing new alpha. For instance, consider the case when certain oil prices went briefly negative in 2020 -- even if your software was so perfect that it was unaffected by that event [hah!] remember that many of the counterparties you actually trade with may not have been as clever. This is just one small example. as markets evolve, they will find ways to turn up new bugs in your code.)
- kranke155 5y ago
  I am aware of it being an adversarial environment, but right now they are on top and they can afford the “maintenance” fees on keeping the machine going, probably with a lot of the top AÍ talent that’s available to recruit (considering their earnings). Mercer himself can just live off the earnings.

balsam 5y ago

They should absolutely get Tom Hardy to play Jim Simons. Who to play De Kapitein, then?

huitzitziltzin 5y ago

Nice review of what sounds like an interesting book. One small objection: "Cleaning data is the fundamental task that defines data science as a role: statisticians and economists buy clean data from somebody." Man I wish. I spent more than a year of my PhD (cumulatively and not all at once) cleaning various kinds of data. I joked w/ an adviser once that I thought I had signed up for an economics PhD but it turned out I was getting a PhD in data cleaning. Finding pre-cleaned, nice data for a project is like finding a $100 bill on the street. Doesn't happen.

netizen-936824 5y ago
I started doing some work with ML models and immediately realised that around 80-90% of my code was dedicated entirely to data cleaning and preparation. Glad I got some practice before diving into a PhD honetly
Mc91 5y ago
I met a Renaissance guy at a party once. His description of data cleaning was "it's like you have a pile of gold bars buried beneath a couple of tons of shit".
bko 5y ago
I really liked the book. I wrote my own review trying to dissect their secret sauce and I believe it to be their data pipeline. The focused on it from the very beginning and wasn't sexy at all. From the book: > Piecing together a custom-built database, Straus purchased historic commodity-price data on magnetic tape from an Indiana-based firm called Dunn & Hargitt, then merged it with the historic information others in the firm already had amassed. ... Using an Apple II computer, Straus and others wrote a program to collect and store their growing data trove. > Over time, Straus and his colleagues created and discovered additional historical pricing data, helping Ax develop new predictive models relying on Carmona’s suggestions. Some of the weekly stocktrading data they’d later find went back as far as the 1800s, reliable information almost no one else had access to. At the time, the team couldn’t do much with the data, but the ability to search history to see how markets reacted to unusual events would later help Simons’s team build models to profit from market collapses and other unexpected events, helping the firm trounce markets during those periods. They also took market prices as nearly devoid of real life meaning. I was originally very skeptical of this approach as it requires immense discipline and rigor to avoid data mining. You see economists do that where they create a signal for something like the 23 month lag of GDP. Why 23 month lag? Because 1-22 were insignificant. But they seem to get away with it due to their strong academic background. > Here’s what was really unique: The paper didn’t try to identify or predict these states using economic theory or other conventional methods, nor did the researchers seek to address why the market entered certain states. Simons and his colleagues used mathematics to determine the set of states best fitting the observed pricing data; their model then made its bets accordingly. The whys didn’t matter, Simons and his colleagues seemed to suggest, just the strategies to take advantage of the inferred states. https://mleverything.substack.com/p/how-renaissance-technologies-solved-the-market-part-1-2814eb271dc3 https://mleverything.substack.com/p/how-renaissance-technolo...

mgaunard 5y ago

So the article basically says that programmers are perfectionist jerks and you shouldn't hire them. Too bad programmers are actually the ones you need the most. Trading is mostly a technical problem, which requires a technical organization to be solved well. You're not going to be successful if you only hire MBA types and cannot build a culture of technological excellence.

professor_v 5y ago
The article was talking about the first computer science guy in a team of mathematicians. I don't know if mathematicians should be called technical, but they're hardly MBA types.

melling 5y ago

“Ed Thorpe’s Princeton Newport and its successor …” He’s been at it for decades. What did Ed figure out half a century ago? Btw, Ed’s a math guy also: https://en.m.wikipedia.org/wiki/Edward_O._Thorp https://en.m.wikipedia.org/wiki/Edward_O._Thorp

AS37 5y ago
Bet sizing, maybe? It's certainly destroyed some big funds. https://en.wikipedia.org/wiki/Edward_O._Thorp#Bibliography https://en.wikipedia.org/wiki/Edward_O._Thorp#Bibliography
auntienomen 5y ago
Thorpe didn't have much success in stat arb after Princeton Newport was broken up. TGS did. I'm not sure I'd credit Thorpe with whatever they were doing.
- cloudbonsai 5y ago
  Thorp operated a small stat arb shop for a decade after P/NP, and IIRC it was quite successful (avg ~20% return with sharpe ratio > 1.5).
deleted 5y ago
[deleted]

pixelgeek 5y ago

I wandered through the rest of the blog. What an amazing asshat. He should be on display somewhere. His rant about US health issues[1] is breathtaking. What sort of person begins a description of autism like this? > Autism. I’m not sure this doesn’t fall into the category of “insanity.” It may also have something to do with hormonal disruption. Either way, the fact that some huge fraction of the population who thinks they don’t have to be kind and thoughtful to others because “muh disability.” I don't want to read the biography he is referring to but I do want to read a psych profile of Scott Locklin. He is the platonic ideal of a cranky, covid-denying asshat. [1]https://scottlocklin.wordpress.com/2021/11/20/us-public-health-problems-which-are-more-serious-than-corona-chan/ https://scottlocklin.wordpress.com/2021/11/20/us-public-heal...

slibhb 5y ago
I believe the author was banned from HN a while back. I find his bluntness funny and somewhat admirable, where I disagree with him and not. Might be a regional thing, we're both New Englanders.
- tptacek 5y ago
  Is calling Mississippi "Amerikwaanza" a New England thing?
  - slibhb 5y ago
    No but not getting offended by it or laughing at the absurdity of it might be.
temporaryi3 5y ago
Politeness > accuracy & honestly is a very common preference, but not mine.
- dahak27 5y ago
  You can be honest and accurate without being needlessly nasty/mocking as the quote seems to be Your sentiment is a really common and insidious one that way too many people use to feel smug and "rational" about being mean

drewcoo 5y ago

It lost me with the first line. How does this make sense? Did word definitions shift on me again? > Most systematic hedge funds are a racket; they either got lucky, or have a strategy that only works in one market regime.

pixelgeek 5y ago
I believe he is trying to say that most multi-market hedge funds (systemic hedge funds) are a fraud because the models that get generated typically only work in the market that the data modelling was done for.
Kranar 5y ago
Are you objecting to the word racket? Most hedge funds are not legally fraudulent, but they are fraudulent in that they sell an expensive product that is by pretty much all measure worse than the cheap off-the-shelf product. Selling ion bracelets to give you increased energy is not legal fraud either, but most people would say it's a fraud nonetheless. The same goes for most hedge funds.
- FredPret 5y ago
  It’s not really that simple. Some funds outperform the market consistently (BRK isn’t a hedge fund but is a good example of outperformance) Others stick to their name and hedge against certain things. It’s so hard to truly evaluate performance by all metrics, it might as well be impossible. For the average person, the best way is probably to stick their money in the S&P 500 and forget about it. But what about outliers, like founders with 95% of their wealth in company x? They might want to hedge their bets as it were.

cryptica 5y ago

I could never get into serious trading because current markets are irrational. Being a good trader these days is not about knowing what's going on; it's about being irrational in the same way as the most irrational rich fools... But it's difficult to tell which group of rich fools you should imitate. Which completely insane narrative will become the new reality? There seems to be no method to the madness. It's just plain madness. I'm kind of diversified right now. Some of my money is in value-adding businesses and some is in speculative crypto ranging from "This project has potential" to "This project is 100% hype-based and has no viable technological future but could be big financially because insanity is becoming a reliable trend."

mccorrinall 5y ago
Engineers can determine the cause of a bridge collapse because there’s agreement that if a certain amount of force is applied to a certain area, that area will break. Physics isn’t controversial. It’s guided by laws. Finance is different. It’s guided by people’s behaviors. And how I behave might make sense to me but look crazy to you. Morgan Housel. The Psychology of Money.
- cryptica 5y ago
  I guess that hits the nail on the head. Most people, especially rich people, seem crazy to me.
Mikeb85 5y ago
Trading has little to do with financial or economic models. And it definitely can be irrational. But it can be predictable if you can predict the behaviour of other traders, because price is determined by supply and demand for the asset. The underlying value is a small part of determining how other traders will value it.
- EricE 5y ago
  lol - you left out emotion. If traders were only motivated by pure supply/demand. Ha!

bensouthworth 5y ago

You need an editor. It was impossible to read, it felt a like a bunch of notes written as you read it. That's fine, but do not call it a review. The RE, the after, the post,the after, this is poor. Sorry.

deleted 5y ago

[deleted]