5 ms·

I'm a quant trader and I agree. There's a big difference between being able to predict what will happen, and having a sensible strategy for how to behave. One

I'm a quant trader and I agree. There's a big difference between being able to predict what will happen, and having a sensible strategy for how to behave. One notable place this happens is backtesting. Particularly for passive strategies, you don't actually know from historical data what would have happened if you had been in the market at a given time. Would your prices move the market? Maybe. Would you get filled disproportionately when it's bad to get filled? Fair chance of it. So don't act like your presence would not be felt. Another angle is risk management. You may have high confidence in your model, but you should have a plan for what to do if things don't go as expected. It seems crazy to me that it could get so far out of hand that a quarter of the company gets sacked. Why not set a budget and dip your toes? Then you can watch out for the adverse selection issues as you grow the portfolio and make adjustments. Finally, it's very normal for quant strategies to look good when they're first proposed, then look a bit worse when traded in real life, and perhaps then degrade over time. Another reason to take it easy when sizing the bets. You're right about Lucas critique. It's fundamental that models affect the thing they're modelling.

treis 5y ago
The big difference here is that the counterparty is deciding which orders get filled. Zillow could be right 99% of the time, but that 1% of the time where they're overpaying are the transactions that for sure will go through. And indeed if you go to /r/realestate anecdotally that is what happened. People would get offers from Zillow/Opendoor etc and talk to agents to see what they would list it at. If Zillow was higher than what the agents said they went with Zillow. If not they'd list it and sell it the old fashion way.
- lordnacho 5y ago
  Yeah, it's remarkable. You'd think that "I'll buy your house immediately" should be at a worse price than the guy who insists on the old school inspection and due diligence. In fact it might not be a bad strategy to simply start with that... see if there's desperate people needing the money fast, and use your corporate heft to both sweep up the deal (brand) and patiently dump it back into the ordinary market, pocketing the spread as a liquidity premium. One thing I remember my first boss saying is that it's hard to get a good trade on. Attractive deals have several interested parties. If you're getting a lot of trades, they might not be good ones.
  - treis 5y ago
    Zillow's ambitions were bigger than that. They wanted to get enough inventory so that they could capture the 6% of the transaction going to agents. Plus a chunk of the miscellaneous fees that go to Mortgage brokers, inspectors, appraisers, and so on. For that they needed volume and the way to get volume in real estate is pay more than other people. I find it hard to believe that everyone there didn't realize that they were burning money. I saw it personally when looking to buy a house a couple years ago. There were a couple Zillow flips out there where the numbers obviously didn't add up. Whether or not they truly believed that it would work eventually or if it was just a way towards higher compensation/promotions/stock price or whatever is anyone's guess. But I suspect pretty much everyone involved from the person banging out ML models up to the CEO knew what was going on.
    - lordnacho 5y ago
      One thing I thought about is alternative scenarios. If the house market had rocketed up for whatever reason, Zillow could have made a pile of cash without having a sensible strategy. Then the articles would be talking about what geniuses they were and ML trampling traditional real estate. They would double down and when the market calmed, all the same issues would reappear. Perhaps that's actually what happened, because the house market has been buoyant recently, and they seem to have had the iBuyer business for a few years?
    - htrp 5y ago
      Startup mentality right? Lose money on each unit and make it up on traction.....
micro_cam 5y ago
I'm a former quant trader as well and this article rang true to me. I've certainly found myself in situations where I developed a model that performed so well in its specific domain that executives wanted to roll it out to a a much broader domain despite my advice. It very much is a sign of immaturity and lack of discipline from leadership. It's too easy to buy into the hype thinking you are being supportive of AI in your org but not realize that truly being supporting an AI strategy involves a lot of nuanced skepticism.
- KKKKkkkk1 5y ago
  Oh man, your comment about the need for nuanced skepticism reminded me of the co-CEO of Waymo who publicly claimed he's willing to trust his kids' lives to a driverless car.
  - sangnoir 5y ago
    "A" driverless car or Waymo's driverless cars? From the outside looking in, it appears Waymo's approach has a lot more self-restraint and is a lot less cavalier about safety compared to the competition.
  - rdtwo 5y ago
    I mean sure why not he probably didn’t raise them. With all that money he could buy another wife kids and nanny care
  - zsmi 5y ago
    > co-CEO of Waymo who publicly claimed he's willing to trust his kids' lives to a driverless car. So are lots of families in Chandler, Arizona

treis 5y ago

The big difference here is that the counterparty is deciding which orders get filled. Zillow could be right 99% of the time, but that 1% of the time where they're overpaying are the transactions that for sure will go through. And indeed if you go to /r/realestate anecdotally that is what happened. People would get offers from Zillow/Opendoor etc and talk to agents to see what they would list it at. If Zillow was higher than what the agents said they went with Zillow. If not they'd list it and sell it the old fashion way.

lordnacho 5y ago
Yeah, it's remarkable. You'd think that "I'll buy your house immediately" should be at a worse price than the guy who insists on the old school inspection and due diligence. In fact it might not be a bad strategy to simply start with that... see if there's desperate people needing the money fast, and use your corporate heft to both sweep up the deal (brand) and patiently dump it back into the ordinary market, pocketing the spread as a liquidity premium. One thing I remember my first boss saying is that it's hard to get a good trade on. Attractive deals have several interested parties. If you're getting a lot of trades, they might not be good ones.
- treis 5y ago
  Zillow's ambitions were bigger than that. They wanted to get enough inventory so that they could capture the 6% of the transaction going to agents. Plus a chunk of the miscellaneous fees that go to Mortgage brokers, inspectors, appraisers, and so on. For that they needed volume and the way to get volume in real estate is pay more than other people. I find it hard to believe that everyone there didn't realize that they were burning money. I saw it personally when looking to buy a house a couple years ago. There were a couple Zillow flips out there where the numbers obviously didn't add up. Whether or not they truly believed that it would work eventually or if it was just a way towards higher compensation/promotions/stock price or whatever is anyone's guess. But I suspect pretty much everyone involved from the person banging out ML models up to the CEO knew what was going on.
  - lordnacho 5y ago
    One thing I thought about is alternative scenarios. If the house market had rocketed up for whatever reason, Zillow could have made a pile of cash without having a sensible strategy. Then the articles would be talking about what geniuses they were and ML trampling traditional real estate. They would double down and when the market calmed, all the same issues would reappear. Perhaps that's actually what happened, because the house market has been buoyant recently, and they seem to have had the iBuyer business for a few years?
  - htrp 5y ago
    Startup mentality right? Lose money on each unit and make it up on traction.....

micro_cam 5y ago

I'm a former quant trader as well and this article rang true to me. I've certainly found myself in situations where I developed a model that performed so well in its specific domain that executives wanted to roll it out to a a much broader domain despite my advice. It very much is a sign of immaturity and lack of discipline from leadership. It's too easy to buy into the hype thinking you are being supportive of AI in your org but not realize that truly being supporting an AI strategy involves a lot of nuanced skepticism.

KKKKkkkk1 5y ago
Oh man, your comment about the need for nuanced skepticism reminded me of the co-CEO of Waymo who publicly claimed he's willing to trust his kids' lives to a driverless car.
- sangnoir 5y ago
  "A" driverless car or Waymo's driverless cars? From the outside looking in, it appears Waymo's approach has a lot more self-restraint and is a lot less cavalier about safety compared to the competition.
- rdtwo 5y ago
  I mean sure why not he probably didn’t raise them. With all that money he could buy another wife kids and nanny care
- zsmi 5y ago
  > co-CEO of Waymo who publicly claimed he's willing to trust his kids' lives to a driverless car. So are lots of families in Chandler, Arizona