4 ms·

They publish some accuracy data. I think this compares the model to WHAT ACTUALLY HAPPENED https://www.zillow.com/z/zestimate/ https://www.zillow.com/z/zestima

They publish some accuracy data. I think this compares the model to WHAT ACTUALLY HAPPENED https://www.zillow.com/z/zestimate/ https://www.zillow.com/z/zestimate/ Based on this I suspect they do check the model based on WHAT ACTUALLY HAPPENED. what makes you think they don't check the model on WHAT ACTUALLY HAPPENED?

TheRealDunkirk 5y ago
> what makes you think they don't check the model on WHAT ACTUALLY HAPPENED? As I said, it seems like a pretty tight feedback loop to check, and then tweak the model. So it's confusing to me how they could be far enough off to ruin the business plan. I'll put it back to you: What am I missing? How could they revise the model based on many months of real-world data, and yet get it wrong enough to nearly destroy the business?
- deleted 5y ago
  [deleted]
- nuclearnice3 5y ago
  Good question. I don’t know. There are a lot of possible interacting causes. Let’s itemize a few. So there is not checking. This seems unlikely given, as you point out, it’s data right at hand and, as I point out, they publish checks. In general, I’m suspicious of WOW PEOPLE ARE STUPID explanations. There is checking and mistranslating model accuracy into business outcomes. The model is 90% accurate. Bid for 1000 houses. Maybe mainly people in the wrong by overpricing segment take you up on it. See the vast literature on adverse selection and winner’s curse. There is tweaking the model because you want the a certain outcome. > Over the last few weeks, it’s become clear that Zillow made a lot of mistakes. It tweaked its algorithm to be more aggressive with offers, winning bidding wars just as the market was starting to cool and overpaying for properties [1] The CEO has commented that the volatility of the prediction was a problem. He mentioned that the unit economics swung from over plus 5% to minus 7%. So the model might be a accurate on average but too noisy for their purpose. [2] Point is it’s not at all obvious that the problem here was accuracy on predictions of house prices. [1] https://www.curbed.com/2021/11/zillows-i-buying-algorithm-house-flipping.html https://www.curbed.com/2021/11/zillows-i-buying-algorithm-ho... [2] https://www.google.com/amp/s/seekingalpha.com/amp/article/4464694-zillow-group-inc-z-ceo-rich-barton-on-q3-2021-results-earnings-call-transcript https://www.google.com/amp/s/seekingalpha.com/amp/article/44...
- gpapilion 5y ago
  They became part of the feedback in their model. Real estate markets are high value with limited inventory, if you add market maker (zilllow) they set the floor. They had too much influence on the price, and didn’t just provide liquidity. Additionally these transactions take time to settle, and Zillow tried to improve their profit buy improving adding additional risk by increasing holding time. Finally im not sure how they hedged the risk. How do you short houses? If you consider mortgages like you holding 20% to protect the bank from losing, the variability can be difficult up to 20%, and leaves you with a hell of a downside.

TheRealDunkirk 5y ago

> what makes you think they don't check the model on WHAT ACTUALLY HAPPENED? As I said, it seems like a pretty tight feedback loop to check, and then tweak the model. So it's confusing to me how they could be far enough off to ruin the business plan. I'll put it back to you: What am I missing? How could they revise the model based on many months of real-world data, and yet get it wrong enough to nearly destroy the business?

deleted 5y ago
[deleted]
nuclearnice3 5y ago
Good question. I don’t know. There are a lot of possible interacting causes. Let’s itemize a few. So there is not checking. This seems unlikely given, as you point out, it’s data right at hand and, as I point out, they publish checks. In general, I’m suspicious of WOW PEOPLE ARE STUPID explanations. There is checking and mistranslating model accuracy into business outcomes. The model is 90% accurate. Bid for 1000 houses. Maybe mainly people in the wrong by overpricing segment take you up on it. See the vast literature on adverse selection and winner’s curse. There is tweaking the model because you want the a certain outcome. > Over the last few weeks, it’s become clear that Zillow made a lot of mistakes. It tweaked its algorithm to be more aggressive with offers, winning bidding wars just as the market was starting to cool and overpaying for properties [1] The CEO has commented that the volatility of the prediction was a problem. He mentioned that the unit economics swung from over plus 5% to minus 7%. So the model might be a accurate on average but too noisy for their purpose. [2] Point is it’s not at all obvious that the problem here was accuracy on predictions of house prices. [1] https://www.curbed.com/2021/11/zillows-i-buying-algorithm-house-flipping.html https://www.curbed.com/2021/11/zillows-i-buying-algorithm-ho... [2] https://www.google.com/amp/s/seekingalpha.com/amp/article/4464694-zillow-group-inc-z-ceo-rich-barton-on-q3-2021-results-earnings-call-transcript https://www.google.com/amp/s/seekingalpha.com/amp/article/44...
gpapilion 5y ago
They became part of the feedback in their model. Real estate markets are high value with limited inventory, if you add market maker (zilllow) they set the floor. They had too much influence on the price, and didn’t just provide liquidity. Additionally these transactions take time to settle, and Zillow tried to improve their profit buy improving adding additional risk by increasing holding time. Finally im not sure how they hedged the risk. How do you short houses? If you consider mortgages like you holding 20% to protect the bank from losing, the variability can be difficult up to 20%, and leaves you with a hell of a downside.