6 ms·

From personal experience, two major contributors to the Zestimate are: 1. The town/city’s assessment of the home’s value for tax purposes 2. Recent sale prices

From personal experience, two major contributors to the Zestimate are: 1. The town/city’s assessment of the home’s value for tax purposes 2. Recent sale prices of homes nearby, mainly as the per square foot price.

TheRealDunkirk 5y ago
Another thing I don't understand. How do you develop a model with VOLUMES of real world data to train it on, and not just check how well it predicted pricing every month, both in aggregate (trending prices) and on individual listings that sold that month? This seems like a very small, closed loop to me. You could tweak the heck out of that algorithm in short order based on WHAT ACTUALLY HAPPENED.
- nuclearnice3 5y ago
  They publish some accuracy data. I think this compares the model to WHAT ACTUALLY HAPPENED https://www.zillow.com/z/zestimate/ https://www.zillow.com/z/zestimate/ Based on this I suspect they do check the model based on WHAT ACTUALLY HAPPENED. what makes you think they don't check the model on WHAT ACTUALLY HAPPENED?
  - TheRealDunkirk 5y ago
    > what makes you think they don't check the model on WHAT ACTUALLY HAPPENED? As I said, it seems like a pretty tight feedback loop to check, and then tweak the model. So it's confusing to me how they could be far enough off to ruin the business plan. I'll put it back to you: What am I missing? How could they revise the model based on many months of real-world data, and yet get it wrong enough to nearly destroy the business?
    - deleted 5y ago
      [deleted]
    - nuclearnice3 5y ago
      Good question. I don’t know. There are a lot of possible interacting causes. Let’s itemize a few. So there is not checking. This seems unlikely given, as you point out, it’s data right at hand and, as I point out, they publish checks. In general, I’m suspicious of WOW PEOPLE ARE STUPID explanations. There is checking and mistranslating model accuracy into business outcomes. The model is 90% accurate. Bid for 1000 houses. Maybe mainly people in the wrong by overpricing segment take you up on it. See the vast literature on adverse selection and winner’s curse. There is tweaking the model because you want the a certain outcome. > Over the last few weeks, it’s become clear that Zillow made a lot of mistakes. It tweaked its algorithm to be more aggressive with offers, winning bidding wars just as the market was starting to cool and overpaying for properties [1] The CEO has commented that the volatility of the prediction was a problem. He mentioned that the unit economics swung from over plus 5% to minus 7%. So the model might be a accurate on average but too noisy for their purpose. [2] Point is it’s not at all obvious that the problem here was accuracy on predictions of house prices. [1] https://www.curbed.com/2021/11/zillows-i-buying-algorithm-house-flipping.html https://www.curbed.com/2021/11/zillows-i-buying-algorithm-ho... [2] https://www.google.com/amp/s/seekingalpha.com/amp/article/4464694-zillow-group-inc-z-ceo-rich-barton-on-q3-2021-results-earnings-call-transcript https://www.google.com/amp/s/seekingalpha.com/amp/article/44...
    - gpapilion 5y ago
      They became part of the feedback in their model. Real estate markets are high value with limited inventory, if you add market maker (zilllow) they set the floor. They had too much influence on the price, and didn’t just provide liquidity. Additionally these transactions take time to settle, and Zillow tried to improve their profit buy improving adding additional risk by increasing holding time. Finally im not sure how they hedged the risk. How do you short houses? If you consider mortgages like you holding 20% to protect the bank from losing, the variability can be difficult up to 20%, and leaves you with a hell of a downside.
- benrapscallion 5y ago
  The problem with that is that many sellers use the Zestimate as a “ground truth” to set the sales price.
annoyingnoob 5y ago
My neighborhood is a mix of single family homes, condos, and apartments. I live in a single family home. Zillow regularly shows nearby condos as comps - well the condos have always sold for less than the single family homes. The condos also have a high HOA that does not show up in the price. Zillow does not know that there are different kinds of buyers for condos vs single family homes, even when they are close by and a similar size. My neighbor's house is the mirror image of my own, same size, same layout, same size lot, right next door. Zillow shows my home currently at $226K more than the one next door that is the same thing. Zillow's Zestimate of my home is more than a recent appraisal - which is odd because until 2021 Zillow had a really low Zestimate for my home, like the house next door. No idea why Zillow decided on a much higher value recently for me but not for my neighbor. And I'll have to say that lenders do consider the Zestimate as part of a loan, especially in a case where needing an appraisal might not be needed - if it on the bubble then the Zestimate might be the difference of whether you get an appraisal or not.

TheRealDunkirk 5y ago

Another thing I don't understand. How do you develop a model with VOLUMES of real world data to train it on, and not just check how well it predicted pricing every month, both in aggregate (trending prices) and on individual listings that sold that month? This seems like a very small, closed loop to me. You could tweak the heck out of that algorithm in short order based on WHAT ACTUALLY HAPPENED.

nuclearnice3 5y ago
They publish some accuracy data. I think this compares the model to WHAT ACTUALLY HAPPENED https://www.zillow.com/z/zestimate/ https://www.zillow.com/z/zestimate/ Based on this I suspect they do check the model based on WHAT ACTUALLY HAPPENED. what makes you think they don't check the model on WHAT ACTUALLY HAPPENED?
- TheRealDunkirk 5y ago
  > what makes you think they don't check the model on WHAT ACTUALLY HAPPENED? As I said, it seems like a pretty tight feedback loop to check, and then tweak the model. So it's confusing to me how they could be far enough off to ruin the business plan. I'll put it back to you: What am I missing? How could they revise the model based on many months of real-world data, and yet get it wrong enough to nearly destroy the business?
  - deleted 5y ago
    [deleted]
  - nuclearnice3 5y ago
    Good question. I don’t know. There are a lot of possible interacting causes. Let’s itemize a few. So there is not checking. This seems unlikely given, as you point out, it’s data right at hand and, as I point out, they publish checks. In general, I’m suspicious of WOW PEOPLE ARE STUPID explanations. There is checking and mistranslating model accuracy into business outcomes. The model is 90% accurate. Bid for 1000 houses. Maybe mainly people in the wrong by overpricing segment take you up on it. See the vast literature on adverse selection and winner’s curse. There is tweaking the model because you want the a certain outcome. > Over the last few weeks, it’s become clear that Zillow made a lot of mistakes. It tweaked its algorithm to be more aggressive with offers, winning bidding wars just as the market was starting to cool and overpaying for properties [1] The CEO has commented that the volatility of the prediction was a problem. He mentioned that the unit economics swung from over plus 5% to minus 7%. So the model might be a accurate on average but too noisy for their purpose. [2] Point is it’s not at all obvious that the problem here was accuracy on predictions of house prices. [1] https://www.curbed.com/2021/11/zillows-i-buying-algorithm-house-flipping.html https://www.curbed.com/2021/11/zillows-i-buying-algorithm-ho... [2] https://www.google.com/amp/s/seekingalpha.com/amp/article/4464694-zillow-group-inc-z-ceo-rich-barton-on-q3-2021-results-earnings-call-transcript https://www.google.com/amp/s/seekingalpha.com/amp/article/44...
  - gpapilion 5y ago
    They became part of the feedback in their model. Real estate markets are high value with limited inventory, if you add market maker (zilllow) they set the floor. They had too much influence on the price, and didn’t just provide liquidity. Additionally these transactions take time to settle, and Zillow tried to improve their profit buy improving adding additional risk by increasing holding time. Finally im not sure how they hedged the risk. How do you short houses? If you consider mortgages like you holding 20% to protect the bank from losing, the variability can be difficult up to 20%, and leaves you with a hell of a downside.
benrapscallion 5y ago
The problem with that is that many sellers use the Zestimate as a “ground truth” to set the sales price.

annoyingnoob 5y ago

My neighborhood is a mix of single family homes, condos, and apartments. I live in a single family home. Zillow regularly shows nearby condos as comps - well the condos have always sold for less than the single family homes. The condos also have a high HOA that does not show up in the price. Zillow does not know that there are different kinds of buyers for condos vs single family homes, even when they are close by and a similar size. My neighbor's house is the mirror image of my own, same size, same layout, same size lot, right next door. Zillow shows my home currently at $226K more than the one next door that is the same thing. Zillow's Zestimate of my home is more than a recent appraisal - which is odd because until 2021 Zillow had a really low Zestimate for my home, like the house next door. No idea why Zillow decided on a much higher value recently for me but not for my neighbor. And I'll have to say that lenders do consider the Zestimate as part of a loan, especially in a case where needing an appraisal might not be needed - if it on the bubble then the Zestimate might be the difference of whether you get an appraisal or not.