7 ms·

I am quite disappointed in the research that went into this article. (1) Author discovers that Zillow has a specific claim about accuracy: "around 90% of the l

by mcherm 10y ago

I am quite disappointed in the research that went into this article. (1) Author discovers that Zillow has a specific claim about accuracy: "around 90% of the listing Zestimate is “Within 20% of Sale Price” of the home". Author then gripes that this is "a really large margin for error". Really? Because I was prepared to praise Zillow for such honest and clear reporting. It only costs a few hundred dollars per home (Zillow covers nearly every home in the US) to get a professional to produce an assessment, and these assessments are usually only a little more accurate than what Zillow is claiming. (2) Author pulls up tables from Zillow showing a median error. Which is an excellent way to analyze this and is (I think) extraordinarily small. Author then proceeds to do other stuff. (3) Author decides to verify the values by checking a total of NINE houses. Is the accuracy of that test expected to impress us? Furthermore, one of these 9 is rejected because "(obviously a data glitch or input error. Dismiss this result)" -- but that's the whole POINT of using median error; it isn't affected by outliers. (4) Author finds that all of the values (all 3 of them) in Staten Island were pretty far off. Author asks professionals who say that the local market o Staten Island is behaving quite unusually at the moment. Reporting on this is the only thing I think the author did right.

dforrestwilson1 10y ago
Yeah people dump on Zillow quite a bit. Probably because home values can be a very emotionally stirring event. Zillow's aggregate economic data is solid, often a better predictor than the Case Shiller Index. I expect that Zestimates will improve over time as the data trickles down and lessons are learned.
- kelukelugames 10y ago
  This is correct. I used to work at Redfin and estimates are a sore point for homeowners. It's one large pissing contest for their egos. Everyone has the same complaint, "Why isn't my home priced as high as that one?!?"
  - irishcoffee 10y ago
    I'm the other guy. I bought my house for 260k a year ago, redfin now lists it at 330k, and there is just no possible way that is correct.
    - gozur88 10y ago
      I'm in the same boat as you. The Zillow estimate in my neighborhood seem to be about 15% higher than the sale price on any particular property. It's kind of academic for me, since I don't plan on selling any time soon, but it would be a mistake to do financial planning based on those estimates. The funny thing is the estimate doesn't change when a sale closes. I would think the surest indication of what you can get for a property is what it sold for last week.
  - chrisan 10y ago
    > "Why isn't my home priced as high as that one?!?" Could Redfin not explain why? I've never bought a home in an area Redfin is in, so I havent used the site before. If Redfin knows the formulas/parameters used for the estimates couldn't they break it down and do side by side comp of $ amounts?
DennisP 10y ago
A long time ago I got a real estate appraiser's license. Here is the procedure for appraising real estate, as taught by my state: 1) Pick three homes that you feel are comparable. 2) Pick out the material differences between those homes and your client's 3) Make up a value for each of those differences, and adjust accordingly (Unofficially speaking) If you don't like the number you come up with, just make different decisions in steps 1-3 until you get the number you want.
- robbyking 10y ago
  In the late 90's I worked for an online mortgage broker, and 9 times out of 10 when we ordered an appraisal, the appraiser would ask us what the selling price was and return an identical figure for their estimate.
  - jaxytee 10y ago
    I purchased a home couple years ago. The appraiser I used did exactly this.
  - leereeves 10y ago
    Amazing that people still pay for that.
    - URSpider94 10y ago
      "People" don't. It's a required part of the due diligence to get a mortgage. No appraisal, no loan. For the most part, the banks and brokers re-sell those loans immediately, so they don't stay on their books. All they need to protect them from lawsuits later is proof that they followed the process to the letter -- which involves getting a written appraisal. No mortgage broker or Realtor ever got rich by picking a fight with the appraiser to lower the appraised value ... there is literally nobody present at the table who has an incentive to push the value down.
      - DennisP 10y ago
hanoz 10y ago
> "around 90% of the listing Zestimate is “Within 20% of Sale Price” of the home" Yes, this sounds pretty good to me. I spent some time trying to do something sophisticated for the estimates on my site https://houseprices.io/ https://houseprices.io/ using repeat sales of similar properties weighted by distance, but I wasn't getting anywhere near that. I gave up and just used the Land Registry index for the county in the end. There are just so many potential local factors which you're never going to capture algorithmically, not least of which is how much the current owner might have spent doing it up. Although I do get the occasional irate message in my Contact Us form telling me exactly how much!
pdubbs90 10y ago
I feel like disappointing research is going to be a given when you have such an obvious conflict. The author has a strong motivation to make Zestimates seem bad (the less reliable the resources you have to buy/sell a home on your own are the more you'll need to contact them) and a million degrees of freedom (which neighborhoods in which cities, price floor for consideration, how many examples to pull ...) to get the answer they want. "Research" of this type should probably never be trusted.
smsm42 10y ago
I'm not sure at all how you can make conclusions from just 9 houses. I've been house hunting recently. Even houses in the same neighborhood are very different, and there are a lot of factors going into valuing a house. Also may be very subjective - some people would pay a lot for things that other people don't care about. Maybe Zillow is uniquely bad at one or two out of 20 factors - e.g., it can not detect a house in a good neighborhood that has been neglected and needs a lot of repair (I've seen houses in very good places that looked great from outside and required tens of thousands dollars of work on the inside just to be livable - maybe that's what happened to that -75% house - maybe it had a recent fire or flooding). Or maybe it doesn't value having a pool correctly. Who knows. But it's impossible to figure that out from only 9 houses, 3 per huge market! You need hundreds of data points to smooth out the individual differences and cancel out noise from all different criteria. That's exactly what Zillow's advantage is supposed to be - to aggregate thousands of data points. Maybe they do a good job, maybe not, but I don't think you can evaluate it by just using mere 3 houses per market of millions!

by mcherm 10y ago

dforrestwilson1 10y ago

Yeah people dump on Zillow quite a bit. Probably because home values can be a very emotionally stirring event. Zillow's aggregate economic data is solid, often a better predictor than the Case Shiller Index. I expect that Zestimates will improve over time as the data trickles down and lessons are learned.

kelukelugames 10y ago
This is correct. I used to work at Redfin and estimates are a sore point for homeowners. It's one large pissing contest for their egos. Everyone has the same complaint, "Why isn't my home priced as high as that one?!?"
- irishcoffee 10y ago
  I'm the other guy. I bought my house for 260k a year ago, redfin now lists it at 330k, and there is just no possible way that is correct.
  - gozur88 10y ago
    I'm in the same boat as you. The Zillow estimate in my neighborhood seem to be about 15% higher than the sale price on any particular property. It's kind of academic for me, since I don't plan on selling any time soon, but it would be a mistake to do financial planning based on those estimates. The funny thing is the estimate doesn't change when a sale closes. I would think the surest indication of what you can get for a property is what it sold for last week.
- chrisan 10y ago
  > "Why isn't my home priced as high as that one?!?" Could Redfin not explain why? I've never bought a home in an area Redfin is in, so I havent used the site before. If Redfin knows the formulas/parameters used for the estimates couldn't they break it down and do side by side comp of $ amounts?

DennisP 10y ago

A long time ago I got a real estate appraiser's license. Here is the procedure for appraising real estate, as taught by my state: 1) Pick three homes that you feel are comparable. 2) Pick out the material differences between those homes and your client's 3) Make up a value for each of those differences, and adjust accordingly (Unofficially speaking) If you don't like the number you come up with, just make different decisions in steps 1-3 until you get the number you want.

robbyking 10y ago
In the late 90's I worked for an online mortgage broker, and 9 times out of 10 when we ordered an appraisal, the appraiser would ask us what the selling price was and return an identical figure for their estimate.
- jaxytee 10y ago
  I purchased a home couple years ago. The appraiser I used did exactly this.
- leereeves 10y ago
  Amazing that people still pay for that.
  - URSpider94 10y ago
    "People" don't. It's a required part of the due diligence to get a mortgage. No appraisal, no loan. For the most part, the banks and brokers re-sell those loans immediately, so they don't stay on their books. All they need to protect them from lawsuits later is proof that they followed the process to the letter -- which involves getting a written appraisal. No mortgage broker or Realtor ever got rich by picking a fight with the appraiser to lower the appraised value ... there is literally nobody present at the table who has an incentive to push the value down.
    - DennisP 10y ago

hanoz 10y ago

> "around 90% of the listing Zestimate is “Within 20% of Sale Price” of the home" Yes, this sounds pretty good to me. I spent some time trying to do something sophisticated for the estimates on my site https://houseprices.io/ https://houseprices.io/ using repeat sales of similar properties weighted by distance, but I wasn't getting anywhere near that. I gave up and just used the Land Registry index for the county in the end. There are just so many potential local factors which you're never going to capture algorithmically, not least of which is how much the current owner might have spent doing it up. Although I do get the occasional irate message in my Contact Us form telling me exactly how much!

pdubbs90 10y ago

I feel like disappointing research is going to be a given when you have such an obvious conflict. The author has a strong motivation to make Zestimates seem bad (the less reliable the resources you have to buy/sell a home on your own are the more you'll need to contact them) and a million degrees of freedom (which neighborhoods in which cities, price floor for consideration, how many examples to pull ...) to get the answer they want. "Research" of this type should probably never be trusted.

smsm42 10y ago

I'm not sure at all how you can make conclusions from just 9 houses. I've been house hunting recently. Even houses in the same neighborhood are very different, and there are a lot of factors going into valuing a house. Also may be very subjective - some people would pay a lot for things that other people don't care about. Maybe Zillow is uniquely bad at one or two out of 20 factors - e.g., it can not detect a house in a good neighborhood that has been neglected and needs a lot of repair (I've seen houses in very good places that looked great from outside and required tens of thousands dollars of work on the inside just to be livable - maybe that's what happened to that -75% house - maybe it had a recent fire or flooding). Or maybe it doesn't value having a pool correctly. Who knows. But it's impossible to figure that out from only 9 houses, 3 per huge market! You need hundreds of data points to smooth out the individual differences and cancel out noise from all different criteria. That's exactly what Zillow's advantage is supposed to be - to aggregate thousands of data points. Maybe they do a good job, maybe not, but I don't think you can evaluate it by just using mere 3 houses per market of millions!