9 ms·

Why Zillow Couldn’t Make Algorithmic House Pricing Work

66 points by Kydlaw 5y ago

notyourday 5y ago
Lack of liquidity. Algorithmic pricing works in liquid markets. Housing is not a liquid market.
mymythisisthis 5y ago
Lots of general contractors, with enough capital to buy a home, do some repairs, and sell it; They have a very good knowledge of the local market. Can do the repairs themselves. Larger companies probably focus on commercial developments and/or condos.
- thehappypm 5y ago
  The network that GCs need is astonishing. I did a renovation this year and my GC had of course his crew, and also hired a plumber, an electrician, a plaster guy, a flooring guy. The electrician and plumber both had their own crews. The network is such a big part of the battle.
supernova87a 5y ago
Isn't another problem that people who would use their automated service to sell their houses will skew towards those whose houses have some value-diminishing problem (which their algorithms have a hard time accounting for) or for other odd reasons have lower than usual market price? People who can fetch a premium for their houses probably wouldn't use an automated algorithm, which tends towards the mean? E.g. a self selection of those who would use this service tends to give Zillow a bunch of houses of lower value than average.
- swalkergibson 5y ago
  Without question this extends to the other iBuyers in the market. Zillow exiting because they could not make it work should be a massive alarm bell. Not only are they no longer buying new properties, they are liquidating their existing assets. You would not do that if you thought the market would come back to you.
  - zebraflask 5y ago
    If social media is to be believed, firing 2,000+ employees is also a bit concerning. At a bare minimum, that does not show much confidence in the organizational structure they put together, either.
  - everybodyknows 5y ago
    So I would think too, yet Opendoor has popped back up within 5% or so of its level prior to the Zillow news: https://www.marketwatch.com/investing/stock/open?mod=over_search https://www.marketwatch.com/investing/stock/open?mod=over_se...
11thEarlOfMar 5y ago
House price volatility is localized to the point that sample sizes are too small. Prices are generally neighborhood-based and the sales in a neighborhood are too infrequent. One quick divorce sale, or one over-eager buyer can skew the data enough to foil this approach. As an anecdotal example, my development has 115 homes. In the last 12 months, 3 have changed hands. The first sold at 5% over the Zestimate, the second, 10% over, the third 15% over. How would the Zillow algorithm predict the price of the next sale?
- short12 5y ago
  It's a very complicated algorithm of weighted distributions of /dev/urandom
- Tommah 5y ago
  The next would be 20% over.
johnthuss 5y ago
There was a great episode of The Indicator from Planet Money about this. One aspect not mentioned in this article is that when you offer an average price for a home, you are only going to get the owners of below-average homes taking you up on the offer, i.e. the Lemons. This is a tricky problem for Zillow to resolve given the mismatch in knowledge between the existing owner and the iBuyer. https://www.npr.org/2021/11/08/1053689886/ibuyers-zillow-and-the-lemons-problem https://www.npr.org/2021/11/08/1053689886/ibuyers-zillow-and...
- maxerickson 5y ago
  Doesn't that beg the question? Zillow can have better information on the market value of a house that it has worse access to information about than the owner.
- sjg007 5y ago
  I disagree. Real estate lemons are quickly identified by home inspectors which is why there is a variable % in the offer to buy. There are plenty of people that want to sell quickly for various reasons if not at market and at a reasonable discount. This is not the problem.
  - deleted 5y ago
    [deleted]
  - auntienomen 5y ago
    Home inspectors, in my experience, are pretty useless. They're running through a checklist, not actually thinking much.
    - oarabbus_ 5y ago
      They don't need to be thinking much, they need to be doing exactly that; running through a checklist of _major_ structural (foundation, roof, walls) or infrastructural (pipes, HVAC, electrical) issues with a house.
huitzitziltzin 5y ago
Interesting topic but this article is very short on the important details. “Buy and hold for a few months” would have been a good strategy for most of the pandemic, but Zillow appears to have overpaid substantially to the point where that didn’t work. Their predictions must have been badly off. (Also: this is a great example of the “winner’s curse” in first-price auctions!) The article quotes a figure of $65,000 “over market price”, but where “market price” comes from for those houses it bought is unclear. The claim of “market manipulation” by Zillow is ludicrous on its face without substantial evidence, which I’m guessing did not appear in a tiktok video. If all iBuyers in the Phoenix market _combined_ were buying 5% of homes, that doesn’t suggest to me that Zillow _alone_ would be able to exert any pricing pressure. I would love to read a more detailed article than this one if any are available.
- cyanydeez 5y ago
  are you just ignoring that zillow is a website that people search for their homes and zillow would be, like facebook, like google, like windows, the pureposefully distort results and dark patterns towars their own wares, skewing the actual perception of the msrket? seems odd to give them any good faith here, given what we have seen with every other xearch platform.
genghisjahn 5y ago
After the failure everyone is able to say why it wouldn't work.
- kyleee 5y ago
  These types of restrospectives are often very interesting examples of human capability to rationalize away culpability. Coldly I think it's good business skill as often people who are good at crafting these types of failure retrospectives seem to fail upwards
- black_13 5y ago
  first as tragedy, then as farce
paulpauper 5y ago
This does not mean the zestiamte estimate is flawed, but that flipping homes is inherently risky in independent of the price estimate algo. The huge, v-shaped recovery of the housing market and also stock market and GDP caught a lot of experts and pundits by surprise. The expectation was that, even with stimulus, that things would take years to recover as the US economy clawed its way out from the depths of the pandemic, but then Game Stop stock went up 100x and suddenly a FOMO unlike the likes ever seen took hold. Glad I didn't sell, and also I added to some stock positions in April 2020. Suddenly everything went from extreme over-supply to extreme shortages in a year.
- anonu 5y ago
  The "extreme shortage" media narrative is misleading. Supply in many areas is at or above pre pandemic levels. What's changed is a massive demand spike as society reshuffles and realigns to post pandemic life. It's a subtle and important point because it means inflation is not transitory. You should buy all and any risk assets to protect against inflation including homes and GME.
  - RhodesianHunter 5y ago
    It is absolutely not misleading. In fact I would say it's understated. We lost a half of builders, a third of tradesmen, and saw a massive drop in people training for the trades after 2008. We only caught up to the peak in new home builds a couple of years ago. Sure, there was a demand spike, but supply is about a decade behind in a market where it takes a very long time for supply to ramp up.
    - anonu 5y ago
      I believe you are correct [1][2] that wage workers in these occupations have declined since 2008. Is that related, however, to physical good supply chain disruptions in the past 12 months? And if you're referring to labor supply - we are basically at the same or better unemployment rate than we were in 2016... which was already a pretty good time in the economy. [1] https://fred.stlouisfed.org/series/LEU0254505500A https://fred.stlouisfed.org/series/LEU0254505500A [2] https://fred.stlouisfed.org/series/LNU04032231 https://fred.stlouisfed.org/series/LNU04032231
ccorda 5y ago
I'd recommend this article for a deeper look at how Zillow and its competitors exhibited different behavior in Phoenix as the market cooled: https://www.mikedp.com/articles/2021/10/19/ibuying-is-hard-zillow-pauses-new-purchases https://www.mikedp.com/articles/2021/10/19/ibuying-is-hard-z... > As the market cooled between August and September, Opendoor and Offerpad purchased fewer houses, while Zillow purchased more. > The iBuyers also adjusted to changing market conditions by paying less for houses. The median purchase price in Phoenix peaked in August. Opendoor and Offerpad's median purchase price also peaked in August before tracking the market and declining in September. But Zillow kept paying more and more.
kwertyoowiyop 5y ago
They have to offer the seller more than the seller thinks the winning bid would’ve been if they sold it without Zillow. And it has to be less (including their transaction and improvement costs) than the actual winning bid will be, when they turn around and sell it themselves.
Lazare 5y ago
I think Matt Levine's take was pretty interesting: https://www.bloomberg.com/opinion/articles/2021-11-03/zillow-is-done-trading-houses https://www.bloomberg.com/opinion/articles/2021-11-03/zillow...
moshun 5y ago
This whole issue is fascinating to me and no doubt will be studied in schools in the coming years. One of the lead product designers on Zillow Offers wrote a retrospective yesterday that outlines, broken user journeys, greed and the attempt to algorithmically one-size-fits-all the whole market. He lays a lot of the blame squarely at the feet of incompetent executive leadership From his notes: “They valued old-school growth hacks from 2007 over taking user-centered, data-informed, research-backed approaches. They spent more time flexing and building decks that drove an internal narrative of Sellers being happy, than actually addressing real and persistent product problems that ultimately led most sellers down a path where they didn't know what to expect, didn't understand the process and were inundated with calls and scheduling.” https://www.linkedin.com/posts/bardlavens_zillowoffers-zillowfailure-activity-6864367665520234496-yuld https://www.linkedin.com/posts/bardlavens_zillowoffers-zillo...
- ilamont 5y ago
  He adds: Many of these leaders, were leaders not on their merit, success or competency, but simply because they stayed at the company long enough that their titles grew as the company did. Curious how people at FAANG and other midsized to large technology companies feel about this. Can we look at a company like Google and find senior people who rose through the ranks not so much because of talent or intelligence or leadership qualities but rather because they stuck around long enough? Or is this something found mainly in toxic companies like the author described Zillow to be?
- jimj 5y ago
  If I didn't know he was talking about Zillow in that quote I would have thought he worked at my company.
- danrocks 5y ago
  I read it and while I appreciate the speaking up, this retro does not look nearly as good as the poster thinks it does. First, it is written from a perspective of "I/we know better, they didn't listen to us, they're incompetent", as in here: > We knew back then how many home sellers felt jilted, offended, etc. by this product. Then he doesn't clarify anything about why sellers would feel this way, what does "many" mean - is it all? A significant majority? 0.3%? Quantifying this would have brought his point across better. Then there's an assertion that they "should have taken a nuanced approach engaging with local communities" again without any justification as to why they needed to do it, with a mention to failure in new markets vs. success in markets already served by Opendoor without any context or even reason to be there. What does that even mean? The descent into the "toxic, abusive, etc.", "meritless leaders" name-calling is poor form and takes away from the best paragraph of this, which is where the author starts hinting at some of the real problems - sellers inundated with calls - he wants to discuss. Obviously the Zillow leadership screwed up on this, but this rant does little to illuminate why and frankly hits at some political ideology disagreement in addition to bad product design.
duxup 5y ago
>They tend to offer lower prices than traditional buyers, but attract sellers by promising faster, all-cash deals. That was the deal? They thought they'd get a lower price by offering cash and quick closing? Even in a down market (last time I bought) I offered quick closing and all cash. Sellers didn't like it. It took around 9 months before I got a call back from one house I wanted saying "Is that offer still good?" In the meantime they had accepted 3 other/ better offers ...but pending the other buyer selling their house, all of those feel through. And that was in a down market...
- mrfusion 5y ago
  Why didn’t sellers like it?
  - duxup 5y ago
    They did like getting offers lower than expexted and seemed to be happy to wait for the possibilty of a better offer, even if less reliable ( not cash,not a quick closing).
    - kcplate 5y ago
      Yep, sellers always want that extra $10k especially if not in a hurry. However, I’d bet their agents would love to sell that home for $10k under asking just to get rid of it fast rather than having to market it across months. That $10k to them is $300 on a split commission.
- maxerickson 5y ago
  They were acting as their own realtor, so they also had some margin compared to people paying for a realtor.
sjg007 5y ago
The simple answer is they didn't fact check with local realtors.
pragmaticalien8 5y ago
Rumor has it that this tiktok video[0] was the cause of the pause and eventual disbandment of iBuyer program. [0]https://www.tiktok.com/@seangotcher/video/7007855978309848325 https://www.tiktok.com/@seangotcher/video/700785597830984832...
- gpapilion 5y ago
  No matter how popular, it’s hard to see a single video doing that. The financials from last quarter are probably enough.
michael1999 5y ago
ctrl-F “adversarial environment” - no hits. Markets are not physics - there is no neutral POV.
MR4D 5y ago
Because models <> reality. That’s it. Full stop. Anyone who thinks otherwise will eventually face this same fate. For instance, from the sub-heading, “But the pandemic messed up its predictions” should really be this instead, “but their algorithms couldn’t handle a situation they didn’t anticipate”.
bradknowles 5y ago
Any metric, once it becomes a target, ceases to be a good metric. Seems to me like the guys at Zillow never heard of this one.
Iv 5y ago
2020-2021 was a very weird period. The fact that a given predictive model failed during these years is not really a strong indictment against the model. It makes sense to suspend the algorithmic trading during that time until the situation goes back to normal.

66 points by Kydlaw 5y ago

Lack of liquidity. Algorithmic pricing works in liquid markets. Housing is not a liquid market.

Lots of general contractors, with enough capital to buy a home, do some repairs, and sell it; They have a very good knowledge of the local market. Can do the repairs themselves. Larger companies probably focus on commercial developments and/or condos.

thehappypm 5y ago
The network that GCs need is astonishing. I did a renovation this year and my GC had of course his crew, and also hired a plumber, an electrician, a plaster guy, a flooring guy. The electrician and plumber both had their own crews. The network is such a big part of the battle.

supernova87a 5y ago

Isn't another problem that people who would use their automated service to sell their houses will skew towards those whose houses have some value-diminishing problem (which their algorithms have a hard time accounting for) or for other odd reasons have lower than usual market price? People who can fetch a premium for their houses probably wouldn't use an automated algorithm, which tends towards the mean? E.g. a self selection of those who would use this service tends to give Zillow a bunch of houses of lower value than average.

swalkergibson 5y ago
Without question this extends to the other iBuyers in the market. Zillow exiting because they could not make it work should be a massive alarm bell. Not only are they no longer buying new properties, they are liquidating their existing assets. You would not do that if you thought the market would come back to you.
- zebraflask 5y ago
  If social media is to be believed, firing 2,000+ employees is also a bit concerning. At a bare minimum, that does not show much confidence in the organizational structure they put together, either.
- everybodyknows 5y ago
  So I would think too, yet Opendoor has popped back up within 5% or so of its level prior to the Zillow news: https://www.marketwatch.com/investing/stock/open?mod=over_search https://www.marketwatch.com/investing/stock/open?mod=over_se...

11thEarlOfMar 5y ago

House price volatility is localized to the point that sample sizes are too small. Prices are generally neighborhood-based and the sales in a neighborhood are too infrequent. One quick divorce sale, or one over-eager buyer can skew the data enough to foil this approach. As an anecdotal example, my development has 115 homes. In the last 12 months, 3 have changed hands. The first sold at 5% over the Zestimate, the second, 10% over, the third 15% over. How would the Zillow algorithm predict the price of the next sale?

short12 5y ago
It's a very complicated algorithm of weighted distributions of /dev/urandom
Tommah 5y ago
The next would be 20% over.

johnthuss 5y ago

There was a great episode of The Indicator from Planet Money about this. One aspect not mentioned in this article is that when you offer an average price for a home, you are only going to get the owners of below-average homes taking you up on the offer, i.e. the Lemons. This is a tricky problem for Zillow to resolve given the mismatch in knowledge between the existing owner and the iBuyer. https://www.npr.org/2021/11/08/1053689886/ibuyers-zillow-and-the-lemons-problem https://www.npr.org/2021/11/08/1053689886/ibuyers-zillow-and...

maxerickson 5y ago
Doesn't that beg the question? Zillow can have better information on the market value of a house that it has worse access to information about than the owner.
sjg007 5y ago
I disagree. Real estate lemons are quickly identified by home inspectors which is why there is a variable % in the offer to buy. There are plenty of people that want to sell quickly for various reasons if not at market and at a reasonable discount. This is not the problem.
- deleted 5y ago
  [deleted]
- auntienomen 5y ago
  Home inspectors, in my experience, are pretty useless. They're running through a checklist, not actually thinking much.
  - oarabbus_ 5y ago
    They don't need to be thinking much, they need to be doing exactly that; running through a checklist of _major_ structural (foundation, roof, walls) or infrastructural (pipes, HVAC, electrical) issues with a house.

huitzitziltzin 5y ago

Interesting topic but this article is very short on the important details. “Buy and hold for a few months” would have been a good strategy for most of the pandemic, but Zillow appears to have overpaid substantially to the point where that didn’t work. Their predictions must have been badly off. (Also: this is a great example of the “winner’s curse” in first-price auctions!) The article quotes a figure of $65,000 “over market price”, but where “market price” comes from for those houses it bought is unclear. The claim of “market manipulation” by Zillow is ludicrous on its face without substantial evidence, which I’m guessing did not appear in a tiktok video. If all iBuyers in the Phoenix market _combined_ were buying 5% of homes, that doesn’t suggest to me that Zillow _alone_ would be able to exert any pricing pressure. I would love to read a more detailed article than this one if any are available.

cyanydeez 5y ago
are you just ignoring that zillow is a website that people search for their homes and zillow would be, like facebook, like google, like windows, the pureposefully distort results and dark patterns towars their own wares, skewing the actual perception of the msrket? seems odd to give them any good faith here, given what we have seen with every other xearch platform.

genghisjahn 5y ago

After the failure everyone is able to say why it wouldn't work.

kyleee 5y ago
These types of restrospectives are often very interesting examples of human capability to rationalize away culpability. Coldly I think it's good business skill as often people who are good at crafting these types of failure retrospectives seem to fail upwards
black_13 5y ago
first as tragedy, then as farce

paulpauper 5y ago

This does not mean the zestiamte estimate is flawed, but that flipping homes is inherently risky in independent of the price estimate algo. The huge, v-shaped recovery of the housing market and also stock market and GDP caught a lot of experts and pundits by surprise. The expectation was that, even with stimulus, that things would take years to recover as the US economy clawed its way out from the depths of the pandemic, but then Game Stop stock went up 100x and suddenly a FOMO unlike the likes ever seen took hold. Glad I didn't sell, and also I added to some stock positions in April 2020. Suddenly everything went from extreme over-supply to extreme shortages in a year.

anonu 5y ago
The "extreme shortage" media narrative is misleading. Supply in many areas is at or above pre pandemic levels. What's changed is a massive demand spike as society reshuffles and realigns to post pandemic life. It's a subtle and important point because it means inflation is not transitory. You should buy all and any risk assets to protect against inflation including homes and GME.
- RhodesianHunter 5y ago
  It is absolutely not misleading. In fact I would say it's understated. We lost a half of builders, a third of tradesmen, and saw a massive drop in people training for the trades after 2008. We only caught up to the peak in new home builds a couple of years ago. Sure, there was a demand spike, but supply is about a decade behind in a market where it takes a very long time for supply to ramp up.
  - anonu 5y ago
    I believe you are correct [1][2] that wage workers in these occupations have declined since 2008. Is that related, however, to physical good supply chain disruptions in the past 12 months? And if you're referring to labor supply - we are basically at the same or better unemployment rate than we were in 2016... which was already a pretty good time in the economy. [1] https://fred.stlouisfed.org/series/LEU0254505500A https://fred.stlouisfed.org/series/LEU0254505500A [2] https://fred.stlouisfed.org/series/LNU04032231 https://fred.stlouisfed.org/series/LNU04032231

ccorda 5y ago

I'd recommend this article for a deeper look at how Zillow and its competitors exhibited different behavior in Phoenix as the market cooled: https://www.mikedp.com/articles/2021/10/19/ibuying-is-hard-zillow-pauses-new-purchases https://www.mikedp.com/articles/2021/10/19/ibuying-is-hard-z... > As the market cooled between August and September, Opendoor and Offerpad purchased fewer houses, while Zillow purchased more. > The iBuyers also adjusted to changing market conditions by paying less for houses. The median purchase price in Phoenix peaked in August. Opendoor and Offerpad's median purchase price also peaked in August before tracking the market and declining in September. But Zillow kept paying more and more.

kwertyoowiyop 5y ago

They have to offer the seller more than the seller thinks the winning bid would’ve been if they sold it without Zillow. And it has to be less (including their transaction and improvement costs) than the actual winning bid will be, when they turn around and sell it themselves.

Lazare 5y ago

I think Matt Levine's take was pretty interesting: https://www.bloomberg.com/opinion/articles/2021-11-03/zillow-is-done-trading-houses https://www.bloomberg.com/opinion/articles/2021-11-03/zillow...

moshun 5y ago

This whole issue is fascinating to me and no doubt will be studied in schools in the coming years. One of the lead product designers on Zillow Offers wrote a retrospective yesterday that outlines, broken user journeys, greed and the attempt to algorithmically one-size-fits-all the whole market. He lays a lot of the blame squarely at the feet of incompetent executive leadership From his notes: “They valued old-school growth hacks from 2007 over taking user-centered, data-informed, research-backed approaches. They spent more time flexing and building decks that drove an internal narrative of Sellers being happy, than actually addressing real and persistent product problems that ultimately led most sellers down a path where they didn't know what to expect, didn't understand the process and were inundated with calls and scheduling.” https://www.linkedin.com/posts/bardlavens_zillowoffers-zillowfailure-activity-6864367665520234496-yuld https://www.linkedin.com/posts/bardlavens_zillowoffers-zillo...

ilamont 5y ago
He adds: Many of these leaders, were leaders not on their merit, success or competency, but simply because they stayed at the company long enough that their titles grew as the company did. Curious how people at FAANG and other midsized to large technology companies feel about this. Can we look at a company like Google and find senior people who rose through the ranks not so much because of talent or intelligence or leadership qualities but rather because they stuck around long enough? Or is this something found mainly in toxic companies like the author described Zillow to be?
jimj 5y ago
If I didn't know he was talking about Zillow in that quote I would have thought he worked at my company.
danrocks 5y ago
I read it and while I appreciate the speaking up, this retro does not look nearly as good as the poster thinks it does. First, it is written from a perspective of "I/we know better, they didn't listen to us, they're incompetent", as in here: > We knew back then how many home sellers felt jilted, offended, etc. by this product. Then he doesn't clarify anything about why sellers would feel this way, what does "many" mean - is it all? A significant majority? 0.3%? Quantifying this would have brought his point across better. Then there's an assertion that they "should have taken a nuanced approach engaging with local communities" again without any justification as to why they needed to do it, with a mention to failure in new markets vs. success in markets already served by Opendoor without any context or even reason to be there. What does that even mean? The descent into the "toxic, abusive, etc.", "meritless leaders" name-calling is poor form and takes away from the best paragraph of this, which is where the author starts hinting at some of the real problems - sellers inundated with calls - he wants to discuss. Obviously the Zillow leadership screwed up on this, but this rant does little to illuminate why and frankly hits at some political ideology disagreement in addition to bad product design.

duxup 5y ago

>They tend to offer lower prices than traditional buyers, but attract sellers by promising faster, all-cash deals. That was the deal? They thought they'd get a lower price by offering cash and quick closing? Even in a down market (last time I bought) I offered quick closing and all cash. Sellers didn't like it. It took around 9 months before I got a call back from one house I wanted saying "Is that offer still good?" In the meantime they had accepted 3 other/ better offers ...but pending the other buyer selling their house, all of those feel through. And that was in a down market...

mrfusion 5y ago
Why didn’t sellers like it?
- duxup 5y ago
  They did like getting offers lower than expexted and seemed to be happy to wait for the possibilty of a better offer, even if less reliable ( not cash,not a quick closing).
  - kcplate 5y ago
    Yep, sellers always want that extra $10k especially if not in a hurry. However, I’d bet their agents would love to sell that home for $10k under asking just to get rid of it fast rather than having to market it across months. That $10k to them is $300 on a split commission.
maxerickson 5y ago
They were acting as their own realtor, so they also had some margin compared to people paying for a realtor.

sjg007 5y ago

The simple answer is they didn't fact check with local realtors.

pragmaticalien8 5y ago

Rumor has it that this tiktok video[0] was the cause of the pause and eventual disbandment of iBuyer program. [0]https://www.tiktok.com/@seangotcher/video/7007855978309848325 https://www.tiktok.com/@seangotcher/video/700785597830984832...

gpapilion 5y ago
No matter how popular, it’s hard to see a single video doing that. The financials from last quarter are probably enough.

michael1999 5y ago

ctrl-F “adversarial environment” - no hits. Markets are not physics - there is no neutral POV.

MR4D 5y ago

Because models <> reality. That’s it. Full stop. Anyone who thinks otherwise will eventually face this same fate. For instance, from the sub-heading, “But the pandemic messed up its predictions” should really be this instead, “but their algorithms couldn’t handle a situation they didn’t anticipate”.

bradknowles 5y ago

Any metric, once it becomes a target, ceases to be a good metric. Seems to me like the guys at Zillow never heard of this one.

Iv 5y ago

2020-2021 was a very weird period. The fact that a given predictive model failed during these years is not really a strong indictment against the model. It makes sense to suspend the algorithmic trading during that time until the situation goes back to normal.