5 ms·
How to Scrape Zillow with Python and Scrapy
- gravypod 4y agoIs there any cleaner data source for home listings? I wish I could just setup a system to automatically filter listings by criteria that are important to me.
- xfour 4y agoUnfortunately no, the underlying systems companies such as Zillow use require basically joining to access to the local Multiple Listing Services (MLS) of a certain market. All of these have different rules of how data can be used.
- cute_boi 4y agoI tried to scrape zillow in past (2019) and it was giving me captcha headache. Didn't author faced similar issues?
- slezyr 4y ago> As of 2022, Zillow uses PerimeterX for scraping countermeasures. This can be bypassed by using ISP proxies from BrightData. Why ask without reading the article?
- cute_boi 4y agoWell, I used luminati( Currently BrightData). And, it was still giving me captcha. In fact, it was giving me more captcha in BrightData. So, I am not sure.
- Kimcha 4y agoThere are services that use humans to solve captchas for a very reasonable price, such as 2Captcha. Some of the protections are better than others. For example, you may need to use a real browser with Java script support instead of just HTTP requests. Browserless works well if you enable the stealth functions.
- throwaway193948 4y agowhat’s the point of scraping a scraper?
- Tarq0n 4y agoYou want a machine-readable dataset rather than a website, and the aggregator has already done some of the work for you. Scraping all of Zillow's sources instead would explode the complexity of the project.