The scraping example seems poorly chosen. The original blog post describing that example is no longer online, but archive.org has a copy: https://web.archive.or

The scraping example seems poorly chosen. The original blog post describing that example is no longer online, but archive.org has a copy: https://web.archive.org/web/20180822034920/https://blog.seanssmith.com/posts/pywren-web-scraping.html https://web.archive.org/web/20180822034920/https://blog.sean... If the author just wanted to fetch pages in parallel, they could have done better than 8 hours even on their own laptop (you can run more than one chromium process at a time). The real benefit they got from using AWS Lambda is that the requests weren't throttled or ghosted by Redfin, probably because the processes were running on enough different machines, with different IP addresses.