Guido van Rossum, the creator of Python, wrote a web crawler as a motivating example for asyncio. You can find the code for it here: https://github.com/aosaboo

Guido van Rossum, the creator of Python, wrote a web crawler as a motivating example for asyncio. You can find the code for it here: https://github.com/aosabook/500lines/tree/master/crawler https://github.com/aosabook/500lines/tree/master/crawler And a detailed post about its design, co-written with A. Jesse Jiryu Davis, here: http://aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html http://aosabook.org/en/500L/a-web-crawler-with-asyncio-corou...