This exercise was more about learning Go, and not so much writing something directly work-related. We don't have a specific use case for a web crawler. We do, h

This exercise was more about learning Go, and not so much writing something directly work-related. We don't have a specific use case for a web crawler. We do, however, have a need for parsing and manipulating HTML (for dynamic templating), making HTTP requests, and writing concurrent code, so this exercise actually taught me a lot of relevant things. I started off with a crawler in mind (hence the function named "crawl" (good catch)), but I stopped a bit short so that I could write a comprehensible tutorial (I find writing tutorials of things I've just learned to really help my understanding). I haven't looked at asyncio yet, but it does look interesting. We've mostly been using gevent (py 2.7) and have found it to be a bit too magical (ie monkey-patching core libraries) and hard to track down specific bottlenecks.