I thought this was going to be an article about the difficulties in writing a web spider in PHP (which is not PHP's strong suit). I once wrote a crawler loosel

I thought this was going to be an article about the difficulties in writing a web spider in PHP (which is not PHP's strong suit). I once wrote a crawler loosely based off Snoopy (http://snoopy.sourceforge.net/ http://snoopy.sourceforge.net/) and it was a nightmare.