None of the ones I know are that simple - but Check out Heritrix at http://crawler.archive.org http://crawler.archive.org and Nutch at http://nutch.apache.org h

None of the ones I know are that simple - but Check out Heritrix at http://crawler.archive.org http://crawler.archive.org and Nutch at http://nutch.apache.org http://nutch.apache.org Also worth checking out is http://80legs.com http://80legs.com