The article is meant to be taken as a general guide. The author does suggest that we use a pre-programmed crawler rather than code one ourselves. Refer to this

The article is meant to be taken as a general guide. The author does suggest that we use a pre-programmed crawler rather than code one ourselves. Refer to this passage: "Crawler. If you don't use an open source crawler, my advice is a super-simple multistep crawler. This is very important advice that will cut months off your development time, so if you ignore everything else, don't ignore this."