Sorry I never looked at the perf aspect of it since my needs were modest (less than 50 web pages per day). The first thing that would come to my mind is to para

Sorry I never looked at the perf aspect of it since my needs were modest (less than 50 web pages per day). The first thing that would come to my mind is to parallelize it but then it's going to become pretty complex.