4 ms·
PHP and Python. Current count - 902873 I have also scrapped all the websites from Quantcast but haven't scanned it yet. The website is build using PHP includi
by samaybhavsar 10y ago
PHP and Python. Current count - 902873
I have also scrapped all the websites from Quantcast but haven't scanned it yet.
The website is build using PHP including the crawler which works for the new websites not in the list.
For scanning multiple websites, the crawler is in Python which uses multiprocessing.