3 ms·
> so it is better to find a way to cache your website's HTML so you're not bombarded Of course, scrapers should identify themselves and then respect robots.txt
by jarofgreen 10mo ago
> so it is better to find a way to cache your website's HTML so you're not bombarded
Of course, scrapers should identify themselves and then respect robots.txt.