4 ms·
instead of scraping then, they could pay the fsf for a dump of the site or some API access or something, right? why overload the servers normally.
by nemomarx 3mo ago
instead of scraping then, they could pay the fsf for a dump of the site or some API access or something, right? why overload the servers normally.
- GoblinSlayer 3mo agoThat's what commoncrawl does.
- nemomarx 3mo agocommon crawl pays the sites they crawl?