4 ms·
In the readme of the project it indicates headers by which sites can opt out. But right after shows a flag by which the scrapers can ignore such headers. Seem
by darepublic 4y ago
In the readme of the project it indicates headers by which sites can opt out. But right after shows a flag by which the scrapers can ignore such headers. Seems like the author of this is basically telling everyone to deal with it.