2 ms·
Doesn't seem like it, since it apparently sends a user agent for an old version of Windows Chrome and not a crawler user agent https://archive.ph/4xptU https:/
by bhaney 2y ago
Doesn't seem like it, since it apparently sends a user agent for an old version of Windows Chrome and not a crawler user agent
https://archive.ph/4xptU https://archive.ph/4xptU
I guess it could have a whitelist of paywalled sites that it pretends to be a crawler for, and just uses the Chrome UA for everything else.