Thanks - wasn't aware. (why: certificate-transparency, open-research-datasets, webwidecrawl) I still don't fathom why they just _ignore_ the request not to be

Thanks - wasn't aware. (why: certificate-transparency, open-research-datasets, webwidecrawl) I still don't fathom why they just _ignore_ the request not to be scraped with the above headers. It's rude.