An algorithm may have incorrectly marked a site as malicious, or crawling might be failing because of an malformed http header etc. Who knows... When there is

An algorithm may have incorrectly marked a site as malicious, or crawling might be failing because of an malformed http header etc. Who knows... When there is feedback loops involved (in the context of an integrity system, for example), it doesn't outright block everything. In the past work i had our systems blocked some publishers traffic wrongfully, while not blocking others when it should(i was in display ads).