Does anyone have a solution for when the spider or backend content aggregator gets its IP address blacklisted? I think that Facebook got around this back in th

Does anyone have a solution for when the spider or backend content aggregator gets its IP address blacklisted? I think that Facebook got around this back in the day by having the user's device do the initial scraping. A side effect of this is that sometimes I'd post an article I found, but the preview image was blank because I submitted it too quickly or something. Until we have a free publicly downloadable cache of all websites, similar to CoralCDN (is this defunct?), then building our own search engines is probably a nonstarter.