For now the solution is proof-of-work systems like Anubis combined with cookie-based rate limiting: you get throttled if your session cookie indicates you scrap

For now the solution is proof-of-work systems like Anubis combined with cookie-based rate limiting: you get throttled if your session cookie indicates you scraped here before, and if you throw the cookie out you get the POW challenge again. I don't know how long this will continue to work, but for my site at least it seems to be holding back the deluge, for the moment.