It does two things: Force everyone (including scrapers) to run a real JS engine, and force everyone to solve the challenge. The first effect is great, because

It does two things: Force everyone (including scrapers) to run a real JS engine, and force everyone to solve the challenge. The first effect is great, because it's a lot more annoying to bring up a full browser environment in your scraper than just run a curl command. But the actual proof of work only takes about 10ms on a server in native code, while it can take multiple seconds on a low-end phone. Given the companies in questions are building entire data centers to house all their GPUs, an extra 10ms per web-page is not a problem for them. They're going to spend orders of magnitude more compute actually training on the content they scraped, than solving the challenge. It's mostly the inconvenience of adapting to Anubis's JS requirements that held them back for a while, but the PoW difficulty mostly slowed down real users.