One issue is that the newspapers believe that they are providing a service by allowing search engines to link to their page. From the perspective of the search

One issue is that the newspapers believe that they are providing a service by allowing search engines to link to their page. From the perspective of the search engine (and us readers), it feels like a bait-and-switch to present the full article to a crawler robot in exchange for a better search ranking, only to hide it and demand payment from a human. But over in Australia, old media organizations have successfully lobbied for a "link tax" which allows them to demand payment from search providers who link to them. We've still got a long way to go before we reach a consensus on the independence of search platforms and the idea that human viewers should be served the same content as web spiders.