How could the crawler discover the pay-walled content? If sites explicitly allow the crawler to enter, what would be their argument to prevent that the search e

How could the crawler discover the pay-walled content? If sites explicitly allow the crawler to enter, what would be their argument to prevent that the search engine shows a cached version, especially if that's the search engine's policy from the start? That said, search engine for pay-walled sites with integrated micro payment isn't a bad outcome. What else could create an alternative to the ad-based business model?