3 ms·
It's not that complicated. Google indexes webpages. Webpages are identified by a URL. If, for any client, without any context or previous interaction, the URL
by jorams 7y ago
It's not that complicated. Google indexes webpages. Webpages are identified by a URL. If, for any client, without any context or previous interaction, the URL results in anything other than the indexed content, it should not appear in search results.
If a webpage requires login credentials, which the crawler obviously does not have, the crawler should have received a login page.
- skybrian 7y agoSure, that's basically giving up on indexing anything behind a gateway. Google could do that, but it's not at all clear they should.