4 ms·
Essentially yes - but "previewing" link and sending that link to Bingbot to crawl and index is another matter. Imagine you send someone a "private" link to a f
by rsbadger 4y ago
Essentially yes - but "previewing" link and sending that link to Bingbot to crawl and index is another matter.
Imagine you send someone a "private" link to a file...Bing sees that and indexes it for the world to see. Not cool.
- Gigachad 4y ago“Private links” should be covered by robots.txt. The only case I see this happening is for those “anyone with link” shares and those are easy to cover.
- rsbadger 4y agoAFAIK, most of those "private" links are actually just "unlisted" but they're still public. I'm sure Bing is indexing those too...
- Gigachad 4y agoAnything private should ideally be put behind authentication. If that isn't possible, than robots.txt. Search engines are _meant_ to index everything that is publicly accessible and not blacklisted on robots.txt.
- swiftcoder 4y agoIs this actually the Bingbot crawling for the index, or do they just use the same bingbot code to generate link previews? If I have a well-tested crawler sitting here, I don't see why I'd write a brand new one just to fetch previews in outlook...