4 ms·
Ugh, Pinterest is the experts-exchange of images. But there's a very straightforward way for Google to deal with Pinterext, and Medium, and Quora, and sign-up-
by SeanLuke 5y ago
Ugh, Pinterest is the experts-exchange of images. But there's a very straightforward way for Google to deal with Pinterext, and Medium, and Quora, and sign-up-only newspapers and magazines. Google just needs a preferences checkbox that says "Remove all sites which present themselves differently if they know GoogleBot is accessing them."
- dmitrygr 5y agoBack in the day (grumble grumble) this was grounds for being delisted from google entirely.
- dylan604 5y agoBut how does googlebot know that? after indexing with googlebot, rescan as pinky-swear-not-googlebot?
- danuker 5y agoYou could look for special treatment in robots.txt. But that wouldn't show you what the server would say if you had a different user-agent.
- paxys 5y agoYeah, why not? Index every page twice. If the non-bot result is materially different, rank the site lower. If the site is not accessible as a regular user, remove it from results entirely.
- dylan604 5y agoWould you run that from different servers/IPs in case site owners attempt to deny pinky-swear-not-googlebot?
- deckard1 5y agoObviously. This is not some technical marvel we're talking about. Google already tells you how to verify the site visitor is Googlebot[1], practically telling the world how to do cloaking. Google employees know Quora, Pinterest, etc. are gaming their system. This has nothing to do with trivial technical solutions and everything to do with business incentives. [1] https://developers.google.com/search/docs/advanced/crawling/verifying-googlebot https://developers.google.com/search/docs/advanced/crawling/...
- dazc 5y agoMaybe have some way of people reporting this behaviour and taking action when a critical mass of complaints point to one publisher? Rather than having a system that looks like what I've just outlined and taking no notice whatsoever.
- dylan604 5y agoSorry, but that's putting a heck of a lot of trust/faith/expectations in G actually paying attention to said feedback from said system. Expecting G to self police is just a non-starter as we clearly see people getting away with things now. Self-policing never works.
- Xenoamorphous 5y agoI think “cloaking” is the term.