5 ms·

Google used to have a policy of sites being required to show Google the same page that they serve users. It seems that has been eroded. I'm not sure how that s

Google used to have a policy of sites being required to show Google the same page that they serve users. It seems that has been eroded. I'm not sure how that serves Google's interests, except perhaps that it keeps them out of legal hot water vs the news industry?

Ozzie_osman 2y ago
It's called cloaking and it's still looked down upon from an SEO perspective. That said, there's a huge gray area. Purposefully cloaking a result to trick a search engine would get penalized. "Updating" a page with newer content periodically is harder to assess.
- lelandfe 2y ago
  There's also "dynamic rendering," in which you serve Google/crawlers "similar" content to, in theory, avoid JS-related SEO issues. However, it can just be a way to do what the parent commenter dislikes: render a mini-blurb unfound on the actual page. Shoot, even a meta description qualifies for that - thankfully Google uses them less and less.
  - frde_me 2y ago
    Google will reliably index dynamic sites rendered using JS. And other search engines do the same. There's really no good reason to do this if you want to be indexed on search engines.
    - lelandfe 2y ago
      Agreed. Yet whether it should be done is different than whether it is done. Google was recommending it in 2018, and degraded it to a "workaround" just two years ago. Sites still do it, SaaS products still tout its benefits, and Google does not penalize sites for it. GP's gripe about SERP blurbs being missing is still very much extant and blessed by Google.
izacus 2y ago
People on HN have repeatedly stated that Google is "stealing their content" from their websites so it seems like this is a natural extension to that widespread opinion. Isn't this the web we want? One where big corporations don't steal our websites? Right?
- meiraleal 2y ago
  People on HN will always find a way to see a good side from Google's terrible product and engineering
  - izacus 2y ago
    Which is great that there's one less of their products, it's terrible anyway.
- ysofunny 2y ago
  ownership over websites does not work the way people expect nor want I'm tired of saying this, yelling at clouds
- dspillett 2y ago
  > People on HN have repeatedly stated It is definitely an area where there is no single "people of HN" - opinion varies widely and is often more nuanced than a binary for/against⁰ matter. From an end user PoV it is (was) a very useful feature, one that kept me using Google by default¹, and I think that many like me used it as a backup when content was down at source. The key problem with giving access to cached copies like this is when it effectively becomes the default view, holding users in the search providers garden instead of the content provider being particularly acknowledged never mind visited and the search service making money from that through adverts and related stalking. I have sympathy for honest sites when their content is used this way, though those that give search engines full text but paywall most of it when I look who complain about the search engine showing fuller text, can do one. Also those who do the "turn off your stalker blocker or well not show you anything" thing. ---- [0] Or ternary for/indifferent/against one. [1] I'm now finally moving away, currently experimenting with Kagi, as a number of little things that kept me there are no longer true and more and more irritations² keep appearing. [2] Like most of the first screen full of a result being adverts and an AI summary that I don't want, just give me the relevant links please…
mtkd 2y ago
Content delivery is becoming so dynamic and targeted, there is no way that can work effectively now -- even for first impression as one or more MVTs may be in place
luckylion 2y ago
That policy was never actually enforced that way, however. They'd go after you if you had entirely different content for google vs for users, but large scientific publishers already had "full pdf to google, HTML abstract + pay wall to users" 20 years ago and it was never an issue. It makes some sense, too because the edges are blurry. If a user from France receives a french version on the same URL where a US-user would receive an english version, is that already different content? What if (as it usually happens), one language gets prioritized and the other only receives updates once in a while? And while Google recommends to treat them like you'd treat any other user when it comes to e.g. geo-targeting, in reality that's not possible if you do anything that requires compliance and isn't available in California. They do Smartphone and Desktop-crawling, but they don't do any state- or even country-level crawling. Which is understandable as well, few sites really need to or want to do that, and it would require _a lot_ more crawling (e.g. in the US you'd need to hit each URL once per state), and there's no protocol to indicate it (and there probably won't be one because it's too rare).
- kaoD 2y ago
  > It makes some sense, too because the edges are blurry. If a user from France receives a french version on the same URL where a US-user would receive an english version, is that already different content? The recommended (or rather correct) way to do this is to have multiple language-scoped URLs, be it a path fragment or entirely different (sub)domains. Then you cross-link each other with <link> tags with rel="alternate" and hreflang (for SEO purposes) and give the user some affordance to switch between them (only if they want to do so). https://developers.google.com/search/docs/specialty/international/localized-versions https://developers.google.com/search/docs/specialty/internat... Public URLs should never show different content depending on anything else than the URL and current server state. If you really need to do this, 302 Redirect into a different URL. But really, don't do that. If the URL is language-qualified but it doesn't match whatever language/region you guessed for the user (which might very well be wrong and/or conflicting, e.g. my language and IP's country don't match, people travel, etc.) just let the user know they can switch URLs manually if they want to do so. You're just going to annoy me if you redirect me away to a language I don't want just because you tried being too smart.
Retr0id 2y ago
I most commonly run into this issue when the search keyword was found in some dynamically retrieved non-body text - maybe it was on a site's "most recent comments" panel or something.

Ozzie_osman 2y ago

It's called cloaking and it's still looked down upon from an SEO perspective. That said, there's a huge gray area. Purposefully cloaking a result to trick a search engine would get penalized. "Updating" a page with newer content periodically is harder to assess.

lelandfe 2y ago
There's also "dynamic rendering," in which you serve Google/crawlers "similar" content to, in theory, avoid JS-related SEO issues. However, it can just be a way to do what the parent commenter dislikes: render a mini-blurb unfound on the actual page. Shoot, even a meta description qualifies for that - thankfully Google uses them less and less.
- frde_me 2y ago
  Google will reliably index dynamic sites rendered using JS. And other search engines do the same. There's really no good reason to do this if you want to be indexed on search engines.
  - lelandfe 2y ago
    Agreed. Yet whether it should be done is different than whether it is done. Google was recommending it in 2018, and degraded it to a "workaround" just two years ago. Sites still do it, SaaS products still tout its benefits, and Google does not penalize sites for it. GP's gripe about SERP blurbs being missing is still very much extant and blessed by Google.

izacus 2y ago

People on HN have repeatedly stated that Google is "stealing their content" from their websites so it seems like this is a natural extension to that widespread opinion. Isn't this the web we want? One where big corporations don't steal our websites? Right?

meiraleal 2y ago
People on HN will always find a way to see a good side from Google's terrible product and engineering
- izacus 2y ago
  Which is great that there's one less of their products, it's terrible anyway.
ysofunny 2y ago
ownership over websites does not work the way people expect nor want I'm tired of saying this, yelling at clouds
dspillett 2y ago
> People on HN have repeatedly stated It is definitely an area where there is no single "people of HN" - opinion varies widely and is often more nuanced than a binary for/against⁰ matter. From an end user PoV it is (was) a very useful feature, one that kept me using Google by default¹, and I think that many like me used it as a backup when content was down at source. The key problem with giving access to cached copies like this is when it effectively becomes the default view, holding users in the search providers garden instead of the content provider being particularly acknowledged never mind visited and the search service making money from that through adverts and related stalking. I have sympathy for honest sites when their content is used this way, though those that give search engines full text but paywall most of it when I look who complain about the search engine showing fuller text, can do one. Also those who do the "turn off your stalker blocker or well not show you anything" thing. ---- [0] Or ternary for/indifferent/against one. [1] I'm now finally moving away, currently experimenting with Kagi, as a number of little things that kept me there are no longer true and more and more irritations² keep appearing. [2] Like most of the first screen full of a result being adverts and an AI summary that I don't want, just give me the relevant links please…

mtkd 2y ago

Content delivery is becoming so dynamic and targeted, there is no way that can work effectively now -- even for first impression as one or more MVTs may be in place

luckylion 2y ago

That policy was never actually enforced that way, however. They'd go after you if you had entirely different content for google vs for users, but large scientific publishers already had "full pdf to google, HTML abstract + pay wall to users" 20 years ago and it was never an issue. It makes some sense, too because the edges are blurry. If a user from France receives a french version on the same URL where a US-user would receive an english version, is that already different content? What if (as it usually happens), one language gets prioritized and the other only receives updates once in a while? And while Google recommends to treat them like you'd treat any other user when it comes to e.g. geo-targeting, in reality that's not possible if you do anything that requires compliance and isn't available in California. They do Smartphone and Desktop-crawling, but they don't do any state- or even country-level crawling. Which is understandable as well, few sites really need to or want to do that, and it would require _a lot_ more crawling (e.g. in the US you'd need to hit each URL once per state), and there's no protocol to indicate it (and there probably won't be one because it's too rare).

kaoD 2y ago
> It makes some sense, too because the edges are blurry. If a user from France receives a french version on the same URL where a US-user would receive an english version, is that already different content? The recommended (or rather correct) way to do this is to have multiple language-scoped URLs, be it a path fragment or entirely different (sub)domains. Then you cross-link each other with <link> tags with rel="alternate" and hreflang (for SEO purposes) and give the user some affordance to switch between them (only if they want to do so). https://developers.google.com/search/docs/specialty/international/localized-versions https://developers.google.com/search/docs/specialty/internat... Public URLs should never show different content depending on anything else than the URL and current server state. If you really need to do this, 302 Redirect into a different URL. But really, don't do that. If the URL is language-qualified but it doesn't match whatever language/region you guessed for the user (which might very well be wrong and/or conflicting, e.g. my language and IP's country don't match, people travel, etc.) just let the user know they can switch URLs manually if they want to do so. You're just going to annoy me if you redirect me away to a language I don't want just because you tried being too smart.

Retr0id 2y ago

I most commonly run into this issue when the search keyword was found in some dynamically retrieved non-body text - maybe it was on a site's "most recent comments" panel or something.