5 ms·

I HATE clicking on a Google link and discovering that it's paywalled. It's a massive waste of time.

wodenokoto 5y ago
Why is this even allowed by google? It’s a bait and switch similar to serving visitors and search engines different content.
- joegahona 5y ago
  How is it a bait and switch? If the content is relevant, it's relevant. If you clicked through to something that said it was about NBA basketball and it was instead about knitting, _that_ would be a bait and switch.
  - zajio1am 5y ago
    First, if i get paywall page instead of the article then it is not relevant, because it is about paying and not about the original query. Second, it does not make sense - if i use a public search engine, i expect publicly available information. If i have subscription to specific newspaper i would use search dialog in that newspaper. If a public search engine returns me paywall for a specific newspaper, it is minimal chance that it is the same newspaper i am already subscribed to.
    - joegahona 5y ago
      So if a user subscribes to the paywall site, that article is "about paying"? The article's content is distinct from the method by which the publication chooses to earn revenue. A paywall article is "about paying" as much as an ad-ridden article is "about belly-fat ads." They're both designed to make money, only one shifts the cost to the end user. I'm all for putting a lock icon next to it in Google searches, or doing whatever's necessary to signal that it's paywalled, I guess, but don't blame the search engine for attempting to organize the world's information.
  - PeterisP 5y ago
    I'm searching for publicly available content published on the internet. You're telling Google that there is certain public content available at this URL - that's the "bait" part. Afterwards you're telling me that no, this content is not actually available to the public - that's the "switch" part. I mean, there's nothing wrong with paywalls, but you have to be honest and explicit about what you're offering. If the page says "Latest NBA game - pay $1 to find out who won", that's fine, but then that's what you should provide to my search engine; and if you want my search engine to index "It was close but no cigar for the Nets as they fell 111-107 to the Chicago Bulls this past Saturday." , then this should the actual content that I get when I go to that page.
    - joegahona 5y ago
      When I search "30 Rock," four of the top results are pay services to consume this content. (Netflix, Hulu, Amazon, and NBC.) Do you view those as bait and switch? I.e. should those shows either be free, or Google should banish these from search? I see your point, but I'm not sure Google ever made the promise that what appears in its search engine was free to consume. Perhaps the expectation is there, I get it.
      - PeterisP 5y ago
        No, that's the exact equivalent to what I said above "if the page says "Latest NBA game - pay $1 to find out who won", that's fine, but then that's what you should provide to my search engine." - that's delivering searches based on the publicly available data, that page is about "30 Rock" and that's what I'm searching for. This dissonance is not really applicable to video or audio, it's a problem specific to text content, because for video we don't have widespread search for actual content like we do for text, we're searching just on the metadata so the availability of the actual content does not affect searches. For text content, if I search for a keyword and a phrase, it gets (or should get) searched on the actual content that's available. My beef is if the site wants google to index the actually unavailable content - for example, if you're selling a book and you're making just the title and abstract available to the public without buying that book, then searches should be based on this metadata. But if you tell google "here's also the content of page 313 with an interesting factoid about 30 Rock (which is the only place in our book which mentions 30 Rock), please index that so that searchers find our book not only when searching for its title but also when searching for 30 Rock" then I would consider it acceptable if and only if that page 313 with that 30 Rock related content is actually available to public; and if you want to keep that text hidden, then that's your prerogative but in that case you don't get to use that text as "advertising" to draw in searchers.
- bduerst 5y ago
  Many news websites won't paywall the first article from a Google Search (WSJ is one though). I'd be willing to bet in these instances you're using a traffic blocker or privacy extension that's clearing HTTP referer or some other means by which the website detects you're from Google instead of being direct (paywall) traffic.
  - kevin_thibedeau 5y ago
    Google has an official policy against this sort of behavior.
    - bduerst 5y ago
      Which behavior? First Click Free was ditched years ago, but websites will still allow x number of free articles. Some don't, and still get indexed.
freediver 5y ago
How is Google supposed to know if you have access or not? (because if you have, and it doesn't show you the relevant result, it is doing a disservice to you)
- mitchdoogle 5y ago
  I don't think they are blaming Google. It's the news org's fault in that case, for showing the Google bot a full article while actual people get a paywall.
  - freediver 5y ago
    Well again, the people that have a subscription do not get a paywall, they get content when they click the same Google link and are probably very happy with that outocme. How is Google supposed to know who has access?
- oh_sigh 5y ago
  Google could log which sites have a paywall, and put an icon next to search result that you might need to pay money to access the article. I could also let google know which sites I subscribe to and if I have a WSJ subscription, it could rank those articles a little higher to similar results on sites I don't have a subscription to. For example, something similar happens when you google TV shows, e.g. https://www.google.com/search?q=mad+men https://www.google.com/search?q=mad+men There is a panel on the right hand side showing which streaming services stream the show. I can tell google which ones I have and it will improve the recommendations based on that.
- erik_seaberg 5y ago
  Even if I were to subscribe to a hundred newspapers (which nobody does), that would leave tens of thousands I don’t. Google should avoid showing me results that are 99% useless.
- PeterisP 5y ago
  Google is indexing public pages as "someone" who does not have any specific access. Google has chosen not to know or care whether I have special access, and index the content as if it's not there - perhaps it's a disservice, but that's the service they're offering to their users. And the actual news sites do not and should not have any say in this, the decision of what to index and what to show to users should remain between each search engine and their users. So if a search engine chooses to index the "content-as-if-you're-anonymous-and-not-logged-in" and if you try to trick this service into showing something else than what's actually published to everyone (serving visitors and search engines different content), then it's quite reasonable for google to treat it as a cheating attempt (because, really, it is one) and choose to penalize it in their search policies - because that's what I as a search engine user would want.

Why is this even allowed by google? It’s a bait and switch similar to serving visitors and search engines different content.

joegahona 5y ago
How is it a bait and switch? If the content is relevant, it's relevant. If you clicked through to something that said it was about NBA basketball and it was instead about knitting, _that_ would be a bait and switch.
- zajio1am 5y ago
  First, if i get paywall page instead of the article then it is not relevant, because it is about paying and not about the original query. Second, it does not make sense - if i use a public search engine, i expect publicly available information. If i have subscription to specific newspaper i would use search dialog in that newspaper. If a public search engine returns me paywall for a specific newspaper, it is minimal chance that it is the same newspaper i am already subscribed to.
  - joegahona 5y ago
    So if a user subscribes to the paywall site, that article is "about paying"? The article's content is distinct from the method by which the publication chooses to earn revenue. A paywall article is "about paying" as much as an ad-ridden article is "about belly-fat ads." They're both designed to make money, only one shifts the cost to the end user. I'm all for putting a lock icon next to it in Google searches, or doing whatever's necessary to signal that it's paywalled, I guess, but don't blame the search engine for attempting to organize the world's information.
- PeterisP 5y ago
  I'm searching for publicly available content published on the internet. You're telling Google that there is certain public content available at this URL - that's the "bait" part. Afterwards you're telling me that no, this content is not actually available to the public - that's the "switch" part. I mean, there's nothing wrong with paywalls, but you have to be honest and explicit about what you're offering. If the page says "Latest NBA game - pay $1 to find out who won", that's fine, but then that's what you should provide to my search engine; and if you want my search engine to index "It was close but no cigar for the Nets as they fell 111-107 to the Chicago Bulls this past Saturday." , then this should the actual content that I get when I go to that page.
  - joegahona 5y ago
    When I search "30 Rock," four of the top results are pay services to consume this content. (Netflix, Hulu, Amazon, and NBC.) Do you view those as bait and switch? I.e. should those shows either be free, or Google should banish these from search? I see your point, but I'm not sure Google ever made the promise that what appears in its search engine was free to consume. Perhaps the expectation is there, I get it.
    - PeterisP 5y ago
      No, that's the exact equivalent to what I said above "if the page says "Latest NBA game - pay $1 to find out who won", that's fine, but then that's what you should provide to my search engine." - that's delivering searches based on the publicly available data, that page is about "30 Rock" and that's what I'm searching for. This dissonance is not really applicable to video or audio, it's a problem specific to text content, because for video we don't have widespread search for actual content like we do for text, we're searching just on the metadata so the availability of the actual content does not affect searches. For text content, if I search for a keyword and a phrase, it gets (or should get) searched on the actual content that's available. My beef is if the site wants google to index the actually unavailable content - for example, if you're selling a book and you're making just the title and abstract available to the public without buying that book, then searches should be based on this metadata. But if you tell google "here's also the content of page 313 with an interesting factoid about 30 Rock (which is the only place in our book which mentions 30 Rock), please index that so that searchers find our book not only when searching for its title but also when searching for 30 Rock" then I would consider it acceptable if and only if that page 313 with that 30 Rock related content is actually available to public; and if you want to keep that text hidden, then that's your prerogative but in that case you don't get to use that text as "advertising" to draw in searchers.
bduerst 5y ago
Many news websites won't paywall the first article from a Google Search (WSJ is one though). I'd be willing to bet in these instances you're using a traffic blocker or privacy extension that's clearing HTTP referer or some other means by which the website detects you're from Google instead of being direct (paywall) traffic.
- kevin_thibedeau 5y ago
  Google has an official policy against this sort of behavior.
  - bduerst 5y ago
    Which behavior? First Click Free was ditched years ago, but websites will still allow x number of free articles. Some don't, and still get indexed.

freediver 5y ago

How is Google supposed to know if you have access or not? (because if you have, and it doesn't show you the relevant result, it is doing a disservice to you)

mitchdoogle 5y ago
I don't think they are blaming Google. It's the news org's fault in that case, for showing the Google bot a full article while actual people get a paywall.
- freediver 5y ago
  Well again, the people that have a subscription do not get a paywall, they get content when they click the same Google link and are probably very happy with that outocme. How is Google supposed to know who has access?
oh_sigh 5y ago
Google could log which sites have a paywall, and put an icon next to search result that you might need to pay money to access the article. I could also let google know which sites I subscribe to and if I have a WSJ subscription, it could rank those articles a little higher to similar results on sites I don't have a subscription to. For example, something similar happens when you google TV shows, e.g. https://www.google.com/search?q=mad+men https://www.google.com/search?q=mad+men There is a panel on the right hand side showing which streaming services stream the show. I can tell google which ones I have and it will improve the recommendations based on that.
erik_seaberg 5y ago
Even if I were to subscribe to a hundred newspapers (which nobody does), that would leave tens of thousands I don’t. Google should avoid showing me results that are 99% useless.
PeterisP 5y ago
Google is indexing public pages as "someone" who does not have any specific access. Google has chosen not to know or care whether I have special access, and index the content as if it's not there - perhaps it's a disservice, but that's the service they're offering to their users. And the actual news sites do not and should not have any say in this, the decision of what to index and what to show to users should remain between each search engine and their users. So if a search engine chooses to index the "content-as-if-you're-anonymous-and-not-logged-in" and if you try to trick this service into showing something else than what's actually published to everyone (serving visitors and search engines different content), then it's quite reasonable for google to treat it as a cheating attempt (because, really, it is one) and choose to penalize it in their search policies - because that's what I as a search engine user would want.