25 ms·

Makes sense from Google's point of view. You are no longer providing value to our users. You will be quickly replaced with something that provides more value t

Makes sense from Google's point of view. You are no longer providing value to our users. You will be quickly replaced with something that provides more value to our users.

revmoo 9y ago
Agreed. Seems simple to me; if you stop providing the content those users are coming for, they stop coming. Why is this hard for NYT to understand?
GhostVII 9y ago
It also makes sense because they can't crawl a page that is behind a paywall.
- 2_listerine_pls 9y ago
  Of course they let in google's crawler.
  - CydeWeys 9y ago
    The whole purpose of making a webcrawler for a search engine is that you are crawling the content that a user will see upon clicking a link in search results.
    - 2_listerine_pls 9y ago
      Even if it's behind a paywall, there is benefit in crawling it. For instance, if you happened to be a logged WSJ user, then google could show you the result based on your cookies.
      - dragonwriter 9y ago
        Googlebot does not crawl the web separately for each user with a copy of that user's credentials borrowed from their browser. Aside from the privacy and security issues, it would require Google to multiply it's search resources by the number of users.
        2_listerine_pls 9y ago
        > Googlebot does not crawl the web separately for each user with a copy of that user's credentials borrowed from their browser. I did not say that. I said google could display the results to which you have access, based on your cookies. Assuming they had access to crawl the whole article.
        CydeWeys 9y ago
        That would require Google to be in the business of knowing every site that you have a membership in. It is not possible for Google to get access to cookies for other sites, anyway. This is a pretty fundamentally important security restriction that browsers implement to protect you from nefarious sites. So it isn't possible for Google to know which sites you have paid accounts with unless you explicitly tell it.
      - Flenser 9y ago
        > Even if it's behind a paywall, there is benefit in crawling it. How would that work? Would Google create or be given a login for WSJ? That would benefit WSJ as an incumbent news provider at the expense of startups too new or small to get special treatment from Google. How would a new website indicate to all search crawlers (so google doesn't befit at the expense of other search engines) how to get access to it's pay-for content in such a way that end-users can not also pretend to be seach crawlers and get access to the same content?
  - DougWebb 9y ago
    Possible, but seems unlikely. To set that up the WSJ website would have to allow Googlebot access while denying others. Any filtering based on the url or HTTP headers would be discovered and abused by others. An approach based on a security token or IP filter could work, but would be un-managable on Google's side because of the scale of their spidering operation. It would be much more effective for them to use their position to force the WSJ to be an open website, or to accept that their paywalled content does not get indexed.
    - devicenull 9y ago
      Just allow access from all of Google's IP space, as long as the user-agent contains "googlebot'. It's pretty trivial to do... Sure, some people could set up GCP instances and proxy it, but that's a very tiny percentage of people.
      - DougWebb 9y ago
        It might start out as a tiny percentage, but all it takes is one person setting it up and letting the world know about it. Then pretty soon the WSJ is faced with millions of people getting free content again. They're complaining about people accessing their articles via a Google search and then clearing cookies to reset their counters. That's hardly mainstream; browsers have been burying the clear-cookies functionality deeper and deeper over the years because it's seen as an advanced-user-only kind of thing. And yet the WSJ has millions of people doing it, enough to make an impact on their bottom line.
        devicenull 9y ago
        I'd bet it's browing in incognito mode versus actually clearing cookies.
      - obstinate 9y ago
        Googlebot's IP range and GCP's IP ranges are disjoint.
      - detaro 9y ago
  - hackcasual 9y ago
    That's a no-no. https://support.google.com/webmasters/answer/66355?hl=en https://support.google.com/webmasters/answer/66355?hl=en
    - 2_listerine_pls 9y ago
      I went through the whole thing and it seems, from the video, that they could avoid cloaking by redirecting non-paid users to the sing-up page. This way WSJ would be showing the same (www.wsj.com/Bezos-with-hair) to every user. *based on the video
      - jacquesm 9y ago
        > sing-up page. That's a new one. Buy a subscription to the WSJ now for a song.
      - squeaky-clean 9y ago
        In the video, he stresses pretty hard that all forms of cloaking are disallowed, even if not malicious or deceiving. Unless the Googlebot has a paid subscription to WSJ that's still cloaking, as you're showing Googlebot a different page than a regular user. Google's rules and help documents are spread all over, but here's some from Google News about subscriptions: "If you prefer this option, please display a snippet of your article that is at least 80 words long and includes either an excerpt or a summary of the specific article. Since we do not permit "cloaking" -- the practice of showing Googlebot a full version of your article while showing users the subscription or registration version -- we will only crawl and display your content based on the article snippets you provide." edit, forgot link: https://support.google.com/news/publisher/answer/40543?hl=en https://support.google.com/news/publisher/answer/40543?hl=en
    - nso 9y ago
      I guess what this one infact states is that what they were doing was okay. In the video the guy explains that while you are not allowed to treat gbots request in a way no other people are treated, it is okay to differentiate between "boxes" of users. In their example it is the country USA, but if you define a "google user country" and let all users of google in it is ok to bundle the gbot with those. Grey area for sure, but might makes right.
  - seanp2k2 9y ago
    Yep, that's literally https://en.m.wikipedia.org/wiki/Cloaking https://en.m.wikipedia.org/wiki/Cloaking . If they were a smaller site, they'd probably get outright banned by google.
  - snickerbockers 9y ago
    There used to be a lot of sites that did that, and the result was that people would set their user-agents to match google's and get all the paywalled content for free.
- Cakez0r 9y ago
  Not sure why you're being downvoted... It says in the article: After the Journal’s free articles went behind a paywall, Google’s bot only saw the first few paragraphs and started ranking them lower, limiting the Journal’s viewership.
omegaworks 9y ago
How do you plan on balancing this overly strict definition of value with a sustainable model of funding (critically necessary) investigative journalism? All this helps is the proliferation of ideology-reinforcing propaganda and clickbait. Are we going to shut out real, substantial content from the online space? The commoditization of search can't come soon enough. Google shouldn't be able to monopolize this space.
- rfrank 9y ago
  That's WSJ's job to figure out, not search providers. From google's perspective, a search which leads users to a page that doesn't load, which the user immediately abandons, is a bad result. WSJ isn't entitle to traffic because they're an old organization.
  - omegaworks 9y ago
    It's not about "entitlement" it's about the structural incentives put in place by the online media playing field. To go out in the streets and talk to people, dig deep into obfuscated government archives, and actually make sense of the world takes real work. It's often work that results in content that isn't always just what their readers want to see.
    - rfrank 9y ago
      None of that influences the way Google's spiders see their site, bad results are bad results. WSJ could leave their content freely available and sell banner ads, vs. locking themselves into the subscription model they're familiar with. It's not Google's fault WSJ doesn't want to change.
      - dragonwriter 9y ago
        Or they could seek patronage without a paywall, like the Guardian. If they want subscriber exclusivity, they shouldn't be surprised that search engines serving people that are mostly not WSJ subscribers rank their content based on what is visible to non-subscribers.
amelius 9y ago
But what if you are the kind of person that wants to pay for good journalism? Will Google figure this out from your history, and rank WSJ higher?
- ng12 9y ago
  If you're paying for good journalism why are you using Google search to find it?
  - mturmon 9y ago
    If you're running a top-notch search engine, why are you excluding the results I care about most?
    - duskwuff 9y ago
      Because Google can only index what they can see. As noted in the article, the Googlebot only gets to see the first few paragraphs of WSJ articles, so those pages are less likely to rank on searches.
      - sowbug 9y ago
        Indexing more of the content (which would be possible by providing the full content to web crawlers) seems to violate Google's cloaking guideline as well: https://support.google.com/webmasters/answer/66355 https://support.google.com/webmasters/answer/66355
    - dragonwriter 9y ago
      It's not excluded. They present less content to Google users and are ranked based on the actual presented content. If News Corp wants search ads for it's paid service, it can by then like anyone else instead of expecting Google to treat them specially, which is what they are really asking for.
  - SilasX 9y ago
    Because a) google is better at indexing a given source, and b) every user shouldn't have to build their own search aggregator across the n sites they have privileged access to; that's the role of a search engine (except for it covering all sources not just a few).
tannhaeuser 9y ago
I don't know about Google search being as great and flawless as you're implying here. I run a website which is dedicated to one particular topic, and Google won't list it at all ("no more results after like 198 hits" of lots of outdated and superficial content), even though the site is registered via Google search console. On other topics, I get pointless "123000 matches in 0.23 ms" results. Not impressed at all; a naive keyword count as rank criterion would do better You can check it out yourself by searching for "SGML" and compare my site http://sgmljs.net/docs/sgmlrefman.html http://sgmljs.net/docs/sgmlrefman.html.
- pkteison 9y ago
  Your site shows up fine if I search for SGML Syntax Reference or SGML syntax. Meanwhile, it doesn't appear to generally be about learning basic SGML concepts, so it seems quite reasonable that it doesn't show up when just searching for SGML. And why blame Google when Bing does the same? It seems like your complaint is that adding one page of reference info wasn't enough to serve as an ad for your business. It doesn't seem like a valid complaint to me.
- whatever_dude 9y ago
  While your point still stands, don't you think there's things you could do to help your site index better on that topic? It has no meta data like keywords and descriptions, no robots.txt/sitemap.xml, the links to the /docs pages are hidden under an overlay that requires JavaScript to show, that URL has 10 different <H1> tags, etc.
- daddyo 9y ago
  - There are other websites with way more authority talking about SGML. (Like the W3C website). - You have no/little sites or forums linking to your content. - Your page titles are uninformative. Biggest offender is probably the homepage, with a page title of "index". But even for your reference page, it it is just "Syntax Reference" (way too general), and Google actually uses your page headings to repair this to "SGML Syntax Reference". Try inverse breadcrumb style "SGML Syntax Reference | Docs | SGML.js". BTW: you ranked 2nd for "SGML Syntax Reference". - Suspicion: Content not visible (like those in the content slider) is ranked lower than always visible content. Chrome headless crawler can detect this. Add this slider content as regular text to your homepage, and also try to expand content there. Include links to your latest blogs. - I prefer hierarchical headings, not just sections and <h1> for everything. This, because hierarchical headings can not hurt, but non-hierarchical headings could hurt. - Finally, SGML being a standard, there are simply a lot of competitors for this keyword. These competitors are not commercial competitors, but authoritative websites with lots of informative content. Exactly the sites that Google likes to rank high. If you want to rank for SGML, you may be fighting an uphill battle.
whiddershins 9y ago
I find the turn of phrase interesting: "Google's" users. Yes, of course technically that is an accurate description. But although I am a user of Google, I don't like the idea of them thinking of me as their asset, even though I obviously am. And irrationally because Google feels like such an omni-present utility, my intuitive expectation is that they index the web in a non preferential way. Which still doesn't make any sense, because the entire utility of their search results is because they weight what they index. There's some cognitive dissonance here, but I can't put my finger on exactly what it is.
- hrayr 9y ago
  >> There's some cognitive dissonance here, but I can't put my finger on exactly what it is. It's the cake... you can't have your cake, and eat it too.
  - phkahler 9y ago
    The cake is a lie.
- johan_larson 9y ago
  The possessive in English doesn't necessarily indicate actual or presumed ownership. It can also indicate looser affiliations and even relationships where the subject is not the controlling party. It's my shovel and my horse, sure, but it's also my country, my boss, and my God.
SilasX 9y ago
Because every Google user lacks a WSJ subscription and will only be satisfied with news articles they can read immediately for free? From the fact that I'm searching for e.g. a black & white old Western movie, it doesn't follow that I only want movies I can view on the internet right this second. Edit: I could very well be satisfied with "here are the names of popular ones, but you'd have to go to their publishers since they're long out of print.
- sgift 9y ago
  Which is the reason Google still shows WSJ in its search results. So, you can decide to still go there even though it is a subscription website. But Google only works with things its bot can see. What you suggest amounts to boosting WSJs search position despite only seeing a small part of their content, because "we know that they are good". Doesn't sound like a great idea.
phkahler 9y ago
Googles job IMHO is to find relevant information, not judge it based on costs. How often do we get second hand news where an article is simply parroting what another site wrote? I'd rather get information as close to the source as possible in most cases. Perhaps that means paying for it, but that choice should be mine to make. Put the most relevant at the top and if people want to read a free second hand ripoff of some news they can select a lower ranked search result.
- rndmwlk 9y ago
  >Put the most relevant at the top and if people want to read a free second hand ripoff of some news they can select a lower ranked search result. Couldn't you argue that if the free "ripoff" has the same information and is more easily accessible it should be considered more relevant?
- PietdeVries 9y ago
  Now how would google find the relevant info (and index it) yet at the same time a 'regular' user would be restricted by the paywall? That might be a solution you can implement technically, but will it also work for your users? If users don't have access to info because it is restricted to paying users, google won't have access either.
  - WhiteOwlLion 9y ago
    It is technically possible via web cloaking. It is the same technology that tricked GoogleBot think you are a legitimate content site and get served pharma ads for big blue pills.

Makes sense from Google's point of view. You are no longer providing value to our users. You will be quickly replaced with something that provides more value to our users.

revmoo 9y ago

Agreed. Seems simple to me; if you stop providing the content those users are coming for, they stop coming. Why is this hard for NYT to understand?

GhostVII 9y ago

It also makes sense because they can't crawl a page that is behind a paywall.

2_listerine_pls 9y ago
Of course they let in google's crawler.
- CydeWeys 9y ago
  The whole purpose of making a webcrawler for a search engine is that you are crawling the content that a user will see upon clicking a link in search results.
  - 2_listerine_pls 9y ago
    Even if it's behind a paywall, there is benefit in crawling it. For instance, if you happened to be a logged WSJ user, then google could show you the result based on your cookies.
    - dragonwriter 9y ago
      Googlebot does not crawl the web separately for each user with a copy of that user's credentials borrowed from their browser. Aside from the privacy and security issues, it would require Google to multiply it's search resources by the number of users.
      - 2_listerine_pls 9y ago
        > Googlebot does not crawl the web separately for each user with a copy of that user's credentials borrowed from their browser. I did not say that. I said google could display the results to which you have access, based on your cookies. Assuming they had access to crawl the whole article.
        CydeWeys 9y ago
        That would require Google to be in the business of knowing every site that you have a membership in. It is not possible for Google to get access to cookies for other sites, anyway. This is a pretty fundamentally important security restriction that browsers implement to protect you from nefarious sites. So it isn't possible for Google to know which sites you have paid accounts with unless you explicitly tell it.
    - Flenser 9y ago
      > Even if it's behind a paywall, there is benefit in crawling it. How would that work? Would Google create or be given a login for WSJ? That would benefit WSJ as an incumbent news provider at the expense of startups too new or small to get special treatment from Google. How would a new website indicate to all search crawlers (so google doesn't befit at the expense of other search engines) how to get access to it's pay-for content in such a way that end-users can not also pretend to be seach crawlers and get access to the same content?
- DougWebb 9y ago
  Possible, but seems unlikely. To set that up the WSJ website would have to allow Googlebot access while denying others. Any filtering based on the url or HTTP headers would be discovered and abused by others. An approach based on a security token or IP filter could work, but would be un-managable on Google's side because of the scale of their spidering operation. It would be much more effective for them to use their position to force the WSJ to be an open website, or to accept that their paywalled content does not get indexed.
  - devicenull 9y ago
    Just allow access from all of Google's IP space, as long as the user-agent contains "googlebot'. It's pretty trivial to do... Sure, some people could set up GCP instances and proxy it, but that's a very tiny percentage of people.
    - DougWebb 9y ago
      It might start out as a tiny percentage, but all it takes is one person setting it up and letting the world know about it. Then pretty soon the WSJ is faced with millions of people getting free content again. They're complaining about people accessing their articles via a Google search and then clearing cookies to reset their counters. That's hardly mainstream; browsers have been burying the clear-cookies functionality deeper and deeper over the years because it's seen as an advanced-user-only kind of thing. And yet the WSJ has millions of people doing it, enough to make an impact on their bottom line.
      - devicenull 9y ago
        I'd bet it's browing in incognito mode versus actually clearing cookies.
    - obstinate 9y ago
      Googlebot's IP range and GCP's IP ranges are disjoint.
    - detaro 9y ago
- hackcasual 9y ago
  That's a no-no. https://support.google.com/webmasters/answer/66355?hl=en https://support.google.com/webmasters/answer/66355?hl=en
  - 2_listerine_pls 9y ago
    I went through the whole thing and it seems, from the video, that they could avoid cloaking by redirecting non-paid users to the sing-up page. This way WSJ would be showing the same (www.wsj.com/Bezos-with-hair) to every user. *based on the video
    - jacquesm 9y ago
      > sing-up page. That's a new one. Buy a subscription to the WSJ now for a song.
    - squeaky-clean 9y ago
      In the video, he stresses pretty hard that all forms of cloaking are disallowed, even if not malicious or deceiving. Unless the Googlebot has a paid subscription to WSJ that's still cloaking, as you're showing Googlebot a different page than a regular user. Google's rules and help documents are spread all over, but here's some from Google News about subscriptions: "If you prefer this option, please display a snippet of your article that is at least 80 words long and includes either an excerpt or a summary of the specific article. Since we do not permit "cloaking" -- the practice of showing Googlebot a full version of your article while showing users the subscription or registration version -- we will only crawl and display your content based on the article snippets you provide." edit, forgot link: https://support.google.com/news/publisher/answer/40543?hl=en https://support.google.com/news/publisher/answer/40543?hl=en
  - nso 9y ago
    I guess what this one infact states is that what they were doing was okay. In the video the guy explains that while you are not allowed to treat gbots request in a way no other people are treated, it is okay to differentiate between "boxes" of users. In their example it is the country USA, but if you define a "google user country" and let all users of google in it is ok to bundle the gbot with those. Grey area for sure, but might makes right.
- seanp2k2 9y ago
  Yep, that's literally https://en.m.wikipedia.org/wiki/Cloaking https://en.m.wikipedia.org/wiki/Cloaking . If they were a smaller site, they'd probably get outright banned by google.
- snickerbockers 9y ago
  There used to be a lot of sites that did that, and the result was that people would set their user-agents to match google's and get all the paywalled content for free.
Cakez0r 9y ago
Not sure why you're being downvoted... It says in the article: After the Journal’s free articles went behind a paywall, Google’s bot only saw the first few paragraphs and started ranking them lower, limiting the Journal’s viewership.

omegaworks 9y ago

How do you plan on balancing this overly strict definition of value with a sustainable model of funding (critically necessary) investigative journalism? All this helps is the proliferation of ideology-reinforcing propaganda and clickbait. Are we going to shut out real, substantial content from the online space? The commoditization of search can't come soon enough. Google shouldn't be able to monopolize this space.

rfrank 9y ago
That's WSJ's job to figure out, not search providers. From google's perspective, a search which leads users to a page that doesn't load, which the user immediately abandons, is a bad result. WSJ isn't entitle to traffic because they're an old organization.
- omegaworks 9y ago
  It's not about "entitlement" it's about the structural incentives put in place by the online media playing field. To go out in the streets and talk to people, dig deep into obfuscated government archives, and actually make sense of the world takes real work. It's often work that results in content that isn't always just what their readers want to see.
  - rfrank 9y ago
    None of that influences the way Google's spiders see their site, bad results are bad results. WSJ could leave their content freely available and sell banner ads, vs. locking themselves into the subscription model they're familiar with. It's not Google's fault WSJ doesn't want to change.
    - dragonwriter 9y ago
      Or they could seek patronage without a paywall, like the Guardian. If they want subscriber exclusivity, they shouldn't be surprised that search engines serving people that are mostly not WSJ subscribers rank their content based on what is visible to non-subscribers.

amelius 9y ago

But what if you are the kind of person that wants to pay for good journalism? Will Google figure this out from your history, and rank WSJ higher?

ng12 9y ago
If you're paying for good journalism why are you using Google search to find it?
- mturmon 9y ago
  If you're running a top-notch search engine, why are you excluding the results I care about most?
  - duskwuff 9y ago
    Because Google can only index what they can see. As noted in the article, the Googlebot only gets to see the first few paragraphs of WSJ articles, so those pages are less likely to rank on searches.
    - sowbug 9y ago
      Indexing more of the content (which would be possible by providing the full content to web crawlers) seems to violate Google's cloaking guideline as well: https://support.google.com/webmasters/answer/66355 https://support.google.com/webmasters/answer/66355
  - dragonwriter 9y ago
    It's not excluded. They present less content to Google users and are ranked based on the actual presented content. If News Corp wants search ads for it's paid service, it can by then like anyone else instead of expecting Google to treat them specially, which is what they are really asking for.
- SilasX 9y ago
  Because a) google is better at indexing a given source, and b) every user shouldn't have to build their own search aggregator across the n sites they have privileged access to; that's the role of a search engine (except for it covering all sources not just a few).

tannhaeuser 9y ago

I don't know about Google search being as great and flawless as you're implying here. I run a website which is dedicated to one particular topic, and Google won't list it at all ("no more results after like 198 hits" of lots of outdated and superficial content), even though the site is registered via Google search console. On other topics, I get pointless "123000 matches in 0.23 ms" results. Not impressed at all; a naive keyword count as rank criterion would do better You can check it out yourself by searching for "SGML" and compare my site http://sgmljs.net/docs/sgmlrefman.html http://sgmljs.net/docs/sgmlrefman.html.

pkteison 9y ago
Your site shows up fine if I search for SGML Syntax Reference or SGML syntax. Meanwhile, it doesn't appear to generally be about learning basic SGML concepts, so it seems quite reasonable that it doesn't show up when just searching for SGML. And why blame Google when Bing does the same? It seems like your complaint is that adding one page of reference info wasn't enough to serve as an ad for your business. It doesn't seem like a valid complaint to me.
whatever_dude 9y ago
While your point still stands, don't you think there's things you could do to help your site index better on that topic? It has no meta data like keywords and descriptions, no robots.txt/sitemap.xml, the links to the /docs pages are hidden under an overlay that requires JavaScript to show, that URL has 10 different <H1> tags, etc.
daddyo 9y ago
- There are other websites with way more authority talking about SGML. (Like the W3C website). - You have no/little sites or forums linking to your content. - Your page titles are uninformative. Biggest offender is probably the homepage, with a page title of "index". But even for your reference page, it it is just "Syntax Reference" (way too general), and Google actually uses your page headings to repair this to "SGML Syntax Reference". Try inverse breadcrumb style "SGML Syntax Reference | Docs | SGML.js". BTW: you ranked 2nd for "SGML Syntax Reference". - Suspicion: Content not visible (like those in the content slider) is ranked lower than always visible content. Chrome headless crawler can detect this. Add this slider content as regular text to your homepage, and also try to expand content there. Include links to your latest blogs. - I prefer hierarchical headings, not just sections and <h1> for everything. This, because hierarchical headings can not hurt, but non-hierarchical headings could hurt. - Finally, SGML being a standard, there are simply a lot of competitors for this keyword. These competitors are not commercial competitors, but authoritative websites with lots of informative content. Exactly the sites that Google likes to rank high. If you want to rank for SGML, you may be fighting an uphill battle.

whiddershins 9y ago

I find the turn of phrase interesting: "Google's" users. Yes, of course technically that is an accurate description. But although I am a user of Google, I don't like the idea of them thinking of me as their asset, even though I obviously am. And irrationally because Google feels like such an omni-present utility, my intuitive expectation is that they index the web in a non preferential way. Which still doesn't make any sense, because the entire utility of their search results is because they weight what they index. There's some cognitive dissonance here, but I can't put my finger on exactly what it is.

hrayr 9y ago
>> There's some cognitive dissonance here, but I can't put my finger on exactly what it is. It's the cake... you can't have your cake, and eat it too.
- phkahler 9y ago
  The cake is a lie.
johan_larson 9y ago
The possessive in English doesn't necessarily indicate actual or presumed ownership. It can also indicate looser affiliations and even relationships where the subject is not the controlling party. It's my shovel and my horse, sure, but it's also my country, my boss, and my God.

SilasX 9y ago

Because every Google user lacks a WSJ subscription and will only be satisfied with news articles they can read immediately for free? From the fact that I'm searching for e.g. a black & white old Western movie, it doesn't follow that I only want movies I can view on the internet right this second. Edit: I could very well be satisfied with "here are the names of popular ones, but you'd have to go to their publishers since they're long out of print.

sgift 9y ago
Which is the reason Google still shows WSJ in its search results. So, you can decide to still go there even though it is a subscription website. But Google only works with things its bot can see. What you suggest amounts to boosting WSJs search position despite only seeing a small part of their content, because "we know that they are good". Doesn't sound like a great idea.

phkahler 9y ago

Googles job IMHO is to find relevant information, not judge it based on costs. How often do we get second hand news where an article is simply parroting what another site wrote? I'd rather get information as close to the source as possible in most cases. Perhaps that means paying for it, but that choice should be mine to make. Put the most relevant at the top and if people want to read a free second hand ripoff of some news they can select a lower ranked search result.

rndmwlk 9y ago
>Put the most relevant at the top and if people want to read a free second hand ripoff of some news they can select a lower ranked search result. Couldn't you argue that if the free "ripoff" has the same information and is more easily accessible it should be considered more relevant?
PietdeVries 9y ago
Now how would google find the relevant info (and index it) yet at the same time a 'regular' user would be restricted by the paywall? That might be a solution you can implement technically, but will it also work for your users? If users don't have access to info because it is restricted to paying users, google won't have access either.
- WhiteOwlLion 9y ago
  It is technically possible via web cloaking. It is the same technology that tricked GoogleBot think you are a legitimate content site and get served pharma ads for big blue pills.