6 ms·

> they seem to be losing the content junk spam SEO fight Google clearly isn't even trying. So many of these sites are polluting search results for months. It

> they seem to be losing the content junk spam SEO fight Google clearly isn't even trying. So many of these sites are polluting search results for months. It isn't a case of sites that pop up for a few hours until Google notices and blacklists them. Google Search has gone so far downhill. I'm not sure what they're optimising for. Long-term irrelevance, it seems.

MrWiffles 5y ago
Well I think you already nailed it with “not even trying”. You don’t need to try or optimize when you’re a monopoly. Competition forces that function, but a lack thereof enables mediocrity and complacency.
- kristiandupont 5y ago
  Oh come on! I am no fan of Google and their privacy policies etc., and I agree that results like the sites mentioned in this thread are annoying but I am not so cocky as to think I have any idea what Google is fighting. There are a lot of very smart people from all over the world putting everything they have into getting to the top of that site. It's not a trivial task for them.
  - kizer 5y ago
    Fair point. But again, that would be less of an issue if there were like 3 or 4 big search engines with different algorithms, all changing and adapting on their own schedules. Would be harder to game the system then.
  - motoxpro 5y ago
    You're so right. It's sad that even if you remove 99% of something, to the outside it looks like you are removing 0% because you have no reference.
  - Nextgrid 5y ago
    I disagree. Google has absolutely dropped the ball - I'm not saying that it's easy to algorithmically rank good/bad content, but they're not even trying. A stop-gap solution would be to simply penalize anything with ads. Legitimate websites will still have enough "SEO juice" (for the lack of a better term) to offset the penalty, but brand new copycats with otherwise no inbound links to them from other legitimate websites (and no other business model beyond scammy ads) will never be able to rank high, essentially removing the incentive from setting up these sites in the first place. Not to mention, these problems can be identified, prioritized and then tackled manually one-by-one. Stackoverflow copycats can be dealt with by downloading the SO data dump, parsing it and then severely penalizing any website where the bulk of the content matches the dump. Pinterest can be penalized by simply excluding it from image searches until they actually display the searched image without asking for login. So on and so forth. You can't win them all, and you can't train an algorithm to win every time either, but you can manually observe what's happening and deal with the biggest problems. The problem however is that the current status-quo is good for Google. They've got the monopoly on search (every other one is typically even worse when it comes to these issues), and spammy copycat websites happen to have Google ads or analytics so Google actually benefits from them too.
    - Kiro 5y ago
      How do you know Google is not already doing all those things? You said it yourself, you can't win them all and we're witnessing the result of that. If Google truly did nothing it would be an unimaginable chaos.
      - Nextgrid 5y ago
        Because I’m not seeing the results. Let’s ignore SO copycats for now because that solution requires at least a slight amount of effort and look at the elephant in the room - Pinterest. Dealing with it is a simple “if image_search and domain == pinterest.com then skip” and they’re not even doing that. They have a site that’s been polluting image search results for years, isn’t even trying to hide (unlike the SO copycats which could technically churn out an infinite amount of domains to work around bans) and they can’t even deal with that.
  - MrWiffles 5y ago
    Consider it from a management/budgetary point of view. Super smart people? Oh hell yeah, and I don't fault them for this one bit. But bean counters? Why the hell invest in hiring those people, or budgeting for search quality improvements, when you have essentially no competition to speak of? Makes more sense to put those same smart people on projects that correlate people's activities outside of Google with activities they can observe either directly or indirectly to create a more wholistic picture of the consumer for advertisers. Those are much harder problems to solve, especially now that the public is finally becoming slowly more aware of the massive abuses of privacy the bean counters like I described above have been pushing for years. At the end of the day Google doesn't exist for the good of the public, they exist to return the highest possible investment for shareholders. It's literally the opposite of their job to improve search quality when competition doesn't force them to (because investors see that as a waste of money that could have been spent on further eroding people's privacy to enable higher and higher ad revenues.)
- rhdunn 5y ago
  One of the problems with things like this is how do you know which site has copied from another, especially if you don't want a list of hard-coded exceptions. Related is if you have a lot of content from git repositories that are mirrored from different locations (GitHub, GitLab, etc.), all of which are showing the same content. Or if different sites are hosting versions of public domain texts. You don't want to derank those results, even if they are similar to the "copy a popular site" websites.
  - lazyeye 5y ago
    Google knows instantly how much time you spend on a page and how quickly you return to the search results because it is a useless copy/paste page from SO.
    - cutemonster 5y ago
      And probably they track content age too? Since they somehow can lookup by content (they can detect duplicated content), there's probably a date remembered too So they could know if a code snippet was already X years old? And from where, originally. (Unless it got edited a lot but that's not the case here?)
  - MrWiffles 5y ago
    > One of the problems with things like this is how do you know which site has copied from another, especially if you don't want a list of hard-coded exceptions. The core function of that is actually pretty simple: 1. Strip all X/HTML tags 2. Run `diff` Sure, it's not perfect, but an organization that pursues academic quantum computing research can sure as hell afford to run the results of the above against an AI to check for similarities.
    - rhdunn 5y ago
      The issue is not figuring out if site X has a copy of a page or text from site Y. The issue is how do you know which site is the source of the information. In other words, the technical bit is easy but determining what to do with that is hard. Especially if you don't want a lot of hard coded exceptions. In the case of things like other sites copying from stack overflow, github, etc. you can figure out which is the source of the information. Lets day you have decided to make github the source of that information, and derank any other sites that also have that information. As a result, you will derank gitlab for having git mirrors of projects, as the source will match that on github. You will also derank sites like lkml as that contains the commit message descriptions, and the patches will partially match content from the kernel source hosted on github. Lets also say you've decided to make wikipedia and other wikimedia sites the source of information. Congratulations, you've now deranked project gutenberg for hosting the same public domain texts as wikimedia, along with other sites like the official sites for authors like Jack London. Plus any blog that includes portions of these to discuss or analyze them. Do you want a blackbox AI making these decisions?
vadfa 5y ago
Is it legally a good idea for Google to go and blacklist individual sites?
- GhettoComputers 5y ago
  Child porn is the extreme. www.chillingeffects.org is the other.
- briffle 5y ago
  Exactly, change the word blacklist to censor, and then it seems wrong.
  - labster 5y ago
    Change the flag button on this post to censor and I bet people wouldn’t click it. But people want some level of censorship, so long as they convince themselves they’re not against free speech.
    - SrslyJosh 5y ago
      Blacklisting by a private entity is not censorship. It's even sillier when you consider that what's being removed are copies and the original content is still there.
  - SrslyJosh 5y ago
    But that's not censorship. Google is not a government, and no one's speech is being suppressed by filtering out literal copies of something and leaving the original.
    - snerbles 5y ago
      Censorship can be conducted by private or public entities. https://www.aclu.org/other/what-censorship https://www.aclu.org/other/what-censorship
- kevin_thibedeau 5y ago
gwittel 5y ago
Pretty much this. It’s likely down to the lack of incentives for leaders and line workers. Ie: search spam volume isn’t a measure by which search team performance is graded
Nasrudith 5y ago
Isn't that a bit narcissistic to think that your one problem in particular out of millions is a sign that they aren't doing anything at all?
testudovictoria 5y ago
> Google Search has gone so far downhill. I'm not sure what they're optimising for. Long-term irrelevance, it seems. I assume they're optimizing for the most common denominator: standard, non-power users. Early search didn't return great results for human-like questions such as What are the wavelengths of the colors on the visible spectrum? The result might be within the first two pages, but that query had too many irrelevant search terms. A better query would have been wavelengths color visible spectrum. That query only has the necessary key terms. Sometimes queries required the user to know search operators (e.g. exact match, date range, synonyms) just to get relevant results. The average person probably didn't know that early searches gave better results when constructed in the second way. Google changed search to adapt to how normal people search. Now the human-like query will return good results. Combine that with locales, search history, and personal interests, even the most basic user can get worthwhile results from asking Google question. The cost is that power users who understand operators and the power of key terms get less relevant results but likely still correct.
- stevesimmons 5y ago
  If Google actually used its search history, it would see I was a sw eng power user, and return the types of results that users like me want to see. Then again, I have been using Google Maps for 16 years and it still cannot show me distances in km rather than miles. A boolean setting ffs that I've had to manually switch probably a thousand times. If billions of dollars of AI R&D can't even figure that out...
  - Kiro 5y ago
    Funny how people complain about Google tracking them and then get angry when they are not being tracked enough.
- acdha 5y ago
  I'm skeptical that it's all that much better for any group of user. I search for normal things all of the time and it doesn't do a good job at returning relevant results for things like businesses, recent movies / TV shows, etc. and natural language syntax doesn't make the results less bad. What it looks suspiciously like to me is a lack of an effective feedback loop for user frustration — if it takes a number of queries to get correct results or someone doesn't stop using Google search entirely, this would be easy to confuse with improved engagement and I'd especially believe that managers whose job it is to get a number to go up are not in a hurry to question whether that growth is meaningful.

MrWiffles 5y ago

Well I think you already nailed it with “not even trying”. You don’t need to try or optimize when you’re a monopoly. Competition forces that function, but a lack thereof enables mediocrity and complacency.

kristiandupont 5y ago
Oh come on! I am no fan of Google and their privacy policies etc., and I agree that results like the sites mentioned in this thread are annoying but I am not so cocky as to think I have any idea what Google is fighting. There are a lot of very smart people from all over the world putting everything they have into getting to the top of that site. It's not a trivial task for them.
- kizer 5y ago
  Fair point. But again, that would be less of an issue if there were like 3 or 4 big search engines with different algorithms, all changing and adapting on their own schedules. Would be harder to game the system then.
- motoxpro 5y ago
  You're so right. It's sad that even if you remove 99% of something, to the outside it looks like you are removing 0% because you have no reference.
- Nextgrid 5y ago
  I disagree. Google has absolutely dropped the ball - I'm not saying that it's easy to algorithmically rank good/bad content, but they're not even trying. A stop-gap solution would be to simply penalize anything with ads. Legitimate websites will still have enough "SEO juice" (for the lack of a better term) to offset the penalty, but brand new copycats with otherwise no inbound links to them from other legitimate websites (and no other business model beyond scammy ads) will never be able to rank high, essentially removing the incentive from setting up these sites in the first place. Not to mention, these problems can be identified, prioritized and then tackled manually one-by-one. Stackoverflow copycats can be dealt with by downloading the SO data dump, parsing it and then severely penalizing any website where the bulk of the content matches the dump. Pinterest can be penalized by simply excluding it from image searches until they actually display the searched image without asking for login. So on and so forth. You can't win them all, and you can't train an algorithm to win every time either, but you can manually observe what's happening and deal with the biggest problems. The problem however is that the current status-quo is good for Google. They've got the monopoly on search (every other one is typically even worse when it comes to these issues), and spammy copycat websites happen to have Google ads or analytics so Google actually benefits from them too.
  - Kiro 5y ago
    How do you know Google is not already doing all those things? You said it yourself, you can't win them all and we're witnessing the result of that. If Google truly did nothing it would be an unimaginable chaos.
    - Nextgrid 5y ago
      Because I’m not seeing the results. Let’s ignore SO copycats for now because that solution requires at least a slight amount of effort and look at the elephant in the room - Pinterest. Dealing with it is a simple “if image_search and domain == pinterest.com then skip” and they’re not even doing that. They have a site that’s been polluting image search results for years, isn’t even trying to hide (unlike the SO copycats which could technically churn out an infinite amount of domains to work around bans) and they can’t even deal with that.
- MrWiffles 5y ago
  Consider it from a management/budgetary point of view. Super smart people? Oh hell yeah, and I don't fault them for this one bit. But bean counters? Why the hell invest in hiring those people, or budgeting for search quality improvements, when you have essentially no competition to speak of? Makes more sense to put those same smart people on projects that correlate people's activities outside of Google with activities they can observe either directly or indirectly to create a more wholistic picture of the consumer for advertisers. Those are much harder problems to solve, especially now that the public is finally becoming slowly more aware of the massive abuses of privacy the bean counters like I described above have been pushing for years. At the end of the day Google doesn't exist for the good of the public, they exist to return the highest possible investment for shareholders. It's literally the opposite of their job to improve search quality when competition doesn't force them to (because investors see that as a waste of money that could have been spent on further eroding people's privacy to enable higher and higher ad revenues.)
rhdunn 5y ago
One of the problems with things like this is how do you know which site has copied from another, especially if you don't want a list of hard-coded exceptions. Related is if you have a lot of content from git repositories that are mirrored from different locations (GitHub, GitLab, etc.), all of which are showing the same content. Or if different sites are hosting versions of public domain texts. You don't want to derank those results, even if they are similar to the "copy a popular site" websites.
- lazyeye 5y ago
  Google knows instantly how much time you spend on a page and how quickly you return to the search results because it is a useless copy/paste page from SO.
  - cutemonster 5y ago
    And probably they track content age too? Since they somehow can lookup by content (they can detect duplicated content), there's probably a date remembered too So they could know if a code snippet was already X years old? And from where, originally. (Unless it got edited a lot but that's not the case here?)
- MrWiffles 5y ago
  > One of the problems with things like this is how do you know which site has copied from another, especially if you don't want a list of hard-coded exceptions. The core function of that is actually pretty simple: 1. Strip all X/HTML tags 2. Run `diff` Sure, it's not perfect, but an organization that pursues academic quantum computing research can sure as hell afford to run the results of the above against an AI to check for similarities.
  - rhdunn 5y ago
    The issue is not figuring out if site X has a copy of a page or text from site Y. The issue is how do you know which site is the source of the information. In other words, the technical bit is easy but determining what to do with that is hard. Especially if you don't want a lot of hard coded exceptions. In the case of things like other sites copying from stack overflow, github, etc. you can figure out which is the source of the information. Lets day you have decided to make github the source of that information, and derank any other sites that also have that information. As a result, you will derank gitlab for having git mirrors of projects, as the source will match that on github. You will also derank sites like lkml as that contains the commit message descriptions, and the patches will partially match content from the kernel source hosted on github. Lets also say you've decided to make wikipedia and other wikimedia sites the source of information. Congratulations, you've now deranked project gutenberg for hosting the same public domain texts as wikimedia, along with other sites like the official sites for authors like Jack London. Plus any blog that includes portions of these to discuss or analyze them. Do you want a blackbox AI making these decisions?

vadfa 5y ago

Is it legally a good idea for Google to go and blacklist individual sites?

GhettoComputers 5y ago
Child porn is the extreme. www.chillingeffects.org is the other.
briffle 5y ago
Exactly, change the word blacklist to censor, and then it seems wrong.
- labster 5y ago
  Change the flag button on this post to censor and I bet people wouldn’t click it. But people want some level of censorship, so long as they convince themselves they’re not against free speech.
  - SrslyJosh 5y ago
    Blacklisting by a private entity is not censorship. It's even sillier when you consider that what's being removed are copies and the original content is still there.
- SrslyJosh 5y ago
  But that's not censorship. Google is not a government, and no one's speech is being suppressed by filtering out literal copies of something and leaving the original.
  - snerbles 5y ago
    Censorship can be conducted by private or public entities. https://www.aclu.org/other/what-censorship https://www.aclu.org/other/what-censorship
kevin_thibedeau 5y ago

gwittel 5y ago

Pretty much this. It’s likely down to the lack of incentives for leaders and line workers. Ie: search spam volume isn’t a measure by which search team performance is graded

Nasrudith 5y ago

Isn't that a bit narcissistic to think that your one problem in particular out of millions is a sign that they aren't doing anything at all?

testudovictoria 5y ago

> Google Search has gone so far downhill. I'm not sure what they're optimising for. Long-term irrelevance, it seems. I assume they're optimizing for the most common denominator: standard, non-power users. Early search didn't return great results for human-like questions such as What are the wavelengths of the colors on the visible spectrum? The result might be within the first two pages, but that query had too many irrelevant search terms. A better query would have been wavelengths color visible spectrum. That query only has the necessary key terms. Sometimes queries required the user to know search operators (e.g. exact match, date range, synonyms) just to get relevant results. The average person probably didn't know that early searches gave better results when constructed in the second way. Google changed search to adapt to how normal people search. Now the human-like query will return good results. Combine that with locales, search history, and personal interests, even the most basic user can get worthwhile results from asking Google question. The cost is that power users who understand operators and the power of key terms get less relevant results but likely still correct.

stevesimmons 5y ago
If Google actually used its search history, it would see I was a sw eng power user, and return the types of results that users like me want to see. Then again, I have been using Google Maps for 16 years and it still cannot show me distances in km rather than miles. A boolean setting ffs that I've had to manually switch probably a thousand times. If billions of dollars of AI R&D can't even figure that out...
- Kiro 5y ago
  Funny how people complain about Google tracking them and then get angry when they are not being tracked enough.
acdha 5y ago
I'm skeptical that it's all that much better for any group of user. I search for normal things all of the time and it doesn't do a good job at returning relevant results for things like businesses, recent movies / TV shows, etc. and natural language syntax doesn't make the results less bad. What it looks suspiciously like to me is a lack of an effective feedback loop for user frustration — if it takes a number of queries to get correct results or someone doesn't stop using Google search entirely, this would be easy to confuse with improved engagement and I'd especially believe that managers whose job it is to get a number to go up are not in a hurry to question whether that growth is meaningful.