3 ms·

Bullet #3: Yes they keep blacklists, I worked on the web crawler for many years. But the article does not understand or differentiate between blacklists for co

Bullet #3: Yes they keep blacklists, I worked on the web crawler for many years. But the article does not understand or differentiate between blacklists for content farms, spam domains, link farms, infinite spaces (that aren't calendars) etc. Blacklists are low level url regexps. In ~2015 some spammer in China overnight created 100 million websites to boost priority and each page had 1000 links. Google saved them all! This will literally crush the web crawler, slowing it by 20000x or more, like a snake swallowing an elephant. Or it will crash the crawler and we'd have to write manual code to cleanse the search logs and add blacklists to keep it from happening again, right away. Blacklisted domains are blacklisted forever, because they crash the crawler. This happens 1x - 3x times per year. Changes here are all tracked and only owners of the first stage of search (who have no connection to the ranking algorithm) have change rights. Bullet #5: search quality is assessed by thousands of contractors worldwide and you can become a part of the crawl quality team, although it doesn't pay that well in the USA. The book they follow is 180 pages and available on the Google website for many years. It has guidelines for how to determine sexual, offensive, or illegal (or child porn) content in ALL countries. It has guidelines on how to rank news source reputation and credibility.

creaghpatr 7y ago
Did you miss this part of bullet #3? >These moves are separate from those that block sites as required by U.S. or foreign law, such as those featuring child abuse or with copyright infringement, and from changes designed to demote spam sites, which attempt to game the system to appear higher in results. The article directly refutes what you are claiming here.
- pas 7y ago
  How exactly does the article contradict that? The article has very few specific examples, a few mentions of some draft and "suspicious" discrepancies between bing and google. None of which really mean much, except that it sounds like the typical liberal bias meme.
baybal2 7y ago
> Bullet #5: search quality is assessed by thousands of contractors worldwide and you can become a part of the crawl quality team, although it doesn't pay that well in the USA. The book they follow is 180 pages and available on the Google website for many years. It has guidelines for how to determine sexual, offensive, or illegal (or child porn) content in ALL countries. It has guidelines on how to rank news source reputation and credibility. Funny factoid for you: Google hires those guys in Russia through recruiters and innocuously sounding shell companies in Saint Petersburg. The guy can work for the shell for years without realising that he works for Google.
- jhanschoo 7y ago
  > Funny factoid for you: Google hires those guys in Russia through recruiters and innocuously sounding shell companies in Saint Petersburg. > The guy can work for the shell for years without realising that he works for Google. Your comment makes this practice sound shady. I don't see how this practice is even probably shady, if the general thrust of you say is true. First, you have to set up a different company for each country where you're doing business. Second, it's not a shell company if it hires employees or contractors and has clients. Finally, if the subsidiary doesn't engage in your primary business activity you don't call the subsidiary by a name similar to the primary company's name. It's a very common corporation structure.
  - baybal2 7y ago
    Well, at one point a recruiting company acting on their behalf tried recruiting me for an ops position. After seeing a few metre long NDA in perfect English, and Russian, and them being extremely tight lipped for whom and what for the job is, it was very clear to me that it was Google given that I knew that the recruiter is one of the few with whom Google works in Russia. I asked "Google?" It raised their eyebrows, but they said they can't answer that.
    - austhrow743 7y ago
      That's common recruiter behaviour. A lot of them are very afraid of being cut out of the deal.

creaghpatr 7y ago

Did you miss this part of bullet #3? >These moves are separate from those that block sites as required by U.S. or foreign law, such as those featuring child abuse or with copyright infringement, and from changes designed to demote spam sites, which attempt to game the system to appear higher in results. The article directly refutes what you are claiming here.

pas 7y ago
How exactly does the article contradict that? The article has very few specific examples, a few mentions of some draft and "suspicious" discrepancies between bing and google. None of which really mean much, except that it sounds like the typical liberal bias meme.

baybal2 7y ago

> Bullet #5: search quality is assessed by thousands of contractors worldwide and you can become a part of the crawl quality team, although it doesn't pay that well in the USA. The book they follow is 180 pages and available on the Google website for many years. It has guidelines for how to determine sexual, offensive, or illegal (or child porn) content in ALL countries. It has guidelines on how to rank news source reputation and credibility. Funny factoid for you: Google hires those guys in Russia through recruiters and innocuously sounding shell companies in Saint Petersburg. The guy can work for the shell for years without realising that he works for Google.

jhanschoo 7y ago
> Funny factoid for you: Google hires those guys in Russia through recruiters and innocuously sounding shell companies in Saint Petersburg. > The guy can work for the shell for years without realising that he works for Google. Your comment makes this practice sound shady. I don't see how this practice is even probably shady, if the general thrust of you say is true. First, you have to set up a different company for each country where you're doing business. Second, it's not a shell company if it hires employees or contractors and has clients. Finally, if the subsidiary doesn't engage in your primary business activity you don't call the subsidiary by a name similar to the primary company's name. It's a very common corporation structure.
- baybal2 7y ago
  Well, at one point a recruiting company acting on their behalf tried recruiting me for an ops position. After seeing a few metre long NDA in perfect English, and Russian, and them being extremely tight lipped for whom and what for the job is, it was very clear to me that it was Google given that I knew that the recruiter is one of the few with whom Google works in Russia. I asked "Google?" It raised their eyebrows, but they said they can't answer that.
  - austhrow743 7y ago
    That's common recruiter behaviour. A lot of them are very afraid of being cut out of the deal.