6 ms·

Google undeleted whocalled.us after a front page article on Hacker News

125 points by whocalledus 12y ago

Last week I posted on HN that 'Google deleted whocalled.us for “Pure Spam” and replaced it with spam' (https://news.ycombinator.com/item?id=9050343). I appreciated reading your thoughts, and it made me feel better because the worst part of it all was feeling like nobody could hear my protests that they made a mistake. I made 8 reconsideration requests, which are messages you send Google to ask for a human to manually review the penalty. All of them were denied with the same automated response. It did not feel like any humans actually read or thought about the situation, so I felt voiceless. As usual, if you make a public scene, a large corporation will often notice, and look closer at that individual's issue. I think that's what happened in this case. I happened to notice today that whocalled.us was back in Google, and the "Pure Spam" manual action had disappeared. There was no notice or explanation, and it feels like someone at Google saw my post, and quietly fixed it. I thought it was important to notify HN about this. I don't know how to feel about this. I already removed AdSense from the site, and gave up on it. From a shortsighted view, I should be happy, and thank Google. But this experience showed me that Google has more power over my website than I do.

solve 12y ago
Google search is FAR more human-driven than they'll ever admit to - for legal and PR reasons.
- lingben 12y ago
  I think most people who are even remotely interested in SEO know this to be true.
  - deleted 12y ago
    [deleted]
  - solve 12y ago
    Or remotely familiar with modern machine learning. With a company of that budget, all important systems ultimately go through a supervised learning layer. Practically nothing important uses unsupervised learning end-to-end.
geetee 12y ago
Fixing mistakes without accepting blame or acknowledging the error always bothers me. Does it stem from the fear of a lawsuit-happy society, or embarrassment?
- nostrademons 12y ago
  I suspect that a lawsuit-happy society is a good part of it. Having been on the other side a few times (the one who quietly fixed an error without acknowledgement), there are a number of other factors at work as well: 1. Sometimes, fixing the error is within my power but talking to the customer is not. For example, when I was at Google, I was an engineer - fixing a bug was something I could do merely with a code review, but talking to the outside world was forbidden by company policy, and I didn't even know who the people were that were authorized to speak to the outside world on company behalf. 2. Very often, the error is one of several dozen in a bug queue, and you just want to fix it and move on. Talking to someone - particularly a customer - is a big mental context switch; it means that 5 other people won't get their errors fixed at all. 3. Sometimes the error is a distraction from your normal job duties, you feel like you can do a good deed by taking 5 minutes of your day to tweak a config file, but you don't want it to blow up any larger than that, and usually once a user has your attention they feel free to ask you to fix all their other annoyances. This is a big problem with working on open-source software; I've never yet had a good deed take less than 5x the amount of time I budgeted for it, and usually involve a few people mad at me. (I often find out later that many other people are very appreciative, but the latter never seem to speak up). I think the fundamental problem is a mismatch between our expectations as human beings, which are shaped by millenia of face-to-face interactions, and the reality of modern society, where we are just a ticket number to a faceless corporation. Internet companies like Google and Facebook have roughly a million users per engineer working on the product; Whatsapp has 50 million. Imagine the scale of insignificance of any one problem; by the numbers, a user of a consumer product is roughly as consequential as a hair on your head. That could be why we get such shitty service from basically every major consumer company out there.
  - scottcanoni 12y ago
    This seems likely: "fixing the error is within my power but talking to the customer is not."
malchow 12y ago
I am the cofounder of a company that serves as an alternative to AdSense. Get in touch with me: joe@publir.com.
- lingben 12y ago
  what does this have to do with adsense or its competition?
  - pearjuice 12y ago
    OP gave up on AdSense.
Bahamut 12y ago
It could also be that someone from Google saw the post and put in a request for reconsideration on your behalf too. Unfortunately, a perk of working at Google means that any requests by Google employees get extra consideration (at least with several issues friends have had to deal with involving Google taking down videos/channels on YouTube)
compbio 12y ago
I saw your previous post and could not find egregious blackhat SEO stuff. The backlink profile was not too healthy though, with around 250 domains responsible for over 7000 backlinks. You said you got a spam warning on Google Webmaster Tools: These penalties are hardly ever permanent. > As usual, if you make a public scene, a large corporation will often notice, and look closer at that individual's issue. I think that's what happened in this case. > I made 8 reconsideration requests .. all of them were denied. Probably because you did not show a good will effort to clean up your act. At least you do not tell us what were the contents of these reconsideration requests. For all we know it was: "Reconsider me!" From your previous post I gather that you were heavily annoyed with the unnatural links actions and warnings. Instead of cleaning up your act for Google you focused on a suggested unnatural link to claim it was all BS. > As usual, if you make a public scene, a large corporation will often notice, and look closer at that individual's issue. I think that's what happened in this case. I do think that the people at Google notice a prominent post on HackerNews. But I do not think that they manually removed a spam penalty to avoid bad publicity. If anything: allowing spammy sites in the index without even a slap on the wrist is not good for publicity at all (and very visible to the users). I think the pre-set penalty period was over, or a recrawl found a cleaner site. > But this experience showed me that Google has more power over my website than I do. Google has more power over their search engine index than you do. You are free to do whatever you want to do with your website. I do have a question: The DuckDuckGo widget you had on your site (now changed to a general link), was that an official widget? Did money exchange hands for placing such a widget/link on every page of your site? I am wondering why a site-wide link to a commercial page is not 'nofollow'-ed, and if that may have brought you unwanted quality guidelines attention.
- jschwartzi 12y ago
  A lot of the "spamminess" comes from practices that were perfectly acceptable back when Google came out, or which are encouraged by the standards which define how the Web operates. For example, each phone number in his database has a specific URL which you can use to access it. Many of these pages have similar features, being that the whole point of the site is to aggregate discussions about suspect phone numbers. This is a similar profile to the "download" sites that have proliferated, except that in this case each URL links to something useful, where the URLs on the download sites link to nothing useful. Further, the whole point of the Web is to link pages together. That Google penalizes one site author for what other site authors do is a problem with Google, not a problem with the author. In the previous posting he indicated that Google asked him to contact the various site authors and ask them to remove their links. Why should he have to do this?
mdesq 12y ago
There's a possible case to be made that Google should be regulated in some way, perhaps as a public utility, with requirements for human interaction. It's unacceptable that they have as much power over the Internet as they do and still hide behind the facade of a machine.
- warmfuzzykitten 12y ago
  You might make a case for regulation if a) you could show that Google is a monopoly and b) Google used its monopoly power to shut out competition or control market pricing. But, at least in the U.S., the former is not true - a market share of 75% is necessary to establish a per se monopoly - unless the latter is true, and the latter is patently untrue. "Attract a lot of eyeballs" is not, thank goodness, grounds for regulation, or Twitter and Facebook would surely be regulated. There is no facade. Google search is algorithmically driven, and no one has ever shown that its algorithms discriminate unfairly.
chdir 12y ago
Your experience is scary-pleasant. Perhaps someone should start a peer-reviewed list : why-google-blacklisted.me where the top voted ones would stand a chance of getting reviewed by non-bots at Google. In all likelihood, a situation like this gives nightmares to entrepreneurs on a daily basis. Edit: Another recent similar incident: https://news.ycombinator.com/item?id=8873149 https://news.ycombinator.com/item?id=8873149
- lingben 12y ago
  So in essence you want thousands of people to volunteer their time and energy to do the work that a multi-billion dollar corporation should be doing in the first place? "Don't be evil" at this point is a punch line
  - chdir 12y ago
    You didn't give an alternate solution? Someone mentioned here there's a million to 1 user to engineer ratio. As much as I would want Google to fix the issue, I don't expect that happening anytime soon. If all of us can collectively help the community fix bugs then why not? Just like countless other crowdsourced intelligence tools (e.g. wot browser extension, ad blockers etc.).
    - lingben 12y ago
      Isn't it obvious? The alternative solution is that google takes responsibility for the quality of their search results and actually pays people to work on it. I know they like to pretend that everything is automated but we all know it clearly isn't. And no, they don't need to hire a $300,000 a year engineer to do this work. You don't need advanced programming knowledge or experience to be able to judge whether a website has a real case when they submit a reconsideration request.
dm2 12y ago
Isn't it likely that they just made a minor "algorithm" update that fixed the real reason why your site was penalized? Maybe they reset your penalty as a "thank you" for bringing it to their attention?

125 points by whocalledus 12y ago

solve 12y ago

Google search is FAR more human-driven than they'll ever admit to - for legal and PR reasons.

lingben 12y ago
I think most people who are even remotely interested in SEO know this to be true.
- deleted 12y ago
  [deleted]
- solve 12y ago
  Or remotely familiar with modern machine learning. With a company of that budget, all important systems ultimately go through a supervised learning layer. Practically nothing important uses unsupervised learning end-to-end.

geetee 12y ago

Fixing mistakes without accepting blame or acknowledging the error always bothers me. Does it stem from the fear of a lawsuit-happy society, or embarrassment?

nostrademons 12y ago
I suspect that a lawsuit-happy society is a good part of it. Having been on the other side a few times (the one who quietly fixed an error without acknowledgement), there are a number of other factors at work as well: 1. Sometimes, fixing the error is within my power but talking to the customer is not. For example, when I was at Google, I was an engineer - fixing a bug was something I could do merely with a code review, but talking to the outside world was forbidden by company policy, and I didn't even know who the people were that were authorized to speak to the outside world on company behalf. 2. Very often, the error is one of several dozen in a bug queue, and you just want to fix it and move on. Talking to someone - particularly a customer - is a big mental context switch; it means that 5 other people won't get their errors fixed at all. 3. Sometimes the error is a distraction from your normal job duties, you feel like you can do a good deed by taking 5 minutes of your day to tweak a config file, but you don't want it to blow up any larger than that, and usually once a user has your attention they feel free to ask you to fix all their other annoyances. This is a big problem with working on open-source software; I've never yet had a good deed take less than 5x the amount of time I budgeted for it, and usually involve a few people mad at me. (I often find out later that many other people are very appreciative, but the latter never seem to speak up). I think the fundamental problem is a mismatch between our expectations as human beings, which are shaped by millenia of face-to-face interactions, and the reality of modern society, where we are just a ticket number to a faceless corporation. Internet companies like Google and Facebook have roughly a million users per engineer working on the product; Whatsapp has 50 million. Imagine the scale of insignificance of any one problem; by the numbers, a user of a consumer product is roughly as consequential as a hair on your head. That could be why we get such shitty service from basically every major consumer company out there.
- scottcanoni 12y ago
  This seems likely: "fixing the error is within my power but talking to the customer is not."

malchow 12y ago

I am the cofounder of a company that serves as an alternative to AdSense. Get in touch with me: joe@publir.com.

lingben 12y ago
what does this have to do with adsense or its competition?
- pearjuice 12y ago
  OP gave up on AdSense.

Bahamut 12y ago

It could also be that someone from Google saw the post and put in a request for reconsideration on your behalf too. Unfortunately, a perk of working at Google means that any requests by Google employees get extra consideration (at least with several issues friends have had to deal with involving Google taking down videos/channels on YouTube)

compbio 12y ago

I saw your previous post and could not find egregious blackhat SEO stuff. The backlink profile was not too healthy though, with around 250 domains responsible for over 7000 backlinks. You said you got a spam warning on Google Webmaster Tools: These penalties are hardly ever permanent. > As usual, if you make a public scene, a large corporation will often notice, and look closer at that individual's issue. I think that's what happened in this case. > I made 8 reconsideration requests .. all of them were denied. Probably because you did not show a good will effort to clean up your act. At least you do not tell us what were the contents of these reconsideration requests. For all we know it was: "Reconsider me!" From your previous post I gather that you were heavily annoyed with the unnatural links actions and warnings. Instead of cleaning up your act for Google you focused on a suggested unnatural link to claim it was all BS. > As usual, if you make a public scene, a large corporation will often notice, and look closer at that individual's issue. I think that's what happened in this case. I do think that the people at Google notice a prominent post on HackerNews. But I do not think that they manually removed a spam penalty to avoid bad publicity. If anything: allowing spammy sites in the index without even a slap on the wrist is not good for publicity at all (and very visible to the users). I think the pre-set penalty period was over, or a recrawl found a cleaner site. > But this experience showed me that Google has more power over my website than I do. Google has more power over their search engine index than you do. You are free to do whatever you want to do with your website. I do have a question: The DuckDuckGo widget you had on your site (now changed to a general link), was that an official widget? Did money exchange hands for placing such a widget/link on every page of your site? I am wondering why a site-wide link to a commercial page is not 'nofollow'-ed, and if that may have brought you unwanted quality guidelines attention.

jschwartzi 12y ago
A lot of the "spamminess" comes from practices that were perfectly acceptable back when Google came out, or which are encouraged by the standards which define how the Web operates. For example, each phone number in his database has a specific URL which you can use to access it. Many of these pages have similar features, being that the whole point of the site is to aggregate discussions about suspect phone numbers. This is a similar profile to the "download" sites that have proliferated, except that in this case each URL links to something useful, where the URLs on the download sites link to nothing useful. Further, the whole point of the Web is to link pages together. That Google penalizes one site author for what other site authors do is a problem with Google, not a problem with the author. In the previous posting he indicated that Google asked him to contact the various site authors and ask them to remove their links. Why should he have to do this?

mdesq 12y ago

There's a possible case to be made that Google should be regulated in some way, perhaps as a public utility, with requirements for human interaction. It's unacceptable that they have as much power over the Internet as they do and still hide behind the facade of a machine.

warmfuzzykitten 12y ago
You might make a case for regulation if a) you could show that Google is a monopoly and b) Google used its monopoly power to shut out competition or control market pricing. But, at least in the U.S., the former is not true - a market share of 75% is necessary to establish a per se monopoly - unless the latter is true, and the latter is patently untrue. "Attract a lot of eyeballs" is not, thank goodness, grounds for regulation, or Twitter and Facebook would surely be regulated. There is no facade. Google search is algorithmically driven, and no one has ever shown that its algorithms discriminate unfairly.

chdir 12y ago

Your experience is scary-pleasant. Perhaps someone should start a peer-reviewed list : why-google-blacklisted.me where the top voted ones would stand a chance of getting reviewed by non-bots at Google. In all likelihood, a situation like this gives nightmares to entrepreneurs on a daily basis. Edit: Another recent similar incident: https://news.ycombinator.com/item?id=8873149 https://news.ycombinator.com/item?id=8873149

lingben 12y ago
So in essence you want thousands of people to volunteer their time and energy to do the work that a multi-billion dollar corporation should be doing in the first place? "Don't be evil" at this point is a punch line
- chdir 12y ago
  You didn't give an alternate solution? Someone mentioned here there's a million to 1 user to engineer ratio. As much as I would want Google to fix the issue, I don't expect that happening anytime soon. If all of us can collectively help the community fix bugs then why not? Just like countless other crowdsourced intelligence tools (e.g. wot browser extension, ad blockers etc.).
  - lingben 12y ago
    Isn't it obvious? The alternative solution is that google takes responsibility for the quality of their search results and actually pays people to work on it. I know they like to pretend that everything is automated but we all know it clearly isn't. And no, they don't need to hire a $300,000 a year engineer to do this work. You don't need advanced programming knowledge or experience to be able to judge whether a website has a real case when they submit a reconsideration request.

dm2 12y ago

Isn't it likely that they just made a minor "algorithm" update that fixed the real reason why your site was penalized? Maybe they reset your penalty as a "thank you" for bringing it to their attention?