5 ms·

But if Google shuts those domains out, then the content farmers can just pick up new ones. Furthermore, Google would need to have some sort of process for dete

by sethg 16y ago

But if Google shuts those domains out, then the content farmers can just pick up new ones. Furthermore, Google would need to have some sort of process for determining which domains are content-farm spam before shutting them out. As with spam, the bad guys can be far more agile than the good guys. The point of having algorithms is that once you have a good algorithm, your search-quality staff doesn’t have to scale linearly with the number of pages on the Web. (Disclaimer: I work in the search division of Nokia, i.e., I work for one of Google’s competitors.)

ekanes 16y ago
Still, a power law applies and you can get huge gains by punishing the biggest offenders and moving down from there. And yes, spammers can always restart, but the sandbox helps with that.
roadnottaken 16y ago
Google has 23,000 employees and ~$10B in yearly operating income. I'm pretty sure they could stay ahead of the bad guys if they wanted to. EDIT: Additionally, it is hard to game pagerank rapidly because the rest of the web needs to link to your site. So, even if you switch your knock-off wikipedia site to a new domain, it would take weeks/months to rise through the rankings. I'm pretty sure it would only take a few employees (at most) to stay ahead of these huge spam sites.
- rokhayakebe 16y ago
  it is hard to game pagerank rapidly because the rest of the web needs to link to your site Not really. Some people own different websites, purchased through different accounts, hosted on different servers. They, then, link between those sites.
- gregholmberg 16y ago
  Google has 23,000 employees ... . I'm pretty sure they could stay ahead of the bad guys if they wanted to. In the past there were people within Google who had the skill, knowledge, organizational connections and authority to quickly and gracefully reduce the problem of link spam without harming the bottom line too much. I think we all loved their early work, and our love helped propel their little company to multibillions-per-year. I don't think that kind of unique early experience and subsequent problem-solving effectiveness is snap-in interchangeable. Some of those "rock stars" work at Facebook now. Google might not be capable of fixing the problem gracefully without them.
  - roadnottaken 16y ago
    That's depressing... I hope you're wrong.
- nostrademons 16y ago
  Compared to how many millions of people making money on the web, all of whom have an incentive to boost their own traffic regardless of whether that's the best thing for users? Joy's Law comes into play here: "No matter who you are, most of the smartest people work for someone else." 23,000 employees sounds huge, until you compare it with the million+ people who make their living as full-time E-bay sellers, and the however many million people who make their living off AdSense.
bad_user 16y ago
GMail doesn't rely solely on algorithms, but also on learning from users' flagging of spam. The only problem is that sometimes it has false positives. I would be happy if Google provided an "is spam" button that filtered the content only for me (i.e. without consequences to other users).
- roadnottaken 16y ago
  Don't Google engineers actually use Google? They shouldn't really need flags to become aware of this sort of thing.
  - sethg 16y ago
    In my experience, the path from “engineer recognizes that queries X, Y, and Z return crap search results” to “search engine improves its performance with queries X, Y, and Z without creating more crap somewhere else” is more difficult than a lot of people realize.
- encoderer 16y ago
  well, yeah, that's how the algorithms work... by using a training dataset where a human actually says "is spam." And as for the "is spam" button... a given email isn't automatically added to the spam training dataset the minute 1 guy hits the spam button. it goes automatically into YOUR spam folder but not automatically into the training dataset. That takes many "votes" from many users.
  - bad_user 16y ago
    That's why I would be happy with a personal spam filter: training by users' votes can be gamed, and producing false positives can take legitimate websites out of business. I see there are some browser plugins floating around: I'm not happy with those because I use multiple workstations, and because hacking around a product's deficiencies is not "voting with my wallet".
bigfudge 16y ago
Without wishing to be rude does Nokia's search division really compete with google?
- wazoox 16y ago
  I suppose it does in the mobile space, at least.
- sethg 16y ago
  High-end Nokia phones have a local-search client that competes with Google Maps. The front end for people using regular browsers is here: http://maps.ovi.com/ http://maps.ovi.com/
  - bigfudge 16y ago
    Fair enough, although that's not really competing in search is it? Doesn't work on an iPad so I can't see it right now.

by sethg 16y ago

ekanes 16y ago

Still, a power law applies and you can get huge gains by punishing the biggest offenders and moving down from there. And yes, spammers can always restart, but the sandbox helps with that.

roadnottaken 16y ago

Google has 23,000 employees and ~$10B in yearly operating income. I'm pretty sure they could stay ahead of the bad guys if they wanted to. EDIT: Additionally, it is hard to game pagerank rapidly because the rest of the web needs to link to your site. So, even if you switch your knock-off wikipedia site to a new domain, it would take weeks/months to rise through the rankings. I'm pretty sure it would only take a few employees (at most) to stay ahead of these huge spam sites.

rokhayakebe 16y ago
it is hard to game pagerank rapidly because the rest of the web needs to link to your site Not really. Some people own different websites, purchased through different accounts, hosted on different servers. They, then, link between those sites.
gregholmberg 16y ago
Google has 23,000 employees ... . I'm pretty sure they could stay ahead of the bad guys if they wanted to. In the past there were people within Google who had the skill, knowledge, organizational connections and authority to quickly and gracefully reduce the problem of link spam without harming the bottom line too much. I think we all loved their early work, and our love helped propel their little company to multibillions-per-year. I don't think that kind of unique early experience and subsequent problem-solving effectiveness is snap-in interchangeable. Some of those "rock stars" work at Facebook now. Google might not be capable of fixing the problem gracefully without them.
- roadnottaken 16y ago
  That's depressing... I hope you're wrong.
nostrademons 16y ago
Compared to how many millions of people making money on the web, all of whom have an incentive to boost their own traffic regardless of whether that's the best thing for users? Joy's Law comes into play here: "No matter who you are, most of the smartest people work for someone else." 23,000 employees sounds huge, until you compare it with the million+ people who make their living as full-time E-bay sellers, and the however many million people who make their living off AdSense.

bad_user 16y ago

GMail doesn't rely solely on algorithms, but also on learning from users' flagging of spam. The only problem is that sometimes it has false positives. I would be happy if Google provided an "is spam" button that filtered the content only for me (i.e. without consequences to other users).

roadnottaken 16y ago
Don't Google engineers actually use Google? They shouldn't really need flags to become aware of this sort of thing.
- sethg 16y ago
  In my experience, the path from “engineer recognizes that queries X, Y, and Z return crap search results” to “search engine improves its performance with queries X, Y, and Z without creating more crap somewhere else” is more difficult than a lot of people realize.
encoderer 16y ago
well, yeah, that's how the algorithms work... by using a training dataset where a human actually says "is spam." And as for the "is spam" button... a given email isn't automatically added to the spam training dataset the minute 1 guy hits the spam button. it goes automatically into YOUR spam folder but not automatically into the training dataset. That takes many "votes" from many users.
- bad_user 16y ago
  That's why I would be happy with a personal spam filter: training by users' votes can be gamed, and producing false positives can take legitimate websites out of business. I see there are some browser plugins floating around: I'm not happy with those because I use multiple workstations, and because hacking around a product's deficiencies is not "voting with my wallet".

bigfudge 16y ago

Without wishing to be rude does Nokia's search division really compete with google?

wazoox 16y ago
I suppose it does in the mobile space, at least.
sethg 16y ago
High-end Nokia phones have a local-search client that competes with Google Maps. The front end for people using regular browsers is here: http://maps.ovi.com/ http://maps.ovi.com/
- bigfudge 16y ago
  Fair enough, although that's not really competing in search is it? Doesn't work on an iPad so I can't see it right now.