4 ms·

I found searching minus the top 10k websites to be most effective for finding new stuff. It seems that after that point, a lot of the 'crap SEO' stuff is gone.

I found searching minus the top 10k websites to be most effective for finding new stuff. It seems that after that point, a lot of the 'crap SEO' stuff is gone. One problem, the sort of baby with the bath water issue, is that the quality of content also goes down. The search engines still do a great job for the most part in surfacing relevant content, mostly because the barrier to entry to 'gaming' them is high enough ( that most people dont' really notice spam ). I don't know if seaching this way will be an effective strategy for me going forward, or even if for the most popular categories the spammers start a new SEO campaign, ( let's optimize these queries for after the top X domains. ), but it was definitely an interesting thought/search experiment. And only in an hour? impressive.

greglindahl 14y ago
I bet there are types of searches for which excluding popular sites works well -- for example, trying to find less popular stuff, which is something that popular websites don't do that good of a job of covering. "Indie" topics, if you will. Would you try to discuss an indie movie on a website chock full of discussions about the latest blockbuster? Then when you search, don't you want to look at less popular movie websites to find the good discussion about your indie movie?
- stcredzero 14y ago
  Already showed up here: http://millionshort.com/ http://millionshort.com/
  - greglindahl 14y ago
    You're right -- I mention that right at the top of this blog posting as what inspired me to do this 1 hour hack.
    - stcredzero 14y ago
      'doh!
      - tripzilch 14y ago
        Aw come on, if HN's UI would be "advanced"[0] enough that it would collapse low-scoring comments[1], it would actually make sense to downvote this. But now it just seems petty. [0] you know, like Reddit. [1] or if its HTML wouldn't be so horrid to make me give up in despair every time I think to script myself such a thing (as none of the existing 3rd party solutions seem to have gone through the trouble of testing on Opera)
ChuckMcM 14y ago
I find the really strange / off-topic / ranty stuff, the dust bunnies of the Internet, fascinating in its own way.
stcredzero 14y ago
I found searching minus the top 10k websites to be most effective for finding new stuff. It seems that after that point, a lot of the 'crap SEO' stuff is gone. One problem, the sort of baby with the bath water issue, is that the quality of content also goes down. What if some search company curated the top 10k websites, blocking those with crap SEO stuff? This might work for a niche player. I don't know if searching this way will be an effective strategy for me going forward, or even if for the most popular categories the spammers start a new SEO campaign, ( let's optimize these queries for after the top X domains. ) What if the top cutoff varied randomly over time?
- greglindahl 14y ago
  One challenge of getting rid of spam this way is that spam sites are both big and small. blekko essentially already curates the top 10,000 websites for spam and terrible content. That's not good enough to get rid of marginal content at an overall acceptable website. I suspect this technique is most useful for finding niche stuff.

greglindahl 14y ago

I bet there are types of searches for which excluding popular sites works well -- for example, trying to find less popular stuff, which is something that popular websites don't do that good of a job of covering. "Indie" topics, if you will. Would you try to discuss an indie movie on a website chock full of discussions about the latest blockbuster? Then when you search, don't you want to look at less popular movie websites to find the good discussion about your indie movie?

stcredzero 14y ago
Already showed up here: http://millionshort.com/ http://millionshort.com/
- greglindahl 14y ago
  You're right -- I mention that right at the top of this blog posting as what inspired me to do this 1 hour hack.
  - stcredzero 14y ago
    'doh!
    - tripzilch 14y ago
      Aw come on, if HN's UI would be "advanced"[0] enough that it would collapse low-scoring comments[1], it would actually make sense to downvote this. But now it just seems petty. [0] you know, like Reddit. [1] or if its HTML wouldn't be so horrid to make me give up in despair every time I think to script myself such a thing (as none of the existing 3rd party solutions seem to have gone through the trouble of testing on Opera)

ChuckMcM 14y ago

I find the really strange / off-topic / ranty stuff, the dust bunnies of the Internet, fascinating in its own way.

stcredzero 14y ago

I found searching minus the top 10k websites to be most effective for finding new stuff. It seems that after that point, a lot of the 'crap SEO' stuff is gone. One problem, the sort of baby with the bath water issue, is that the quality of content also goes down. What if some search company curated the top 10k websites, blocking those with crap SEO stuff? This might work for a niche player. I don't know if searching this way will be an effective strategy for me going forward, or even if for the most popular categories the spammers start a new SEO campaign, ( let's optimize these queries for after the top X domains. ) What if the top cutoff varied randomly over time?

greglindahl 14y ago
One challenge of getting rid of spam this way is that spam sites are both big and small. blekko essentially already curates the top 10,000 websites for spam and terrible content. That's not good enough to get rid of marginal content at an overall acceptable website. I suspect this technique is most useful for finding niche stuff.