13 ms·

Dear Google: please let me ban sites from results

438 points by nervechannel 16y ago

Given the current high-ranking thread about spammy sites in Google results, it strikes me that a very simple solution would be to let logged-in users blacklist sites. Bam, no more wareseeker or efreedom. This would solve a lot of people's complaints in one fell swoop. There are greasemonkey etc. scripts to do this, but they're tied to a single browser on a single machine. A global filter (like in gmail) would be so much more useful. Would this be particularly hard to do?

iwwr 16y ago
In the interim, you can do your searches by adding -wareseeker -efreedom to the search string.
- nervechannel 16y ago
  I've discovered you can also set up a Custom Search Engine, with no included sites (default to everything), and specifically exclude the sites you don't want. Then do all your searches through this. http://www.google.com/cse/ http://www.google.com/cse/ Usability-wise, though, it's not nearly as much use as a 'ban' button next to each result would be. But it shows Google already have the infrastructure and code that would allow this -- they just need to make it instant to use. EDIT: The other downside of this is you lose a load of bells & whistles, e.g. previews, "pages from the UK" (without typing), icons for images/news/etc. Time will tell if I miss those.
  - srean 16y ago
    But its not free, right ?
    - duck 16y ago
      It is free.
    - JoachimSchipper 16y ago
      From the signup page: "Standard edition - ads are required on search pages" There's an ad-free premium version as well, but you definitely can get it for free.
    - wccrawford 16y ago
      It's free if you show ads on the pages. It costs money if you don't.
    - nervechannel 16y ago
      Basic ad-supported version is free.
  - tszyn 16y ago
    Yes, it works and it's completely free. Here's an example Custom Search Engine I set up: http://www.antimoon.com/ce/ http://www.antimoon.com/ce/ It includes only sites which are known to use (mostly) good English. It's designed for English learners/teachers who want to find correct usage examples without risking exposure to Yahoo-Answers-style English.
  - jmckib 16y ago
    I tried this, and if you don't enter any websites to include, you get the message: You must enter 1 or more websites.
foljs 16y ago
And no more bloody experts-exchange...
- shrikant 16y ago
  I honestly don't mind that site - about half the time I search for an issue, I find it's been resolved by someone over at Experts Exchange. Do people realise that if Google is your referrer, you can scroll all the way down and see the solutions to the question?
  - duck 16y ago
    I have always known that, but I don't think a lot of people do. A while back I was working with a high dollar consultant who really thought he knew it all, but he seemed really surprised that I didn't need to use his Experts Exchange account to see the answer.
    - nervechannel 16y ago
      It didn't used to be like that -- you used to have to 'view source' to see it <facepalm> Then I think they went through a stage of not revealing it at all. But in any case, it's far less useful than SO/SF etc.
      - hessenwolf 16y ago
        Depends on what you are doing. I sold out to greysuitland and ee is better for all my retarded vb, vba, c# questions.
  - jawns 16y ago
    It seems like every few months, EE tries to cloak its site in a new way ... then Google catches them and they revert to merely misleading people (by putting the answers several page-scrolls down) into thinking that they need to pay to see the answers. Actually, I sort of sympathize with the predicament EE faces. They want to show up high on Google search results, because that's how they get new customers ... but they don't want to give away their content for free. Here's an opportunity for a search engine start-up: Allow users to search (by default) for only free content -- but also allow them to search, if they so choose, for content that's behind a paywall. Pay-for-access sites would love something like this.
alexobenauer 16y ago
Although it's sad because it speaks volumes that we're fed up with all the garbage in many of our search queries. I do hope those working on the algorithm are taking note.
aquilax 16y ago
Wasn't this a problem Google Search Wiki tried to solve? http://googleblog.blogspot.com/2008/11/searchwiki-make-search-your-own.html http://googleblog.blogspot.com/2008/11/searchwiki-make-searc...
- joeyo 16y ago
  Yeah. I miss searchwiki. :-(
al_james 16y ago
Yes that would be good. They could then look at the number of people blocking certain domains and de-weight them in the global results. Traditionally google seem against human powered editing (as this would be), but I think as the black hat SEOs run rings around them, its needed badly.
- thushan 16y ago
  Anyone have any ideas why the clone sites like efreedom are ranking above Stack Overflow when SO's inbound links and reputation values are likely far better than efreedom's in Google's algorithm? I'm surprised that search engine optimization could do THAT much to a site's ranking. Also it's not like SO isn't doing the same kind of SEO themselves. What I'm trying to get at is, with all things equal, let's say Stack Overflow and efreedom's SEO is on par with each other, shouldn't SO's reputation/inbound link ranks automatically trump things?
  - hessenwolf 16y ago
    Efreedom is deliberately manufacturing links, whereas SO may be just hoping for the best.
    - thushan 16y ago
      Shouldn't the sites hosting deliberately manufacturing links also fall to the algorithm's reputation/inbound link values as well. We all know they don't, and that seems like that is one of the fundamental issues here. As much as I'd like to have a site black list, that doesn't really help the major issue at hand.
    - jqueryin 16y ago
      Efreedom has been the bane of my recent programming related searches, polluting my results to no end.
      - mattmanser 16y ago
        Do what I've taken to doing nowdays, for programming questions go straight to stackoverflow.com before Google. Or use google with site:stackoverflow.com
eliben 16y ago
Can't this be done with a browser plugin?
- nervechannel 16y ago
  Like I said in the OP: "There are gresemonkey etc. scripts to do this, but they're tied to a single browser on a single machine. A global filter (like in gmail) would be so much more useful."
djhworld 16y ago
I think the worst culprits are the ones that skim StackOverflow questions and rehash them into their own supposed original "question and answer" site
- deleted 16y ago
  [deleted]
- bartl 16y ago
  Jeff Atwood discusses this in his most recent Coding Horror blog post (http://www.codinghorror.com/blog/2011/01/trouble-in-the-house-of-google.html http://www.codinghorror.com/blog/2011/01/trouble-in-the-hous...), in which he states that he doesn't really mind people copying StackOverflow questions (and answers), but that he does mind that the copies get a higher Google ranking than the original.
  - retube 16y ago
    How does that happen though? efreedom must be doing something special with SEO to get higher than SO given that SO must have huge page rank score.
    - iuytgyuikoiuy 16y ago
      Pagerank is based on number/quality of links. Nobody links to an SO question, except perhaps mentioning it in a blog post. Scrapping sites create lots of links to the question, even if they are low quality ones
      - retube 16y ago
        Yeah but does anyone link to efreedom question pages? And SO as a whole must have way more inbounds than efreedom, surely?
- ergo98 16y ago
  What do you think most StackOverflow answers are? It's a karma-paid labor pool where you can post questions and a lot of under-employed people will rush out and do the necessary Google searches, collating and slightly rewriting the results to yield the most votes. Everyone is ripping off someone's content. And just to be accurate here, SO content is creative commons (created by the community). Are those just cheap words?
pixelbeat 16y ago
Google were experimenting with voting on results: http://techcrunch.com/2007/11/28/straight-out-of-left-field-google-experimenting-with-digg-style-voting-on-search-results/ http://techcrunch.com/2007/11/28/straight-out-of-left-field-... Also there is this form for reporting spam sites: https://www.google.com/webmasters/tools/spamreport https://www.google.com/webmasters/tools/spamreport Integrating the above into standard search results would be difficult unless it was restricted to users with a good "karma". That might be possible in our increasingly socially networked world
- cosmicray 16y ago
  You could also restrict it to people who buy, and sell, on Google Checkout. Putting real money/goods on the table tends to weed out the fake accounts.
- CWuestefeld 16y ago
  The thing is, the SO scrapers like efreedom aren't spam, strictly speaking. It's just that they clone existing content without adding value, and as such are just noise in the results. Perhaps we need to frame the discussion differently, considering what the searcher wants, rather than "spam-free hits".
  - nervechannel 16y ago
    That was my point really. I don't want to see eFreedom hits, I consider them spammy, so I'd like to be able to click-ban them from my results. If Google use that information to gradually adjust their ranking overall, then fair enough -- won't affect me, I can't see them anyway. EDIT: Even if they don't let that affect everyone else's results (because of gaming), then I still don't care, I still don't see the crap in my results ever again.
krschultz 16y ago
I'd ban eHow.
- slig 16y ago
  And mahalo.
  - bradly 16y ago
    And ExpertsExchage and About.com
    - travisp 16y ago
      ExpertsExchange is at least original content (as far as I know)
      - jasonkester 16y ago
        ExpertsExchange used to be nearly 100% scraped usenet. Then eventually a few poor souls stumbled in thinking it was an actual site and started posting questions there directly. Eventually people even started answering them. Had they not had such a terrible site design and revenue scheme, they might have actually made the transition from scraping bottom feeder to respectable site.
        moxiemk1 16y ago
        I never knew that. I always assumed that someone out there was paying for all these questions. Good to know.
    - grok2 16y ago
      I notice that nowadays, I don't really care where the answers come from so long as they they answer my question. While I tend to look for and click on StackOverflow links and skip efreedom links (just because I want to be loyal to them), I find that I still use eHow and About.com and ExpertsExchange results since I find that the answers there most times help me with what I am seeking -- for me it doesn't matter really that it is the original source of the info or not. I think these sites are better than the true spammy sites that really have no content on the answer to your question, but somehow manage to get up there in the search results.
hessenwolf 16y ago
How many gmail accounts do we need to band together to lower the rank of stack overflow against our super-duper question-and-answer site QandAdsWithMe.annoying.com?
- Sandman 16y ago
  An excellent point. Blacklisting works both ways, and there's nothing stopping spammers from creating hundreds, maybe thousands, or even more, of throwaway Google accounts just to blacklist the original site. Sure, the logged in Google users wouldn't see the spam site (if they blacklisted it), but it would still appear, and outrank the original site, in standard search results for those that aren't logged in.
  - nervechannel 16y ago
    So don't use the blacklisting stats for re-ranking everyone else's then. I didn't consider that when I put the post up. An instant way to filter sites just for yourself would solve 90% of people's complaints straight away.
forkrulassail 16y ago
YES. Like the useless chromeextensions.org This would be an awesome feature.
dawgr 16y ago
That will never happen, if they ever did that it would be an admission that there is something inherently wrong with their algorithm. They won't do it.
- shimonamit 16y ago
  On the contrary. As suggested by another commenter, it could be used to improve their algorithm. Greater number of exclusions would reduce their relevance.
- nervechannel 16y ago
  Nah. They let people 'report spam' in gmail -- they're happy to admit that algorithm doesn't always guess right. Also there's already a "give us feedback" mechanism for reporting bad search results, it's just too slow and manual.
  - wahnfrieden 16y ago
    People are used to spam buttons, and would realize that the lack of "report spam" would mean that Google might be marking too many things as spam if they think the user doesn't need a "Mark as spam" button. People are also a lot less tolerable of spam in their inbox than they are of irrelevant search results.
Pewpewarrows 16y ago
Gmail already does it, and the global system uses an algorithm to look at reported spam results in order to automatically move future emails from that party to the spam folder automatically, not just for the person that reported it, but for everyone. If they're not looking into integrating that nicely into the existing search results page (not a separate form that the average user will never find or use), especially after all the internet chatter about it recently, then they definitely should make that a top priority in 2011. I definitely don't want them to do a rush job on it though. I don't want competitors to start reporting each other as spam in search results to try and game the system even further. I'm assuming they have anti-gaming measures in place for Gmail, so they won't be completely starting that from scratch...
- mtkd 16y ago
  I don't see how you could anti-game this, the SEOs would just use mechanical turk to hire 100s of people (with valid Google Accounts) to do the reporting. At best G could use the information as a list of potential spammers and filter domains manually, but I really can't see this being automated without giving the SEOs another weapon.
  - moe 16y ago
    I don't think anyone wants to filter sites that could be gamed with "100s of votes". We want to filter sites that will require tens of thousands of votes to get rid of.
    - roc 16y ago
      And even then it should consider the votes themselves to be suspect and watch for blocks of users who only vote in unison (qualitatively and temporally).
Luc 16y ago
Also, I would like '[any widget] review' to take me to an actual review, not pages upon pages of spam. I usually end up looking at comments on a few trusted sites (e.g. Amazon). This seems broken...
- CWuestefeld 16y ago
  Yes, most of the results for this query wind up pointing to pages saying "be the first to review [any widget]". As a workaround, try searching for "[any widget] sucks" and "[any widget] good". EDIT: tying this to other discussions on the topic, it's a symptom of Patio11's observation that natural language search doesn't work very well. If you want to find something, you need to paint a picture of what it looks like, rather than asking a question about it.
  - tokenadult 16y ago
    I've noticed for years that "[product] hosed" brings up good results on how to work around bugs in various software products.
  - tesseract 16y ago
    automated version of this: http://www.sucks-rocks.com/ http://www.sucks-rocks.com/
deleted 16y ago
[deleted]
- retube 16y ago
  I believe blekko does
shimonamit 16y ago
Maybe this could be implemented in the way of sticky search operators? So for example, I could define -site:efreedom.com as an operator to be applied silently for every search I make.
ScottWhigham 16y ago
For those wanting Google to put a penalty on the sites who are banned/removed from the user's view, what's to stop someone from gaming that system via Mech. Turk (or some other way)? Just pay people $0.12 to open gmail accounts and ban a competitor or whatever. That's the only negative I can think of - other than that, I say bring it!
davidk0101 16y ago
I'm not sure how this would be implemented. Where would the blacklist be held and how would it influence the search results? I know that they already do a lot of search customization but most of it is just aggregate statistical computations. It's not that they return results specifically tailored to you but more like results tailored to a very fuzzy average version of you. A blacklist seems way too specific to each user to be susceptible to meaningful aggregate statistical operations like spam filtering which is one of the reasons that spam filtering in google is so good. Each user contributes something and everyone benefits. I don't see that happening with blacklists. I think to make it worthwhile they would need to figure out how to feed the information from blacklists into providing more meaningful results for everyone.
- nervechannel 16y ago
  That's the point, what I want is exactly a user-specific blacklist. I can even do that already with Google's Custom Search, all that's missing is a little 'block this site' button. Instead I have to go and configure Custom Search manually for each URL mask.
  - jimmyswimmy 16y ago
    You could write a little GreaseMonkey script or extension to do that, shouldn't be too hard.
- gwern 16y ago
  > how would it influence the search results? Presumably it would do the same exact thing as '-site:foobar.com'.
hoofish 16y ago
the problem I have with this is that some black hat people can do this to any site they feel they are competing with. what would prevent someone from blacklisting a legitimate blog or website just because they did not like the content?
- nervechannel 16y ago
  So what? All that would mean is that the site wouldn't show up in their results. I never said anything about it affecting other people's results...
pragmatic 16y ago
Proof that true AI is a long way off? If the best and brightest (arguably) on the planet can't figure out how to filter out search with algorithms, what makes us think we can mimic true human intelligence any time soon. (I think it will happen, just not as soon as some claim)
- stcredzero 16y ago
  We are already able to (badly) mimic true human intelligence. The problem is when our mimics compete with humans in general contexts. There, being smarter means you can cream the competition. Basically, it's the difference between PvP and PvE.
- tel 16y ago
  Or maybe it just means that the day we can algorithmically emulate human intelligence is the day spam becomes useful (or the day we get impossibly good at fooling ourselves).
SimonPStevens 16y ago
No, it's not particular hard, but it will make the problem worse. Why? 99% of users are non-tech oriented. Those users will not really be aware of the specific problems with the search results, they won't understand the concept of a good vs bad result and they certainly won't bother to tweak/ban/filter their results. The 1% that do care and are currently being vocal about it will start filtering their results and they will perceive that the problem is solved. They will stop making a fuss. So now, the complaints have gone away, but 99% of users are still using the broken system, so the good sites that create good original content are still ranking below the scrapers and spam results for 99% of the users. The problem must be solved for all (or at least the majority) of users. (And you can't take the 1%s filtering and apply it to all users in some kind of social search because the spammers will just join the 1% and game the system)
- nervechannel 16y ago
  Do you think 99% of users are too stupid to click 'report spam' when they get a spam email?
  - SimonPStevens 16y ago
    Not "too stupid" no. I think 99% of email users have not been adequately trained in why or how they should report spam, and even if they were I think most of them would still not care enough to actually do it with any regularity. When pushed many may acknowledge that they know it exists, they will probably even be able to find the button when asked if given a chance. But they won't remember to do it when they see spam, they'll just ignore it and move on to the messages from people they know.
    - seabee 16y ago
      Do you think this could be improved by alternate wording? Instead of reporting 'spam', ask the user 'was this useful?' or 'did you want this?'. After all, the real goal is giving people a better, more relevant experience, detecting and removing spam is just one facet of that. Whether it's email or search.
podperson 16y ago
simply add -site:foo.com to your search request. And no, this doesn't solve the problem.
deleted 16y ago
[deleted]
AndrewO 16y ago
I see a lot of people asking what happens when a group of people downvote a site just to ruin its ranking. Sure that's a problem, but there's an easy solution on Google's end: your blacklist only affects you. Yes, that means all of us have to hide efreedom ourselves. Doesn't seem like a problem to me... Plus, we are talking about a company whose core business demands that it can identify groups of bad-faith voters. Given time, they may find a way to incorporate this data safely into the ranking data (if anyone could, it would be Google). And I know there are extensions to do this (mine mysteriously stopped working recently), but doing this on the client-side in a way that's bound to a single browser install just seems wrong to me, especially for Google.
- nervechannel 16y ago
  This is exactly what I meant. Lots of people have added the idea about crowd-sourced re-ranking based on blacklisting, then said "it'll never work..."
  - AndrewO 16y ago
    It seems to be a particular affliction among our group (by that I mean anyone who spends a lot of time here) that once we learn to apply one revolutionary/disruptive idea, we can't stop even when we probably should. I'd say it's because we're used to trying to think of how to scale-up every idea, but I've been accused of armchair-psychology before... :)
- peterhoffmann 16y ago
  The blacklist does not only have to affect me, just throw in the blacklists of my social graph too. These are the people I trust.
  - coderdude 16y ago
    I don't know about you, but my social graph is pretty diverse. I wouldn't trust all those very different people to make an important decision like "what sites should be visible to me in search engines."
Sukotto 16y ago
I want a search results page similar to the "Priority Inbox" we got recently in gmail. Set sane defaults and let me override them with "Important/Notimportant" buttons (or thumbs up/down or whatever) next to results. Let it learn what I think is a good result for my needs. If you make it a little bit social, make sure you weight other people's opinions by how much they agree with my own in other areas (making it harder for sockpuppets to muddy the waters)
pilom 16y ago
Startup idea: Create a service around google custom search. Select the "Search the entire web but emphasize the selected sites" Then create a gui to allow people to prioritize or ban their search results.
Tichy 16y ago
Didn't Google have downvotes for results - shouldn't they be sufficient to achieve the result you want? Presumably Google would learn that you consistently downvote wareseeker and exclude it from results in the future. I haven't used it because I don't want Google to remember my search history. But if you are willing to stay logged into Google (which would be required for your proposal), it would not be an issue.
joshrule 16y ago
It seems that it might be more helpful to whitelist sites. The web grows too quickly, and the mass of spam sites overwhelmingly so. If I had some way to blacklist sites, I'd end up spending a lot of time doing so. In fact, it could quickly take up most of my search time. If, though, we could whitelist sites, it seems that results would get cleaner faster. I don't care about how many bad sites are out there, as long as helpful sites make it to the top. Plus, I typically use just a few sites to access reliable information anyway (the number's about 7, right?), so if I can whitelist results from those sites, I'll probably find my desired content more quickly. What about the case when there are 30 spam sites listed before 1 good site? That hasn't happened too often for me. Instead, the results I'm looking for are usually just 4 or 5 spots down the front page, and very occasionally on the second page. White listing seems like it would still be faster and easier for now.
svlla 16y ago
I'd like to see an option for searching only ad-free sites, or perhaps just sites that don't use AdSense, as well. Surely Google would have no problem with that.
jeffg1 16y ago
It doesn't seem like it would be hard, but if the rankings aren't driven by money, then there will be attempts to game the system. The problem I feel is Money. As long as everyone has to compete for it (meaning money doesn't work for the people, people work for money - in a system owned by the few), we'll have shady marketers, shady products, spammers etc... so, I think that it will remain a cat and mouse game.
twir 16y ago
Looks like a lot of people are assuming a solution would some sort of voting system like stackexchange, etc. Why not allow individual users to hide sites from their own search results and save the info in their google account? For example, provide a "hide this site from my results" link next to each result. Each person decides which site they don't want to see and SEO and global results remain unaffected.
- teach 16y ago
  I feel like I'm taking crazy pills. Am I the only one that remembers this EXACT feature on Google about a year ago? You had to be logged in to iGoogle, and each search result had a small [X] to the right of it that would appear on hover. If you clicked it, that result wouldn't appear for you again. I used it all the time. Then, lately it's gone. Maybe I was part of a small, randomly-selected test group?
  - mthoms 16y ago
    That was part of Google's SearchWiki experiment. http://googleblog.blogspot.com/2008/11/searchwiki-make-search-your-own.html http://googleblog.blogspot.com/2008/11/searchwiki-make-searc... That experiment was replaced by Google "Stars" in March 2010 because, according to Google: > In our testing, we learned that people really liked the idea of marking a website for future reference, but they didn't like changing the order of Google's organic search results. http://googleblog.blogspot.com/2010/03/stars-make-search-more-personal.html http://googleblog.blogspot.com/2010/03/stars-make-search-mor... I personally think there is much more going on here than Google admits.
balakk 16y ago
How about decentralizing the search page? Hear me out for a bit. My theory is that these complaints are coming from specific interest groups, not the general public. For example, spammy-content is created and targeted at a developer/programmer audience, and that is the source of some of these complaints. So my suggestion is Google should platformize their search; and give out dedicated search instances to specific communities. The community should have enough levers to govern/influence what is spam or not. In addition, the community can promote certain high-value resources, which are otherwise unfairly listed in search results. Invite some high-profile communities for a test-run, and let the communities make their own choices. The public Google can still handle the general public. This can also bring in some transparency in the way spam is determined.
coffee 16y ago
"This would solve a lot of people's complaints in one fell swoop." And doing this would spawn a lot of people's complaints in one fell swoop. If you owned a site, and created enemies, they could band together and flag your site as spam.
- emef 16y ago
  I don't think you understood his suggestion. "Banning" a site would be local to your signed-in google account, not a global ban from results (which would indeed suffer the backfire you mentioned.)
  - coffee 16y ago
    doh! You are totally correct. I completely missed that. Maybe I should drink a bit of coffee and wake up ;) Although... I have a suspicion that at some point it would effect non-logged in users. Many logged in users banning a site is a signal that may effect the global results. Maybe in the same way marking spam in Gmail...
- scotty79 16y ago
  Now they can bot-click your adsense ads to cause google to ban you. If you are ad sponsored this effectively puts you out of business.
diegob 16y ago
Wouldn't implementing this feature be a tacit admission that there's a problem with search results?
coffeedrinker 16y ago
As programmers, our typical complaints are for sites that bog us down in common (expert's exchange, stackoverflow scrapers, etc.). What I found interesting: I was doing a search on something I normally have no interest in (a sewing machine manual for my wife) and I was amazed by the level of spam I was encountering. We have no idea how bad the problem is for others whose topics we do not usually see. The web is far more full of spam than we even realize.
charlesju 16y ago
Here is a conspiracy theory for you guys. 1. How does Google make money? Search Ads. 2. How do people click on search ads? Bad real search results.
radley 16y ago
Google does provide this service: it's called Google Custom Search. You can prioritize or blacklist sites and it's pretty easy to add it to your browser searchbar. I don't always use it, but I'll switch to it when I encounter a spammy topic, usually dev-related searches. http://radleymarx.com/blog/better-search-results/ http://radleymarx.com/blog/better-search-results/
- dejb 16y ago
  I can't believe this isn't the most popular comment. Kinda makes HN look a bit like a knowledge vacuum with all these recent discussions of how to ban results when a google service that's existed for nearly 5 years can do the job. The format of the results isn't quite a nice as the normal google search though.
  - deleted 16y ago
    [deleted]
  - nervechannel 16y ago
    Err, I mentioned this well before the comment above: http://news.ycombinator.com/item?id=2075437 http://news.ycombinator.com/item?id=2075437 It's not so much that it's a knowledge vacuum, just that someone didn't read the whole thread before replying.
  - radley 16y ago
    I'm on the "wrong" side of the Flash/HTML5 debate, so my average post value is really low. It wasn't a big deal until the HN algorithm got tweaked recently.
- jimmyswimmy 16y ago
  Agree with the sibling poster - this is the best comment in the thread. Do you have any interest in sharing your hard work with us? If you go to the CSE website and select 'Advanced' and then download annotations, you can export the list of sites you've excluded. Further you can make the exclusion list ("annotation list") into a feed - so it is entirely possible to implement the kind of user-generated blacklist of sites which has been discussed here.
richbradshaw 16y ago
Just use Google SearchWiki. Oh, yeah – they pulled it.
Rhapso 16y ago
It seems like a obvious answer, but why not just use "-site:annoyingpage.com" in you search? In fact "-TotallyUnRelated" has helped me narrow down searches effectively too. You are asking for a feature that only a small subset of the users will benefit from and use, it makes more sense for google just to find a way to rank sites better then it does to build a additional filter on top of the current system.
- jonhendry 16y ago
  Because I'd want to add a list of excluded sites for pretty much every single query I do. Would you want to type out a string of 20 or 30 excluded sites every time you search Google? Ranking clearly isn't going to be good enough, because algorithms can be worked around and gamed.
andrewljohnson 16y ago
I'd definitely make use of this feature. Some ancillary features might include: a) Google could warn you if it thinks the sites you have blacklisted seemed to have regained credibility. b) Google could suggest additional sites you may wish to blacklist, based on other user blacklists. c) Google could allow outside parties to curate blacklists. d) Google could list the most commonly black-listed sites publicly. For the webmasters that find themselves listed who want to run an actual honest business, this is a good sign they should change their tactics. For the folks that aim to spam and profit... well screw those guys.
stretchwithme 16y ago
great idea. Let this be the first question asked at any Google event. In fact, let there be a sea of hands all gesticulating wildly to present it.
ajayjapan 16y ago
My question is why stackoverflow hasn't banned efreedom yet?
- kqueue 16y ago
  SO provides their database content for free under creative commons license. efreedom is not doing anything illegal.
serveboy 16y ago
I use a Chrome extension called Google Search Filter which solves this exact problem - https://chrome.google.com/extensions/detail/eidhkmnbiahhgbgpjpiimdogfidfikgf https://chrome.google.com/extensions/detail/eidhkmnbiahhgbgp... It lets me sync my config accross multiple machines. Has nice hacker-ish config. Basically a text file you can share with others. This is my current config: # Make these domains stand out in results +en.wikipedia.org +stackoverflow.com +github.com +api.rubyonrails.org +apple.com +ruby-doc.org +codex.wordpress.org +imdb.com +alternativeto.net # SPAM - never show these results experts-exchange.com ezinearticles
michaelhart 16y ago
Google Domain Blocker: (userscript/greasemonkey), for those interested. http://userscripts.org/scripts/show/33156 http://userscripts.org/scripts/show/33156 You can also sync them for Firefox across multiple machines using Dropbox, as the preferences are stored in your profile (IIRC, in a javascript file).
RP_Joe 16y ago
So what we are talking about is censorship. You are suggesting a non-traditional type where a government does not do the censoring, but a few people do. How many votes would it take to put a website on a blacklist? 50, 100? Who decides if a site is spam? So is free speech dead under your proposal? What is I build a site that criticizes the Governor of your state. Or a federal agency. What would prevent my site from being blacklisted in your proposal? Even if I had great content (your argument is about poor quality content) my could be voted into a black hole in a few hours. Lets think about this carefully. Is that the price we are willing to pay to get rid of EE?
- nervechannel 16y ago
  Did I say anything in the OP about my blacklist affecting other users? Please read before ranting.
alnayyir 16y ago
https://chrome.google.com/extensions/detail/ddgjlkmkllmpdhegaliddgplookikmjf https://chrome.google.com/extensions/detail/ddgjlkmkllmpdheg... Is there something I'm missing here? It's not in Google's financial interest to provide this feature, but it already exists rather trivially.
- nervechannel 16y ago
  Still tied to a specific browser though. Which isn't available on all platforms.
  - alnayyir 16y ago
    Which platforms are that? I'm using Chrome on Windows, Mac and Linux. It can be run on FreeBSD if you're willing to deal with the bridge troll running the show there. http://i.imgur.com/1Amu3.png http://i.imgur.com/1Amu3.png Do you really think this feature doesn't exist for Firefox? Further, it'll even sync with your google account making it global if you give it access.
    - nervechannel 16y ago
      Well iOS for one. Also much as I like Chrome, I can't run it on my Linux box because the font rendering is terrible and it lets sites' font choices supersede the user's. That means I can't overrule their painful font choices with ones that look good, like I can in Firefox. But that's another rant...
      - alnayyir 16y ago
        >iOS for one whistles and pops finger > it lets sites' font choices supersede the user's Googled, found it 5 seconds. Sure there are at least 3 other ways to do it, one of them while rubbing your tummy and patting your head. http://www.google.com/support/forum/p/Chrome/thread?tid=21218cb9950ed044&hl=en http://www.google.com/support/forum/p/Chrome/thread?tid=2121...
GrandMasterBirt 16y ago
Use duckduckgo.com. Its pretty good with excluding spam. And with a new service there is an indicator of how spammy a site is.
AussieChris 16y ago
blekko . com is doing this and much more
scotty79 16y ago
In the old days we had killfile. Why can't we PLONK content sources like authors or sites by handles like nicks or domain names? There should be some standard protocol for that. Httplonk.
thinkbohemian 16y ago
Does anyone remember when google had this feature? Well sortof, you could block individual responses from coming up under a specific search term. There was a little x by each result if you were signed into google and it said "never show this result again" Not enough people used the feature for it to stick around... I would love this ability but google please, good UI and consumer education. I love your features but don't love when they get taken away because users don't know they exist.
pilooch 16y ago
you can do it with seeks... http://www.seeks-project.info/ http://www.seeks-project.info/ http://www.seeks.fr/ http://www.seeks.fr/ on your local machine and/or remote server... and it's free software. blekko ? try this query, http://blekko.com/ws/?q=debian http://blekko.com/ws/?q=debian duh ?
cygwin98 16y ago
Sounds to me the web search is not yet a solved problem. As the hardware (storage and memory) is getting cheaper and cheaper, and the emerging enabling technologies such as cloud computing, building your own search engine may not sound impossible any longer. Wonder how feasible it is to apply anti-spam algorithms that work well on emails to web pages.
byron8 16y ago
me gusta esta idea, muchos de los resultados iniciales son spam, y los resultados que de verdad me sirven aparecen dos o tres paginas después, apreciaría mucho que se pudiera banear los resultados alejados o que considere spam, thxs

438 points by nervechannel 16y ago

iwwr 16y ago

In the interim, you can do your searches by adding -wareseeker -efreedom to the search string.

nervechannel 16y ago
I've discovered you can also set up a Custom Search Engine, with no included sites (default to everything), and specifically exclude the sites you don't want. Then do all your searches through this. http://www.google.com/cse/ http://www.google.com/cse/ Usability-wise, though, it's not nearly as much use as a 'ban' button next to each result would be. But it shows Google already have the infrastructure and code that would allow this -- they just need to make it instant to use. EDIT: The other downside of this is you lose a load of bells & whistles, e.g. previews, "pages from the UK" (without typing), icons for images/news/etc. Time will tell if I miss those.
- srean 16y ago
  But its not free, right ?
  - duck 16y ago
    It is free.
  - JoachimSchipper 16y ago
    From the signup page: "Standard edition - ads are required on search pages" There's an ad-free premium version as well, but you definitely can get it for free.
  - wccrawford 16y ago
    It's free if you show ads on the pages. It costs money if you don't.
  - nervechannel 16y ago
    Basic ad-supported version is free.
- tszyn 16y ago
  Yes, it works and it's completely free. Here's an example Custom Search Engine I set up: http://www.antimoon.com/ce/ http://www.antimoon.com/ce/ It includes only sites which are known to use (mostly) good English. It's designed for English learners/teachers who want to find correct usage examples without risking exposure to Yahoo-Answers-style English.
- jmckib 16y ago
  I tried this, and if you don't enter any websites to include, you get the message: You must enter 1 or more websites.

foljs 16y ago

And no more bloody experts-exchange...

shrikant 16y ago
I honestly don't mind that site - about half the time I search for an issue, I find it's been resolved by someone over at Experts Exchange. Do people realise that if Google is your referrer, you can scroll all the way down and see the solutions to the question?
- duck 16y ago
  I have always known that, but I don't think a lot of people do. A while back I was working with a high dollar consultant who really thought he knew it all, but he seemed really surprised that I didn't need to use his Experts Exchange account to see the answer.
  - nervechannel 16y ago
    It didn't used to be like that -- you used to have to 'view source' to see it <facepalm> Then I think they went through a stage of not revealing it at all. But in any case, it's far less useful than SO/SF etc.
    - hessenwolf 16y ago
      Depends on what you are doing. I sold out to greysuitland and ee is better for all my retarded vb, vba, c# questions.
- jawns 16y ago
  It seems like every few months, EE tries to cloak its site in a new way ... then Google catches them and they revert to merely misleading people (by putting the answers several page-scrolls down) into thinking that they need to pay to see the answers. Actually, I sort of sympathize with the predicament EE faces. They want to show up high on Google search results, because that's how they get new customers ... but they don't want to give away their content for free. Here's an opportunity for a search engine start-up: Allow users to search (by default) for only free content -- but also allow them to search, if they so choose, for content that's behind a paywall. Pay-for-access sites would love something like this.

alexobenauer 16y ago

Although it's sad because it speaks volumes that we're fed up with all the garbage in many of our search queries. I do hope those working on the algorithm are taking note.

aquilax 16y ago

Wasn't this a problem Google Search Wiki tried to solve? http://googleblog.blogspot.com/2008/11/searchwiki-make-search-your-own.html http://googleblog.blogspot.com/2008/11/searchwiki-make-searc...

joeyo 16y ago
Yeah. I miss searchwiki. :-(

al_james 16y ago

Yes that would be good. They could then look at the number of people blocking certain domains and de-weight them in the global results. Traditionally google seem against human powered editing (as this would be), but I think as the black hat SEOs run rings around them, its needed badly.

thushan 16y ago
Anyone have any ideas why the clone sites like efreedom are ranking above Stack Overflow when SO's inbound links and reputation values are likely far better than efreedom's in Google's algorithm? I'm surprised that search engine optimization could do THAT much to a site's ranking. Also it's not like SO isn't doing the same kind of SEO themselves. What I'm trying to get at is, with all things equal, let's say Stack Overflow and efreedom's SEO is on par with each other, shouldn't SO's reputation/inbound link ranks automatically trump things?
- hessenwolf 16y ago
  Efreedom is deliberately manufacturing links, whereas SO may be just hoping for the best.
  - thushan 16y ago
    Shouldn't the sites hosting deliberately manufacturing links also fall to the algorithm's reputation/inbound link values as well. We all know they don't, and that seems like that is one of the fundamental issues here. As much as I'd like to have a site black list, that doesn't really help the major issue at hand.
  - jqueryin 16y ago
    Efreedom has been the bane of my recent programming related searches, polluting my results to no end.
    - mattmanser 16y ago
      Do what I've taken to doing nowdays, for programming questions go straight to stackoverflow.com before Google. Or use google with site:stackoverflow.com

eliben 16y ago

Can't this be done with a browser plugin?

nervechannel 16y ago
Like I said in the OP: "There are gresemonkey etc. scripts to do this, but they're tied to a single browser on a single machine. A global filter (like in gmail) would be so much more useful."

djhworld 16y ago

I think the worst culprits are the ones that skim StackOverflow questions and rehash them into their own supposed original "question and answer" site

deleted 16y ago
[deleted]
bartl 16y ago
Jeff Atwood discusses this in his most recent Coding Horror blog post (http://www.codinghorror.com/blog/2011/01/trouble-in-the-house-of-google.html http://www.codinghorror.com/blog/2011/01/trouble-in-the-hous...), in which he states that he doesn't really mind people copying StackOverflow questions (and answers), but that he does mind that the copies get a higher Google ranking than the original.
- retube 16y ago
  How does that happen though? efreedom must be doing something special with SEO to get higher than SO given that SO must have huge page rank score.
  - iuytgyuikoiuy 16y ago
    Pagerank is based on number/quality of links. Nobody links to an SO question, except perhaps mentioning it in a blog post. Scrapping sites create lots of links to the question, even if they are low quality ones
    - retube 16y ago
      Yeah but does anyone link to efreedom question pages? And SO as a whole must have way more inbounds than efreedom, surely?
ergo98 16y ago
What do you think most StackOverflow answers are? It's a karma-paid labor pool where you can post questions and a lot of under-employed people will rush out and do the necessary Google searches, collating and slightly rewriting the results to yield the most votes. Everyone is ripping off someone's content. And just to be accurate here, SO content is creative commons (created by the community). Are those just cheap words?

pixelbeat 16y ago

Google were experimenting with voting on results: http://techcrunch.com/2007/11/28/straight-out-of-left-field-google-experimenting-with-digg-style-voting-on-search-results/ http://techcrunch.com/2007/11/28/straight-out-of-left-field-... Also there is this form for reporting spam sites: https://www.google.com/webmasters/tools/spamreport https://www.google.com/webmasters/tools/spamreport Integrating the above into standard search results would be difficult unless it was restricted to users with a good "karma". That might be possible in our increasingly socially networked world

cosmicray 16y ago
You could also restrict it to people who buy, and sell, on Google Checkout. Putting real money/goods on the table tends to weed out the fake accounts.
CWuestefeld 16y ago
The thing is, the SO scrapers like efreedom aren't spam, strictly speaking. It's just that they clone existing content without adding value, and as such are just noise in the results. Perhaps we need to frame the discussion differently, considering what the searcher wants, rather than "spam-free hits".
- nervechannel 16y ago
  That was my point really. I don't want to see eFreedom hits, I consider them spammy, so I'd like to be able to click-ban them from my results. If Google use that information to gradually adjust their ranking overall, then fair enough -- won't affect me, I can't see them anyway. EDIT: Even if they don't let that affect everyone else's results (because of gaming), then I still don't care, I still don't see the crap in my results ever again.

krschultz 16y ago

I'd ban eHow.

slig 16y ago
And mahalo.
- bradly 16y ago
  And ExpertsExchage and About.com
  - travisp 16y ago
    ExpertsExchange is at least original content (as far as I know)
    - jasonkester 16y ago
      ExpertsExchange used to be nearly 100% scraped usenet. Then eventually a few poor souls stumbled in thinking it was an actual site and started posting questions there directly. Eventually people even started answering them. Had they not had such a terrible site design and revenue scheme, they might have actually made the transition from scraping bottom feeder to respectable site.
      - moxiemk1 16y ago
        I never knew that. I always assumed that someone out there was paying for all these questions. Good to know.
  - grok2 16y ago
    I notice that nowadays, I don't really care where the answers come from so long as they they answer my question. While I tend to look for and click on StackOverflow links and skip efreedom links (just because I want to be loyal to them), I find that I still use eHow and About.com and ExpertsExchange results since I find that the answers there most times help me with what I am seeking -- for me it doesn't matter really that it is the original source of the info or not. I think these sites are better than the true spammy sites that really have no content on the answer to your question, but somehow manage to get up there in the search results.

hessenwolf 16y ago

How many gmail accounts do we need to band together to lower the rank of stack overflow against our super-duper question-and-answer site QandAdsWithMe.annoying.com?

Sandman 16y ago
An excellent point. Blacklisting works both ways, and there's nothing stopping spammers from creating hundreds, maybe thousands, or even more, of throwaway Google accounts just to blacklist the original site. Sure, the logged in Google users wouldn't see the spam site (if they blacklisted it), but it would still appear, and outrank the original site, in standard search results for those that aren't logged in.
- nervechannel 16y ago
  So don't use the blacklisting stats for re-ranking everyone else's then. I didn't consider that when I put the post up. An instant way to filter sites just for yourself would solve 90% of people's complaints straight away.

forkrulassail 16y ago

YES. Like the useless chromeextensions.org This would be an awesome feature.

dawgr 16y ago

That will never happen, if they ever did that it would be an admission that there is something inherently wrong with their algorithm. They won't do it.

shimonamit 16y ago
On the contrary. As suggested by another commenter, it could be used to improve their algorithm. Greater number of exclusions would reduce their relevance.
nervechannel 16y ago
Nah. They let people 'report spam' in gmail -- they're happy to admit that algorithm doesn't always guess right. Also there's already a "give us feedback" mechanism for reporting bad search results, it's just too slow and manual.
- wahnfrieden 16y ago
  People are used to spam buttons, and would realize that the lack of "report spam" would mean that Google might be marking too many things as spam if they think the user doesn't need a "Mark as spam" button. People are also a lot less tolerable of spam in their inbox than they are of irrelevant search results.

Pewpewarrows 16y ago

Gmail already does it, and the global system uses an algorithm to look at reported spam results in order to automatically move future emails from that party to the spam folder automatically, not just for the person that reported it, but for everyone. If they're not looking into integrating that nicely into the existing search results page (not a separate form that the average user will never find or use), especially after all the internet chatter about it recently, then they definitely should make that a top priority in 2011. I definitely don't want them to do a rush job on it though. I don't want competitors to start reporting each other as spam in search results to try and game the system even further. I'm assuming they have anti-gaming measures in place for Gmail, so they won't be completely starting that from scratch...

mtkd 16y ago
I don't see how you could anti-game this, the SEOs would just use mechanical turk to hire 100s of people (with valid Google Accounts) to do the reporting. At best G could use the information as a list of potential spammers and filter domains manually, but I really can't see this being automated without giving the SEOs another weapon.
- moe 16y ago
  I don't think anyone wants to filter sites that could be gamed with "100s of votes". We want to filter sites that will require tens of thousands of votes to get rid of.
  - roc 16y ago
    And even then it should consider the votes themselves to be suspect and watch for blocks of users who only vote in unison (qualitatively and temporally).

Luc 16y ago

Also, I would like '[any widget] review' to take me to an actual review, not pages upon pages of spam. I usually end up looking at comments on a few trusted sites (e.g. Amazon). This seems broken...

CWuestefeld 16y ago
Yes, most of the results for this query wind up pointing to pages saying "be the first to review [any widget]". As a workaround, try searching for "[any widget] sucks" and "[any widget] good". EDIT: tying this to other discussions on the topic, it's a symptom of Patio11's observation that natural language search doesn't work very well. If you want to find something, you need to paint a picture of what it looks like, rather than asking a question about it.
- tokenadult 16y ago
  I've noticed for years that "[product] hosed" brings up good results on how to work around bugs in various software products.
- tesseract 16y ago
  automated version of this: http://www.sucks-rocks.com/ http://www.sucks-rocks.com/

deleted 16y ago

[deleted]

retube 16y ago
I believe blekko does

shimonamit 16y ago

Maybe this could be implemented in the way of sticky search operators? So for example, I could define -site:efreedom.com as an operator to be applied silently for every search I make.

ScottWhigham 16y ago

For those wanting Google to put a penalty on the sites who are banned/removed from the user's view, what's to stop someone from gaming that system via Mech. Turk (or some other way)? Just pay people $0.12 to open gmail accounts and ban a competitor or whatever. That's the only negative I can think of - other than that, I say bring it!

davidk0101 16y ago

I'm not sure how this would be implemented. Where would the blacklist be held and how would it influence the search results? I know that they already do a lot of search customization but most of it is just aggregate statistical computations. It's not that they return results specifically tailored to you but more like results tailored to a very fuzzy average version of you. A blacklist seems way too specific to each user to be susceptible to meaningful aggregate statistical operations like spam filtering which is one of the reasons that spam filtering in google is so good. Each user contributes something and everyone benefits. I don't see that happening with blacklists. I think to make it worthwhile they would need to figure out how to feed the information from blacklists into providing more meaningful results for everyone.

nervechannel 16y ago
That's the point, what I want is exactly a user-specific blacklist. I can even do that already with Google's Custom Search, all that's missing is a little 'block this site' button. Instead I have to go and configure Custom Search manually for each URL mask.
- jimmyswimmy 16y ago
  You could write a little GreaseMonkey script or extension to do that, shouldn't be too hard.
gwern 16y ago
> how would it influence the search results? Presumably it would do the same exact thing as '-site:foobar.com'.

hoofish 16y ago

the problem I have with this is that some black hat people can do this to any site they feel they are competing with. what would prevent someone from blacklisting a legitimate blog or website just because they did not like the content?

nervechannel 16y ago
So what? All that would mean is that the site wouldn't show up in their results. I never said anything about it affecting other people's results...

pragmatic 16y ago

Proof that true AI is a long way off? If the best and brightest (arguably) on the planet can't figure out how to filter out search with algorithms, what makes us think we can mimic true human intelligence any time soon. (I think it will happen, just not as soon as some claim)

stcredzero 16y ago
We are already able to (badly) mimic true human intelligence. The problem is when our mimics compete with humans in general contexts. There, being smarter means you can cream the competition. Basically, it's the difference between PvP and PvE.
tel 16y ago
Or maybe it just means that the day we can algorithmically emulate human intelligence is the day spam becomes useful (or the day we get impossibly good at fooling ourselves).

SimonPStevens 16y ago

No, it's not particular hard, but it will make the problem worse. Why? 99% of users are non-tech oriented. Those users will not really be aware of the specific problems with the search results, they won't understand the concept of a good vs bad result and they certainly won't bother to tweak/ban/filter their results. The 1% that do care and are currently being vocal about it will start filtering their results and they will perceive that the problem is solved. They will stop making a fuss. So now, the complaints have gone away, but 99% of users are still using the broken system, so the good sites that create good original content are still ranking below the scrapers and spam results for 99% of the users. The problem must be solved for all (or at least the majority) of users. (And you can't take the 1%s filtering and apply it to all users in some kind of social search because the spammers will just join the 1% and game the system)

nervechannel 16y ago
Do you think 99% of users are too stupid to click 'report spam' when they get a spam email?
- SimonPStevens 16y ago
  Not "too stupid" no. I think 99% of email users have not been adequately trained in why or how they should report spam, and even if they were I think most of them would still not care enough to actually do it with any regularity. When pushed many may acknowledge that they know it exists, they will probably even be able to find the button when asked if given a chance. But they won't remember to do it when they see spam, they'll just ignore it and move on to the messages from people they know.
  - seabee 16y ago
    Do you think this could be improved by alternate wording? Instead of reporting 'spam', ask the user 'was this useful?' or 'did you want this?'. After all, the real goal is giving people a better, more relevant experience, detecting and removing spam is just one facet of that. Whether it's email or search.

podperson 16y ago

simply add -site:foo.com to your search request. And no, this doesn't solve the problem.

deleted 16y ago

[deleted]

AndrewO 16y ago

I see a lot of people asking what happens when a group of people downvote a site just to ruin its ranking. Sure that's a problem, but there's an easy solution on Google's end: your blacklist only affects you. Yes, that means all of us have to hide efreedom ourselves. Doesn't seem like a problem to me... Plus, we are talking about a company whose core business demands that it can identify groups of bad-faith voters. Given time, they may find a way to incorporate this data safely into the ranking data (if anyone could, it would be Google). And I know there are extensions to do this (mine mysteriously stopped working recently), but doing this on the client-side in a way that's bound to a single browser install just seems wrong to me, especially for Google.

nervechannel 16y ago
This is exactly what I meant. Lots of people have added the idea about crowd-sourced re-ranking based on blacklisting, then said "it'll never work..."
- AndrewO 16y ago
  It seems to be a particular affliction among our group (by that I mean anyone who spends a lot of time here) that once we learn to apply one revolutionary/disruptive idea, we can't stop even when we probably should. I'd say it's because we're used to trying to think of how to scale-up every idea, but I've been accused of armchair-psychology before... :)
peterhoffmann 16y ago
The blacklist does not only have to affect me, just throw in the blacklists of my social graph too. These are the people I trust.
- coderdude 16y ago
  I don't know about you, but my social graph is pretty diverse. I wouldn't trust all those very different people to make an important decision like "what sites should be visible to me in search engines."

Sukotto 16y ago

I want a search results page similar to the "Priority Inbox" we got recently in gmail. Set sane defaults and let me override them with "Important/Notimportant" buttons (or thumbs up/down or whatever) next to results. Let it learn what I think is a good result for my needs. If you make it a little bit social, make sure you weight other people's opinions by how much they agree with my own in other areas (making it harder for sockpuppets to muddy the waters)

pilom 16y ago

Startup idea: Create a service around google custom search. Select the "Search the entire web but emphasize the selected sites" Then create a gui to allow people to prioritize or ban their search results.

Tichy 16y ago

Didn't Google have downvotes for results - shouldn't they be sufficient to achieve the result you want? Presumably Google would learn that you consistently downvote wareseeker and exclude it from results in the future. I haven't used it because I don't want Google to remember my search history. But if you are willing to stay logged into Google (which would be required for your proposal), it would not be an issue.

joshrule 16y ago

It seems that it might be more helpful to whitelist sites. The web grows too quickly, and the mass of spam sites overwhelmingly so. If I had some way to blacklist sites, I'd end up spending a lot of time doing so. In fact, it could quickly take up most of my search time. If, though, we could whitelist sites, it seems that results would get cleaner faster. I don't care about how many bad sites are out there, as long as helpful sites make it to the top. Plus, I typically use just a few sites to access reliable information anyway (the number's about 7, right?), so if I can whitelist results from those sites, I'll probably find my desired content more quickly. What about the case when there are 30 spam sites listed before 1 good site? That hasn't happened too often for me. Instead, the results I'm looking for are usually just 4 or 5 spots down the front page, and very occasionally on the second page. White listing seems like it would still be faster and easier for now.

svlla 16y ago

I'd like to see an option for searching only ad-free sites, or perhaps just sites that don't use AdSense, as well. Surely Google would have no problem with that.

jeffg1 16y ago

It doesn't seem like it would be hard, but if the rankings aren't driven by money, then there will be attempts to game the system. The problem I feel is Money. As long as everyone has to compete for it (meaning money doesn't work for the people, people work for money - in a system owned by the few), we'll have shady marketers, shady products, spammers etc... so, I think that it will remain a cat and mouse game.

twir 16y ago

Looks like a lot of people are assuming a solution would some sort of voting system like stackexchange, etc. Why not allow individual users to hide sites from their own search results and save the info in their google account? For example, provide a "hide this site from my results" link next to each result. Each person decides which site they don't want to see and SEO and global results remain unaffected.

teach 16y ago
I feel like I'm taking crazy pills. Am I the only one that remembers this EXACT feature on Google about a year ago? You had to be logged in to iGoogle, and each search result had a small [X] to the right of it that would appear on hover. If you clicked it, that result wouldn't appear for you again. I used it all the time. Then, lately it's gone. Maybe I was part of a small, randomly-selected test group?
- mthoms 16y ago
  That was part of Google's SearchWiki experiment. http://googleblog.blogspot.com/2008/11/searchwiki-make-search-your-own.html http://googleblog.blogspot.com/2008/11/searchwiki-make-searc... That experiment was replaced by Google "Stars" in March 2010 because, according to Google: > In our testing, we learned that people really liked the idea of marking a website for future reference, but they didn't like changing the order of Google's organic search results. http://googleblog.blogspot.com/2010/03/stars-make-search-more-personal.html http://googleblog.blogspot.com/2010/03/stars-make-search-mor... I personally think there is much more going on here than Google admits.

balakk 16y ago

How about decentralizing the search page? Hear me out for a bit. My theory is that these complaints are coming from specific interest groups, not the general public. For example, spammy-content is created and targeted at a developer/programmer audience, and that is the source of some of these complaints. So my suggestion is Google should platformize their search; and give out dedicated search instances to specific communities. The community should have enough levers to govern/influence what is spam or not. In addition, the community can promote certain high-value resources, which are otherwise unfairly listed in search results. Invite some high-profile communities for a test-run, and let the communities make their own choices. The public Google can still handle the general public. This can also bring in some transparency in the way spam is determined.

coffee 16y ago

"This would solve a lot of people's complaints in one fell swoop." And doing this would spawn a lot of people's complaints in one fell swoop. If you owned a site, and created enemies, they could band together and flag your site as spam.

emef 16y ago
I don't think you understood his suggestion. "Banning" a site would be local to your signed-in google account, not a global ban from results (which would indeed suffer the backfire you mentioned.)
- coffee 16y ago
  doh! You are totally correct. I completely missed that. Maybe I should drink a bit of coffee and wake up ;) Although... I have a suspicion that at some point it would effect non-logged in users. Many logged in users banning a site is a signal that may effect the global results. Maybe in the same way marking spam in Gmail...
scotty79 16y ago
Now they can bot-click your adsense ads to cause google to ban you. If you are ad sponsored this effectively puts you out of business.

diegob 16y ago

Wouldn't implementing this feature be a tacit admission that there's a problem with search results?

coffeedrinker 16y ago

As programmers, our typical complaints are for sites that bog us down in common (expert's exchange, stackoverflow scrapers, etc.). What I found interesting: I was doing a search on something I normally have no interest in (a sewing machine manual for my wife) and I was amazed by the level of spam I was encountering. We have no idea how bad the problem is for others whose topics we do not usually see. The web is far more full of spam than we even realize.

charlesju 16y ago

Here is a conspiracy theory for you guys. 1. How does Google make money? Search Ads. 2. How do people click on search ads? Bad real search results.

radley 16y ago

Google does provide this service: it's called Google Custom Search. You can prioritize or blacklist sites and it's pretty easy to add it to your browser searchbar. I don't always use it, but I'll switch to it when I encounter a spammy topic, usually dev-related searches. http://radleymarx.com/blog/better-search-results/ http://radleymarx.com/blog/better-search-results/

dejb 16y ago
I can't believe this isn't the most popular comment. Kinda makes HN look a bit like a knowledge vacuum with all these recent discussions of how to ban results when a google service that's existed for nearly 5 years can do the job. The format of the results isn't quite a nice as the normal google search though.
- deleted 16y ago
  [deleted]
- nervechannel 16y ago
  Err, I mentioned this well before the comment above: http://news.ycombinator.com/item?id=2075437 http://news.ycombinator.com/item?id=2075437 It's not so much that it's a knowledge vacuum, just that someone didn't read the whole thread before replying.
- radley 16y ago
  I'm on the "wrong" side of the Flash/HTML5 debate, so my average post value is really low. It wasn't a big deal until the HN algorithm got tweaked recently.
jimmyswimmy 16y ago
Agree with the sibling poster - this is the best comment in the thread. Do you have any interest in sharing your hard work with us? If you go to the CSE website and select 'Advanced' and then download annotations, you can export the list of sites you've excluded. Further you can make the exclusion list ("annotation list") into a feed - so it is entirely possible to implement the kind of user-generated blacklist of sites which has been discussed here.

richbradshaw 16y ago

Just use Google SearchWiki. Oh, yeah – they pulled it.

Rhapso 16y ago

It seems like a obvious answer, but why not just use "-site:annoyingpage.com" in you search? In fact "-TotallyUnRelated" has helped me narrow down searches effectively too. You are asking for a feature that only a small subset of the users will benefit from and use, it makes more sense for google just to find a way to rank sites better then it does to build a additional filter on top of the current system.

jonhendry 16y ago
Because I'd want to add a list of excluded sites for pretty much every single query I do. Would you want to type out a string of 20 or 30 excluded sites every time you search Google? Ranking clearly isn't going to be good enough, because algorithms can be worked around and gamed.

andrewljohnson 16y ago

I'd definitely make use of this feature. Some ancillary features might include: a) Google could warn you if it thinks the sites you have blacklisted seemed to have regained credibility. b) Google could suggest additional sites you may wish to blacklist, based on other user blacklists. c) Google could allow outside parties to curate blacklists. d) Google could list the most commonly black-listed sites publicly. For the webmasters that find themselves listed who want to run an actual honest business, this is a good sign they should change their tactics. For the folks that aim to spam and profit... well screw those guys.

stretchwithme 16y ago

great idea. Let this be the first question asked at any Google event. In fact, let there be a sea of hands all gesticulating wildly to present it.

ajayjapan 16y ago

My question is why stackoverflow hasn't banned efreedom yet?

kqueue 16y ago
SO provides their database content for free under creative commons license. efreedom is not doing anything illegal.

serveboy 16y ago

I use a Chrome extension called Google Search Filter which solves this exact problem - https://chrome.google.com/extensions/detail/eidhkmnbiahhgbgpjpiimdogfidfikgf https://chrome.google.com/extensions/detail/eidhkmnbiahhgbgp... It lets me sync my config accross multiple machines. Has nice hacker-ish config. Basically a text file you can share with others. This is my current config: # Make these domains stand out in results +en.wikipedia.org +stackoverflow.com +github.com +api.rubyonrails.org +apple.com +ruby-doc.org +codex.wordpress.org +imdb.com +alternativeto.net # SPAM - never show these results experts-exchange.com ezinearticles

michaelhart 16y ago

Google Domain Blocker: (userscript/greasemonkey), for those interested. http://userscripts.org/scripts/show/33156 http://userscripts.org/scripts/show/33156 You can also sync them for Firefox across multiple machines using Dropbox, as the preferences are stored in your profile (IIRC, in a javascript file).

RP_Joe 16y ago

So what we are talking about is censorship. You are suggesting a non-traditional type where a government does not do the censoring, but a few people do. How many votes would it take to put a website on a blacklist? 50, 100? Who decides if a site is spam? So is free speech dead under your proposal? What is I build a site that criticizes the Governor of your state. Or a federal agency. What would prevent my site from being blacklisted in your proposal? Even if I had great content (your argument is about poor quality content) my could be voted into a black hole in a few hours. Lets think about this carefully. Is that the price we are willing to pay to get rid of EE?

nervechannel 16y ago
Did I say anything in the OP about my blacklist affecting other users? Please read before ranting.

alnayyir 16y ago

https://chrome.google.com/extensions/detail/ddgjlkmkllmpdhegaliddgplookikmjf https://chrome.google.com/extensions/detail/ddgjlkmkllmpdheg... Is there something I'm missing here? It's not in Google's financial interest to provide this feature, but it already exists rather trivially.

nervechannel 16y ago
Still tied to a specific browser though. Which isn't available on all platforms.
- alnayyir 16y ago
  Which platforms are that? I'm using Chrome on Windows, Mac and Linux. It can be run on FreeBSD if you're willing to deal with the bridge troll running the show there. http://i.imgur.com/1Amu3.png http://i.imgur.com/1Amu3.png Do you really think this feature doesn't exist for Firefox? Further, it'll even sync with your google account making it global if you give it access.
  - nervechannel 16y ago
    Well iOS for one. Also much as I like Chrome, I can't run it on my Linux box because the font rendering is terrible and it lets sites' font choices supersede the user's. That means I can't overrule their painful font choices with ones that look good, like I can in Firefox. But that's another rant...
    - alnayyir 16y ago
      >iOS for one whistles and pops finger > it lets sites' font choices supersede the user's Googled, found it 5 seconds. Sure there are at least 3 other ways to do it, one of them while rubbing your tummy and patting your head. http://www.google.com/support/forum/p/Chrome/thread?tid=21218cb9950ed044&hl=en http://www.google.com/support/forum/p/Chrome/thread?tid=2121...

GrandMasterBirt 16y ago

Use duckduckgo.com. Its pretty good with excluding spam. And with a new service there is an indicator of how spammy a site is.

AussieChris 16y ago

blekko . com is doing this and much more

scotty79 16y ago

In the old days we had killfile. Why can't we PLONK content sources like authors or sites by handles like nicks or domain names? There should be some standard protocol for that. Httplonk.

thinkbohemian 16y ago

Does anyone remember when google had this feature? Well sortof, you could block individual responses from coming up under a specific search term. There was a little x by each result if you were signed into google and it said "never show this result again" Not enough people used the feature for it to stick around... I would love this ability but google please, good UI and consumer education. I love your features but don't love when they get taken away because users don't know they exist.

pilooch 16y ago

you can do it with seeks... http://www.seeks-project.info/ http://www.seeks-project.info/ http://www.seeks.fr/ http://www.seeks.fr/ on your local machine and/or remote server... and it's free software. blekko ? try this query, http://blekko.com/ws/?q=debian http://blekko.com/ws/?q=debian duh ?

cygwin98 16y ago

Sounds to me the web search is not yet a solved problem. As the hardware (storage and memory) is getting cheaper and cheaper, and the emerging enabling technologies such as cloud computing, building your own search engine may not sound impossible any longer. Wonder how feasible it is to apply anti-spam algorithms that work well on emails to web pages.

byron8 16y ago

me gusta esta idea, muchos de los resultados iniciales son spam, y los resultados que de verdad me sirven aparecen dos o tres paginas después, apreciaría mucho que se pudiera banear los resultados alejados o que considere spam, thxs