13 ms·

DuckDuckGo: Escape your search engine filter bubble

548 points by senko 15y ago

Luyt 15y ago
I noticed a while ago that a collegue and I got different results when we googled for our company name, but I didn't know that it's now happening in such an extent. I've added DuckDuck to my FireFox search engines.
- wisty 15y ago
  I've set it as default on Chrome. Maybe google is better - I'll get more python libraries and less news stories about sheep (or Darwin laureates) getting swallowed by pythons, but I can live with that. Besides, I'm pretty good at typing "google" if I need it.
  - epi0Bauqu 15y ago
    You can just add !g to the end (or beg) of your search and it will take you right there.
    - colinprince 15y ago
      Actually, this is super handy! When I know I want the wikipedia article I just add !w at the end. Ditto for youtube (!yt) etc etc
      - wattsbaat 15y ago
        fyi, you can also use !v to search youtube via ddg ('v' for video)
- yanw 15y ago
  It's not happening to any extent, it's just a new FUD concept that is meant to sell books an confuse people, if you seek more info you can dig deeper into any set of initial results. Personalization == relevance.
  - windsurfer 15y ago
    That's funny, when I search for "filter bubble" I can't find anyone refuting it.
- deleted 15y ago
  [deleted]
yanw 15y ago
This assumes that 'filter bubble' is something more than a nonsense term. There is little difference between personalization and the relevance of search results. How would you go about ranking then? alphabetically?! it's a matter of tuning the relevance 'dials' and it's all in early stages so a solution to this imaginary problems is more research and not to create bullshit terminology in order to sell some books.
- TheBranca18 15y ago
  Most people don't realize that Google and other companies are doing this. That's my main problem. It's not about selling books in my mind as much as it about communicating why something is in a search list for person A vs. person B. I don't want my Internet censored.
  - yanw 15y ago
    People don't realize many things about search engine from indices to ranking algorithms, they do realize however that a search engine is to return the most relevant results for them and that is were personalization/relevance fits in.
  - joesb 15y ago
    Do you see search result #1,000,000 on your first page? Congratulation, your internet is censored.
- elehack 15y ago
  There is a major difference between personalization and "pure" relevance, at least as used here. However, it is more of a question of inputs than actual terms. "Pure" relevance is solely a function of the query and the document - R(q,d). Personalization involves the user and possibly the context - R(q,d,u,x). Now, if you consider the user and their context to be part of the query, then yes, it's the same thing as relevance. So the real question: should the user's identity/history/profile and/or current context be considered a part of the relevance function or not? DDG says no, Google says yes. Thinking of it in this way makes the real question clearer. Unfortunately, it will probably make little sense to most Real Users.
ThomPete 15y ago
It is the ultimate quest for serendipity. If you filter by what you want you don't get what you didn't know you wanted. I will give DDG the benefit of the doubt and try it out for a couple of months. Starting now.
- jasonlotito 15y ago
  Long time DDG user here. I highly encourage people to try and make the switch. I often find DDG gives me the needed information without having to actually having to visit the site. I still find myself going to Google sometimes if DDG doesn't give me what I want, but it's smart enough to get me the most relevant information faster most of the time. Often enough that I don't feel the need to switch back. In making the switch, I highly advise you to spend a little bit of time learning the keyboard shortcuts, especially the !Bang feature (https://duckduckgo.com/bang.html https://duckduckgo.com/bang.html). You'll love the HN search options. =) There are so many, you won't be able to learn them all. You can also still use Google if you want, or even Bing (or other engines). Basically, the !Bang syntax would mean even if you aren't using DDG, it enables you to quickly use whatever engine you really want. I'm just a fan of DDG. Hope you find it as useful as I do. =)
- rcamera 15y ago
  I'm glad I have been using DDG for over 6 months now. For the new user, here are some advices. DDG is my default search engine. Basically, all of my searches are completed in DDG, only very few times I have to go to google to find what I am looking for. My suggestion is that you spend 5 minutes learning the bang! syntax, as it speeds up your search by alot, then set DDG as your default search engine for a couple weeks. You will see that the bang! syntax and the 0-click-info really makes the difference on search speed, and their results are pretty damn good.
- Pieces 15y ago
  Also a long time DDG user. I have it set as my default search provider in chrome, and it has sped up my browsing considerably. Shortcuts like "!w some article" or "!a some product" (takes you directly to Wikipedia or Amazon, respectively) are very handy. The only times I've had to revert to Google are for very specific forum trawling.
the-kenny 15y ago
The first example is totally borked. They first search for "climate change" (notice the ") and then search for climate change (without quotes). Of course the search engine shows different results for different queries.
- epi0Bauqu 15y ago
  I didn't notice that, but will fix ASAP. The base source of that image was from: http://www.thefilterbubble.com/what-is-the-internet-hiding-lets-find-out http://www.thefilterbubble.com/what-is-the-internet-hiding-l... Update: the quotes vs no quotes doesn't change the top results on Bing (at least for this search).
retube 15y ago
Yeah this is an important issue. Google apparently ranks results (for you personally) based on some 57 inputs, even when not logged into Google services. In short results suffer from a self-reinforcing feedback loop, forever constraining what you see. I wonder how easy it is to get "clean" or "default" results from Google? And I know it's been discussed many times , but just how easy would it be to maintain real anonymity across the web?
- senko 15y ago
  Incognito/private browsing + going through DuckDuckGo (even for Google queries, with !g prefix) works for me. (Edit to add since I can't reply: going through DDG avoids Google redirecting to country-specific variation).
  - scoot 15y ago
    Whit, what? Doesn't going through DDG with !g not just redirect to google, where your normal filter will be applied (private browsing notwithstanding)...
  - darklajid 15y ago
    I hate that country specific thing. Somewhere, I think even on HN, I was taught about google.com/ncr - which should solve that problem (being in Country A and arriving on the localized, unreadable landing page) once and for all. That said: I've DDG as my homepage and I'm a happy user.
- yanw 15y ago
  No it's a non-issue, and by the way Google probably uses much more that 57 'inputs' (signals) to determine relevance. Personalization is just a different term for relevant results and this is just FUD, the source of which is people who don't understand the technical aspects.
  - retube 15y ago
    I'm gonna humbly disagree. There are many potential ranking factors in a search result: how well the query matches the document text, how important that text is in the document, how important the document is (PR) and so on. What I don't necessarily want is further ordering by what a machine thinks is my political inclination or world view.
mathly 15y ago
We often forget that, although it's obvious search engines filter results, the information we see on social sites is also filtered. Consider users of Reddit. Now most of them would consider themselves very open minded and enlightened, yet their is active discouragement for radical ideas without due consideration as to their merits. It's just easier to downvote and look at Mario cake. Overall, I think in a way we NEED filters to remove the faff, but be careful to keep a social circle which encourages radical ideas to be bought out into the light of logic and due consideration,
mkr-hn 15y ago
I've been using DDG exclusively for a little while. Half the time the results are identical to Google. The other half is usually a collection of links from sites I would have searched on to research the topic.
- util 15y ago
  http://searchengineland.com/google-bing-is-cheating-copying-our-search-results-62914 http://searchengineland.com/google-bing-is-cheating-copying-... http://duckduckgo.com/faq.html#results http://duckduckgo.com/faq.html#results
davidmat 15y ago
In all seriousness, this is actually my main use case for the private browsing mode in chrome: to search google without the filter bubble(1) It's quite shocking to see just how much those results differ from the ones I'm usually served, actually. I know it's actually supposed to be 'awesome' to have every search tailored to _you_, but it just makes me feel uncomfortable that I'm not seeing the internet "the way it's supposed to be seen" - if that makes any sense. (1): Or at least a smaller bubble, considering it still knows my location - even though i use google.com, my os, my browser, etc...
igorgue 15y ago
I heard about this book on NPR a couple of weeks ago [1]. I really don't like this biased view of ML. In summary what he said on the show was something like: At least the news shows (and news pappers, radio) gives the same information to everyone, so instead of showing Kardashians news they do show you Bin Laden news, even though, they know the Kardashians are more profitable... this is not the case with Google Search or Netflix, Yahoo, Bing, etc (all attacked by this author). I think that ML helps more than it hurts and viewing it in a non-technical way is wrong, the author gives the impression that Google (the company and their execs) manually (via algorithms, but very manageable in his opinion) change the search results to not show things that they don't like, then it raises the question "do we really trust one company?". Its arguable that the search results in Google or Netflix are optimized for profits, but how do you make profits in the customer industry? IMHO you do that by making their happier, showing useful results, for them, not for everyone. I'm waiting for the time when I google: "what channel and time is Conan on" [2] and I get "channel 43 11pm" as the result. Of course that is very personalized, and the result will be just for me... but again I'm the one searching and I'm the one needing the results. [1] http://thedianerehmshow.org/shows/2011-05-17/eli-pariser-filter-bubble http://thedianerehmshow.org/shows/2011-05-17/eli-pariser-fil... [2] At the moment I get this 1st result: http://articles.boston.com/2010-11-10/ae/29330376_1_conan-andy-richter-anti-nbc http://articles.boston.com/2010-11-10/ae/29330376_1_conan-an... which is not what I searched for nor wanted...
- yanw 15y ago
  Exactly! it's just nonsense peddled by those who don't understand the technical aspects of search, people who don't realize that a search engine's prime function is to filter the millions of results for each query to down to the most relevant results for the users, and not the same 10 results are relevant to each and every user. And user regardless of nay personalization can dig through any initial results. It's just annoying an ignorant bullshit being disguised as a real issue.
- qjz 15y ago
  I think there's reason to complain that the reality is far from the ideal. Take Netflix, for example. I rated hundreds of movies, filled out the taste preferences to narrow down the genres I was most interested in, but the "Suggestions for You" were underwhelming, to say the least. Even worse, they simply never changed. So I went back into my taste preferences and checked that I "Often" watched every single mood and genre listed. Now I get a vastly improved range of suggestions, exposing me to some great movies, simply because Netflix is no longer hiding them from my view. I may have a unique individual taste for movies, but Netflix sure hasn't fathomed my criteria with their algorithm.
groby_b 15y ago
Of course, I might want my search results to be filtered. To take but one example: Neither FOX nor CNN are providing any valuable insights at all, so filtering them out is a good idea for me. Or, to get all pithy about it, the problem is not the filter bubble, it's the crap bubble. If there's a problem with relevance sorting at all, it's the issue that it is based on history, not intent.
markkat 15y ago
I think this is a real issue, and I am glad that DDG is addressing it. This is a more-compelling take on the tracking issue IMO. What I'd really like to see, is a search engine to allow me to do both. I'd like to have a profile (that didn't use my name), and when I wanted to, I could click a result as 'useful'. This would go into my personal algorithm. I could then toggle between filtered search, or unfiltered search, whenever I like. It might even be useful to build search filter categories. But I'd keep that a bit buried for the power users.
- andrenotgiant 15y ago
  Use Incognito mode in Chrome when you would like to search the un-"filtered" Google.
  - huckfinnaafb 15y ago
    Unless I'm mistaken, Incognito mode simply disables local history recording, and has not much (if at all) to do with search results or the ability for Google to track your movements. I suggest Chromium for the paranoid!
    - rryan 15y ago
      Incognito mode isn't just disabled history recording. It does not use any of your cookies from main session, and it deletes the cookies when it is closed. It also isolates your extensions from running incognito unless you elect for them to be.
  - aticris 15y ago
    Irony: Use of a Google product to circumvent Google's lack of privacy or anonymity in one of their other products.
chaselee 15y ago
I just had to explain this to my co-founder last night after he freaked out because our old site with a different name was showing up first in his search results with our new name as the query . With web history off the old site wasn't even on the first page.Now I just have to explain that no one searches for early-stage startups let alone cares if they change their names :)
loup-vaillant 15y ago
Confirmation bias is bad enough already, it's really a shame that powerful companies like Google reinforces it just for the sake of giving you more pleasing search results. (Pleasing and Good overlap, but they're not equal.) http://en.wikipedia.org/wiki/Confirmation_bias http://en.wikipedia.org/wiki/Confirmation_bias http://lesswrong.com/lw/iw/positive_bias_look_into_the_dark/ http://lesswrong.com/lw/iw/positive_bias_look_into_the_dark/ Edit: I didn't intend to bash Google specifically. But they are faced with a choice in which their own interest conflict with those of their users. And as Capitalist Bastards are more and more accepted in our society, we don't blame them for the selfish choice. Maybe not a shame then, but at the very least a pity.
- jarek 15y ago
  Yes, after all the primary job of a search engine is to fight the user's own biases, and who wants a product that's pleasing to use anyhow?
  - loup-vaillant 15y ago
    My phrasing was a bit harsh. I understand that Google (and others) are mainly out to make money. To do that they have to please users most. Anyway, they probably can't make the difference between "relevant" and "pleasant", because user's behaviour only grant them access to "pleasant". As long as this is done in a neutral way (by delivering the same result to everyone), any confirmation bias will be averaged across entire populations, so this should be okay. Personalized results however make the results noticeably more pleasant, and significantly more biased (this is probably unavoidable). Of course Google, Bing, and Co would shun that bias thing. Who can blame them? I don't want blame Google specifically. I want to point out this old, common moral dilemma: make money, or don't hurt anyone? Google took the money. Many do. I'm not sure to what extent we should blame them, but clearly, the System™ has room for improvement.
    - widget 15y ago
      I cannot see a problem here. Who exactly is being hurt by the "filter bubble"? The end user is fine - they are more likely to see results they are actually interested in. If a user doesn't trust a source and won't click on their links, they'll soon not have to bother scrolling past them. The sites themselves actually benefit as well. Sure, they may be bumped from the first page of results for users that are unlikely to visit their site, anyway, but the tradeoff is that they get a higher position for the users who may actually visit their site. It's an ideal trade for those being filtered. I suppose that leaves the idea that the end result is a "biased" internet. I don't buy it. Google is not removing sites that disagree with them, they are re-ordering them for different users. If your profile wasn't factored in, then what options do they have? They could order on popularity, but biasing towards popular opinion isn't any better than biasing towards my opinion. They could randomize the order, this would be without bias, but absolutely useless to anyone. They could judge the objective truth of sites, but that's far more biased than any of the other options.
yalogin 15y ago
DDG is quite good in regular search. I like their commitment to privacy and so switched my default search engine to DDG. But I noticed more and more that I use google maps almost as much search. I search for places and directions on maps end up going to google a lot. That is brilliance of Google. They built search verticals and so its tough to not use them. If only DDG could do something about that.
- epi0Bauqu 15y ago
  We are working on integrating OpenStreetMap, and just yesterday launched some upgrades, e.g. https://duckduckgo.com/?q=Valley+Forge%2C+PA https://duckduckgo.com/?q=Valley+Forge%2C+PA or https://duckduckgo.com/?q=black+lab+bistro https://duckduckgo.com/?q=black+lab+bistro (map in right column expands to view). This goes with address detection (https://duckduckgo.com/?q=3+ames+st%2C+boston%2C+ma https://duckduckgo.com/?q=3+ames+st%2C+boston%2C+ma) and you can always force an OSM query by adding map to the end. And of course adding !gm will take you right to Google Maps.
  - yuvipanda 15y ago
    Problem with OSM is that it's nowhere near as complete as Google Maps, and I don't see it growing the way Wikipedia did (contributing to Wikipedia is much easier than contributing to OSM)
- Habbie 15y ago
  Just add !gm to your query. If you do it a few times, DDG will remember it and make it easy to add !gm just by pushing ! to open the dropdown.
yanw 15y ago
Shame on DDG for endorsing this nonsense. I understand the quest for a larger userbase but please don't use this sort of FUD that is being peddled by those who don't understand the technical aspects of search and are trying to sell their books. A search engine's prime function is to filter the millions of results for each query down to the most relevant results for each individual user, and never the same 10 results are relevant to each and every user. There is little difference between personalization and the relevance of search results. How would you go about ranking then? alphabetically?! it's a matter of tuning the relevance 'dials' and it's all in early stages so a solution to this imaginary problem is more research and not to hide behind bullshit terminology. And as a bonus a user (regardless of any personalization) can dig through any initial set of results if she seeks more information. So please don’t buy into this misleading and ignorant bullshit that is being disguised as a real issue.
- ThomPete 15y ago
  Sorry but this is a real issue, whether DDG solves it or not. At least to me.
  - yanw 15y ago
    It's not an issue just because you want to believe it is. Media outlets always filtered stuff for you but now the algorithms are 'filtering' results according to your interaction to find what's relevant to you, so you have a 'vote' here. You also can dig through any initial results (click the news vertical to find more news for example) you couldn't do that with the media of yore.
    - ThomPete 15y ago
      It's not not an issue just because it doesn't apply to you. If you want to understand where I am coming from then this is why I think it's an issue http://000fff.org/slaves-of-the-feed-this-is-not-the-realtime-weve-been-looking-for/ http://000fff.org/slaves-of-the-feed-this-is-not-the-realtim...
  - jannes 15y ago
deleted 15y ago
[deleted]
cfontes 15y ago
I would like to use it as default, but I tryed it and it's very very slow... takes me 5 secs to get a search back. in Google I take 0.035s So I will stay information sided for a while
tathagatadg 15y ago
From the search engine revenue perspective, how much can be the loss in case they provide a preference option to show "generic results" (like "safe search option off") - with an easy to switch ui between the two?
- Matt_Cutts 15y ago
  Normally we don't like to crowd up our preferences page with a bunch of options that people don't use much, but you can turn off personalization by adding "&pws=0" to the end of the Google search url.
taylorbuley 15y ago
Does hitting the logout button pop the filter bubble?
hollerith 15y ago
I heard you can add "&pws=0" to a search query to turn off personalization: http://google.com/search?q=paul+graham&pws=0 http://google.com/search?q=paul+graham&pws=0
- citricsquid 15y ago
  Checking through a few different search terms, that works for me (provides different results)
- odiroot 15y ago
  It's funny how "Paul Graham" is Hacker News' foo bar. Also, works well for me.
Kylekramer 15y ago
It is well known that people seek out information that reinforces their own beliefs. And while it may not be ideal for an enlightened population, why should Google/Bing fight this natural inclination? Their job is to ultimately give their users what they are searching for, not challenge their users' opinions. Especially since there are tons of benefits to personalized search, and it may be hard to not throw the baby out with the bathwater.
- epi0Bauqu 15y ago
  There are other options, i.e. opt-in vs opt-out, separate sections (only show in sidebar), instant on/off, and overall greater transparency.
  - Uchikoma 15y ago
    Surfing in anonymous mode with Chrome should work as an opt-out, or shouldn't it?
    - idonthack 15y ago
      only as long as you don't log in to google or anything else before you search.
  - speckledjim 15y ago
    There's obvious opt-out things for the miniscule amount of people who care. Exactly the same as the "search term leakage" rubbish.
- noahlt 15y ago
  This mentality ("give people what they want") results in the empty, sensationalist journalism that most people dislike. When I read the news, I don't want to be entertained, I want to be informed. The same goes for Internet searches.
widget 15y ago
Any filter at all will result in a "Filter Bubble" as defined here (except, I suppose, returning a randomly sorted list of all sites on the internet). Whether the filter is personalized or not doesn't change that fact. What is the actual benefit of everyone getting the same search results when looking for a particular term? Search engines are not news sites or research papers: biasing towards relevant results is not a bad thing.
Uchikoma 15y ago
People that don't want "filtering", also want their search results in German, Chinese or Japanese? Or is "filtering" on language ok?
- qjz 15y ago
  I think it's fair to assume that people want results in the same language as the search terms they entered.
  - Uchikoma 15y ago
    Like Computer, oh well, no Computer in German is Computer. Try Biergarten, oh well.
elb0w 15y ago
I noticed this a long time ago. Suddenly I could ask general programming questions and it popped up results in the language I use primarily as the top results (Without me specifying it). While some may think that this is hiding, do we not often create algorithms to asses and aide? It seems like disabling this would be against the advancement of a learning system. I for one think its a great feature and have no desire to disable it.
alexandros 15y ago
What about the filter bubble of me only searching for what i am interested in? Will DDG throw in some random results in every search to save me from myself?
- adamdecaf 15y ago
  DDG won't build a profile about you; it only returns the most relevant links to the keywords entered. The original Google search, only on today's web.
  - khafra 15y ago
    That's not what he asked. He asked (facetiously) about the higher-order search bubble: ie, if I only ever search for articles about Erlang innards, I'm never going to read about The Kardashians, which is relevant to a sizeable portion of contemporary American culture. To truly pop my bubble, DDG ought to throw in completely random results every now and then.
mquander 15y ago
The purpose of search engines is filtering content to give you what you want, which might be information, or it might be discussion, or polemics, or gossip. If you want to learn about some topic that has some element of subjectivity, familiarize yourself with the different points of view and read the writing of whoever you are interested in. There are lots of tools on the Internet to facilitate this -- Wikipedia, for example, is built around an ideal of giving people an objective survey of different things. If you type "climate change" or "Barack Obama" into Google and form an opinion based on the top results then fuck you.
- c4urself 15y ago
  I agree with you shouldn't form an opinion based on Google's top results: But I believe that's _exactly_ what most people do.
- toddh 15y ago
  How to do I know what filters form the top results if they aren't transparent? What would lead you to believe that familiarizing yourself with a different point of view would take you down a meaningfully different path through the search graph? None that I can see. Every search has a non-objective filter. The original page rank is one such. What would be useful is to make the tree obvious and manipulable as a separate object itself.
  - mquander 15y ago
    I agree that it would be cool to expose the different criteria that Google (for instance) is using to help reorder your results, to the degree that those criteria can be discretely identified, but I'm not surprised that they don't; that's a pretty significant portion of their secret sauce they would be publicizing. I also don't think that mere transparency would really help solve any search engine "filter bubble" problem, if such a thing is real. Nothing would make Joe Google User take five minutes off whatever he came to search for to fiddle with some tree full of sliders on his result page.
    - toddh 15y ago
Hisoka 15y ago
I suspect DDG is having trouble making money hence the need to make a commotion about filter bubbles and generating some linkbait. No surprise... < 1% of a search engine market is not gonna even make you much at the end of the day.
btipling 15y ago
Except duckduckgo can't find my project and Google can: http://duckduckgo.com/?q=sphela+github http://duckduckgo.com/?q=sphela+github https://encrypted.google.com/search?q=sphela https://encrypted.google.com/search?q=sphela
- ComputerGuru 15y ago
  Doesn't that say more about your project than DDG?
  - btipling 15y ago
    Right because you'd only ever want to find things that were notable.
- mkr-hn 15y ago
  "Try: !github sphela (Searches GitHub.com using our !bang syntax" !gh works too.
- adamdecaf 15y ago
  http://duckduckgo.com/?q=!github+sphela http://duckduckgo.com/?q=!github+sphela
- epi0Bauqu 15y ago
  Thx -- we do have our own github crawl, but it needs to be updated more frequently. Duly noted and will fix.
c4urself 15y ago
In the end (1) technical people will find a way around in when they need to, (2) The conspiracy-theory loving will use the "Filter bubble"as an argument for their own ends, and (3) The rest-most people will just not care... I think the more interesting question is whether this third group will veer off to become more biased to their own world-view as the article suggests.
Matt_Cutts 15y ago
If I had to tackle the notion of over-personalization in ~5 minutes, I'd say: - If someone prefers to search Google without personalization, add "&pws=0" (the "pws" stands for "personalized web search") to the end of the Google search url to turn it off, or use the incognito version of Chrome. Personalization tends to be a nice relevance improvement overall, but it doesn't trigger that much--when it launched, the impact was on the order of one search result above the fold for one in five search results. - personalization has much less impact than localization, which takes things like your IP address into account when determining the best search results. You can change localization by going to country-specific versions of Google (e.g. search for [bank] on google.co.uk vs. google.co.nz), or on google.com you can click "change location" on the left sidebar to enter a different city or zip code in the U.S. - We do have algorithms in place designed specifically to promote variety in the results page. For example, you can imagine limiting the number of results returned from one single site to allow other results to show up instead. That helps with the diversity of the search results. When trying to find the best search results, we look at relevance, diversity, personalization, localization, as well as serendipity and try to find the best balance we can. I saw Eli Pariser's talk at TED and was skeptical, although I did enjoy his example of Facebook starting to return only his liberal friends because he only ever clicked on the links his liberal friends shared. I had a number of concerns browsing through Pariser's book, but I would encourage anyone interested in these issues to pick up a copy; it's a thoughtful read.
- pasbesoin 15y ago
  Do you have a pointer to a publicly visible reference to the definition of some of the other query string parameters Google uses? Just thought I'd ask, given the context. I've noted various references people have cobbled together, but perhaps there's something a bit more... "canonical". Thanks for the above.
  - Matt_Cutts 15y ago
    http://code.google.com/apis/searchappliance/documentation/610/xml_reference.html http://code.google.com/apis/searchappliance/documentation/61... is a reference for our search appliance, but a lot of parameters are in common. I'm not aware of any other official Google doc available externally, but if you search for [google search url parameters] the top 3-4 results are all quite good.
derwiki 15y ago
Sorry to be pedantic, "beg the question" is being used incorrectly. See http://begthequestion.info/ http://begthequestion.info/ ("This begs the question: what are you missing?"). Doesn't detract from the overall post.
yhlasx 15y ago
Personally, not convinced at all. Looks like saying "We are better because we cannot do it yet"
balloot 15y ago
This entire argument is simply without merit. Google is, and always has been, a filter. If I type in "Barack Obama", it will not show me links about The Green Bay Packers NFL team. This is not because Google is conspiring to keep me from reading about the Green Bay Packers. It is because it is most likely that I am not looking for Packers links and will not click. The general idea is that Google starts with every single piece of content on the internet and filters to get the content it thinks I am looking for. Now, this article complains that Google is flawed because it will more likely show some MSNBC over Fox News, or vice versa. The implication here is that you never click on Fox News when it is presented. Because if you did click on Fox News and its ilk from time to time, Google wouldn't start filtering it in the first place. The problem isn't that the search engine creates the "bubble", it is that the user does! So if the scenario presented in this article offends you, then start changing your behavior and browsing more diverse sites. Otherwise, don't blame Google for noting that you hang out in a very narrow corner of the internet, and presenting you links from that corner. It's just doing its job correctly in that case.
tomlin 15y ago
It almost seems like DDG is trying to get me to think personalization == censorship. Sorry, but that ain't gonna fly. You've just undermined your own search engine's potential to use such a feature in the future -- people will be spitting quotes back at DDG about how opposed on this feature they were. Know your audience.
mitcheme 15y ago
I have Google Web History turned off and I'm seeing a pretty good mix of things in my results. E.g., I'm heavily left-wing, somewhat pro-gun control, with no personal interest in shooting, but my results for 'gun' are pretty much all gun shops and fansites. For Barack Obama I get mostly official sites, although some of the top news is critical (I'm Canadian, if it matters). Egypt gives me a mix of travel, protests, general info and ancient history. I don't recall ever turning it off, so is it off by default? And if so, what's the problem here?
jlft 15y ago
I don't like the direction Google is taking with Search. My view of "improvement" is very different from Google's view. I have the feeling that the more features they add to Search the worst the service gets. I miss the old simple Google Search. I think there is a big oportunity here for another search engine focused on: simplicity, speed, relevance (algorythm) and unfiltered. Back to basics.
ZoFreX 15y ago
You're missing a trick by that near-final slide of the DDG input just being an image - would be neat if it were a fully functional DDG search form.
ryanisinallofus 15y ago
I spent a few hours using Duck Duck Go before commenting. It's been my Chrome default search for 4 hours. • The results are less relevant. Bubble or no, it was harder to find things I look for all the time. For example I had to constantly add the word Seattle to my search terms. • It felt like an old search engine. The results, the display, the choice of Mapquest maps all made me feel like I was using Yahoo or even early Google. • At first the feeling of not being watched was liberating but I forgot about it very quickly. I'm going to continue using it as my default search for the lack of tracking but Iv'e already had to go to Google a few times to find what I was looking for. Convenience over privacy right?
- rnicholson 15y ago
  Interesting. I've been using ddg for several months now, and haven't quite experienced my search results as being "less relevant" or "harder to find things" in comparison to Google or Yahoo!. Ironically, I do use the ddg bang (!) operators quite often. ...like !gm or !gi if I know I want a map or image (for example)
- iamelgringo 15y ago
  Blekko has been my default search engine for the last months, and I really really like it.
- epi0Bauqu 15y ago
  Thx -- in my experience it really takes a week to get into it. If you can stick it out I'd love to get your additional feedback after that point. Point of fact: it isn't MapQuest, it is OpenStreetMap served via OpenMapquest, which uses their resources to forward that project. You can read more about it at http://openstreetmap.com/ http://openstreetmap.com/ (left column) and http://open.mapquestapi.com/ http://open.mapquestapi.com/ - in any case, maps are relatively new and in process.
  - ryanisinallofus 15y ago
    Thanks for clarifying. Open street map is a cool project.
pilooch 15y ago
There's a personalization / privacy trade-off that needs to be considered. It is annoying that personalization cannot be achieved locally on the user machines or browser. Filtering / re-ranking results at home offers much privacy. My little personal project is to have the ML for personalization done at home, seeks the Seeks Project, http://www.seeks-project.info/ http://www.seeks-project.info/ Personal control over the personal bubble matters...
malaka 15y ago
"But aren't you a part of the relevance equation? The ideal results for a search like [bitcoin crash] should be different for a Japanese-speaking searcher in Tokyo vs. a German-speaking searcher in Munich vs. a bitcoin expert vs. a programmer trying to diagnose why compiling bitcoin is crashing vs. my Mom who has never heard of bitcoin before, right?" Maybe, but that is not the point. Besides, you are mixing localization (japanese vs german) with personalization (expert vs naive), and what is worst is that you are assuming that Google knows so well each user so as to be right (and that is either impossible, either extremely creepy), at every single instant of his life (a person can change interests). Furthermore, to grab your example, how can Google know that an expert in bitcoin and expert in bitcoin compilation and crash solver, is not just interested in hearing about the "market" crash of bitcoin? Google CANNOT read the users' mind. And even if it did, there would not be a need for "personalized search", as the "mind reading" would give enough search criteria to nail the results more easily (albeit, most people do not know exactly what they want, so it will still be an iterative process, which is good, as randomness is the seed for evolution). So, back to the point, it is that in the quest for "adequate results" for each person, Google is turning web search into a non-deterministic event (). Imagine the web being a library, and the search being searching for the library's book database, why would the search for a given book return different results to different persons? It should always return the same results, if the person doing the search is not satisfied with the results, then she/he will add more criteria. In other words, let the person do the filtering! Once that Google accepts that in his quest for "better results" (where 'better' is a concept decided by solely Google and whose ranking parameters and algorithm are unknown) there is a potential (probably demonstrable already) for a "filter bubble" with positive feedback loop on user behaviour, which, as with any positive feedback loop, can go out of control, exacerbating certain ideologies and fueling extremisms. And there is a fundamental difference between a "self guided" (as in self controlled) filtering, where users would knowingly filter out results in order to find those that they like, and a "google guided" (as in externally controlled), filtering. () strictly speaking, search will not be deterministic as the web is a dynamic system and it grows, so search results can vary with time, but they should not vary from person to person at a given time.
- widget 15y ago
  Of course Google can't read minds, that does not mean they should ignore information that they have when deciding which results to show or what order to show them in. Sufficient data to provide a better filter for a given user is not the same as mind reading. You are absolutely right that more criteria should be used if a user is interested in better results, but I fail to see why a deterministic base case is superior. If I never click on news links and always click on travel links, it makes perfect sense for Google to assume that my search for "Egypt" is looking for information related to travel to Egypt. If I am not following my usual search patterns, I can look on the second page, or I can disable personalized search, or I can search for "Egypt News" all of which would give me better results. Why is it preferable to always make everyone clarify their searches when there is sufficient information to narrow the search down somewhat without requiring additional intervention by the user? This is usability 101 right here.
jerrya 15y ago
How does duckduckgo.com compare with startpage.com?
KristianSerrano 15y ago
Google should invite Pariser for an Authors@Google event with Cutts conducting the interview.
m_feuz 15y ago
in our research study, we found the impact/presence of personalization very strong: after about 3'000 search queries, in some cases more than every search query received personalised search results. out of the 10 blue links in some cases we found 6.4 personalised: see for yourself (search for Hypothesis 1 to get to the data) http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/3344/2766 http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/ar...
aghilmort 15y ago
Explicitly named "filter bubble" search - http://trap.it/ http://trap.it/ xref: http://www.readwriteweb.com/archives/darpa-born_trapit_wants_to_be_your_personalized_ne.php http://www.readwriteweb.com/archives/darpa-born_trapit_wants...

548 points by senko 15y ago

Luyt 15y ago

I noticed a while ago that a collegue and I got different results when we googled for our company name, but I didn't know that it's now happening in such an extent. I've added DuckDuck to my FireFox search engines.

wisty 15y ago
I've set it as default on Chrome. Maybe google is better - I'll get more python libraries and less news stories about sheep (or Darwin laureates) getting swallowed by pythons, but I can live with that. Besides, I'm pretty good at typing "google" if I need it.
- epi0Bauqu 15y ago
  You can just add !g to the end (or beg) of your search and it will take you right there.
  - colinprince 15y ago
    Actually, this is super handy! When I know I want the wikipedia article I just add !w at the end. Ditto for youtube (!yt) etc etc
    - wattsbaat 15y ago
      fyi, you can also use !v to search youtube via ddg ('v' for video)
yanw 15y ago
It's not happening to any extent, it's just a new FUD concept that is meant to sell books an confuse people, if you seek more info you can dig deeper into any set of initial results. Personalization == relevance.
- windsurfer 15y ago
  That's funny, when I search for "filter bubble" I can't find anyone refuting it.
deleted 15y ago
[deleted]

yanw 15y ago

This assumes that 'filter bubble' is something more than a nonsense term. There is little difference between personalization and the relevance of search results. How would you go about ranking then? alphabetically?! it's a matter of tuning the relevance 'dials' and it's all in early stages so a solution to this imaginary problems is more research and not to create bullshit terminology in order to sell some books.

TheBranca18 15y ago
Most people don't realize that Google and other companies are doing this. That's my main problem. It's not about selling books in my mind as much as it about communicating why something is in a search list for person A vs. person B. I don't want my Internet censored.
- yanw 15y ago
  People don't realize many things about search engine from indices to ranking algorithms, they do realize however that a search engine is to return the most relevant results for them and that is were personalization/relevance fits in.
- joesb 15y ago
  Do you see search result #1,000,000 on your first page? Congratulation, your internet is censored.
elehack 15y ago
There is a major difference between personalization and "pure" relevance, at least as used here. However, it is more of a question of inputs than actual terms. "Pure" relevance is solely a function of the query and the document - R(q,d). Personalization involves the user and possibly the context - R(q,d,u,x). Now, if you consider the user and their context to be part of the query, then yes, it's the same thing as relevance. So the real question: should the user's identity/history/profile and/or current context be considered a part of the relevance function or not? DDG says no, Google says yes. Thinking of it in this way makes the real question clearer. Unfortunately, it will probably make little sense to most Real Users.

ThomPete 15y ago

It is the ultimate quest for serendipity. If you filter by what you want you don't get what you didn't know you wanted. I will give DDG the benefit of the doubt and try it out for a couple of months. Starting now.

jasonlotito 15y ago
Long time DDG user here. I highly encourage people to try and make the switch. I often find DDG gives me the needed information without having to actually having to visit the site. I still find myself going to Google sometimes if DDG doesn't give me what I want, but it's smart enough to get me the most relevant information faster most of the time. Often enough that I don't feel the need to switch back. In making the switch, I highly advise you to spend a little bit of time learning the keyboard shortcuts, especially the !Bang feature (https://duckduckgo.com/bang.html https://duckduckgo.com/bang.html). You'll love the HN search options. =) There are so many, you won't be able to learn them all. You can also still use Google if you want, or even Bing (or other engines). Basically, the !Bang syntax would mean even if you aren't using DDG, it enables you to quickly use whatever engine you really want. I'm just a fan of DDG. Hope you find it as useful as I do. =)
rcamera 15y ago
I'm glad I have been using DDG for over 6 months now. For the new user, here are some advices. DDG is my default search engine. Basically, all of my searches are completed in DDG, only very few times I have to go to google to find what I am looking for. My suggestion is that you spend 5 minutes learning the bang! syntax, as it speeds up your search by alot, then set DDG as your default search engine for a couple weeks. You will see that the bang! syntax and the 0-click-info really makes the difference on search speed, and their results are pretty damn good.
Pieces 15y ago
Also a long time DDG user. I have it set as my default search provider in chrome, and it has sped up my browsing considerably. Shortcuts like "!w some article" or "!a some product" (takes you directly to Wikipedia or Amazon, respectively) are very handy. The only times I've had to revert to Google are for very specific forum trawling.

the-kenny 15y ago

The first example is totally borked. They first search for "climate change" (notice the ") and then search for climate change (without quotes). Of course the search engine shows different results for different queries.

epi0Bauqu 15y ago
I didn't notice that, but will fix ASAP. The base source of that image was from: http://www.thefilterbubble.com/what-is-the-internet-hiding-lets-find-out http://www.thefilterbubble.com/what-is-the-internet-hiding-l... Update: the quotes vs no quotes doesn't change the top results on Bing (at least for this search).

retube 15y ago

Yeah this is an important issue. Google apparently ranks results (for you personally) based on some 57 inputs, even when not logged into Google services. In short results suffer from a self-reinforcing feedback loop, forever constraining what you see. I wonder how easy it is to get "clean" or "default" results from Google? And I know it's been discussed many times , but just how easy would it be to maintain real anonymity across the web?

senko 15y ago
Incognito/private browsing + going through DuckDuckGo (even for Google queries, with !g prefix) works for me. (Edit to add since I can't reply: going through DDG avoids Google redirecting to country-specific variation).
- scoot 15y ago
  Whit, what? Doesn't going through DDG with !g not just redirect to google, where your normal filter will be applied (private browsing notwithstanding)...
- darklajid 15y ago
  I hate that country specific thing. Somewhere, I think even on HN, I was taught about google.com/ncr - which should solve that problem (being in Country A and arriving on the localized, unreadable landing page) once and for all. That said: I've DDG as my homepage and I'm a happy user.
yanw 15y ago
No it's a non-issue, and by the way Google probably uses much more that 57 'inputs' (signals) to determine relevance. Personalization is just a different term for relevant results and this is just FUD, the source of which is people who don't understand the technical aspects.
- retube 15y ago
  I'm gonna humbly disagree. There are many potential ranking factors in a search result: how well the query matches the document text, how important that text is in the document, how important the document is (PR) and so on. What I don't necessarily want is further ordering by what a machine thinks is my political inclination or world view.

mathly 15y ago

We often forget that, although it's obvious search engines filter results, the information we see on social sites is also filtered. Consider users of Reddit. Now most of them would consider themselves very open minded and enlightened, yet their is active discouragement for radical ideas without due consideration as to their merits. It's just easier to downvote and look at Mario cake. Overall, I think in a way we NEED filters to remove the faff, but be careful to keep a social circle which encourages radical ideas to be bought out into the light of logic and due consideration,

mkr-hn 15y ago

I've been using DDG exclusively for a little while. Half the time the results are identical to Google. The other half is usually a collection of links from sites I would have searched on to research the topic.

util 15y ago
http://searchengineland.com/google-bing-is-cheating-copying-our-search-results-62914 http://searchengineland.com/google-bing-is-cheating-copying-... http://duckduckgo.com/faq.html#results http://duckduckgo.com/faq.html#results

davidmat 15y ago

In all seriousness, this is actually my main use case for the private browsing mode in chrome: to search google without the filter bubble(1) It's quite shocking to see just how much those results differ from the ones I'm usually served, actually. I know it's actually supposed to be 'awesome' to have every search tailored to _you_, but it just makes me feel uncomfortable that I'm not seeing the internet "the way it's supposed to be seen" - if that makes any sense. (1): Or at least a smaller bubble, considering it still knows my location - even though i use google.com, my os, my browser, etc...

igorgue 15y ago

I heard about this book on NPR a couple of weeks ago [1]. I really don't like this biased view of ML. In summary what he said on the show was something like: At least the news shows (and news pappers, radio) gives the same information to everyone, so instead of showing Kardashians news they do show you Bin Laden news, even though, they know the Kardashians are more profitable... this is not the case with Google Search or Netflix, Yahoo, Bing, etc (all attacked by this author). I think that ML helps more than it hurts and viewing it in a non-technical way is wrong, the author gives the impression that Google (the company and their execs) manually (via algorithms, but very manageable in his opinion) change the search results to not show things that they don't like, then it raises the question "do we really trust one company?". Its arguable that the search results in Google or Netflix are optimized for profits, but how do you make profits in the customer industry? IMHO you do that by making their happier, showing useful results, for them, not for everyone. I'm waiting for the time when I google: "what channel and time is Conan on" [2] and I get "channel 43 11pm" as the result. Of course that is very personalized, and the result will be just for me... but again I'm the one searching and I'm the one needing the results. [1] http://thedianerehmshow.org/shows/2011-05-17/eli-pariser-filter-bubble http://thedianerehmshow.org/shows/2011-05-17/eli-pariser-fil... [2] At the moment I get this 1st result: http://articles.boston.com/2010-11-10/ae/29330376_1_conan-andy-richter-anti-nbc http://articles.boston.com/2010-11-10/ae/29330376_1_conan-an... which is not what I searched for nor wanted...

yanw 15y ago
Exactly! it's just nonsense peddled by those who don't understand the technical aspects of search, people who don't realize that a search engine's prime function is to filter the millions of results for each query to down to the most relevant results for the users, and not the same 10 results are relevant to each and every user. And user regardless of nay personalization can dig through any initial results. It's just annoying an ignorant bullshit being disguised as a real issue.
qjz 15y ago
I think there's reason to complain that the reality is far from the ideal. Take Netflix, for example. I rated hundreds of movies, filled out the taste preferences to narrow down the genres I was most interested in, but the "Suggestions for You" were underwhelming, to say the least. Even worse, they simply never changed. So I went back into my taste preferences and checked that I "Often" watched every single mood and genre listed. Now I get a vastly improved range of suggestions, exposing me to some great movies, simply because Netflix is no longer hiding them from my view. I may have a unique individual taste for movies, but Netflix sure hasn't fathomed my criteria with their algorithm.

groby_b 15y ago

Of course, I might want my search results to be filtered. To take but one example: Neither FOX nor CNN are providing any valuable insights at all, so filtering them out is a good idea for me. Or, to get all pithy about it, the problem is not the filter bubble, it's the crap bubble. If there's a problem with relevance sorting at all, it's the issue that it is based on history, not intent.

markkat 15y ago

I think this is a real issue, and I am glad that DDG is addressing it. This is a more-compelling take on the tracking issue IMO. What I'd really like to see, is a search engine to allow me to do both. I'd like to have a profile (that didn't use my name), and when I wanted to, I could click a result as 'useful'. This would go into my personal algorithm. I could then toggle between filtered search, or unfiltered search, whenever I like. It might even be useful to build search filter categories. But I'd keep that a bit buried for the power users.

andrenotgiant 15y ago
Use Incognito mode in Chrome when you would like to search the un-"filtered" Google.
- huckfinnaafb 15y ago
  Unless I'm mistaken, Incognito mode simply disables local history recording, and has not much (if at all) to do with search results or the ability for Google to track your movements. I suggest Chromium for the paranoid!
  - rryan 15y ago
    Incognito mode isn't just disabled history recording. It does not use any of your cookies from main session, and it deletes the cookies when it is closed. It also isolates your extensions from running incognito unless you elect for them to be.
- aticris 15y ago
  Irony: Use of a Google product to circumvent Google's lack of privacy or anonymity in one of their other products.

chaselee 15y ago

I just had to explain this to my co-founder last night after he freaked out because our old site with a different name was showing up first in his search results with our new name as the query . With web history off the old site wasn't even on the first page.Now I just have to explain that no one searches for early-stage startups let alone cares if they change their names :)

loup-vaillant 15y ago

Confirmation bias is bad enough already, it's really a shame that powerful companies like Google reinforces it just for the sake of giving you more pleasing search results. (Pleasing and Good overlap, but they're not equal.) http://en.wikipedia.org/wiki/Confirmation_bias http://en.wikipedia.org/wiki/Confirmation_bias http://lesswrong.com/lw/iw/positive_bias_look_into_the_dark/ http://lesswrong.com/lw/iw/positive_bias_look_into_the_dark/ Edit: I didn't intend to bash Google specifically. But they are faced with a choice in which their own interest conflict with those of their users. And as Capitalist Bastards are more and more accepted in our society, we don't blame them for the selfish choice. Maybe not a shame then, but at the very least a pity.

jarek 15y ago
Yes, after all the primary job of a search engine is to fight the user's own biases, and who wants a product that's pleasing to use anyhow?
- loup-vaillant 15y ago
  My phrasing was a bit harsh. I understand that Google (and others) are mainly out to make money. To do that they have to please users most. Anyway, they probably can't make the difference between "relevant" and "pleasant", because user's behaviour only grant them access to "pleasant". As long as this is done in a neutral way (by delivering the same result to everyone), any confirmation bias will be averaged across entire populations, so this should be okay. Personalized results however make the results noticeably more pleasant, and significantly more biased (this is probably unavoidable). Of course Google, Bing, and Co would shun that bias thing. Who can blame them? I don't want blame Google specifically. I want to point out this old, common moral dilemma: make money, or don't hurt anyone? Google took the money. Many do. I'm not sure to what extent we should blame them, but clearly, the System™ has room for improvement.
  - widget 15y ago
    I cannot see a problem here. Who exactly is being hurt by the "filter bubble"? The end user is fine - they are more likely to see results they are actually interested in. If a user doesn't trust a source and won't click on their links, they'll soon not have to bother scrolling past them. The sites themselves actually benefit as well. Sure, they may be bumped from the first page of results for users that are unlikely to visit their site, anyway, but the tradeoff is that they get a higher position for the users who may actually visit their site. It's an ideal trade for those being filtered. I suppose that leaves the idea that the end result is a "biased" internet. I don't buy it. Google is not removing sites that disagree with them, they are re-ordering them for different users. If your profile wasn't factored in, then what options do they have? They could order on popularity, but biasing towards popular opinion isn't any better than biasing towards my opinion. They could randomize the order, this would be without bias, but absolutely useless to anyone. They could judge the objective truth of sites, but that's far more biased than any of the other options.

yalogin 15y ago

DDG is quite good in regular search. I like their commitment to privacy and so switched my default search engine to DDG. But I noticed more and more that I use google maps almost as much search. I search for places and directions on maps end up going to google a lot. That is brilliance of Google. They built search verticals and so its tough to not use them. If only DDG could do something about that.

epi0Bauqu 15y ago
We are working on integrating OpenStreetMap, and just yesterday launched some upgrades, e.g. https://duckduckgo.com/?q=Valley+Forge%2C+PA https://duckduckgo.com/?q=Valley+Forge%2C+PA or https://duckduckgo.com/?q=black+lab+bistro https://duckduckgo.com/?q=black+lab+bistro (map in right column expands to view). This goes with address detection (https://duckduckgo.com/?q=3+ames+st%2C+boston%2C+ma https://duckduckgo.com/?q=3+ames+st%2C+boston%2C+ma) and you can always force an OSM query by adding map to the end. And of course adding !gm will take you right to Google Maps.
- yuvipanda 15y ago
  Problem with OSM is that it's nowhere near as complete as Google Maps, and I don't see it growing the way Wikipedia did (contributing to Wikipedia is much easier than contributing to OSM)
Habbie 15y ago
Just add !gm to your query. If you do it a few times, DDG will remember it and make it easy to add !gm just by pushing ! to open the dropdown.

yanw 15y ago

Shame on DDG for endorsing this nonsense. I understand the quest for a larger userbase but please don't use this sort of FUD that is being peddled by those who don't understand the technical aspects of search and are trying to sell their books. A search engine's prime function is to filter the millions of results for each query down to the most relevant results for each individual user, and never the same 10 results are relevant to each and every user. There is little difference between personalization and the relevance of search results. How would you go about ranking then? alphabetically?! it's a matter of tuning the relevance 'dials' and it's all in early stages so a solution to this imaginary problem is more research and not to hide behind bullshit terminology. And as a bonus a user (regardless of any personalization) can dig through any initial set of results if she seeks more information. So please don’t buy into this misleading and ignorant bullshit that is being disguised as a real issue.

ThomPete 15y ago
Sorry but this is a real issue, whether DDG solves it or not. At least to me.
- yanw 15y ago
  It's not an issue just because you want to believe it is. Media outlets always filtered stuff for you but now the algorithms are 'filtering' results according to your interaction to find what's relevant to you, so you have a 'vote' here. You also can dig through any initial results (click the news vertical to find more news for example) you couldn't do that with the media of yore.
  - ThomPete 15y ago
    It's not not an issue just because it doesn't apply to you. If you want to understand where I am coming from then this is why I think it's an issue http://000fff.org/slaves-of-the-feed-this-is-not-the-realtime-weve-been-looking-for/ http://000fff.org/slaves-of-the-feed-this-is-not-the-realtim...
- jannes 15y ago

deleted 15y ago

[deleted]

cfontes 15y ago

I would like to use it as default, but I tryed it and it's very very slow... takes me 5 secs to get a search back. in Google I take 0.035s So I will stay information sided for a while

tathagatadg 15y ago

From the search engine revenue perspective, how much can be the loss in case they provide a preference option to show "generic results" (like "safe search option off") - with an easy to switch ui between the two?

Matt_Cutts 15y ago
Normally we don't like to crowd up our preferences page with a bunch of options that people don't use much, but you can turn off personalization by adding "&pws=0" to the end of the Google search url.

taylorbuley 15y ago

Does hitting the logout button pop the filter bubble?

hollerith 15y ago

I heard you can add "&pws=0" to a search query to turn off personalization: http://google.com/search?q=paul+graham&pws=0 http://google.com/search?q=paul+graham&pws=0

citricsquid 15y ago
Checking through a few different search terms, that works for me (provides different results)
odiroot 15y ago
It's funny how "Paul Graham" is Hacker News' foo bar. Also, works well for me.

Kylekramer 15y ago

It is well known that people seek out information that reinforces their own beliefs. And while it may not be ideal for an enlightened population, why should Google/Bing fight this natural inclination? Their job is to ultimately give their users what they are searching for, not challenge their users' opinions. Especially since there are tons of benefits to personalized search, and it may be hard to not throw the baby out with the bathwater.

epi0Bauqu 15y ago
There are other options, i.e. opt-in vs opt-out, separate sections (only show in sidebar), instant on/off, and overall greater transparency.
- Uchikoma 15y ago
  Surfing in anonymous mode with Chrome should work as an opt-out, or shouldn't it?
  - idonthack 15y ago
    only as long as you don't log in to google or anything else before you search.
- speckledjim 15y ago
  There's obvious opt-out things for the miniscule amount of people who care. Exactly the same as the "search term leakage" rubbish.
noahlt 15y ago
This mentality ("give people what they want") results in the empty, sensationalist journalism that most people dislike. When I read the news, I don't want to be entertained, I want to be informed. The same goes for Internet searches.

widget 15y ago

Any filter at all will result in a "Filter Bubble" as defined here (except, I suppose, returning a randomly sorted list of all sites on the internet). Whether the filter is personalized or not doesn't change that fact. What is the actual benefit of everyone getting the same search results when looking for a particular term? Search engines are not news sites or research papers: biasing towards relevant results is not a bad thing.

Uchikoma 15y ago

People that don't want "filtering", also want their search results in German, Chinese or Japanese? Or is "filtering" on language ok?

qjz 15y ago
I think it's fair to assume that people want results in the same language as the search terms they entered.
- Uchikoma 15y ago
  Like Computer, oh well, no Computer in German is Computer. Try Biergarten, oh well.

elb0w 15y ago

I noticed this a long time ago. Suddenly I could ask general programming questions and it popped up results in the language I use primarily as the top results (Without me specifying it). While some may think that this is hiding, do we not often create algorithms to asses and aide? It seems like disabling this would be against the advancement of a learning system. I for one think its a great feature and have no desire to disable it.

alexandros 15y ago

What about the filter bubble of me only searching for what i am interested in? Will DDG throw in some random results in every search to save me from myself?

adamdecaf 15y ago
DDG won't build a profile about you; it only returns the most relevant links to the keywords entered. The original Google search, only on today's web.
- khafra 15y ago
  That's not what he asked. He asked (facetiously) about the higher-order search bubble: ie, if I only ever search for articles about Erlang innards, I'm never going to read about The Kardashians, which is relevant to a sizeable portion of contemporary American culture. To truly pop my bubble, DDG ought to throw in completely random results every now and then.

mquander 15y ago

The purpose of search engines is filtering content to give you what you want, which might be information, or it might be discussion, or polemics, or gossip. If you want to learn about some topic that has some element of subjectivity, familiarize yourself with the different points of view and read the writing of whoever you are interested in. There are lots of tools on the Internet to facilitate this -- Wikipedia, for example, is built around an ideal of giving people an objective survey of different things. If you type "climate change" or "Barack Obama" into Google and form an opinion based on the top results then fuck you.

c4urself 15y ago
I agree with you shouldn't form an opinion based on Google's top results: But I believe that's _exactly_ what most people do.
toddh 15y ago
How to do I know what filters form the top results if they aren't transparent? What would lead you to believe that familiarizing yourself with a different point of view would take you down a meaningfully different path through the search graph? None that I can see. Every search has a non-objective filter. The original page rank is one such. What would be useful is to make the tree obvious and manipulable as a separate object itself.
- mquander 15y ago
  I agree that it would be cool to expose the different criteria that Google (for instance) is using to help reorder your results, to the degree that those criteria can be discretely identified, but I'm not surprised that they don't; that's a pretty significant portion of their secret sauce they would be publicizing. I also don't think that mere transparency would really help solve any search engine "filter bubble" problem, if such a thing is real. Nothing would make Joe Google User take five minutes off whatever he came to search for to fiddle with some tree full of sliders on his result page.
  - toddh 15y ago

Hisoka 15y ago

I suspect DDG is having trouble making money hence the need to make a commotion about filter bubbles and generating some linkbait. No surprise... < 1% of a search engine market is not gonna even make you much at the end of the day.

btipling 15y ago

Except duckduckgo can't find my project and Google can: http://duckduckgo.com/?q=sphela+github http://duckduckgo.com/?q=sphela+github https://encrypted.google.com/search?q=sphela https://encrypted.google.com/search?q=sphela

ComputerGuru 15y ago
Doesn't that say more about your project than DDG?
- btipling 15y ago
  Right because you'd only ever want to find things that were notable.
mkr-hn 15y ago
"Try: !github sphela (Searches GitHub.com using our !bang syntax" !gh works too.
adamdecaf 15y ago
http://duckduckgo.com/?q=!github+sphela http://duckduckgo.com/?q=!github+sphela
epi0Bauqu 15y ago
Thx -- we do have our own github crawl, but it needs to be updated more frequently. Duly noted and will fix.

c4urself 15y ago

In the end (1) technical people will find a way around in when they need to, (2) The conspiracy-theory loving will use the "Filter bubble"as an argument for their own ends, and (3) The rest-most people will just not care... I think the more interesting question is whether this third group will veer off to become more biased to their own world-view as the article suggests.

Matt_Cutts 15y ago

If I had to tackle the notion of over-personalization in ~5 minutes, I'd say: - If someone prefers to search Google without personalization, add "&pws=0" (the "pws" stands for "personalized web search") to the end of the Google search url to turn it off, or use the incognito version of Chrome. Personalization tends to be a nice relevance improvement overall, but it doesn't trigger that much--when it launched, the impact was on the order of one search result above the fold for one in five search results. - personalization has much less impact than localization, which takes things like your IP address into account when determining the best search results. You can change localization by going to country-specific versions of Google (e.g. search for [bank] on google.co.uk vs. google.co.nz), or on google.com you can click "change location" on the left sidebar to enter a different city or zip code in the U.S. - We do have algorithms in place designed specifically to promote variety in the results page. For example, you can imagine limiting the number of results returned from one single site to allow other results to show up instead. That helps with the diversity of the search results. When trying to find the best search results, we look at relevance, diversity, personalization, localization, as well as serendipity and try to find the best balance we can. I saw Eli Pariser's talk at TED and was skeptical, although I did enjoy his example of Facebook starting to return only his liberal friends because he only ever clicked on the links his liberal friends shared. I had a number of concerns browsing through Pariser's book, but I would encourage anyone interested in these issues to pick up a copy; it's a thoughtful read.

pasbesoin 15y ago
Do you have a pointer to a publicly visible reference to the definition of some of the other query string parameters Google uses? Just thought I'd ask, given the context. I've noted various references people have cobbled together, but perhaps there's something a bit more... "canonical". Thanks for the above.
- Matt_Cutts 15y ago
  http://code.google.com/apis/searchappliance/documentation/610/xml_reference.html http://code.google.com/apis/searchappliance/documentation/61... is a reference for our search appliance, but a lot of parameters are in common. I'm not aware of any other official Google doc available externally, but if you search for [google search url parameters] the top 3-4 results are all quite good.

derwiki 15y ago

Sorry to be pedantic, "beg the question" is being used incorrectly. See http://begthequestion.info/ http://begthequestion.info/ ("This begs the question: what are you missing?"). Doesn't detract from the overall post.

yhlasx 15y ago

Personally, not convinced at all. Looks like saying "We are better because we cannot do it yet"

balloot 15y ago

This entire argument is simply without merit. Google is, and always has been, a filter. If I type in "Barack Obama", it will not show me links about The Green Bay Packers NFL team. This is not because Google is conspiring to keep me from reading about the Green Bay Packers. It is because it is most likely that I am not looking for Packers links and will not click. The general idea is that Google starts with every single piece of content on the internet and filters to get the content it thinks I am looking for. Now, this article complains that Google is flawed because it will more likely show some MSNBC over Fox News, or vice versa. The implication here is that you never click on Fox News when it is presented. Because if you did click on Fox News and its ilk from time to time, Google wouldn't start filtering it in the first place. The problem isn't that the search engine creates the "bubble", it is that the user does! So if the scenario presented in this article offends you, then start changing your behavior and browsing more diverse sites. Otherwise, don't blame Google for noting that you hang out in a very narrow corner of the internet, and presenting you links from that corner. It's just doing its job correctly in that case.

tomlin 15y ago

It almost seems like DDG is trying to get me to think personalization == censorship. Sorry, but that ain't gonna fly. You've just undermined your own search engine's potential to use such a feature in the future -- people will be spitting quotes back at DDG about how opposed on this feature they were. Know your audience.

mitcheme 15y ago

I have Google Web History turned off and I'm seeing a pretty good mix of things in my results. E.g., I'm heavily left-wing, somewhat pro-gun control, with no personal interest in shooting, but my results for 'gun' are pretty much all gun shops and fansites. For Barack Obama I get mostly official sites, although some of the top news is critical (I'm Canadian, if it matters). Egypt gives me a mix of travel, protests, general info and ancient history. I don't recall ever turning it off, so is it off by default? And if so, what's the problem here?

jlft 15y ago

I don't like the direction Google is taking with Search. My view of "improvement" is very different from Google's view. I have the feeling that the more features they add to Search the worst the service gets. I miss the old simple Google Search. I think there is a big oportunity here for another search engine focused on: simplicity, speed, relevance (algorythm) and unfiltered. Back to basics.

ZoFreX 15y ago

You're missing a trick by that near-final slide of the DDG input just being an image - would be neat if it were a fully functional DDG search form.

ryanisinallofus 15y ago

I spent a few hours using Duck Duck Go before commenting. It's been my Chrome default search for 4 hours. • The results are less relevant. Bubble or no, it was harder to find things I look for all the time. For example I had to constantly add the word Seattle to my search terms. • It felt like an old search engine. The results, the display, the choice of Mapquest maps all made me feel like I was using Yahoo or even early Google. • At first the feeling of not being watched was liberating but I forgot about it very quickly. I'm going to continue using it as my default search for the lack of tracking but Iv'e already had to go to Google a few times to find what I was looking for. Convenience over privacy right?

rnicholson 15y ago
Interesting. I've been using ddg for several months now, and haven't quite experienced my search results as being "less relevant" or "harder to find things" in comparison to Google or Yahoo!. Ironically, I do use the ddg bang (!) operators quite often. ...like !gm or !gi if I know I want a map or image (for example)
iamelgringo 15y ago
Blekko has been my default search engine for the last months, and I really really like it.
epi0Bauqu 15y ago
Thx -- in my experience it really takes a week to get into it. If you can stick it out I'd love to get your additional feedback after that point. Point of fact: it isn't MapQuest, it is OpenStreetMap served via OpenMapquest, which uses their resources to forward that project. You can read more about it at http://openstreetmap.com/ http://openstreetmap.com/ (left column) and http://open.mapquestapi.com/ http://open.mapquestapi.com/ - in any case, maps are relatively new and in process.
- ryanisinallofus 15y ago
  Thanks for clarifying. Open street map is a cool project.

pilooch 15y ago

There's a personalization / privacy trade-off that needs to be considered. It is annoying that personalization cannot be achieved locally on the user machines or browser. Filtering / re-ranking results at home offers much privacy. My little personal project is to have the ML for personalization done at home, seeks the Seeks Project, http://www.seeks-project.info/ http://www.seeks-project.info/ Personal control over the personal bubble matters...

malaka 15y ago

"But aren't you a part of the relevance equation? The ideal results for a search like [bitcoin crash] should be different for a Japanese-speaking searcher in Tokyo vs. a German-speaking searcher in Munich vs. a bitcoin expert vs. a programmer trying to diagnose why compiling bitcoin is crashing vs. my Mom who has never heard of bitcoin before, right?" Maybe, but that is not the point. Besides, you are mixing localization (japanese vs german) with personalization (expert vs naive), and what is worst is that you are assuming that Google knows so well each user so as to be right (and that is either impossible, either extremely creepy), at every single instant of his life (a person can change interests). Furthermore, to grab your example, how can Google know that an expert in bitcoin and expert in bitcoin compilation and crash solver, is not just interested in hearing about the "market" crash of bitcoin? Google CANNOT read the users' mind. And even if it did, there would not be a need for "personalized search", as the "mind reading" would give enough search criteria to nail the results more easily (albeit, most people do not know exactly what they want, so it will still be an iterative process, which is good, as randomness is the seed for evolution). So, back to the point, it is that in the quest for "adequate results" for each person, Google is turning web search into a non-deterministic event (). Imagine the web being a library, and the search being searching for the library's book database, why would the search for a given book return different results to different persons? It should always return the same results, if the person doing the search is not satisfied with the results, then she/he will add more criteria. In other words, let the person do the filtering! Once that Google accepts that in his quest for "better results" (where 'better' is a concept decided by solely Google and whose ranking parameters and algorithm are unknown) there is a potential (probably demonstrable already) for a "filter bubble" with positive feedback loop on user behaviour, which, as with any positive feedback loop, can go out of control, exacerbating certain ideologies and fueling extremisms. And there is a fundamental difference between a "self guided" (as in self controlled) filtering, where users would knowingly filter out results in order to find those that they like, and a "google guided" (as in externally controlled), filtering. () strictly speaking, search will not be deterministic as the web is a dynamic system and it grows, so search results can vary with time, but they should not vary from person to person at a given time.

widget 15y ago
Of course Google can't read minds, that does not mean they should ignore information that they have when deciding which results to show or what order to show them in. Sufficient data to provide a better filter for a given user is not the same as mind reading. You are absolutely right that more criteria should be used if a user is interested in better results, but I fail to see why a deterministic base case is superior. If I never click on news links and always click on travel links, it makes perfect sense for Google to assume that my search for "Egypt" is looking for information related to travel to Egypt. If I am not following my usual search patterns, I can look on the second page, or I can disable personalized search, or I can search for "Egypt News" all of which would give me better results. Why is it preferable to always make everyone clarify their searches when there is sufficient information to narrow the search down somewhat without requiring additional intervention by the user? This is usability 101 right here.

jerrya 15y ago

How does duckduckgo.com compare with startpage.com?

KristianSerrano 15y ago

Google should invite Pariser for an Authors@Google event with Cutts conducting the interview.

m_feuz 15y ago

in our research study, we found the impact/presence of personalization very strong: after about 3'000 search queries, in some cases more than every search query received personalised search results. out of the 10 blue links in some cases we found 6.4 personalised: see for yourself (search for Hypothesis 1 to get to the data) http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/3344/2766 http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/ar...

aghilmort 15y ago

Explicitly named "filter bubble" search - http://trap.it/ http://trap.it/ xref: http://www.readwriteweb.com/archives/darpa-born_trapit_wants_to_be_your_personalized_ne.php http://www.readwriteweb.com/archives/darpa-born_trapit_wants...