8 ms·

I made an app that lets you compare duck-duck-go results with that of google, side by side [0][1]. One thing I found is how bad ddg is at removing conspiracy t

by bko 6y ago

I made an app that lets you compare duck-duck-go results with that of google, side by side [0][1]. One thing I found is how bad ddg is at removing conspiracy theories and other misinformation. Google is, for better or for worse, much more managed. For instance, search "are vaccines dangerous", google replies with .org and .gov sites, while ddg promotes dangersofvaccines.com. Even if you search "dangers of vaccines" in google, it still never produces that site. [0] https://search-compare.netlify.app/ https://search-compare.netlify.app/ [1] https://github.com/breeko/search-compare https://github.com/breeko/search-compare

diggan 6y ago
That's a cool site, you should submit it to "Show HN" if you haven't already :) Not sure if your site is taking the location into account, but when I do the "are vaccines dangerous" search on the search-compare website, I get .gov and .edu domains as the top 5 hits or so.
- bko 6y ago
  Glad you like my site. It was fun to build. I remember bing use to have a site years ago where you would search for a term and see bing vs google results and you'd pick the one you preferred, and apparently 2/3 people preferred bing results. But I couldn't find it, so it gave me an idea to build a site but I didn't build the gamified portion. Yeah, you're right about search results. I'm seeing more normal sites now as well. The results are not deterministic. I'm not sure whats causing it, but I saw different results for the vaccines query earlier. "Flat earth" still gives me a fair amount of flat earth content. From the FAQ: > For Duck Duck Go, search-compare searches the html version (e.g. duckduckgo.com/html/?q=search). The results are different than the canonical duckduckgo search, but its easier to scrape. For Google, search-compare uses the main google site (e.g. google.com/search?hl=en&q=search), but it doesn't pass any personally identifiable information
mellosouls 6y ago
Speaking as somebody who goes with mainstream expert opinion by default, I nevertheless think there's enough censorship from the big tech providers, at the very least let's have such "filter out stuff that annoys us" initiatives as optional.
- dmurray 6y ago
  Google has to choose something to prioritize. It's good that in this case it chose the authoritative site over the conspiracy theory one. Even better if they chose it because it was more authoritative rather than because they calculated they could show you more ads that way, but I won't give them credit for that just yet. They didn't censor the other site completely. If I search with site:dangersofvaccines.com I get results from it. I can't get results from it any other way though, including searching text that only appears on that site - they'd rather tell me there are no results - so I agree that's an unpleasant level of censorship/nannying.
- smaryjerry 6y ago
  I can’t believe anyone is in favor of censorship in the first place. Maybe they just need to experience the harsh reality of rule by majority opinion or mob rule. Reddit is the perfect example. I can’t tell you how many times a top up voted thread or comment is 100% false. The experts aren’t the ones voting is the problem so who gets to the top is only the one most persuasive or with a comment that already agrees with expectations. This is exactly how things work with politics or news as well. I can’t tell you how many news articles I’ve read that make extremely basic mistakes and assumptions because the person writing the article has no experience in that field.
  - antris 6y ago
    But there is space between outright censorship and deliberately building a system that promotes the ideas of those who are the best at information warfare. If you repeat your idea often and loud enough, you can make a large number of people believe anything. Internet curated by algorithms is a perfect tool for spreading misinformation: - You need not to carry the weight of any of the legal structures or responsibilities of a journalist. - You can produce misinformation faster than the news cycle is able to react, meaning your misinformation will be able to poison the well before any conversation is able to take place. - You can segment your audience so that dissenting opinion is nowhere to be seen. - By the time any debunking by actual journalists can be done, your target audience is already exposed to the next cycle and can barely remember anything about the previous one. This is extremely dangerous. I am sure that the people who came up with automobiles weren't trying to create a vehicle of death, but there needs to be some kind of safety check akin to seat belts for the information propagation that takes place online. I am not sure what the answer is, but acknowledging the problem and trying to do something shouldn't be dismissed with a mere cry of "censorship" when information can and is being weaponized in a way and at a scale that has been impossible so far.
abc-xyz 6y ago
> One thing I found is how bad ddg is at removing conspiracy theories and other misinformation. You should probably blame Microsoft for that. I mean DDG is just a Bing API wrapper.
- Semaphor 6y ago
  Yes, that’s why [0] and [1] show the same results. Only, they don’t. They are more similar than DDG vs. Google, but not the same and DDG has (for me) more relevant results closer to the top. [0]: https://www.bing.com/search?q=css%20center%20text https://www.bing.com/search?q=css%20center%20text [1]: https://duckduckgo.com/?q=css+center+text https://duckduckgo.com/?q=css+center+text
  - nvrspyx 6y ago
    Weird. The results are in the exact same order for me when I clicked both of your links. The only difference is that the bootstrapshuffle.com result is below (or rather, on the second page) the stackoverflow result on Bing. Considering the former is specific to Bootstrap, it's less relevant than the stackoverflow result. Anyways, DDG does mainly use Bing, but they also do their own modifications to the rankings, which is sometimes for better and sometimes for worse.
    - pythux 6y ago
      > [...] but they also do their own modifications to the rankings, which is sometimes for better and sometimes for worse. Could you maybe share a reference for this?
      - nvrspyx 6y ago
        I'm a couple of days late to this, but no, I cannot provide a reference (anymore). DDG used to have a page that described in pretty great detail how the results were sourced and ranked, but it seems the current page is buried pretty deep and has less information specific to ranking. IIRC, the Bing results (and previously Yahoo/Verizon Media) rankings were augmented by their own crawler (DDBot) and aggregated usage of their search. I believe they also pull the results that their "instant answers" are derived from higher up based on the search. If, for example, you did a technical search and their "instant answer" was from a stackoverflow post, that particular post would be pulled higher up the list (but not necessarily to the top) even if it was lower on Bing.
Semaphor 6y ago
Cool, I’ll see about doing searches with it more often to get a better read on quality differences :) It’s interesting to me that despite the widespread complaint about the bad quality of DDG (which I, excepting long-tail results, disagree with), at least for my quick checks of "ddg" and "css center text" from your suggestions, I prefer the DDG results in both cases (especially for the css search).
textgel 6y ago
Some people are fine with making decisions for themselves and don't need a net nanny to say "mummy thinks you shouldn't be reading this".
- bko 6y ago
  I agree in general. But what if I did want to know if vaccines are dangerous, so I search "are vaccines dangerous". If I want to find an anti-vaccine site, I should be able to do so, but the search term "are vaccines dangerous" should be viewed neutrally, in which case I would find links from cdc or .gov sites to be more useful that anti-vaccines sites
  - textgel 6y ago
    That's fair; I suppose it's inevitable that search engines move more in a similar direction to what we know as the main high street when it comes to high profile searches, keeping more mainstream stuff while losing the off the wall areas but it does seem a shame to loose some of the weirder side streets in the process (the sort of weirdness from the early 2000s internet for example).
    - intricatedetail 6y ago
      It's a bad idea. There are examples where mainstream was wrong. Usually when it's about big money. E.g. decades of anti-cannabis propaganda.
      - textgel 6y ago
        Now you mention it, the official position on the correct response to coronavirus flip flopping and changing from location to location is a good recent example too.
  - smaryjerry 6y ago
    We go to school in the US for usually at least 12 years. Any person should already be educated on topics such as anti vaccination. In particular in my state you can’t even go to school if you haven’t had your vaccines. Not only that but we learn how to learn in school. The argument for censorship against stupidity assumes exactly that, that the majority of people are stupid which they are not.
disown 6y ago
> One thing I found is how bad ddg is at removing conspiracy theories and other misinformation. Can't be any worse than google since they give preferential treatment to conspiracy theorists like CNN, MBNBC, Foxnews, etc. > For instance, search "are vaccines dangerous", google replies with .org and .gov sites So google should just push government and special interest groups funded by the wealthy? > Even if you search "dangers of vaccines" in google, it still never produces that site. You make it sound like that's a good thing... Holy christ man. You are right, let google just show government sites. Big Brother knows best.
- valec 6y ago
  > conspiracy theorists like CNN, MBNBC ??? >let google just show government sites. Big Brother knows best. .org is not a government domain. even so, maybe if they and other tech companies had done this earlier we wouldn't have gotten into this trump + coronavirus mess
  - disown 6y ago
    > ??? Exactly. This is the problem with the country. Half the crazies believe one set of conspiracies is the truth and the other half believe in another set of conspiracies. > .org is not a government domain. Right, I said they were special interests funded by the wealthy. > maybe if they and other tech companies had done this earlier we wouldn't have gotten into this trump + coronavirus mess Right... And in a few years, the other half of the crazies will be talking about the biden mess. Half the morons the last 4 years, Trump stole the election. Half the morons now, Biden stole the election. Figure it out. That anyone would defend any mainstream media after the last 4 years is being intentionally blind. Right or left.
    - drenvuk 6y ago
      See a therapist, please.
- ThePreacherMan 6y ago
  You are an idiot.
ghthor 6y ago
It's not a conspiracy theory that people are hurt by vaccines. That's a fact. The question is, are people hurt more by the vaccine or the disease itself. From the evidence I've researched, I think it's the vaccines that are more dangerous.
- drummer 6y ago
  Yep, my own research also shows that vaccines are dangerous in various ways. Also the power of big pharma and big tech when it comes to money and censorship.
  - alexandrvicente 6y ago
    I don’t think we’d be seeing new measles outbreaks if big tech and big pharma were that effective at censoring anti-vaxxers.
    - drummer 6y ago
      You'll find plenty of examples online of cases where the vaccine is responsible for the outbreak of the disease it supposedly protect against.
      - esarbe 6y ago
        I can find plenty of examples online of cases where aliens anally probe Dakota farmers. That doesn't mean it's true.
        drummer 6y ago
        Tip of the ice berg " LONDON (AP) — The World Health Organization says a new polio outbreak in Sudan is linked to an ongoing vaccine-sparked epidemic in Chad — a week after the U.N. health agency declared the African continent free of the wild polio virus. " https://apnews.com/619efb65b9eeec5650f011b960a152e9 https://apnews.com/619efb65b9eeec5650f011b960a152e9
miked85 6y ago
I don't want a search engine removing whatever they deem a "conspiracy theory" or "misinformation". I can decide on my own.
- criddell 6y ago
  Isn't that essentially saying you don't want your search engine ranking by relevance and quality?
  - anaganisk 6y ago
    Quality, the same "AI" recommends me irrelevant videos, cant recognise me as a human if i log in from any browser other than chrome, and that issues takedowns randomly for users. The ine that spams, pinterest like shit site quality on front page. Yeah, no, I dont want it to decide.
    - hiccuphippo 6y ago
      The first thing I do when searching for images is `-site:pinterest.com` The quality of results increases immensely.
      - spurgu 6y ago
        IIRC Google used to have a dropdown in the search results similar to "Don't show results from this site". Luckily I don't see Yahoo Answers results anymore but would be nice to have Pinterest removed.
        Normille 6y ago
        If you use Greasemonkey / Tampermonkey, etc. you might want to check out 'Google hit Hider by domain'[0] [in spite of the name, it works on several search engines, not just Google]. It provides a really simple interface to ban [with various levels of extreme prejudice] sites from ever again appearing in your search results. [0] https://greasyfork.org/en/scripts/1682-google-hit-hider-by-domain-search-filter-block-sites https://greasyfork.org/en/scripts/1682-google-hit-hider-by-d...
aezakmi 6y ago
How on Earth are people like you allowed to breathe. You're worse than cattle.
coldtea 6y ago
>One thing I found is how bad ddg is at removing conspiracy theories and other misinformation. In all kinds of regimes, from dictatorships to western democracies, all kinds of valid (and sometimes proven in court, or historically, later) stories have been officially called "conspiracy theories" and/or misinformation. And inversely, in all kinds of regimes, from dictatorships to western democracies, all kinds of bogus state propaganda has been officially called "the truth". I'd rather not have a search engine decide for me...
srg0 6y ago
I searched for "are vaccines dangerous" in ddg, and the results are reasonable. https://duckduckgo.com/?q=are+vaccines+dangerous&t=fpas&ia=web https://duckduckgo.com/?q=are+vaccines+dangerous&t=fpas&ia=w... The first page of the US search results: vaccines.gov, chop.edu, cdc.gov, livescience.com, scientificamerican.com, nytimes.com, newsweek.com, healthline.com, who.int, mayoclinic.org. That's a good mix of the official outlets and some popular explanations. Results change of you specify a different region or no region at all. For reference, these are results in Italian, many of them are rebuttals of a minister claiming that vaccines are useless and dangerous: https://duckduckgo.com/?q=vaccini+pericolosi&t=fpas&ia=web https://duckduckgo.com/?q=vaccini+pericolosi&t=fpas&ia=web But if you try to search English terms in a non-English region, expect ask kind of SEO garbage.

by bko 6y ago

diggan 6y ago

That's a cool site, you should submit it to "Show HN" if you haven't already :) Not sure if your site is taking the location into account, but when I do the "are vaccines dangerous" search on the search-compare website, I get .gov and .edu domains as the top 5 hits or so.

bko 6y ago
Glad you like my site. It was fun to build. I remember bing use to have a site years ago where you would search for a term and see bing vs google results and you'd pick the one you preferred, and apparently 2/3 people preferred bing results. But I couldn't find it, so it gave me an idea to build a site but I didn't build the gamified portion. Yeah, you're right about search results. I'm seeing more normal sites now as well. The results are not deterministic. I'm not sure whats causing it, but I saw different results for the vaccines query earlier. "Flat earth" still gives me a fair amount of flat earth content. From the FAQ: > For Duck Duck Go, search-compare searches the html version (e.g. duckduckgo.com/html/?q=search). The results are different than the canonical duckduckgo search, but its easier to scrape. For Google, search-compare uses the main google site (e.g. google.com/search?hl=en&q=search), but it doesn't pass any personally identifiable information

mellosouls 6y ago

Speaking as somebody who goes with mainstream expert opinion by default, I nevertheless think there's enough censorship from the big tech providers, at the very least let's have such "filter out stuff that annoys us" initiatives as optional.

dmurray 6y ago
Google has to choose something to prioritize. It's good that in this case it chose the authoritative site over the conspiracy theory one. Even better if they chose it because it was more authoritative rather than because they calculated they could show you more ads that way, but I won't give them credit for that just yet. They didn't censor the other site completely. If I search with site:dangersofvaccines.com I get results from it. I can't get results from it any other way though, including searching text that only appears on that site - they'd rather tell me there are no results - so I agree that's an unpleasant level of censorship/nannying.
smaryjerry 6y ago
I can’t believe anyone is in favor of censorship in the first place. Maybe they just need to experience the harsh reality of rule by majority opinion or mob rule. Reddit is the perfect example. I can’t tell you how many times a top up voted thread or comment is 100% false. The experts aren’t the ones voting is the problem so who gets to the top is only the one most persuasive or with a comment that already agrees with expectations. This is exactly how things work with politics or news as well. I can’t tell you how many news articles I’ve read that make extremely basic mistakes and assumptions because the person writing the article has no experience in that field.
- antris 6y ago
  But there is space between outright censorship and deliberately building a system that promotes the ideas of those who are the best at information warfare. If you repeat your idea often and loud enough, you can make a large number of people believe anything. Internet curated by algorithms is a perfect tool for spreading misinformation: - You need not to carry the weight of any of the legal structures or responsibilities of a journalist. - You can produce misinformation faster than the news cycle is able to react, meaning your misinformation will be able to poison the well before any conversation is able to take place. - You can segment your audience so that dissenting opinion is nowhere to be seen. - By the time any debunking by actual journalists can be done, your target audience is already exposed to the next cycle and can barely remember anything about the previous one. This is extremely dangerous. I am sure that the people who came up with automobiles weren't trying to create a vehicle of death, but there needs to be some kind of safety check akin to seat belts for the information propagation that takes place online. I am not sure what the answer is, but acknowledging the problem and trying to do something shouldn't be dismissed with a mere cry of "censorship" when information can and is being weaponized in a way and at a scale that has been impossible so far.

abc-xyz 6y ago

> One thing I found is how bad ddg is at removing conspiracy theories and other misinformation. You should probably blame Microsoft for that. I mean DDG is just a Bing API wrapper.

Semaphor 6y ago
Yes, that’s why [0] and [1] show the same results. Only, they don’t. They are more similar than DDG vs. Google, but not the same and DDG has (for me) more relevant results closer to the top. [0]: https://www.bing.com/search?q=css%20center%20text https://www.bing.com/search?q=css%20center%20text [1]: https://duckduckgo.com/?q=css+center+text https://duckduckgo.com/?q=css+center+text
- nvrspyx 6y ago
  Weird. The results are in the exact same order for me when I clicked both of your links. The only difference is that the bootstrapshuffle.com result is below (or rather, on the second page) the stackoverflow result on Bing. Considering the former is specific to Bootstrap, it's less relevant than the stackoverflow result. Anyways, DDG does mainly use Bing, but they also do their own modifications to the rankings, which is sometimes for better and sometimes for worse.
  - pythux 6y ago
    > [...] but they also do their own modifications to the rankings, which is sometimes for better and sometimes for worse. Could you maybe share a reference for this?
    - nvrspyx 6y ago
      I'm a couple of days late to this, but no, I cannot provide a reference (anymore). DDG used to have a page that described in pretty great detail how the results were sourced and ranked, but it seems the current page is buried pretty deep and has less information specific to ranking. IIRC, the Bing results (and previously Yahoo/Verizon Media) rankings were augmented by their own crawler (DDBot) and aggregated usage of their search. I believe they also pull the results that their "instant answers" are derived from higher up based on the search. If, for example, you did a technical search and their "instant answer" was from a stackoverflow post, that particular post would be pulled higher up the list (but not necessarily to the top) even if it was lower on Bing.

Semaphor 6y ago

Cool, I’ll see about doing searches with it more often to get a better read on quality differences :) It’s interesting to me that despite the widespread complaint about the bad quality of DDG (which I, excepting long-tail results, disagree with), at least for my quick checks of "ddg" and "css center text" from your suggestions, I prefer the DDG results in both cases (especially for the css search).

textgel 6y ago

Some people are fine with making decisions for themselves and don't need a net nanny to say "mummy thinks you shouldn't be reading this".

bko 6y ago
I agree in general. But what if I did want to know if vaccines are dangerous, so I search "are vaccines dangerous". If I want to find an anti-vaccine site, I should be able to do so, but the search term "are vaccines dangerous" should be viewed neutrally, in which case I would find links from cdc or .gov sites to be more useful that anti-vaccines sites
- textgel 6y ago
  That's fair; I suppose it's inevitable that search engines move more in a similar direction to what we know as the main high street when it comes to high profile searches, keeping more mainstream stuff while losing the off the wall areas but it does seem a shame to loose some of the weirder side streets in the process (the sort of weirdness from the early 2000s internet for example).
  - intricatedetail 6y ago
    It's a bad idea. There are examples where mainstream was wrong. Usually when it's about big money. E.g. decades of anti-cannabis propaganda.
    - textgel 6y ago
      Now you mention it, the official position on the correct response to coronavirus flip flopping and changing from location to location is a good recent example too.
- smaryjerry 6y ago
  We go to school in the US for usually at least 12 years. Any person should already be educated on topics such as anti vaccination. In particular in my state you can’t even go to school if you haven’t had your vaccines. Not only that but we learn how to learn in school. The argument for censorship against stupidity assumes exactly that, that the majority of people are stupid which they are not.

disown 6y ago

> One thing I found is how bad ddg is at removing conspiracy theories and other misinformation. Can't be any worse than google since they give preferential treatment to conspiracy theorists like CNN, MBNBC, Foxnews, etc. > For instance, search "are vaccines dangerous", google replies with .org and .gov sites So google should just push government and special interest groups funded by the wealthy? > Even if you search "dangers of vaccines" in google, it still never produces that site. You make it sound like that's a good thing... Holy christ man. You are right, let google just show government sites. Big Brother knows best.

valec 6y ago
> conspiracy theorists like CNN, MBNBC ??? >let google just show government sites. Big Brother knows best. .org is not a government domain. even so, maybe if they and other tech companies had done this earlier we wouldn't have gotten into this trump + coronavirus mess
- disown 6y ago
  > ??? Exactly. This is the problem with the country. Half the crazies believe one set of conspiracies is the truth and the other half believe in another set of conspiracies. > .org is not a government domain. Right, I said they were special interests funded by the wealthy. > maybe if they and other tech companies had done this earlier we wouldn't have gotten into this trump + coronavirus mess Right... And in a few years, the other half of the crazies will be talking about the biden mess. Half the morons the last 4 years, Trump stole the election. Half the morons now, Biden stole the election. Figure it out. That anyone would defend any mainstream media after the last 4 years is being intentionally blind. Right or left.
  - drenvuk 6y ago
    See a therapist, please.
ThePreacherMan 6y ago
You are an idiot.

ghthor 6y ago

It's not a conspiracy theory that people are hurt by vaccines. That's a fact. The question is, are people hurt more by the vaccine or the disease itself. From the evidence I've researched, I think it's the vaccines that are more dangerous.

drummer 6y ago
Yep, my own research also shows that vaccines are dangerous in various ways. Also the power of big pharma and big tech when it comes to money and censorship.
- alexandrvicente 6y ago
  I don’t think we’d be seeing new measles outbreaks if big tech and big pharma were that effective at censoring anti-vaxxers.
  - drummer 6y ago
    You'll find plenty of examples online of cases where the vaccine is responsible for the outbreak of the disease it supposedly protect against.
    - esarbe 6y ago
      I can find plenty of examples online of cases where aliens anally probe Dakota farmers. That doesn't mean it's true.
      - drummer 6y ago
        Tip of the ice berg " LONDON (AP) — The World Health Organization says a new polio outbreak in Sudan is linked to an ongoing vaccine-sparked epidemic in Chad — a week after the U.N. health agency declared the African continent free of the wild polio virus. " https://apnews.com/619efb65b9eeec5650f011b960a152e9 https://apnews.com/619efb65b9eeec5650f011b960a152e9

miked85 6y ago

I don't want a search engine removing whatever they deem a "conspiracy theory" or "misinformation". I can decide on my own.

criddell 6y ago
Isn't that essentially saying you don't want your search engine ranking by relevance and quality?
- anaganisk 6y ago
  Quality, the same "AI" recommends me irrelevant videos, cant recognise me as a human if i log in from any browser other than chrome, and that issues takedowns randomly for users. The ine that spams, pinterest like shit site quality on front page. Yeah, no, I dont want it to decide.
  - hiccuphippo 6y ago
    The first thing I do when searching for images is `-site:pinterest.com` The quality of results increases immensely.
    - spurgu 6y ago
      IIRC Google used to have a dropdown in the search results similar to "Don't show results from this site". Luckily I don't see Yahoo Answers results anymore but would be nice to have Pinterest removed.
      - Normille 6y ago
        If you use Greasemonkey / Tampermonkey, etc. you might want to check out 'Google hit Hider by domain'[0] [in spite of the name, it works on several search engines, not just Google]. It provides a really simple interface to ban [with various levels of extreme prejudice] sites from ever again appearing in your search results. [0] https://greasyfork.org/en/scripts/1682-google-hit-hider-by-domain-search-filter-block-sites https://greasyfork.org/en/scripts/1682-google-hit-hider-by-d...

aezakmi 6y ago

How on Earth are people like you allowed to breathe. You're worse than cattle.

coldtea 6y ago

>One thing I found is how bad ddg is at removing conspiracy theories and other misinformation. In all kinds of regimes, from dictatorships to western democracies, all kinds of valid (and sometimes proven in court, or historically, later) stories have been officially called "conspiracy theories" and/or misinformation. And inversely, in all kinds of regimes, from dictatorships to western democracies, all kinds of bogus state propaganda has been officially called "the truth". I'd rather not have a search engine decide for me...

srg0 6y ago

I searched for "are vaccines dangerous" in ddg, and the results are reasonable. https://duckduckgo.com/?q=are+vaccines+dangerous&t=fpas&ia=web https://duckduckgo.com/?q=are+vaccines+dangerous&t=fpas&ia=w... The first page of the US search results: vaccines.gov, chop.edu, cdc.gov, livescience.com, scientificamerican.com, nytimes.com, newsweek.com, healthline.com, who.int, mayoclinic.org. That's a good mix of the official outlets and some popular explanations. Results change of you specify a different region or no region at all. For reference, these are results in Italian, many of them are rebuttals of a minister claiming that vaccines are useless and dangerous: https://duckduckgo.com/?q=vaccini+pericolosi&t=fpas&ia=web https://duckduckgo.com/?q=vaccini+pericolosi&t=fpas&ia=web But if you try to search English terms in a non-English region, expect ask kind of SEO garbage.