15 ms·

Which search engine is the least censored?

74 points by celdon25 5y ago

i67vw3 5y ago
Bogus test
celdon25 5y ago
Personally, I don't think this was a complete/valid test - Yandex could be censoring other things. The input test data primarily covered things that would fit a specific narrative, not one for the 'other side'. Has anyone done a more complete test?
- sacrosancty 5y ago
  Yes. I think the conclusion is that you would do well to use both Yandex and a typical western one for anything political.
- mrozbarry 5y ago
  I think the validity is mostly scoped the current major world-wide events. It would be interesting to have some sort of topic censorship ranking website; give it a topic, and it ranks various search engines.
- bjourne 5y ago
  Can you give examples of queries for the "other side"? I'd like to test them out.
  - marcusverus 5y ago
    Zelensky is a hero -- should return five results in which the author considers Zelensky to be a hero. Russia thousands arrested -- should return five results about arrests at recent war protests in Russia Russian casualties in ukraine -- should return five results listing figures in the thousands.
    - kordlessagain 5y ago
      This is why the index shouldn't be full of crap. A smaller collection, curated by the user, is where it's at. Maybe you should do a pull request!
      - robbedpeter 5y ago
        This is a little silly when you consider volume. A search engine should crawl millions of pages and sites, and over time develop quality indicators so the crap can be filtered out. Anything that works equivalently to Google's algorithm will suffer from the same crap infection as Google results, and to a lesser degree any of the other top 20 engines and meta-search tools. Something like ublock lists and p2p sharing in closed communities could provide semantic indexing and gradual manual curation, so with a few thousand people you might get novelty + quality in an engine. If you stick to a small, curated list of sites, you won't get site level novelty, and quality will eventually suffer relative to searches that scan over new things. Even then, consider reddit - build a reddit search that's as good as Google. That updates once an hour. Sharing bookmarks isn't feasible when you want the functionality of a general purpose search engine.
    - bjourne 5y ago
      Oddly enough, Yandex pass these tests with flying colors.
- anothernewdude 5y ago
  I expect any search engine worth using to filter out spam and low effort crap. Yandex does neither.
  - asddubs 5y ago
    neither does google these days
sharemywin 5y ago
I found lots of articles from top media outlets about "Obama Ukraine biolab". I just put "fact check" in front of it. as I do with most random blog and/or facebook posts. https://www.factcheck.org/2022/03/social-media-posts-misrepresent-u-s-ukraine-threat-reduction-program/ https://www.factcheck.org/2022/03/social-media-posts-misrepr... https://www.usatoday.com/story/news/factcheck/2022/02/25/fact-check-claim-us-biolabs-ukraine-disinformation/6937923001/ https://www.usatoday.com/story/news/factcheck/2022/02/25/fac... https://www.bloomberg.com/news/articles/2022-03-08/china-pushes-russia-conspiracy-theory-about-u-s-labs-in-ukraine https://www.bloomberg.com/news/articles/2022-03-08/china-pus...
- egberts1 5y ago
  And often you will get factcheck’d by others for your said citations.
jrm4 5y ago
But this gets me thinking. "Censored" is definitely the wrong word here; it implies that there is some objective pool of information that we could all be getting to if it weren't for the search engines getting it wrong, and that's just not it. It's definitely much closer to -- imagine 5 hypothetical bookstores owned by different people. Each has to make decisions on what to choose to carry based on their business interests, and so they do. But there's nothing I would call "censorship" about that. And if we're deciding it is, then we have to be much more serious and intelligent in what to do about it. I'm thinking something like the EPA or FDA for search engines. If they have to tell us whats in the ketchup, then they have to tell us what's in the "search engine algorithms."
- celdon25 5y ago
  When a company gets large enough, whether in the US or Russia, "good relations with the government" becomes one of your aforementioned "business interests". Call it whatever you want.
- gruez 5y ago
  >"Censored" is definitely the wrong word here; it implies that there is some objective pool of information that we could all be getting to if it weren't for the search engines getting it wrong, and that's just not it. 1. I think that when it comes to "search engine censorship", most people know it's about search engines downranking/hiding certain results, not them literally going around burning books or blocking/taking down websites. 2. The definition for "censorship" is "The use of state or group power to control freedom of expression or press, such as passing laws to prevent media from being published or propagated". When it comes to the internet, getting downranked/hidden from search engines does a pretty good job at preventing such information from being "propagated", even though you could still access it via the deep web. As a thought experiment, suppose the government banned 1984 from being sold/read anywhere, save for you going to the library of congress in person. You can theoretically still read it, but at great difficulty. It's certainly more work than reading harry potter or whatever. Would you say such measures are not "censorship"?
- s1artibartfast 5y ago
  I think there are pretty clear examples of censorship that would cross the line. The most widely recognized example is when major search engines delisted pictures and results for the Tiananmen Square massacre last year. The bookstore analogy would be if you came in and asked for a book on the massacre, and the owner said "sorry, I don't know what you are talking about" when they have a whole shelf in stock.
sundarurfriend 5y ago
This is interesting because I tried Yandex for the first time after this DDG debacle, and I found it solved a major problem I have with every other popular search engine - Yandex respected my actual query and gave me the correct results every time, instead of trying to mind read me and mangle the results like the other ones. Google always does it, DDG started doing it some time ago, and unfortunately Brave search also seems to do it. I understand there's probably a reason others do it - maybe that that's what's needed to serve a general audience, they're optimising the results for my 70 yr old dad and 10 yr old niece. But that does mean that using them is a frustrating experience for me half of the time, of quoting everything and micromanaging the query terms so that they understand that I do mean this specific thing rather than this overgeneralised idea they seem to parse from it. Yandex was a breath of fresh air in that respect.
- Biganon 5y ago
  In search engines and in so many other applications, we desperately need a checkbox that says "I know what I'm doing, don't assume anything, give it to me straight". Expert modes, literal modes, they need to be everywhere.
  - horsawlarway 5y ago
    Google has sort of added this - but they hide it so well I didn't even know it existed until a few weeks back. If you do a search, click tools, click the "more" dropdown, and choose "Verbatim" then the results are actually pretty close to "don't fucking try to read my mind". They claim the following about Verbatim (although this info is from 2011, and it may be different now) ---- This verbatim search removes personalized, corrected, suggested, related, and non-inclusive results. On the verbatim page, users will see only results that: Include all their search terms. Match their exact spelling. Use the same tense (e.g., “is” and “was” will be seen as distinct). Use the same verb form (e.g., “swimming” and “swim” will be seen as distinct). Use the same plural vs singular form (e.g., “hat” and “hats” will be seen as distinct).
    - sundarurfriend 5y ago
shoulderfake 5y ago
tehaugmenter 5y ago
I figure to be a more complete test you'd have to include Baidu the Chinese search engine, which is in no doubt censored. Or old school engines like ask.com, and Yahoo even though they serve Bing results to my understanding it would be nice to include them. Either way, good showcase, great example queries. Although I do think some should be catering the MSM narrative to see if the results give contrary results included.
- moltke 5y ago
  I think in the future people will just resort to checking yandex/baidu/duckduckgo every time they think the results might be censored. The opposing regimes are likely to censor complimentary parts of the web.
  - ThalesX 5y ago
    I've been doing this for awhile. Most importantly, I do this with my news. If there's a war in Europe, I don't even bother to check the BBC anymore, or German outlets. I try to grok Al Jazeera, India Times, Channel News Asia, see what's common between them.
personasdfghjkl 5y ago
> Results for "evidence of election fraud 2020" - at least 5 results showing major election fraud took place. Shouldn't an "objective" search engine return results that are true (that there was no systemic election fraud) instead of results that fit his personal biases?
- deleted 5y ago
  [deleted]
- splintercell 5y ago
  That query is biased to begin with. Let’s say I want to understand the flat earth arguments. Should the search engine fill my feed with “earth is not flat” when I’m trying to search for “proof that earth is flat”. This is different from searching “is earth flat” which is an attempt to discover the truth. Your claim that the search engine should show results which indicate that there was no systematic election fraud, is a wishful thinking that you don’t want people to believe in it, not a good decision making regarding what search engines should display.
  - freediver 5y ago
    > Should the search engine fill my feed with “earth is not flat” when I’m trying to search for “proof that earth is flat”. What should it do? Article titled "There is no proof that the earth is flat" also satisfies the query completely from a keyword perspective, although not from a semantic. But if there is no actual proof that earth is flat, any article claiming there is would satisfy the query only from a keyword perspective but technically not semantically. It is a good question.
- StanislavPetrov 5y ago
  An objective search engine should return the results that are most relevant to your query without attempting to be the arbiter of "truth". An objective search engine doesn't try to weigh in on disputed events like the existence of systemic election fraud (which has been a regular occurrence in every election since before any one of us was born, despite the exhortations of the DC blob and legacy media outlets). For example, a Special Counsel in Wisconsin recently did a report on 2020 election fraud. If you Google "Wisconsin report on election fraud" the front page is comprised entirely of DC blob legacy media outlet editorials about why the Wisconsin report is debunked without a single link to the report itself. You are within your rights to think that the Wisconsin report is total garbage, but if you go to a search engine and search for "Wisconsin report on election fraud" an objective search engine will have a link to the actual report on the front page, if not as the first result.
tomerv 5y ago
> No search engine, not even the Russian based Yandex, returned an objective article that included the reasons laid out in Putin's invasion speech. [...] The level of disinformation I observed for this test was terrifying. Based on what I saw, I don't believe it is possible for an uninformed person to learn the actual reasons Russia invaded from a web search on any search engine. Really, you consider Putin's stated reasons to be the actual reasons for the invasion? And the linked article in that paragraph is just a pile of misinformation (even calling it subjective would be a complement).
- supergirl 5y ago
  a search engine is meant for searching text about whatever I ask it. I don’t need it to vet the result based on geopolitics. If I search for “Putin invasion speech” I want articles that show me what Putin said about the invasion ranked by various technical metrics but not by whether the world thinks Putin is a liar and then giving me articles with what others said about Putin’s speech
  - jrmg 5y ago
    I do believe that Google’s ‘quality’ of results has been declining in recent years, but: This article is biased, cherry-picked nonsense. Is anyone actually looking at the results of his example queries and evaluating the quality themselves before responding? To go with your example: the first three results a Google search returns for me on ‘Putin Invasion Speech’ are two NY Times articles analyzing the speech, and then a full translated transcript from Bloomberg. It’s hardly hidden. On the author’s own example, “Why did Russia invade Ukraine? 2022”, the first article is from the BBC and tries to lay out the full historical context - and mentions Putin’s speech. The rest are from mainstream news sources of various descriptions, again laying out historical context. The author may not _like_ that these are trusted, mainstream, popular sources - but that’s what Google has _always_ strived to return. Pagerank was never an ‘unbiased’ thing that searched solely for page content - and, wow, can you imagine the toxic SEO nightmare we’d be in if it was…
    - supergirl 5y ago
TehCorwiz 5y ago
Search engines turn searches into money. Some do this by selling ads, some by selling your data. Unless you’re paying for it you’re bound with their database as the product and are mostly irrelevant to them besides spending more time there. They have an incentive to delay you finding what you want.
- jamjamjamjamjam 5y ago
  Yet here we are with no usable alternatives because competitors get crushed
wallrussianera 5y ago
I suggest anyone who read this post to scroll down the webpage, and once past the comment section - see the first article that was previously posted by the author, titled "Why did Russia attack Ukraine...". Reading that post should tell you how smoothly you're being manipulated.
elephanlemon 5y ago
I’ve found in the past that Yandex’s reverse image search is vastly better than Google’s. I am almost always able to find things with Yandex that I cannot with Google.
jamjamjamjamjam 5y ago
Is deranking considered censorship? On what basis does RT need to be the top result? Now of course I know its been purposely deranked. But couldn’t google claim its not censored if it exists but buried in results? I want to know what legal basis we have for asserting certain sites must have the highest rank on google for certain searches. I’m assuming there isn’t one.

74 points by celdon25 5y ago

Bogus test

Personally, I don't think this was a complete/valid test - Yandex could be censoring other things. The input test data primarily covered things that would fit a specific narrative, not one for the 'other side'. Has anyone done a more complete test?

sacrosancty 5y ago
Yes. I think the conclusion is that you would do well to use both Yandex and a typical western one for anything political.
mrozbarry 5y ago
I think the validity is mostly scoped the current major world-wide events. It would be interesting to have some sort of topic censorship ranking website; give it a topic, and it ranks various search engines.
bjourne 5y ago
Can you give examples of queries for the "other side"? I'd like to test them out.
- marcusverus 5y ago
  Zelensky is a hero -- should return five results in which the author considers Zelensky to be a hero. Russia thousands arrested -- should return five results about arrests at recent war protests in Russia Russian casualties in ukraine -- should return five results listing figures in the thousands.
  - kordlessagain 5y ago
    This is why the index shouldn't be full of crap. A smaller collection, curated by the user, is where it's at. Maybe you should do a pull request!
    - robbedpeter 5y ago
      This is a little silly when you consider volume. A search engine should crawl millions of pages and sites, and over time develop quality indicators so the crap can be filtered out. Anything that works equivalently to Google's algorithm will suffer from the same crap infection as Google results, and to a lesser degree any of the other top 20 engines and meta-search tools. Something like ublock lists and p2p sharing in closed communities could provide semantic indexing and gradual manual curation, so with a few thousand people you might get novelty + quality in an engine. If you stick to a small, curated list of sites, you won't get site level novelty, and quality will eventually suffer relative to searches that scan over new things. Even then, consider reddit - build a reddit search that's as good as Google. That updates once an hour. Sharing bookmarks isn't feasible when you want the functionality of a general purpose search engine.
  - bjourne 5y ago
    Oddly enough, Yandex pass these tests with flying colors.
anothernewdude 5y ago
I expect any search engine worth using to filter out spam and low effort crap. Yandex does neither.
- asddubs 5y ago
  neither does google these days

sharemywin 5y ago

I found lots of articles from top media outlets about "Obama Ukraine biolab". I just put "fact check" in front of it. as I do with most random blog and/or facebook posts. https://www.factcheck.org/2022/03/social-media-posts-misrepresent-u-s-ukraine-threat-reduction-program/ https://www.factcheck.org/2022/03/social-media-posts-misrepr... https://www.usatoday.com/story/news/factcheck/2022/02/25/fact-check-claim-us-biolabs-ukraine-disinformation/6937923001/ https://www.usatoday.com/story/news/factcheck/2022/02/25/fac... https://www.bloomberg.com/news/articles/2022-03-08/china-pushes-russia-conspiracy-theory-about-u-s-labs-in-ukraine https://www.bloomberg.com/news/articles/2022-03-08/china-pus...

egberts1 5y ago
And often you will get factcheck’d by others for your said citations.

jrm4 5y ago

But this gets me thinking. "Censored" is definitely the wrong word here; it implies that there is some objective pool of information that we could all be getting to if it weren't for the search engines getting it wrong, and that's just not it. It's definitely much closer to -- imagine 5 hypothetical bookstores owned by different people. Each has to make decisions on what to choose to carry based on their business interests, and so they do. But there's nothing I would call "censorship" about that. And if we're deciding it is, then we have to be much more serious and intelligent in what to do about it. I'm thinking something like the EPA or FDA for search engines. If they have to tell us whats in the ketchup, then they have to tell us what's in the "search engine algorithms."

celdon25 5y ago
When a company gets large enough, whether in the US or Russia, "good relations with the government" becomes one of your aforementioned "business interests". Call it whatever you want.
gruez 5y ago
>"Censored" is definitely the wrong word here; it implies that there is some objective pool of information that we could all be getting to if it weren't for the search engines getting it wrong, and that's just not it. 1. I think that when it comes to "search engine censorship", most people know it's about search engines downranking/hiding certain results, not them literally going around burning books or blocking/taking down websites. 2. The definition for "censorship" is "The use of state or group power to control freedom of expression or press, such as passing laws to prevent media from being published or propagated". When it comes to the internet, getting downranked/hidden from search engines does a pretty good job at preventing such information from being "propagated", even though you could still access it via the deep web. As a thought experiment, suppose the government banned 1984 from being sold/read anywhere, save for you going to the library of congress in person. You can theoretically still read it, but at great difficulty. It's certainly more work than reading harry potter or whatever. Would you say such measures are not "censorship"?
s1artibartfast 5y ago
I think there are pretty clear examples of censorship that would cross the line. The most widely recognized example is when major search engines delisted pictures and results for the Tiananmen Square massacre last year. The bookstore analogy would be if you came in and asked for a book on the massacre, and the owner said "sorry, I don't know what you are talking about" when they have a whole shelf in stock.

sundarurfriend 5y ago

This is interesting because I tried Yandex for the first time after this DDG debacle, and I found it solved a major problem I have with every other popular search engine - Yandex respected my actual query and gave me the correct results every time, instead of trying to mind read me and mangle the results like the other ones. Google always does it, DDG started doing it some time ago, and unfortunately Brave search also seems to do it. I understand there's probably a reason others do it - maybe that that's what's needed to serve a general audience, they're optimising the results for my 70 yr old dad and 10 yr old niece. But that does mean that using them is a frustrating experience for me half of the time, of quoting everything and micromanaging the query terms so that they understand that I do mean this specific thing rather than this overgeneralised idea they seem to parse from it. Yandex was a breath of fresh air in that respect.

Biganon 5y ago
In search engines and in so many other applications, we desperately need a checkbox that says "I know what I'm doing, don't assume anything, give it to me straight". Expert modes, literal modes, they need to be everywhere.
- horsawlarway 5y ago
  Google has sort of added this - but they hide it so well I didn't even know it existed until a few weeks back. If you do a search, click tools, click the "more" dropdown, and choose "Verbatim" then the results are actually pretty close to "don't fucking try to read my mind". They claim the following about Verbatim (although this info is from 2011, and it may be different now) ---- This verbatim search removes personalized, corrected, suggested, related, and non-inclusive results. On the verbatim page, users will see only results that: Include all their search terms. Match their exact spelling. Use the same tense (e.g., “is” and “was” will be seen as distinct). Use the same verb form (e.g., “swimming” and “swim” will be seen as distinct). Use the same plural vs singular form (e.g., “hat” and “hats” will be seen as distinct).
  - sundarurfriend 5y ago

shoulderfake 5y ago

tehaugmenter 5y ago

I figure to be a more complete test you'd have to include Baidu the Chinese search engine, which is in no doubt censored. Or old school engines like ask.com, and Yahoo even though they serve Bing results to my understanding it would be nice to include them. Either way, good showcase, great example queries. Although I do think some should be catering the MSM narrative to see if the results give contrary results included.

moltke 5y ago
I think in the future people will just resort to checking yandex/baidu/duckduckgo every time they think the results might be censored. The opposing regimes are likely to censor complimentary parts of the web.
- ThalesX 5y ago
  I've been doing this for awhile. Most importantly, I do this with my news. If there's a war in Europe, I don't even bother to check the BBC anymore, or German outlets. I try to grok Al Jazeera, India Times, Channel News Asia, see what's common between them.

personasdfghjkl 5y ago

> Results for "evidence of election fraud 2020" - at least 5 results showing major election fraud took place. Shouldn't an "objective" search engine return results that are true (that there was no systemic election fraud) instead of results that fit his personal biases?

deleted 5y ago
[deleted]
splintercell 5y ago
That query is biased to begin with. Let’s say I want to understand the flat earth arguments. Should the search engine fill my feed with “earth is not flat” when I’m trying to search for “proof that earth is flat”. This is different from searching “is earth flat” which is an attempt to discover the truth. Your claim that the search engine should show results which indicate that there was no systematic election fraud, is a wishful thinking that you don’t want people to believe in it, not a good decision making regarding what search engines should display.
- freediver 5y ago
  > Should the search engine fill my feed with “earth is not flat” when I’m trying to search for “proof that earth is flat”. What should it do? Article titled "There is no proof that the earth is flat" also satisfies the query completely from a keyword perspective, although not from a semantic. But if there is no actual proof that earth is flat, any article claiming there is would satisfy the query only from a keyword perspective but technically not semantically. It is a good question.
StanislavPetrov 5y ago
An objective search engine should return the results that are most relevant to your query without attempting to be the arbiter of "truth". An objective search engine doesn't try to weigh in on disputed events like the existence of systemic election fraud (which has been a regular occurrence in every election since before any one of us was born, despite the exhortations of the DC blob and legacy media outlets). For example, a Special Counsel in Wisconsin recently did a report on 2020 election fraud. If you Google "Wisconsin report on election fraud" the front page is comprised entirely of DC blob legacy media outlet editorials about why the Wisconsin report is debunked without a single link to the report itself. You are within your rights to think that the Wisconsin report is total garbage, but if you go to a search engine and search for "Wisconsin report on election fraud" an objective search engine will have a link to the actual report on the front page, if not as the first result.

tomerv 5y ago

> No search engine, not even the Russian based Yandex, returned an objective article that included the reasons laid out in Putin's invasion speech. [...] The level of disinformation I observed for this test was terrifying. Based on what I saw, I don't believe it is possible for an uninformed person to learn the actual reasons Russia invaded from a web search on any search engine. Really, you consider Putin's stated reasons to be the actual reasons for the invasion? And the linked article in that paragraph is just a pile of misinformation (even calling it subjective would be a complement).

supergirl 5y ago
a search engine is meant for searching text about whatever I ask it. I don’t need it to vet the result based on geopolitics. If I search for “Putin invasion speech” I want articles that show me what Putin said about the invasion ranked by various technical metrics but not by whether the world thinks Putin is a liar and then giving me articles with what others said about Putin’s speech
- jrmg 5y ago
  I do believe that Google’s ‘quality’ of results has been declining in recent years, but: This article is biased, cherry-picked nonsense. Is anyone actually looking at the results of his example queries and evaluating the quality themselves before responding? To go with your example: the first three results a Google search returns for me on ‘Putin Invasion Speech’ are two NY Times articles analyzing the speech, and then a full translated transcript from Bloomberg. It’s hardly hidden. On the author’s own example, “Why did Russia invade Ukraine? 2022”, the first article is from the BBC and tries to lay out the full historical context - and mentions Putin’s speech. The rest are from mainstream news sources of various descriptions, again laying out historical context. The author may not _like_ that these are trusted, mainstream, popular sources - but that’s what Google has _always_ strived to return. Pagerank was never an ‘unbiased’ thing that searched solely for page content - and, wow, can you imagine the toxic SEO nightmare we’d be in if it was…
  - supergirl 5y ago

TehCorwiz 5y ago

Search engines turn searches into money. Some do this by selling ads, some by selling your data. Unless you’re paying for it you’re bound with their database as the product and are mostly irrelevant to them besides spending more time there. They have an incentive to delay you finding what you want.

jamjamjamjamjam 5y ago
Yet here we are with no usable alternatives because competitors get crushed

wallrussianera 5y ago

I suggest anyone who read this post to scroll down the webpage, and once past the comment section - see the first article that was previously posted by the author, titled "Why did Russia attack Ukraine...". Reading that post should tell you how smoothly you're being manipulated.

elephanlemon 5y ago

I’ve found in the past that Yandex’s reverse image search is vastly better than Google’s. I am almost always able to find things with Yandex that I cannot with Google.

jamjamjamjamjam 5y ago

Is deranking considered censorship? On what basis does RT need to be the top result? Now of course I know its been purposely deranked. But couldn’t google claim its not censored if it exists but buried in results? I want to know what legal basis we have for asserting certain sites must have the highest rank on google for certain searches. I’m assuming there isn’t one.