7 ms·

Suppose I go off and make a search engine, with my own algorithm for ranking search results. At first it works great. And then five years later, I find that,

by Imnimo 5y ago

Suppose I go off and make a search engine, with my own algorithm for ranking search results. At first it works great. And then five years later, I find that, by a quirk in my algorithm, I find that a new site has risen to the top of all my results. This new site was created as a prank, and it contains authoritative-sounding but incorrect answers to common questions. The results returned are still relevant to my users, but they are now factually incorrect. Should I simply accept that this site is the correct top result for my users' queries, since it has been ranked by my unbiased algorithm? Or should I decide to put my finger on the scale, and redesign my algorithm (possibly by simply reweighting results from this particular site) to change the rankings? Does the answer change if the site is not a prank site, but is propoganda? Or an earnest-but-incorrect flat earther?

ricardo81 5y ago
> contains authoritative-sounding but incorrect answers to common questions. The results returned are still relevant to my users, but they are now factually incorrect. IMO the difference is between web-wide decisions vs something else. DDG have stated they're specifically targetting something beyond web pages. If DDG has some way of measuring "misinformation" then fair play, we can assume it isn't specific to Russia because it'd be a universal solution. Some sort of decree from the CEO about what is true and what is not just sounds dangerous. We might not fully understand automation, as in all search engines, but it's a safe wager that he doesn't understand it any better otherwise he would've coded it in already and it'd be a non-story for the searchers of the world.
- Imnimo 5y ago
  So if I could only detect this one prank site, but did not have a way to detect and downrank all prank sites at once, you would say it's invalid for me to just take action against the one I know about, since that's not a universal solution?
  - ricardo81 5y ago
    define prank. dictionary definition (according to Google) >a practical joke or mischievous act. Maybe if that's the definition then all Twitter parody accounts need to be removed. Would you agree?
    - Imnimo 5y ago
      Well that depends. If my position were that you have to act via "web-wide decisions" and "universal solutions", then I might conclude that my only options were to leave up all prank sites, including those damaging the usefulness of my search results, or remove everything including harmless parody accounts. On the other hand, if I were to accept that it's valid to target individual sites or subsets of sites only as they become a problem, then I might conclude that removing this one troublesome prank site that is pointing my users to bad information does not require me to take action against random twitter parody accounts.
      - ricardo81 5y ago
        > I might conclude that my only options I guess the point is that you were able to conclude that in the first place. You concluded there are options. An algorithm/decision removing one side of it means you do not, more so if you do not even get to understand why that choice was removed when you're making that choice.
    - deleted 5y ago
      [deleted]
moltke 5y ago
>Should I simply accept that this site is the correct top result for my users' queries, Yes, your users aren't using your search engine as an oracle for the truth, they're using it to find pages with similar text. It's their job (and specifically not yours) to determine how well the text they find corresponds with reality. Perhaps they're even aware the site is factually incorrect and enjoy reading it for the novelty, perhaps they want to cite it to warn others that sometimes things online are wrong. You have no idea what the people are doing with the information. Google has started censoring their results because they decided people should use their search engine as a truth oracle (with the results you would expect, lets not forget the "when did George Washington go to the moon" thing from a year or so ago.) I think that's a mistake and is a large part of the reason I use DuckDuckGo. You may have caught this one instance but there are likely an unlimited number of others. You're not going to catch everything and you make your search engine worse with every exception you make to the algorithm.
- helen___keller 5y ago
  > Yes, your users aren't using your search engine as an oracle for the truth, they're using it to find pages with similar text. I’m confident this is factually incorrect for a very very large subset of searches. Probably a majority.
  - moltke 5y ago
    It's never going to work for that though. This is like changing a hammer design because some people are hurting themselves using it as a comb.
    - helen___keller 5y ago
      It does work for that. For a large number of searches, on relatively uncontroversial topics, search engines have effectively become oracles of truth. This is how google ended up with their modern design that discourages website clicks, they did research into their users and realized most people searching “Abraham Lincoln’s birthday” want to see the top result “February 12, 1809” and not the top result “https://www.loc.gov/item/today-in-history/february-12” https://www.loc.gov/item/today-in-history/february-12” Obviously I’m aware that not all data is uncontroversial and agreed upon, but it’s a fact that a large percentage of searches are quite literally treating search engines as oracles of truth
kkjjkgjjgg 5y ago
I'd say a good search engine should try to deliver the results the user is looking for. Simple as that. It probably can never be perfect, as the search engine can only guess what the user is looking for. But it can try to do its best. There is no rule that it should rank sites by page rank or number of keywords or whatever. If I don't want to see "Russian Disinformation", I probably won't search for it to begin with. On the other hand, maybe I want to see what all the fuss is about - then why should the search engine stand in my way?
- soundnote 5y ago
  The problem is that the definition of disinformation isn't really about accuracy - it's mostly "what is (woke) California prepared to accept". For example, most of the coronavirus "disinfo" bans have been handed out on a basis of roughly like "is this useful for the goal of making people adhere to covid policy?" If you say vaccines won't prevent transmission before Cali is okay with it, byebye, even though the statement has always been accurate. It just hasn't been part of blue California's story until lately. Talking about lab leak? It's okay now, but has gotten people banned for the last couple years. So what will "Russian disinfo" be like? It's hard to even know what the facts are since truth is the first casualty of war.
- bena 5y ago
  I think you're framing the discussion in the wrong way. Yes, if you're looking for "Russian Disinformation", you should be able to find it. But if I'm looking for "U.S. midterm elections", I shouldn't get Russian disinformation. What is the problem with modifying the algorithm to favor actual information over disinformation? Or sites that are known to propagate disinformation? The only thing I could say is that it's weirdly specific to Russian disinformation. Disinformation should be disfavored no matter the country of origin.
  - username3 5y ago
    I shouldn’t get American disinformation, but your trusted sources are untouchable.
goatcode 5y ago
What would you direct searches for "strawman" to?
lynndotpy 5y ago
I think the role of a search engine is to index the page, and to help users find what they're looking for. It's a difficult task that requires somehow quantifying relevance of answers for a given text string. Whatever quantification is used is an implicit kind of 'censorship', and the 'relevance' of results for a given string is inherently a value-judgement. That's where the value of a search engine comes from. We choose search engines based off how good its value judgements are. So, I wish DDG released more details here, because this announcement otherwise just sounds like standard practice. Abusing "quirks in the algorithm" is just called SEO, and it's been a cat-and-mouse battle between search-engines and abuse. Without details, it's harder to interpret this move generously.
- FastMonkey 5y ago
  That sounds about right. A search engine has to try and intuit what users are going to want. Decisions have to be made to show one thing above another. In order for us to trust the engine though, we should have a way of knowing what values are referred to when making these decisions.

by Imnimo 5y ago

ricardo81 5y ago

> contains authoritative-sounding but incorrect answers to common questions. The results returned are still relevant to my users, but they are now factually incorrect. IMO the difference is between web-wide decisions vs something else. DDG have stated they're specifically targetting something beyond web pages. If DDG has some way of measuring "misinformation" then fair play, we can assume it isn't specific to Russia because it'd be a universal solution. Some sort of decree from the CEO about what is true and what is not just sounds dangerous. We might not fully understand automation, as in all search engines, but it's a safe wager that he doesn't understand it any better otherwise he would've coded it in already and it'd be a non-story for the searchers of the world.

Imnimo 5y ago
So if I could only detect this one prank site, but did not have a way to detect and downrank all prank sites at once, you would say it's invalid for me to just take action against the one I know about, since that's not a universal solution?
- ricardo81 5y ago
  define prank. dictionary definition (according to Google) >a practical joke or mischievous act. Maybe if that's the definition then all Twitter parody accounts need to be removed. Would you agree?
  - Imnimo 5y ago
    Well that depends. If my position were that you have to act via "web-wide decisions" and "universal solutions", then I might conclude that my only options were to leave up all prank sites, including those damaging the usefulness of my search results, or remove everything including harmless parody accounts. On the other hand, if I were to accept that it's valid to target individual sites or subsets of sites only as they become a problem, then I might conclude that removing this one troublesome prank site that is pointing my users to bad information does not require me to take action against random twitter parody accounts.
    - ricardo81 5y ago
      > I might conclude that my only options I guess the point is that you were able to conclude that in the first place. You concluded there are options. An algorithm/decision removing one side of it means you do not, more so if you do not even get to understand why that choice was removed when you're making that choice.
  - deleted 5y ago
    [deleted]

moltke 5y ago

>Should I simply accept that this site is the correct top result for my users' queries, Yes, your users aren't using your search engine as an oracle for the truth, they're using it to find pages with similar text. It's their job (and specifically not yours) to determine how well the text they find corresponds with reality. Perhaps they're even aware the site is factually incorrect and enjoy reading it for the novelty, perhaps they want to cite it to warn others that sometimes things online are wrong. You have no idea what the people are doing with the information. Google has started censoring their results because they decided people should use their search engine as a truth oracle (with the results you would expect, lets not forget the "when did George Washington go to the moon" thing from a year or so ago.) I think that's a mistake and is a large part of the reason I use DuckDuckGo. You may have caught this one instance but there are likely an unlimited number of others. You're not going to catch everything and you make your search engine worse with every exception you make to the algorithm.

helen___keller 5y ago
> Yes, your users aren't using your search engine as an oracle for the truth, they're using it to find pages with similar text. I’m confident this is factually incorrect for a very very large subset of searches. Probably a majority.
- moltke 5y ago
  It's never going to work for that though. This is like changing a hammer design because some people are hurting themselves using it as a comb.
  - helen___keller 5y ago
    It does work for that. For a large number of searches, on relatively uncontroversial topics, search engines have effectively become oracles of truth. This is how google ended up with their modern design that discourages website clicks, they did research into their users and realized most people searching “Abraham Lincoln’s birthday” want to see the top result “February 12, 1809” and not the top result “https://www.loc.gov/item/today-in-history/february-12” https://www.loc.gov/item/today-in-history/february-12” Obviously I’m aware that not all data is uncontroversial and agreed upon, but it’s a fact that a large percentage of searches are quite literally treating search engines as oracles of truth

kkjjkgjjgg 5y ago

I'd say a good search engine should try to deliver the results the user is looking for. Simple as that. It probably can never be perfect, as the search engine can only guess what the user is looking for. But it can try to do its best. There is no rule that it should rank sites by page rank or number of keywords or whatever. If I don't want to see "Russian Disinformation", I probably won't search for it to begin with. On the other hand, maybe I want to see what all the fuss is about - then why should the search engine stand in my way?

soundnote 5y ago
The problem is that the definition of disinformation isn't really about accuracy - it's mostly "what is (woke) California prepared to accept". For example, most of the coronavirus "disinfo" bans have been handed out on a basis of roughly like "is this useful for the goal of making people adhere to covid policy?" If you say vaccines won't prevent transmission before Cali is okay with it, byebye, even though the statement has always been accurate. It just hasn't been part of blue California's story until lately. Talking about lab leak? It's okay now, but has gotten people banned for the last couple years. So what will "Russian disinfo" be like? It's hard to even know what the facts are since truth is the first casualty of war.
bena 5y ago
I think you're framing the discussion in the wrong way. Yes, if you're looking for "Russian Disinformation", you should be able to find it. But if I'm looking for "U.S. midterm elections", I shouldn't get Russian disinformation. What is the problem with modifying the algorithm to favor actual information over disinformation? Or sites that are known to propagate disinformation? The only thing I could say is that it's weirdly specific to Russian disinformation. Disinformation should be disfavored no matter the country of origin.
- username3 5y ago
  I shouldn’t get American disinformation, but your trusted sources are untouchable.

goatcode 5y ago

What would you direct searches for "strawman" to?

lynndotpy 5y ago

I think the role of a search engine is to index the page, and to help users find what they're looking for. It's a difficult task that requires somehow quantifying relevance of answers for a given text string. Whatever quantification is used is an implicit kind of 'censorship', and the 'relevance' of results for a given string is inherently a value-judgement. That's where the value of a search engine comes from. We choose search engines based off how good its value judgements are. So, I wish DDG released more details here, because this announcement otherwise just sounds like standard practice. Abusing "quirks in the algorithm" is just called SEO, and it's been a cat-and-mouse battle between search-engines and abuse. Without details, it's harder to interpret this move generously.

FastMonkey 5y ago
That sounds about right. A search engine has to try and intuit what users are going to want. Decisions have to be made to show one thing above another. In order for us to trust the engine though, we should have a way of knowing what values are referred to when making these decisions.