5 ms·

"Generally speaking I think ddg went with the "we don't store data" route because it's the niche they were left with and that ultimately means that their search

"Generally speaking I think ddg went with the "we don't store data" route because it's the niche they were left with and that ultimately means that their search won't improve beyond a certain point." As they will all be too busy not storing peoples data, or how does this line of reasoning work?

s3r3nity 13y ago
Search is based upon machine learning algorithms that use stored data as signals. Google is not only REALLY good at what they do BECAUSE of the amount of data they have on the individual doing the searching, but the benefits of machine learning are that they will get even BETTER as more and more data is incorporated and stored. Without proper machine learning on the individual level, DDG will never improve beyond a point. They're already kind of shitty, and even if they get millions of more queries they won't be able to improve at the rate that Google does. It's a trade-off ('privacy' vs. 'quality') that 95% of the world doesn't give a shit about. Therefore I see this sort of move to DDG as just short-sighted and unreasonable overall.
- nfoz 13y ago
  Note that DDG is small enough that they're not even doing their search crawling/aggregation. They have to rely on garbage data they get from Yahoo and other crawlers. Their quality, I think, is much less from the lack of smart machine learning algos, as it is their lack of funds in not being able to afford to crawl the web.
  - sveme 13y ago
    Wrong: https://dukgo.com/help/en_US/results/sources https://dukgo.com/help/en_US/results/sources Don't spread wrongness if you don't know it better.
- ChuckMcM 13y ago
  This pretty much captures it. Google knows a lot about you, and it knows a lot about the people you hang out with and the kinds of things you read and the kinds of things you don't read. In knowing all of that stuff they can ascertain that you are part of a community (well a cohort at least) with certain preferences and proclivities, and as soon as a new user identifies as also part of that cohort they can tweak the user experience to "improve" it. And don't get me wrong, it does improve. I'm glad that I'm not bombarded with ads for TJMax on Google properties but think Digikey could back off a bit. From a feedback perspective though it gets weird. And you can see that when you search without any identifying information enabled. I found the whole 'Don't Bubble Me' campaign quite creative. The challenge for search engines though is that being the reference librarian's favorite tool for objectiveness doesn't woo the mass market. And search is a game of numbers if nothing else.
  - engrenage 13y ago
    Google is a great search engine because what it knows about the web. Google is a great advertising business because of what it knows about you.
    - thrownaway2424 13y ago
      No. Google is a great search engine because of what it knows about you. Trivial example: search on DDG for "ramen": useless generic information about ramen. Search on Google for "ramen": general information on the right, TEN laser-accurate local results for ramen shops. Search on DDG for "tires": ten results for sites that sell tires. Search on Google for "tires": shopping links for tires for my motorcycle. Nice.
      - engrenage 13y ago
        Geolocated search results are hardly what it knows about you. However knowing your motorcycle is interesting. How do you think it got that information?
        dylz 13y ago
        Google Now indexes things like your mailbox, calendar, contacts. If you have a local motorcycle shop or something in your contacts or whatever, [.......]
        engrenage 13y ago
        Presumably all available when the NSA wants it.
- akbar501 13y ago
  Recent survey conducted by Washington Post-ABC News: 1. 75% of [Americans] are concerned about personal privacy 2. 40% say it is more important to protect privacy even if that limits the US government’s ability to investigate possible terrorist threats Privacy appears to be a concern to a large percentage of Americans. That said, it's not like all of these people are going to delete their FB account, switch to DDG tomorrow, etc. However, the numbers do suggest that DDG has a sizable opportunity assuming they execute well and that privacy concerns continue. "Therefore I see this sort of move to DDG as just short-sighted and unreasonable overall." What do users of Gnome's browser care about? If users of Gnome's browser are concerned about personal privacy, then the move to switch to DDG for search is a sound choice.
  - thrownaway2424 13y ago
    This analysis relies entirely on DDG's privacy being a material thing, rather than an illusion. Nobody on this site nor anywhere outside of DDG has any idea if DDG is capable of resisting coercion or intrusion by state agencies and other bad people.
natrius 13y ago
User data is commonly used to improve relevance.
tomku 13y ago
You can only make your results so relevant if you limit yourself to returning the same results for everyone for a given query. Google uses its data on you to (among other things) tweak your search results based on what it thinks you specifically are likely to be looking for, which DDG will never be able to do. DDG has (rather successfully) spun that as a critical feature rather than the tradeoff between privacy and utility that it is.

s3r3nity 13y ago

Search is based upon machine learning algorithms that use stored data as signals. Google is not only REALLY good at what they do BECAUSE of the amount of data they have on the individual doing the searching, but the benefits of machine learning are that they will get even BETTER as more and more data is incorporated and stored. Without proper machine learning on the individual level, DDG will never improve beyond a point. They're already kind of shitty, and even if they get millions of more queries they won't be able to improve at the rate that Google does. It's a trade-off ('privacy' vs. 'quality') that 95% of the world doesn't give a shit about. Therefore I see this sort of move to DDG as just short-sighted and unreasonable overall.

nfoz 13y ago
Note that DDG is small enough that they're not even doing their search crawling/aggregation. They have to rely on garbage data they get from Yahoo and other crawlers. Their quality, I think, is much less from the lack of smart machine learning algos, as it is their lack of funds in not being able to afford to crawl the web.
- sveme 13y ago
  Wrong: https://dukgo.com/help/en_US/results/sources https://dukgo.com/help/en_US/results/sources Don't spread wrongness if you don't know it better.
ChuckMcM 13y ago
This pretty much captures it. Google knows a lot about you, and it knows a lot about the people you hang out with and the kinds of things you read and the kinds of things you don't read. In knowing all of that stuff they can ascertain that you are part of a community (well a cohort at least) with certain preferences and proclivities, and as soon as a new user identifies as also part of that cohort they can tweak the user experience to "improve" it. And don't get me wrong, it does improve. I'm glad that I'm not bombarded with ads for TJMax on Google properties but think Digikey could back off a bit. From a feedback perspective though it gets weird. And you can see that when you search without any identifying information enabled. I found the whole 'Don't Bubble Me' campaign quite creative. The challenge for search engines though is that being the reference librarian's favorite tool for objectiveness doesn't woo the mass market. And search is a game of numbers if nothing else.
- engrenage 13y ago
  Google is a great search engine because what it knows about the web. Google is a great advertising business because of what it knows about you.
  - thrownaway2424 13y ago
    No. Google is a great search engine because of what it knows about you. Trivial example: search on DDG for "ramen": useless generic information about ramen. Search on Google for "ramen": general information on the right, TEN laser-accurate local results for ramen shops. Search on DDG for "tires": ten results for sites that sell tires. Search on Google for "tires": shopping links for tires for my motorcycle. Nice.
    - engrenage 13y ago
      Geolocated search results are hardly what it knows about you. However knowing your motorcycle is interesting. How do you think it got that information?
      - dylz 13y ago
        Google Now indexes things like your mailbox, calendar, contacts. If you have a local motorcycle shop or something in your contacts or whatever, [.......]
        engrenage 13y ago
        Presumably all available when the NSA wants it.
akbar501 13y ago
Recent survey conducted by Washington Post-ABC News: 1. 75% of [Americans] are concerned about personal privacy 2. 40% say it is more important to protect privacy even if that limits the US government’s ability to investigate possible terrorist threats Privacy appears to be a concern to a large percentage of Americans. That said, it's not like all of these people are going to delete their FB account, switch to DDG tomorrow, etc. However, the numbers do suggest that DDG has a sizable opportunity assuming they execute well and that privacy concerns continue. "Therefore I see this sort of move to DDG as just short-sighted and unreasonable overall." What do users of Gnome's browser care about? If users of Gnome's browser are concerned about personal privacy, then the move to switch to DDG for search is a sound choice.
- thrownaway2424 13y ago
  This analysis relies entirely on DDG's privacy being a material thing, rather than an illusion. Nobody on this site nor anywhere outside of DDG has any idea if DDG is capable of resisting coercion or intrusion by state agencies and other bad people.

natrius 13y ago

User data is commonly used to improve relevance.

tomku 13y ago

You can only make your results so relevant if you limit yourself to returning the same results for everyone for a given query. Google uses its data on you to (among other things) tweak your search results based on what it thinks you specifically are likely to be looking for, which DDG will never be able to do. DDG has (rather successfully) spun that as a critical feature rather than the tradeoff between privacy and utility that it is.