5 ms·

I've been using DuckDuckGo for months now and over the past few weeks it's results have gotten considerably better. I wonder if this is because a massive amount

I've been using DuckDuckGo for months now and over the past few weeks it's results have gotten considerably better. I wonder if this is because a massive amount of new users are consistently using it overall improving it's results? Overall I think this is a great change though!

BitMastro 13y ago
Honest question: if DuckDuckGo doesn't track the users, how can it improve when more users are using it?
- jvehent 13y ago
  Not tracking the user's personal information doesn't mean you're not tracking the number of clicks, or calculate geoip stats.
  - webdev4242 13y ago
    How is that different from using Google over https without logging in?
    - yk 13y ago
      Google can still track you, cookies are just one example. So DDG uses only the data which can not related to a single user, essentially "users who searched X clicked most often on the second link" while Google builds a profile, that is "the user with cookie value Y usually searches for computers."
    - ams6110 13y ago
      You can dump your google cookies frequently or blacklist google.com from setting cookies.
  - taopao 13y ago
    IIRC DDG was vocally against personalization and "filter bubbling". Even if they collected these stats, they wouldn't be able to refine their results unless they've changed their position.
    - cpeterso 13y ago
      Optimizing their search results globally based on user clicks is not personalization or filter bubbling.
      - magicalist 13y ago
        That's assuming that search preferences are homogeneous, that everyone in the world uses DDG, or that the people currently providing click data are a representative sample of the world. The first and last things might be true for a few terms, but even then, the order of the search results is a very tricky thing (e.g. NYTimes or WSJ editorial page first? the difference in traffic between 1st and even 2nd or 3rd on the search results page is huge). The only real benefit is that you and your neighbor are in the same filter bubble.
      - tomkarlo 13y ago
        There are upper limits to global optimization. We all don't want the same results for a given query; to get beyond some particular bound, you can't treat all users the same.
- mbrubeck 13y ago
  In addition to aggregate usage stats, there's also the indirect (and slower) route: more users -> more ad impressions -> higher revenue -> more employees or other resources available to improve the service.
- fhd2 13y ago
  Working for a company that takes privacy very seriously, I can say that it can be a PITA. Worth doing, but still a PITA. We had to resort to some clever trickery just to count monthly active users without any sort of unique identifier. I bet DDG faces similar challenges.
  - yegg 13y ago
    Spot on, but worth it!
  - grey-area 13y ago
    Out of curiosity, what solution did you hit upon? Google uses a cookie to tie visits to a certain user for analytics. The other solution I've come up with is to use a hash of IP + other identifying info, so not recording the IP or anytihng to tie it to a person, but still recording a relatively unique ID. What sort of techniques are you using?
    - fhd2 13y ago
      Well, we have daily update requests (it's not a web app), so we knew the number of daily users already. But we really wanted to know the monthly users, gives a way better idea of the actual user numbers. What we ended up doing is to transmit the timestamp of the last update check with each update check, that way we can see which requests are the first over an arbitrary period. Granted, the situation is probably not that similar to what DDG faces, but it was just one issue that came up. What we use for web analytics is based on awstats, which operates on access logs with anonymised IPs. Not as good as Google Analytics, but as useful as it gets without JS/tracking, I guess.
      - grey-area 13y ago
        Thanks
- LukeShu 13y ago
  I'm not sure if they are still as responsive on Twitter as they used to be, but if you tweeted at them with a specific search term that gave poor results, they would act directly on that. It was cool to see your own input make a difference.
- deleted 13y ago
  [deleted]
deleted 13y ago
[deleted]
hkmurakami 13y ago
After many months of zero Japanese support, DDG seems to now have support. It's been great!
psbp 13y ago
I've been reading these comments about DDG for what feels like years now. Was it so terrible to begin with or is considerable an overstatement? It can't be improving so quickly and still pretty shitty for such a long period of time.
- LukeShu 13y ago
  I'd say it's improving for different categories of searches. Because for a long time users were largely technical, it's search for tech stuff has been pretty good for a long time (in general, there are a few sub-categories in that that have lagged[1]). Where, searching for an author or book might have terrible results. So, when these people say it has improved, they mean it has improved for the kind of searches that that person does. As more people use it, and suggest improvements[2], it will approach being good for all kinds of searches. [1] It still isn't very good at searching for specific error messages. It definitely used to show obscure projects on GitHub way too often. You'd do a search for 'getty' and some 3-commit project with a name containing getty would be the first hit. Also, I think it's actually gotten worse at finding Rails API stuff. [2] It's been a while since I tried, but if you tweet at them with a search term with a poor result, they will try to figure out how to tweak the algorithm to make it better. It's really cool to see them act so directly on your suggestions.
- GoodIntentions 13y ago
  I don't think it was _terrible_. It just doesn't give as accurate results as Google in some searches, I assume due to the rafts of information collected about users - a trade off I will make happily.
  - benologist 13y ago
    It's not even that they're less accurate, I think for a lot of searches we expect certain results.
    - jaredmcateer 13y ago
      If the search engine isn't giving me results I expect isn't that the same if not worse than it being inaccurate? If I type cucumber in google, it knows since I'm always search technical information I probably mean Cucumber BDD and it gives it to me as the second result right after the wikipedia article about the gourd, that's what I expect. DDG, however, having no clue who I am, gives me 8 results about the gourd before finally, below the fold mind you, giving me Cucumber BDD.

BitMastro 13y ago

Honest question: if DuckDuckGo doesn't track the users, how can it improve when more users are using it?

jvehent 13y ago
Not tracking the user's personal information doesn't mean you're not tracking the number of clicks, or calculate geoip stats.
- webdev4242 13y ago
  How is that different from using Google over https without logging in?
  - yk 13y ago
    Google can still track you, cookies are just one example. So DDG uses only the data which can not related to a single user, essentially "users who searched X clicked most often on the second link" while Google builds a profile, that is "the user with cookie value Y usually searches for computers."
  - ams6110 13y ago
    You can dump your google cookies frequently or blacklist google.com from setting cookies.
- taopao 13y ago
  IIRC DDG was vocally against personalization and "filter bubbling". Even if they collected these stats, they wouldn't be able to refine their results unless they've changed their position.
  - cpeterso 13y ago
    Optimizing their search results globally based on user clicks is not personalization or filter bubbling.
    - magicalist 13y ago
      That's assuming that search preferences are homogeneous, that everyone in the world uses DDG, or that the people currently providing click data are a representative sample of the world. The first and last things might be true for a few terms, but even then, the order of the search results is a very tricky thing (e.g. NYTimes or WSJ editorial page first? the difference in traffic between 1st and even 2nd or 3rd on the search results page is huge). The only real benefit is that you and your neighbor are in the same filter bubble.
    - tomkarlo 13y ago
      There are upper limits to global optimization. We all don't want the same results for a given query; to get beyond some particular bound, you can't treat all users the same.
mbrubeck 13y ago
In addition to aggregate usage stats, there's also the indirect (and slower) route: more users -> more ad impressions -> higher revenue -> more employees or other resources available to improve the service.
fhd2 13y ago
Working for a company that takes privacy very seriously, I can say that it can be a PITA. Worth doing, but still a PITA. We had to resort to some clever trickery just to count monthly active users without any sort of unique identifier. I bet DDG faces similar challenges.
- yegg 13y ago
  Spot on, but worth it!
- grey-area 13y ago
  Out of curiosity, what solution did you hit upon? Google uses a cookie to tie visits to a certain user for analytics. The other solution I've come up with is to use a hash of IP + other identifying info, so not recording the IP or anytihng to tie it to a person, but still recording a relatively unique ID. What sort of techniques are you using?
  - fhd2 13y ago
    Well, we have daily update requests (it's not a web app), so we knew the number of daily users already. But we really wanted to know the monthly users, gives a way better idea of the actual user numbers. What we ended up doing is to transmit the timestamp of the last update check with each update check, that way we can see which requests are the first over an arbitrary period. Granted, the situation is probably not that similar to what DDG faces, but it was just one issue that came up. What we use for web analytics is based on awstats, which operates on access logs with anonymised IPs. Not as good as Google Analytics, but as useful as it gets without JS/tracking, I guess.
    - grey-area 13y ago
      Thanks
LukeShu 13y ago
I'm not sure if they are still as responsive on Twitter as they used to be, but if you tweeted at them with a specific search term that gave poor results, they would act directly on that. It was cool to see your own input make a difference.
deleted 13y ago
[deleted]

deleted 13y ago

[deleted]

hkmurakami 13y ago

After many months of zero Japanese support, DDG seems to now have support. It's been great!

psbp 13y ago

I've been reading these comments about DDG for what feels like years now. Was it so terrible to begin with or is considerable an overstatement? It can't be improving so quickly and still pretty shitty for such a long period of time.

LukeShu 13y ago
I'd say it's improving for different categories of searches. Because for a long time users were largely technical, it's search for tech stuff has been pretty good for a long time (in general, there are a few sub-categories in that that have lagged[1]). Where, searching for an author or book might have terrible results. So, when these people say it has improved, they mean it has improved for the kind of searches that that person does. As more people use it, and suggest improvements[2], it will approach being good for all kinds of searches. [1] It still isn't very good at searching for specific error messages. It definitely used to show obscure projects on GitHub way too often. You'd do a search for 'getty' and some 3-commit project with a name containing getty would be the first hit. Also, I think it's actually gotten worse at finding Rails API stuff. [2] It's been a while since I tried, but if you tweet at them with a search term with a poor result, they will try to figure out how to tweak the algorithm to make it better. It's really cool to see them act so directly on your suggestions.
GoodIntentions 13y ago
I don't think it was _terrible_. It just doesn't give as accurate results as Google in some searches, I assume due to the rafts of information collected about users - a trade off I will make happily.
- benologist 13y ago
  It's not even that they're less accurate, I think for a lot of searches we expect certain results.
  - jaredmcateer 13y ago
    If the search engine isn't giving me results I expect isn't that the same if not worse than it being inaccurate? If I type cucumber in google, it knows since I'm always search technical information I probably mean Cucumber BDD and it gives it to me as the second result right after the wikipedia article about the gourd, that's what I expect. DDG, however, having no clue who I am, gives me 8 results about the gourd before finally, below the fold mind you, giving me Cucumber BDD.