5 ms·

Shame on DDG for endorsing this nonsense. I understand the quest for a larger userbase but please don't use this sort of FUD that is being peddled by those who

by yanw 15y ago

Shame on DDG for endorsing this nonsense. I understand the quest for a larger userbase but please don't use this sort of FUD that is being peddled by those who don't understand the technical aspects of search and are trying to sell their books. A search engine's prime function is to filter the millions of results for each query down to the most relevant results for each individual user, and never the same 10 results are relevant to each and every user. There is little difference between personalization and the relevance of search results. How would you go about ranking then? alphabetically?! it's a matter of tuning the relevance 'dials' and it's all in early stages so a solution to this imaginary problem is more research and not to hide behind bullshit terminology. And as a bonus a user (regardless of any personalization) can dig through any initial set of results if she seeks more information. So please don’t buy into this misleading and ignorant bullshit that is being disguised as a real issue.

ThomPete 15y ago
Sorry but this is a real issue, whether DDG solves it or not. At least to me.
- yanw 15y ago
  It's not an issue just because you want to believe it is. Media outlets always filtered stuff for you but now the algorithms are 'filtering' results according to your interaction to find what's relevant to you, so you have a 'vote' here. You also can dig through any initial results (click the news vertical to find more news for example) you couldn't do that with the media of yore.
  - ThomPete 15y ago
    It's not not an issue just because it doesn't apply to you. If you want to understand where I am coming from then this is why I think it's an issue http://000fff.org/slaves-of-the-feed-this-is-not-the-realtime-weve-been-looking-for/ http://000fff.org/slaves-of-the-feed-this-is-not-the-realtim...
- jannes 15y ago
  Why do you think it is an issue? I think everyone already filters intuitively. It's called having a "bullshit filter". I believe it makes not much difference whether it's automated or everyone does it manually. I think the chance that I would click on a bullshit source like Fox News is very low because I don't trust them. Even if Google brought it up as the first result. My point is that people already only click on search results that they agree with. Nobody wants to see stuff that they disagree with. That is human nature. So living in a filter bubble already happens without automatic filters. It happens in our brains and we call it personality.
qjz 15y ago
So, your argument is that it's bullshit because it's true?
- yanw 15y ago
  It's bullshit because search is filtering and the filtering done by search is being exaggerated by those who peddle this bullshit.
  - qjz 15y ago
    I expect my search to be faithful to the terms I enter. If the results aren't satisfactory, I'll narrow the search with additional terms. Withholding information based on some guess at my political leanings is dishonest. I'd be appalled if a librarian did it; why should I expect less from a search engine?
    - yanw 15y ago
      The 'political leaning results' are just a tool the author used to try to explained his (ill-informed) point, there are no evidence to any such degree of personalization specially not to any 'bubble' inducing extent, it's all very sensationalistic. And for the record the political example was about facebook wall.
    - Uchikoma 15y ago
      What does faithful mean? You want German or Chinese results? You want global or local results (in your language of choice). There is always personalization in search.
    - SoftwareMaven 15y ago
      If you went to a librarian and said "show me Egypt", she would have nowhere to start. A conversation would then ensue about what you are interested in. Keep going to the library, keep having those conversations, and eventually, the librarian will take you straight to the books you want when you ask for Norway. If you want something different, you'll have to explicitly ask. What Google is doing is no different. It just happens to have a lot more conversations with you.
speckledjim 15y ago
Spreading FUD about tracking search terms etc didn't seem to help DDG last time. I doubt this will have any effect either. DDG: If you want to grow, don't keep attacking the big search engines with FUD that only a miniscule % of people care about.
- mkr-hn 15y ago
  The February DDG newsletter reported growth in January from 2.5m searches to 5m. I don't think DDG needs FUD to grow.
  - speckledjim 15y ago
    http://duckduckgo.com/traffic.html http://duckduckgo.com/traffic.html Searches per day in Feb average = 197,687 Searches per day in May average = 205,759 FUD certainly won't grow DDG out of the extreme niche it's in.
    - mkr-hn 15y ago
      That agrees with what I said. Unless you don't think that much growth in searches over such a short period is significant. It's even more significant when you look at where it was a year ago. 1.2 million in April 2010, 5.9 million in April 2011, 6.3 last month, and already 3.8 a little over halfway through this month. This is significant when you consider the lack of marketing and that DDG is run by one person.
      - Hisoka 15y ago
        It's not significant. In the world of search engines, you need to be the #1 or #2 player to earn major revenue. People aren't gonna wanna waste any time/energy advertising on your platform if you're getting just a handful of searches per month for say... mp3 players or cellphones. DDG is just betting on an acquisition, but it's just a wrapper around the Yahoo BOSS API.
        15y ago
Revisor 15y ago
Search queries have different purposes. Personalization based on my preferences and past search works wonders for known-item seeking and for re-finding. But for exploratory search and exhaustive search personalization works as an echo chamber (or filter bubble if you will) . Then there is the problem of change and inertia. People change, their preferences vary in time. Personalization has an inertia that causes the recommendation engine to always be behind the actual preferences. It's more visible the better the recommendation engine. I'm not saying personalization is problematic. There are problems with using it all the time and stealthily, without a clear possibility to turn it off. It's not FUD, but a challenge to overcome.
- yanw 15y ago
  All can be remedied by tweaking the algorithms and that happens all the time, you can also use a certain vertical directly for certain type queries (scholar search etc). The same way that what a user might enjoy can be inferred the inverse of that can be used as a signal in the algorithm as well. Personalization is just relevance and all is just a matter of tuning.
  - Revisor 15y ago
    I agree, it can be remedied by better algorithms - such that can recognize when I'm doing a search for a known item and when I'm performing exploratory search. In the first case, personalize away, in the second case, please show me everything. The challenge is that people switch very quickly between these search modes and don't even think about it.
quinndupont 15y ago
I think you missed the point. Yes DDG ranks, but it may rank differently. Of course, there is an implicit slam on them here: instead of showing you what you want the vast majority of the time, they show you other stuff. This is good on occasion (so you don't live in an echo chamber), but if this was all the time it would mean lower quality search results for simple things.
- yanw 15y ago
  They show you other stuff Do you have any proof that this isn't what all search engines actually do?
- thesz 15y ago
  That's quite a point. My own standing is that most of the time you need what you have to have to complete task at hand. Then you could go at YouTube and refresh "Recomendations" to your heart content. The other way around is to wreak havoc to important parts of your life.
- rogerbraun 15y ago
  This is exactly why I stopped using DDG as my main search engine. My workflow was often like this: - Search for term on DDG - Look at results, find nothing - Try to find better term - Give up and use original term in Google - Find result I tried to use DDG exclusively two time (once 8 months ago, once ~4 months ago), but the result was the same. I don't know how much personalization affects this, but Google just gives me the best results.
  - MetallicCloud 15y ago
    I have had almost the exact same experience. I want to use DDG, but I too regularly had to use Google after I couldn't find the results I was after.
kitsune_ 15y ago
No, no, no, no, no, no, no. "There is little difference between personalization and the relevance of search results." Even if you're a Sailor Moon fan, you might actually be looking for information about the rock rotating around this planet when searching for "moon". In the same vein I might be a liberal but be looking for a multitude of opposing views and ideas when googling "core values of NRA members". Oh btw, we engineers, our algorithms, aren't smart enough to foretell people's inclinations based on their past habits, we also can't read minds. The increasing use of personalization as a factor for relevancy will, imho, lead to user dissatisfaction, precisely because they make shoddy assumptions. Believe it or not, but some people also want to be surprised and learn new things. YES. We exist!
glassx 15y ago
Maybe we don't want a filter that sucks, plain and simple? Whether the filter sucks or not is completely subjective, but it sucks for me, so there. Changing the language not that intuitive in Google, it's a drag. I have to do it all the time when I clean my cookies, and sometimes I still get crappy "pt.wikipedia.org" results on top instead of "en.wikipedia.org" just because my browser isn't in english and I don't want to change it just because of Google. I might want to type www.google.it and search for Berlusconi news in Italia, but it won't let me, because it just show me brazilian news. So yeah, I can't even get to another bubble. Also, apparently Google filters by IP, since searching via Tor or a VPS renders very different results. I dislike this for personal reasons because it breaks the internet for me. Also, I might not like the fact that Google collects stuff about my search habits, which is also a valid concern. So, there, it's an issue for me. Might be an edge case, might be an exception, but I think that I deserve to know why this happens and the alternatives.
gnosis 15y ago
"There is little difference between personalization and the relevance of search results. How would you go about ranking then? alphabetically?!" Oh, you know, maybe using Google's much vaunted "PageRank" algorithm, which is supposed to take in to account things such as links on other pages back to the page being considered for inclusion in the search results.. the more such links there are, the better the rank is supposed to be. The above description is obviously an oversimplification, and I don't have access to Google's PageRank algorithm anyway, so couldn't tell you what it actually was if I wanted to, but it's something along those lines, and need not take in to account your previous search history in any way or "personalize" the search for you.

by yanw 15y ago

ThomPete 15y ago

Sorry but this is a real issue, whether DDG solves it or not. At least to me.

yanw 15y ago
It's not an issue just because you want to believe it is. Media outlets always filtered stuff for you but now the algorithms are 'filtering' results according to your interaction to find what's relevant to you, so you have a 'vote' here. You also can dig through any initial results (click the news vertical to find more news for example) you couldn't do that with the media of yore.
- ThomPete 15y ago
  It's not not an issue just because it doesn't apply to you. If you want to understand where I am coming from then this is why I think it's an issue http://000fff.org/slaves-of-the-feed-this-is-not-the-realtime-weve-been-looking-for/ http://000fff.org/slaves-of-the-feed-this-is-not-the-realtim...
jannes 15y ago
Why do you think it is an issue? I think everyone already filters intuitively. It's called having a "bullshit filter". I believe it makes not much difference whether it's automated or everyone does it manually. I think the chance that I would click on a bullshit source like Fox News is very low because I don't trust them. Even if Google brought it up as the first result. My point is that people already only click on search results that they agree with. Nobody wants to see stuff that they disagree with. That is human nature. So living in a filter bubble already happens without automatic filters. It happens in our brains and we call it personality.

qjz 15y ago

So, your argument is that it's bullshit because it's true?

yanw 15y ago
It's bullshit because search is filtering and the filtering done by search is being exaggerated by those who peddle this bullshit.
- qjz 15y ago
  I expect my search to be faithful to the terms I enter. If the results aren't satisfactory, I'll narrow the search with additional terms. Withholding information based on some guess at my political leanings is dishonest. I'd be appalled if a librarian did it; why should I expect less from a search engine?
  - yanw 15y ago
    The 'political leaning results' are just a tool the author used to try to explained his (ill-informed) point, there are no evidence to any such degree of personalization specially not to any 'bubble' inducing extent, it's all very sensationalistic. And for the record the political example was about facebook wall.
  - Uchikoma 15y ago
    What does faithful mean? You want German or Chinese results? You want global or local results (in your language of choice). There is always personalization in search.
  - SoftwareMaven 15y ago
    If you went to a librarian and said "show me Egypt", she would have nowhere to start. A conversation would then ensue about what you are interested in. Keep going to the library, keep having those conversations, and eventually, the librarian will take you straight to the books you want when you ask for Norway. If you want something different, you'll have to explicitly ask. What Google is doing is no different. It just happens to have a lot more conversations with you.

speckledjim 15y ago

Spreading FUD about tracking search terms etc didn't seem to help DDG last time. I doubt this will have any effect either. DDG: If you want to grow, don't keep attacking the big search engines with FUD that only a miniscule % of people care about.

mkr-hn 15y ago
The February DDG newsletter reported growth in January from 2.5m searches to 5m. I don't think DDG needs FUD to grow.
- speckledjim 15y ago
  http://duckduckgo.com/traffic.html http://duckduckgo.com/traffic.html Searches per day in Feb average = 197,687 Searches per day in May average = 205,759 FUD certainly won't grow DDG out of the extreme niche it's in.
  - mkr-hn 15y ago
    That agrees with what I said. Unless you don't think that much growth in searches over such a short period is significant. It's even more significant when you look at where it was a year ago. 1.2 million in April 2010, 5.9 million in April 2011, 6.3 last month, and already 3.8 a little over halfway through this month. This is significant when you consider the lack of marketing and that DDG is run by one person.
    - Hisoka 15y ago
      It's not significant. In the world of search engines, you need to be the #1 or #2 player to earn major revenue. People aren't gonna wanna waste any time/energy advertising on your platform if you're getting just a handful of searches per month for say... mp3 players or cellphones. DDG is just betting on an acquisition, but it's just a wrapper around the Yahoo BOSS API.
      - 15y ago

Revisor 15y ago

Search queries have different purposes. Personalization based on my preferences and past search works wonders for known-item seeking and for re-finding. But for exploratory search and exhaustive search personalization works as an echo chamber (or filter bubble if you will) . Then there is the problem of change and inertia. People change, their preferences vary in time. Personalization has an inertia that causes the recommendation engine to always be behind the actual preferences. It's more visible the better the recommendation engine. I'm not saying personalization is problematic. There are problems with using it all the time and stealthily, without a clear possibility to turn it off. It's not FUD, but a challenge to overcome.

yanw 15y ago
All can be remedied by tweaking the algorithms and that happens all the time, you can also use a certain vertical directly for certain type queries (scholar search etc). The same way that what a user might enjoy can be inferred the inverse of that can be used as a signal in the algorithm as well. Personalization is just relevance and all is just a matter of tuning.
- Revisor 15y ago
  I agree, it can be remedied by better algorithms - such that can recognize when I'm doing a search for a known item and when I'm performing exploratory search. In the first case, personalize away, in the second case, please show me everything. The challenge is that people switch very quickly between these search modes and don't even think about it.

quinndupont 15y ago

I think you missed the point. Yes DDG ranks, but it may rank differently. Of course, there is an implicit slam on them here: instead of showing you what you want the vast majority of the time, they show you other stuff. This is good on occasion (so you don't live in an echo chamber), but if this was all the time it would mean lower quality search results for simple things.

yanw 15y ago
They show you other stuff Do you have any proof that this isn't what all search engines actually do?
thesz 15y ago
That's quite a point. My own standing is that most of the time you need what you have to have to complete task at hand. Then you could go at YouTube and refresh "Recomendations" to your heart content. The other way around is to wreak havoc to important parts of your life.
rogerbraun 15y ago
This is exactly why I stopped using DDG as my main search engine. My workflow was often like this: - Search for term on DDG - Look at results, find nothing - Try to find better term - Give up and use original term in Google - Find result I tried to use DDG exclusively two time (once 8 months ago, once ~4 months ago), but the result was the same. I don't know how much personalization affects this, but Google just gives me the best results.
- MetallicCloud 15y ago
  I have had almost the exact same experience. I want to use DDG, but I too regularly had to use Google after I couldn't find the results I was after.

kitsune_ 15y ago

No, no, no, no, no, no, no. "There is little difference between personalization and the relevance of search results." Even if you're a Sailor Moon fan, you might actually be looking for information about the rock rotating around this planet when searching for "moon". In the same vein I might be a liberal but be looking for a multitude of opposing views and ideas when googling "core values of NRA members". Oh btw, we engineers, our algorithms, aren't smart enough to foretell people's inclinations based on their past habits, we also can't read minds. The increasing use of personalization as a factor for relevancy will, imho, lead to user dissatisfaction, precisely because they make shoddy assumptions. Believe it or not, but some people also want to be surprised and learn new things. YES. We exist!

glassx 15y ago

Maybe we don't want a filter that sucks, plain and simple? Whether the filter sucks or not is completely subjective, but it sucks for me, so there. Changing the language not that intuitive in Google, it's a drag. I have to do it all the time when I clean my cookies, and sometimes I still get crappy "pt.wikipedia.org" results on top instead of "en.wikipedia.org" just because my browser isn't in english and I don't want to change it just because of Google. I might want to type www.google.it and search for Berlusconi news in Italia, but it won't let me, because it just show me brazilian news. So yeah, I can't even get to another bubble. Also, apparently Google filters by IP, since searching via Tor or a VPS renders very different results. I dislike this for personal reasons because it breaks the internet for me. Also, I might not like the fact that Google collects stuff about my search habits, which is also a valid concern. So, there, it's an issue for me. Might be an edge case, might be an exception, but I think that I deserve to know why this happens and the alternatives.

gnosis 15y ago

"There is little difference between personalization and the relevance of search results. How would you go about ranking then? alphabetically?!" Oh, you know, maybe using Google's much vaunted "PageRank" algorithm, which is supposed to take in to account things such as links on other pages back to the page being considered for inclusion in the search results.. the more such links there are, the better the rank is supposed to be. The above description is obviously an oversimplification, and I don't have access to Google's PageRank algorithm anyway, so couldn't tell you what it actually was if I wanted to, but it's something along those lines, and need not take in to account your previous search history in any way or "personalize" the search for you.