6 ms·

(DuckDuckGo CEO/Founder) Just seeing this and we're looking into this now. This is not intentional. Update: Still investigating, but have made some progress --

by yegg 3y ago

(DuckDuckGo CEO/Founder) Just seeing this and we're looking into this now. This is not intentional. Update: Still investigating, but have made some progress -- Determined that on desktop there was a link to Techdirt up continuously via our About module (when you search for "Techdirt"). And now the traditional web link is back up as well (for desktop and mobile): https://duckduckgo.com/?q=techdirt&ia=web https://duckduckgo.com/?q=techdirt&ia=web.

WheatMillington 3y ago
I thought you barely relied on Bing anymore. At least, that's the claim you've repeatedly made here. This indicates that DDG is little more than a Bing mirror.
- kats 3y ago
  [flagged]
- yegg 3y ago
  I don't believe I have ever made that claim and it isn't true either that we are "little more than a Bing mirror". See other comment on this thread (https://news.ycombinator.com/item?id=36898807 https://news.ycombinator.com/item?id=36898807) but in short it isn't a binary thing and I think the fallacy people keep making is a modern search engine != web index. Over the past fifteen years, search has become "universal" with dozens of indexes and modules throughout the page. Also it is important to understand people click/engage with things on the page roughly half as much each position down, so by the time you get to the bottom it is like 100 times less than the top. This means that things on top, increasingly non-web links, have become more and more important. In this context, as mentioned in the other comment, the largest modules on mobile and desktop we power ourselves, that is, local and knowledge graph. AI will be the same. We do use Bing as the primary source for traditional web links, but not for all and even when we do it sometimes looks different in various ways. In something like this, we can re-insert this link if that is what is needed.
  - Waterluvian 3y ago
    Thanks for looking into this. Would love to learn how a specific query like "site: techdirt.com" got wiped like that. Will you share your findings once it's been identified and resolved?
    - lapcat 3y ago
      I doubt there will be any findings, because the removal from the index was done by Microsoft rather than DuckDuckGo. The postmortem has to come from Microsoft. Likely the only thing that Yegg can do is poke some internal contact at MS and tell them to fix it.
      - Waterluvian 3y ago
        That's a finding, though. "We still depend too much on Bing and so when you search for a website that Microsoft has censored, we can't do any better."
        PeterisP 3y ago
        They can do better, however, it takes time and effort to do that - they can and do augment Bing results with other data sources when needed, but up until recently there was no need to add anything extra with respect to Techdirt; it certainly makes no sense to index all of the internet and try to duplicate all the search results for every search just in case Bing has filtered something out.
        Waterluvian 3y ago
        I feel like that's exactly why they should duplicate all the effort. So that they aren't thrall to Microsoft's choices.
        ribosometronome 3y ago
        I feel like they would love to be able to do that. One potentially relevant line from the article: >I love that first one. Microsoft, a company with a $2.5 trillion market cap, “may not have enough resources” to crawl and index Techdirt? Cool. I have to imagine DDG's valuation wouldn't quite hit 2.5 trillion if it went public. It's not absurd that they aren't able to fully duplicate those efforts (or that they don't need to in order to have a decent product).
- FormerBandmate 3y ago
  Why is Bing doing this? It feels like we’re burying the lede, a $2 trillion company isn’t showing sites that are critical of it on its supposedly unbiased search engine
  - criddell 3y ago
    Does Microsoft actually claim Bing is unbiased?
- pb7 3y ago
  Regardless of what their CEO claims here every single time, if Bing cut off their access, DDG would be dead overnight. All the stuff they claim to do themselves is <1% of whatever value DDG provides. Lucky for them it’s just enough to be able to use terms like “largely” instead of “entirely”, “other sources” instead of “only source”, etc.
  - photonerd 3y ago
    They wouldn’t die overnight at all. They would fast become outdated however, but that’s a different problem. One solvable with another source.
brucethemoose2 3y ago
This is what I love about HN. If some tech kerfuffle is happening, there's a good chance someone involved will see it. Its like a wider ranged subreddit, or a more orderly Twitter.
- rcarmo 3y ago
  Given the "X" rebrand and that HN has a prominent "Y" logo, this made me laugh out loud.
- WheatMillington 3y ago
  Except we're only getting the canned corporate response of "we're looking into it" here.
  - dang 3y ago
    I'm as bored by canned responses as the next person so I have to stick up for yegg here - these aren't canned, they're interesting comments! https://news.ycombinator.com/item?id=36899187 https://news.ycombinator.com/item?id=36899187 https://news.ycombinator.com/item?id=36899072 https://news.ycombinator.com/item?id=36899072 https://news.ycombinator.com/item?id=36898994 https://news.ycombinator.com/item?id=36898994 https://news.ycombinator.com/item?id=36898807 https://news.ycombinator.com/item?id=36898807
  - Springtime 3y ago
    Eh, the HN guidelines say to take things in good faith. Seeing how it plays out seems better than pre-emptively dismissing what could progress things. Edit: results appear on DDG, for me, apparently.
  - HWR_14 3y ago
    I'm not sure how quickly you expect a more detailed response. Acknowledging the report and looking into it is the first step to a resolution.
  - photonerd 3y ago
    Would you like him to write fiction instead? “This shouldn’t happen, we are working out why it did” is probably the only official response possible right now. Elsewhere in the comments he’s addressing misconceptions & outright falsehoods about how DDG works.
mapierce2 3y ago
[flagged]
lnxg33k1 3y ago
Its all fake news/bait titles, good riddance anyway
- dang 3y ago
  Could you please stop posting unsubstantive comments? You've unfortunately been doing it repeatedly. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful. (We definitely don't want fake/bait stuff here either so I share your feeling on that - this just isn't a good way to express it, at least not on HN.)
  - deleted 3y ago
    [deleted]
  - deleted 3y ago
    [deleted]
  - lnxg33k1 3y ago
    Nope I am not really here to satisfy randomers, but feel free to ignore me, or report me to moderators so that they can decide if they want to delete my account? Otherwise just stfu
    - slater 3y ago
      (you're responding to the site's main admin)
      - lnxg33k1 3y ago
        so he could just delete my account without pinching my balls? What are you trying to say? You can't change others to make what you want them to do, you can ignore it, or act in a way that it doesn't affect you, he decided to break my balls with requests even if he could ignore OR EVEN ban me, as he had the power.. unbelievable
bob-09 3y ago
Thank you for the update. I appreciate your considerate answers and time spent here.
andrewshadura 3y ago
Since you're here, I have a bug to report :) I'm in Slovakia. When I look up something that has a related Wikipedia page, the widget cites the English Wikipedia. However, the link leads to the Slovak Wikipedia but to an article that has a name from the English one. Usually things are named differently, so if I click the link, I get a 404.
alexsantos201 3y ago
[dead]
dredmorbius 3y ago
I've found DDG to be extremely responsive to issues in the past. I'd reported a problem with the "lite" interface about two months ago. It was not only acknowledged, but fixed, in just over half an hour: <https://toot.cat/@dredmorbius/110476717889891339 https://toot.cat/@dredmorbius/110476717889891339>
- autoexec 3y ago
  I've had a similar experience, an issue I reported was corrected, but at the same time, they have major problems that have persisted for years but haven't been fixed (the "windows -microsoft" problem) which cause me to G! far more than I'd like.
stanislavb 3y ago
Hi Yegg, I'm sorry to bother you about another issue; however, I'm having a similar experience with DuckDuckGo and Bing. I've exchanged dozens of emails with people from Bing's web team without any success. Long story short - at some people in the past someone proxy-mirrored all content of SaaSHub. You'd open a page like "someshady-proxy-mirror.com/duckduckgo-alternatives", and it will mirror saashub.com/duckduckgo-alternatives. The same happened for all all pages. Soon after that, Bing (and DDG respectively) dropped all SaaSHub links from the index and kept the proxy-mirrored content! WTF. After a week-or-two of "fighting" I managed to block the proxy-mirrors; however, I never got SaaSHub back in Bing's index. I'm a single founder and feel helpless with this Bing/DGG issue. Any help would be appreciated. Thanks.
clove 3y ago
Hey man. I'm DDG user, but video search on DDG has such a poor user experience that I have to bang away to Google and YouTube. Fix it please.

by yegg 3y ago

WheatMillington 3y ago

I thought you barely relied on Bing anymore. At least, that's the claim you've repeatedly made here. This indicates that DDG is little more than a Bing mirror.

kats 3y ago
[flagged]
yegg 3y ago
I don't believe I have ever made that claim and it isn't true either that we are "little more than a Bing mirror". See other comment on this thread (https://news.ycombinator.com/item?id=36898807 https://news.ycombinator.com/item?id=36898807) but in short it isn't a binary thing and I think the fallacy people keep making is a modern search engine != web index. Over the past fifteen years, search has become "universal" with dozens of indexes and modules throughout the page. Also it is important to understand people click/engage with things on the page roughly half as much each position down, so by the time you get to the bottom it is like 100 times less than the top. This means that things on top, increasingly non-web links, have become more and more important. In this context, as mentioned in the other comment, the largest modules on mobile and desktop we power ourselves, that is, local and knowledge graph. AI will be the same. We do use Bing as the primary source for traditional web links, but not for all and even when we do it sometimes looks different in various ways. In something like this, we can re-insert this link if that is what is needed.
- Waterluvian 3y ago
  Thanks for looking into this. Would love to learn how a specific query like "site: techdirt.com" got wiped like that. Will you share your findings once it's been identified and resolved?
  - lapcat 3y ago
    I doubt there will be any findings, because the removal from the index was done by Microsoft rather than DuckDuckGo. The postmortem has to come from Microsoft. Likely the only thing that Yegg can do is poke some internal contact at MS and tell them to fix it.
    - Waterluvian 3y ago
      That's a finding, though. "We still depend too much on Bing and so when you search for a website that Microsoft has censored, we can't do any better."
      - PeterisP 3y ago
        They can do better, however, it takes time and effort to do that - they can and do augment Bing results with other data sources when needed, but up until recently there was no need to add anything extra with respect to Techdirt; it certainly makes no sense to index all of the internet and try to duplicate all the search results for every search just in case Bing has filtered something out.
        Waterluvian 3y ago
        I feel like that's exactly why they should duplicate all the effort. So that they aren't thrall to Microsoft's choices.
        ribosometronome 3y ago
        I feel like they would love to be able to do that. One potentially relevant line from the article: >I love that first one. Microsoft, a company with a $2.5 trillion market cap, “may not have enough resources” to crawl and index Techdirt? Cool. I have to imagine DDG's valuation wouldn't quite hit 2.5 trillion if it went public. It's not absurd that they aren't able to fully duplicate those efforts (or that they don't need to in order to have a decent product).
FormerBandmate 3y ago
Why is Bing doing this? It feels like we’re burying the lede, a $2 trillion company isn’t showing sites that are critical of it on its supposedly unbiased search engine
- criddell 3y ago
  Does Microsoft actually claim Bing is unbiased?
pb7 3y ago
Regardless of what their CEO claims here every single time, if Bing cut off their access, DDG would be dead overnight. All the stuff they claim to do themselves is <1% of whatever value DDG provides. Lucky for them it’s just enough to be able to use terms like “largely” instead of “entirely”, “other sources” instead of “only source”, etc.
- photonerd 3y ago
  They wouldn’t die overnight at all. They would fast become outdated however, but that’s a different problem. One solvable with another source.

brucethemoose2 3y ago

This is what I love about HN. If some tech kerfuffle is happening, there's a good chance someone involved will see it. Its like a wider ranged subreddit, or a more orderly Twitter.

rcarmo 3y ago
Given the "X" rebrand and that HN has a prominent "Y" logo, this made me laugh out loud.
WheatMillington 3y ago
Except we're only getting the canned corporate response of "we're looking into it" here.
- dang 3y ago
  I'm as bored by canned responses as the next person so I have to stick up for yegg here - these aren't canned, they're interesting comments! https://news.ycombinator.com/item?id=36899187 https://news.ycombinator.com/item?id=36899187 https://news.ycombinator.com/item?id=36899072 https://news.ycombinator.com/item?id=36899072 https://news.ycombinator.com/item?id=36898994 https://news.ycombinator.com/item?id=36898994 https://news.ycombinator.com/item?id=36898807 https://news.ycombinator.com/item?id=36898807
- Springtime 3y ago
  Eh, the HN guidelines say to take things in good faith. Seeing how it plays out seems better than pre-emptively dismissing what could progress things. Edit: results appear on DDG, for me, apparently.
- HWR_14 3y ago
  I'm not sure how quickly you expect a more detailed response. Acknowledging the report and looking into it is the first step to a resolution.
- photonerd 3y ago
  Would you like him to write fiction instead? “This shouldn’t happen, we are working out why it did” is probably the only official response possible right now. Elsewhere in the comments he’s addressing misconceptions & outright falsehoods about how DDG works.

mapierce2 3y ago

[flagged]

lnxg33k1 3y ago

Its all fake news/bait titles, good riddance anyway

dang 3y ago
Could you please stop posting unsubstantive comments? You've unfortunately been doing it repeatedly. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful. (We definitely don't want fake/bait stuff here either so I share your feeling on that - this just isn't a good way to express it, at least not on HN.)
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- lnxg33k1 3y ago
  Nope I am not really here to satisfy randomers, but feel free to ignore me, or report me to moderators so that they can decide if they want to delete my account? Otherwise just stfu
  - slater 3y ago
    (you're responding to the site's main admin)
    - lnxg33k1 3y ago
      so he could just delete my account without pinching my balls? What are you trying to say? You can't change others to make what you want them to do, you can ignore it, or act in a way that it doesn't affect you, he decided to break my balls with requests even if he could ignore OR EVEN ban me, as he had the power.. unbelievable

bob-09 3y ago

Thank you for the update. I appreciate your considerate answers and time spent here.

andrewshadura 3y ago

Since you're here, I have a bug to report :) I'm in Slovakia. When I look up something that has a related Wikipedia page, the widget cites the English Wikipedia. However, the link leads to the Slovak Wikipedia but to an article that has a name from the English one. Usually things are named differently, so if I click the link, I get a 404.

alexsantos201 3y ago

[dead]

dredmorbius 3y ago

I've found DDG to be extremely responsive to issues in the past. I'd reported a problem with the "lite" interface about two months ago. It was not only acknowledged, but fixed, in just over half an hour: <https://toot.cat/@dredmorbius/110476717889891339 https://toot.cat/@dredmorbius/110476717889891339>

autoexec 3y ago
I've had a similar experience, an issue I reported was corrected, but at the same time, they have major problems that have persisted for years but haven't been fixed (the "windows -microsoft" problem) which cause me to G! far more than I'd like.

stanislavb 3y ago

Hi Yegg, I'm sorry to bother you about another issue; however, I'm having a similar experience with DuckDuckGo and Bing. I've exchanged dozens of emails with people from Bing's web team without any success. Long story short - at some people in the past someone proxy-mirrored all content of SaaSHub. You'd open a page like "someshady-proxy-mirror.com/duckduckgo-alternatives", and it will mirror saashub.com/duckduckgo-alternatives. The same happened for all all pages. Soon after that, Bing (and DDG respectively) dropped all SaaSHub links from the index and kept the proxy-mirrored content! WTF. After a week-or-two of "fighting" I managed to block the proxy-mirrors; however, I never got SaaSHub back in Bing's index. I'm a single founder and feel helpless with this Bing/DGG issue. Any help would be appreciated. Thanks.

clove 3y ago

Hey man. I'm DDG user, but video search on DDG has such a poor user experience that I have to bang away to Google and YouTube. Fix it please.