6 ms·

Either you’re very misinformed about what a mirror is, or you’re being extraordinarily disingenuous here. Bing is their initial and primary source of data for

Either you’re very misinformed about what a mirror is, or you’re being extraordinarily disingenuous here. Bing is their initial and primary source of data for web links. Not the only one either, but the primary one. They maintain their own index from it. That’s not “a mirror” by any stretch of the imagination. What appears to have happened here is some form of replication bug where in a way yet to be determined a removal from the original data source was unintentionally replicated. Data synchronization is hard when you’re not directly mirroring data. This kind of bug crops up

kikokikokiko 3y ago
If they're not just a mirror, it seems rather strange that when a specific website disappears from Bing's index, it also disappears from DDG. In my very common sense based view of the situation, if DDG really builds it's own "index" based on a variety of sources, then why did TechDirt completely disappeared from it's search results? DDG can argue all day about how much their "local and knowledge graph" is built in-house, yada yada yada. At the end of the day, a search engine must retrieve the most relevant websites for a given query. And nothing can be more relevant to "site:xxx.xxx.xxx" than the site itself. If it doesn't appear on DDG results, then it's not indexed. And then it clearly shows that DDG is nothing but a fancy mirror to Bing search.
- photonerd 3y ago
  > it seems rather strange […] Not really. That’s the point. Data replication is hard, especially when you have to consistently update from the same data source. Bugs happen.
JeremyNT 3y ago
> Bing is their initial and primary source of data for web links. Not the only one either, but the primary one. They maintain their own index from it. What is their other data source? And how did it come to suffer from the same problem? I keep seeing insinuations that DDG has some other secret source than Bing, but it's always hand waved and never explicitly named. Incidents like this seem to strongly imply that only Bing really matters at the end of the day.
- photonerd 3y ago
  > I keep seeing insinuations […] It’s not an insinuation, it’s a straight fact direct from the founder. That you don’t understand data replication problems—or know the inner workings of a private company’s codebase & data funnel—doesn’t make it some conspiracy. It just means you’re uninformed.
  - JeremyNT 3y ago
    > It just means you’re uninformed. I'm glad to be corrected. What is their other data source?
- autoexec 3y ago
  Yeah DDG has always been weirdly evasive about how their search engine works and where results are coming from. If it makes you feel any better, bing has been caught copying results from Google (https://www.wired.com/2011/02/bing-copies-google/ https://www.wired.com/2011/02/bing-copies-google/) so there's a chance you'll get bing results plus results from whatever other search engine bing copies. Personally, I don't even mind if using DDG is the same thing as searching bing so long as it actually works, but they still can't get a simple search like "office -microsoft" or "headphones -best" right. I even thought they acknowledged the problem and were looking into it at one point.

kikokikokiko 3y ago

If they're not just a mirror, it seems rather strange that when a specific website disappears from Bing's index, it also disappears from DDG. In my very common sense based view of the situation, if DDG really builds it's own "index" based on a variety of sources, then why did TechDirt completely disappeared from it's search results? DDG can argue all day about how much their "local and knowledge graph" is built in-house, yada yada yada. At the end of the day, a search engine must retrieve the most relevant websites for a given query. And nothing can be more relevant to "site:xxx.xxx.xxx" than the site itself. If it doesn't appear on DDG results, then it's not indexed. And then it clearly shows that DDG is nothing but a fancy mirror to Bing search.

photonerd 3y ago
> it seems rather strange […] Not really. That’s the point. Data replication is hard, especially when you have to consistently update from the same data source. Bugs happen.

JeremyNT 3y ago

> Bing is their initial and primary source of data for web links. Not the only one either, but the primary one. They maintain their own index from it. What is their other data source? And how did it come to suffer from the same problem? I keep seeing insinuations that DDG has some other secret source than Bing, but it's always hand waved and never explicitly named. Incidents like this seem to strongly imply that only Bing really matters at the end of the day.

photonerd 3y ago
> I keep seeing insinuations […] It’s not an insinuation, it’s a straight fact direct from the founder. That you don’t understand data replication problems—or know the inner workings of a private company’s codebase & data funnel—doesn’t make it some conspiracy. It just means you’re uninformed.
- JeremyNT 3y ago
  > It just means you’re uninformed. I'm glad to be corrected. What is their other data source?
autoexec 3y ago
Yeah DDG has always been weirdly evasive about how their search engine works and where results are coming from. If it makes you feel any better, bing has been caught copying results from Google (https://www.wired.com/2011/02/bing-copies-google/ https://www.wired.com/2011/02/bing-copies-google/) so there's a chance you'll get bing results plus results from whatever other search engine bing copies. Personally, I don't even mind if using DDG is the same thing as searching bing so long as it actually works, but they still can't get a simple search like "office -microsoft" or "headphones -best" right. I even thought they acknowledged the problem and were looking into it at one point.