4 ms·
"This is not to say that DDG is perfect: links you click do seem to be redirected through a /l/ page on their domain ..." I am surprised the user is not compla
by pwdisswordfish2 6y ago
"This is not to say that DDG is perfect: links you click do seem to be redirected through a /l/ page on their domain ..."
I am surprised the user is not complaining about this instead of the favicons. Their privacy policy goes on about the privacy implications of Referer headers and instead of calling out browsers for sending Referer by default, they instead give themselves power to record all the user's clicked results themselves. The Referer problem is something that can be solved by the user at the browser level through, e.g., using a client that does not send Referer, browser extensions/plug-ins that can control headers sent, or perhaps with a local proxy to remove the Referer header.
Unless DDG has changed, these prefixed result URLs are the default. It is possible to get unprefixed result URLs using the "lite" version of DDG however that is not the default. "Privacy-focused" search engine chooses less private default. News at 11.
I recently noticed that DDG has started redirecting queries submitted via POST to /lite/. The redirect is to the same domain. No explanation. I have a custom client that does not follow redirects and I now have to submit two sets of HTTP headers instead of one.
These guys are trying to make money from advertising just like everyone else. They have to be very particular in the methods they use to do it -- check the exceptions in their privacy policy -- but it is the same game. Ads and affiliate links. That sort of business and privacy are always going to be at odds with each other.
- couchand 6y agoUnable to reproduce. Links appear to be plain links and no JS intercepts clicks.
- pwdisswordfish2 6y agoTo get prefixed result URLs, use GET not POST. Try duckduckgo.com/lite/ and lite.duckduckgo.com/lite/ If I recall correctly there is a way to turn off the prefixed result URLs without having to turn on Javascript, by adding/changing a URL parameter, e.g., kh. However that is not the default setting. Not very friendly toward the privacy-conscious user who has Javascript disabled and does not read HN to find out about otherwise undocumented URL parameter usage. For certain queries, I can actually get a different first result based on the HTTP method I use.
- couchand 6y agoI'm not sure what you mean by bringing up HTTP method. The two main ways I know to get to the results page are to enter a query into the website or the browser search bar, and both of those seem to issue GET requests. They also both produce real URLs without any intervention. What defaults are different for you than they are for me?
- pwdisswordfish2 6y agoIf you navigate to https://duckduckgo.com/lite/ https://duckduckgo.com/lite/ or https://lite.duckduckgo.com/lite/ https://lite.duckduckgo.com/lite/, type your query in the form input field, you are using POST. If, OTOH, you were to type your query into the address bar, e.g., https://duckduckgo.com/lite/?q=xyz https://duckduckgo.com/lite/?q=xyz then you are using GET. curl -Huser-agent= https://duckduckgo.com/lite/?q=xyz|grep result-link curl -Huser-agent= -dq=xyz https://duckduckgo.com/lite/|grep result-link
- bscphil 6y agoThis is done only for browsers that don't support setting the referrer policy. https://caniuse.com/referrer-policy https://caniuse.com/referrer-policy. It should not be happening with any modern browser, and (like the other reply) I don't see it in mine (Firefox). See here: https://help.duckduckgo.com/results/rduckduckgocom/ https://help.duckduckgo.com/results/rduckduckgocom/
- pwdisswordfish2 6y agoAre you saying DDG is forcing you to send a User-Agent header? DDG's privacy policy also goes on about the privacy implications of User-Agent headers combined with IP addresses. So let's say I take what they have put in their privacy policy to heart and I stop sending a User-Agent header. In response DDG sends prefixed result URLs? WTF? Using haproxy, for example, I can use a "modern browser" and send no User-Agent header. I still get prefixed result URLs.
- bscphil 6y agoYou can disable this on the settings page. Frankly I'd infinitely rather send my request through a redirect served by DDG than send my referer to every site I visit. I think this is a perfectly sensible default for the average user, who both doesn't block / obscure their user agent and doesn't have referer masking software in place. You're free to disagree, but in that case you can make use of the option to change the default that they provide for you.
- pwdisswordfish2 6y agoThe settings page requires that the user enable Javascript. I would not call haproxy "referer masking software". In any event, a proxy is not even needed. Modern browsers are open source, right? Users can edit the source and remove the code that sends Referer header. Even easier, I wrote my own http client. I can send any header I want, or none at all. According to DDG's privacy policy this is a good thing.