5 ms·

Understood. Can you clarify how you have the data to provide an auto-suggest?

by ronaldx 12y ago

andrewguenther 12y ago
It looks like it is based on sets of bigrams and trigrams pulled from pages which they have indexed.
- ronaldx 12y ago
  Quick example: "Is my password... cats6plus" does not appear to be pulled from any page(?)
pnathan 12y ago
Not yegg, but if you store a database of queries, along with frequencies, you can do a prefix match on what's already in the search box and return that. I'm speculating that DDG does something that. It doesn't take any PII to do that unless the query itself includes PII. E.g., "websites operated by paul nathan" is pretty PII. :)
- ronaldx 12y ago
  Saving - and now seemingly publishing - search queries is saving and publishing PII. http://donttrack.us http://donttrack.us directly acknowledges that search queries can be used to personally identify individuals. I therefore understood from the later statement on that site, "[ddg doesn't] store any personal information at all", that this would include search queries: ddg almost literally advertises itself on the fact that it doesn't save search queries. (According to the small print, I understood this wrong.) It would be a shame if your explanation was the correct one.
  - dekz 12y ago
    But that is in conjunction with a browser fingerprint right? I don't follow that DDG not being able to save anonymous queries disregards their stance on privacy? Surely for the best experience you'd want DDG to do this anonymously.
  - germano 12y ago
    By the grandparent's description, the difference would be that DDG is not saving the ip address and browser information associated with the search query. If they're only saving the frequency of each search query and nothing else, then there would be no way to see for a particular user/browser/ip address what search queries have been performed because that dimension to the data simply doesn't exist.
  - crazypyro 12y ago
    I think the difference here is that there is no association of the search term with a specific search. If they see a term, they simply add 1 to the counter. Of course, that is probably vastly simplifying it. Basically, your search is thrown into a pool with all of the other searches. It possibly never even has an id associated with it. Just a frequency number. For example, say I start to type "movie t", it'll see that often times after the word "movie", "times" appears at a high frequency so that could be one of the suggestions. I don't see how this introduces a privacy issue unless they are saving the searches by some personal identification number (whether thats an ip, user id, computer, etc), which they specifically deny. Without the specific identifier, there is no way to say it was user A who searched for "movie times" 300 times in a row and not 300 different users searching for "movie times"
  - pnathan 12y ago
    I'm not prepared to defend DDG in depth, as I'm not a representative. Nor is my research area inferring PII. But I want to point out that doing a standard ddg search gives me `https://duckduckgo.com/?q=seattle` https://duckduckgo.com/?q=seattle` , which is a GET command, visible across the entire network as it percolates through, unencrypted.
    - dangrossman 12y ago
      That's a secure URL. HTTP requests and responses are encrypted, and nobody other than you and the server know what they contain. Nobody between you on the network can tell what URLs you're accessing over SSL.
      - alister 12y ago
        I'm sure you realize this, but to clarify a bit for the parent commenter: Although the URL is encrypted, anyone watching the connections on the network can still tell what IP address you're connecting to and usually be able to infer the domain name from the IP address. To state it simple terms: HTTPS protects the URL and all the contents of the connection, but does not protect the fact that you're connecting to a particular domain, the duration of the connection, and the volume of data.

Understood. Can you clarify how you have the data to provide an auto-suggest?

by ronaldx 12y ago

Understood. Can you clarify how you have the data to provide an auto-suggest?

andrewguenther 12y ago

It looks like it is based on sets of bigrams and trigrams pulled from pages which they have indexed.

ronaldx 12y ago
Quick example: "Is my password... cats6plus" does not appear to be pulled from any page(?)

pnathan 12y ago

Not yegg, but if you store a database of queries, along with frequencies, you can do a prefix match on what's already in the search box and return that. I'm speculating that DDG does something that. It doesn't take any PII to do that unless the query itself includes PII. E.g., "websites operated by paul nathan" is pretty PII. :)

ronaldx 12y ago
Saving - and now seemingly publishing - search queries is saving and publishing PII. http://donttrack.us http://donttrack.us directly acknowledges that search queries can be used to personally identify individuals. I therefore understood from the later statement on that site, "[ddg doesn't] store any personal information at all", that this would include search queries: ddg almost literally advertises itself on the fact that it doesn't save search queries. (According to the small print, I understood this wrong.) It would be a shame if your explanation was the correct one.
- dekz 12y ago
  But that is in conjunction with a browser fingerprint right? I don't follow that DDG not being able to save anonymous queries disregards their stance on privacy? Surely for the best experience you'd want DDG to do this anonymously.
- germano 12y ago
  By the grandparent's description, the difference would be that DDG is not saving the ip address and browser information associated with the search query. If they're only saving the frequency of each search query and nothing else, then there would be no way to see for a particular user/browser/ip address what search queries have been performed because that dimension to the data simply doesn't exist.
- crazypyro 12y ago
  I think the difference here is that there is no association of the search term with a specific search. If they see a term, they simply add 1 to the counter. Of course, that is probably vastly simplifying it. Basically, your search is thrown into a pool with all of the other searches. It possibly never even has an id associated with it. Just a frequency number. For example, say I start to type "movie t", it'll see that often times after the word "movie", "times" appears at a high frequency so that could be one of the suggestions. I don't see how this introduces a privacy issue unless they are saving the searches by some personal identification number (whether thats an ip, user id, computer, etc), which they specifically deny. Without the specific identifier, there is no way to say it was user A who searched for "movie times" 300 times in a row and not 300 different users searching for "movie times"
- pnathan 12y ago
  I'm not prepared to defend DDG in depth, as I'm not a representative. Nor is my research area inferring PII. But I want to point out that doing a standard ddg search gives me `https://duckduckgo.com/?q=seattle` https://duckduckgo.com/?q=seattle` , which is a GET command, visible across the entire network as it percolates through, unencrypted.
  - dangrossman 12y ago
    That's a secure URL. HTTP requests and responses are encrypted, and nobody other than you and the server know what they contain. Nobody between you on the network can tell what URLs you're accessing over SSL.
    - alister 12y ago
      I'm sure you realize this, but to clarify a bit for the parent commenter: Although the URL is encrypted, anyone watching the connections on the network can still tell what IP address you're connecting to and usually be able to infer the domain name from the IP address. To state it simple terms: HTTPS protects the URL and all the contents of the connection, but does not protect the fact that you're connecting to a particular domain, the duration of the connection, and the volume of data.