5 ms·

In my understanding what Cliqz did, at least in the beginning, was to buy clickstream data and then build an index on top of that. So in a sense they just scrap

In my understanding what Cliqz did, at least in the beginning, was to buy clickstream data and then build an index on top of that. So in a sense they just scraped Googles' search index, as almost all users rely on Google for finding stuff on the web. The clickstream data gives you both the search query and the website(s) users visit after searching, so it's pretty easy to build a search index from that, at least for popular searches (it might be more difficult for the long tail of search queries). A lot of the clickstream data you can buy comes from browser extensions btw, and often gets collected without users knowing about it (looking at you, "Web of Trust"). I think their reliance on such data was the reason Cliqz acquired Ghostery, which also collects a copious amount of "anonymous" data from its users. On one hand it's a neat idea since you're basically standing on Googles' shoulders, on the other hand it's at least questionable for a "privacy-first" company as the generation of the search index is based on personal data mined from (often unwitting) users. That said I don't know how their system evolved, so maybe today they have another way to build their index.

sytse 6y ago
Bing might have also done this to improve their index https://searchengineland.com/google-bing-is-cheating-copying-our-search-results-62914 https://searchengineland.com/google-bing-is-cheating-copying...
- pastrami_panda 6y ago
  That's one side of it. The ironic thing is they probably used exactly the same tactic as the search engine in the article: https://www.quora.com/Did-Bing-intentionally-copy-Googles-search-results https://www.quora.com/Did-Bing-intentionally-copy-Googles-se...
  - BrendanEich 6y ago
    No, Brave Search won't copy search results that users do not click on. You own your queries and clicks. Only users who opt into anonymous logging to help Brave Search send unlinkable records up, and those records are not scraped from unclicked links in SERPs.
    - pastrami_panda 6y ago
      I was referring to this bit in the top level comment: > In my understanding what Cliqz did, at least in the beginning, was to buy clickstream data and then build an index on top of that I don't know if that's what cliqz actually did, but if they did do that it sounds very similar to what bing did.
      - BrendanEich 6y ago
        From https://www.siliconfilter.com/hiybbprqag-google-claims-bing-is-copying-its-search-results/ https://www.siliconfilter.com/hiybbprqag-google-claims-bing-... it seems Google engineers laid a trap by using IE with Bing Toolbar and Bing Search Suggestions enabled. Not clear what was gathering the data, but this article doesn't say whether the Google engineers clicked on the bogus-keyword's result link. If they did, then clickstream as you say. If they didn't, something in the IE+Bing mix scraped links from whole Google results pages.
samstave 6y ago
As mentioned in my previous comment: There is a better way to service users interests; initially it was "keywords" - but now it can be more structured; "I want to learn [topic]" and the response may be a step-by-step how-to on how to learn [topic] TBH this was a subject addressed on NPR this morning.. People staying at home are talking about the old infra of edu where people cant be in person - but nobody is talking about the opportunity on changing the structure of learning at all - there should be seen the opportunity on changing the way in which we learn something.
wly_cdgr 6y ago
I work at Ghostery. Yes, Cliqz bought Ghostery for the Human Web data, since we have so many more users than Cliqz ever did. What gives you the impression that any data we are collecting is not appropriately anonymous? The Ghostery extension is open source, so feel free to link to anything in the code that looks suspect to you
- ThePhysicist 6y ago
  I'm not saying it's not anonymous, just that it's impossible to assert the anonymity. Also, I saw a lot of "anonymous" clickstream data offered by other companies, which was often trivial to de-anonymize. We did a DEF CON 25 talk about it, just google "Dark Data DEF CON 25". Robustly anonymizing high-dimensional data like user clickstreams is practically impossible, and often knowing a combination of 4-7 websites a user regularly visits is enough to identify him/her in a pool of millions of users (see the talk for details), so I'm highly doubtful about any company that claims it can robustly anonymize such data. If you're confident your data is anonymous why not release a large sample and have researchers look at it? So while I'm not saying Ghostery is also doing that I don't have a lot of good faith in these data collection practices in general (also, I think before Cliqz acquired Ghostery it collected a lot of data like cookies from the users). Again, it's a smart way to collect data but I wouldn't call it very privacy-friendly.
  - solso 6y ago
    It is trivial to de-anonymize if records are linkable, which is the case you mention on Dark Data DEFCON25. Another famous case was the de-anonymization of the Netflix data set. However, you are assuming that HumanWeb data collection is record-linkable, which is not the case, precisely to avoid this attack. If what is being collected is linkable: e.g. (user_id, url_1), ... (urser_id, url_n). No matter how you anonymize user_id, it will eventually leak. A single url containing personal identifiable information, e.g. a username, will compromise the whole session. No matter how sophisticated the user_id generation is. The real problem, privacy-wise, is the fact that record can be linked to the same origin. An attacker (or the collector) has the ability to know if two records have the same origin. The anonymization of HumanWeb, however, ensures that linkability across data points is not present. Hence, an attacker cannot know if two records come from the same origin. As a consequence, the fact that one url might give away user data, for instance a username, it would not compromise all the urls sent by that person. If you are interested in more details I recommend this article: https://0x65.dev/blog/2019-12-03/human-web-collecting-data-in-a-socially-responsible-manner.html https://0x65.dev/blog/2019-12-03/human-web-collecting-data-i... [Disclaimer I'm one of the authors]

sytse 6y ago

Bing might have also done this to improve their index https://searchengineland.com/google-bing-is-cheating-copying-our-search-results-62914 https://searchengineland.com/google-bing-is-cheating-copying...

pastrami_panda 6y ago
That's one side of it. The ironic thing is they probably used exactly the same tactic as the search engine in the article: https://www.quora.com/Did-Bing-intentionally-copy-Googles-search-results https://www.quora.com/Did-Bing-intentionally-copy-Googles-se...
- BrendanEich 6y ago
  No, Brave Search won't copy search results that users do not click on. You own your queries and clicks. Only users who opt into anonymous logging to help Brave Search send unlinkable records up, and those records are not scraped from unclicked links in SERPs.
  - pastrami_panda 6y ago
    I was referring to this bit in the top level comment: > In my understanding what Cliqz did, at least in the beginning, was to buy clickstream data and then build an index on top of that I don't know if that's what cliqz actually did, but if they did do that it sounds very similar to what bing did.
    - BrendanEich 6y ago
      From https://www.siliconfilter.com/hiybbprqag-google-claims-bing-is-copying-its-search-results/ https://www.siliconfilter.com/hiybbprqag-google-claims-bing-... it seems Google engineers laid a trap by using IE with Bing Toolbar and Bing Search Suggestions enabled. Not clear what was gathering the data, but this article doesn't say whether the Google engineers clicked on the bogus-keyword's result link. If they did, then clickstream as you say. If they didn't, something in the IE+Bing mix scraped links from whole Google results pages.

samstave 6y ago

As mentioned in my previous comment: There is a better way to service users interests; initially it was "keywords" - but now it can be more structured; "I want to learn [topic]" and the response may be a step-by-step how-to on how to learn [topic] TBH this was a subject addressed on NPR this morning.. People staying at home are talking about the old infra of edu where people cant be in person - but nobody is talking about the opportunity on changing the structure of learning at all - there should be seen the opportunity on changing the way in which we learn something.

wly_cdgr 6y ago

I work at Ghostery. Yes, Cliqz bought Ghostery for the Human Web data, since we have so many more users than Cliqz ever did. What gives you the impression that any data we are collecting is not appropriately anonymous? The Ghostery extension is open source, so feel free to link to anything in the code that looks suspect to you

ThePhysicist 6y ago
I'm not saying it's not anonymous, just that it's impossible to assert the anonymity. Also, I saw a lot of "anonymous" clickstream data offered by other companies, which was often trivial to de-anonymize. We did a DEF CON 25 talk about it, just google "Dark Data DEF CON 25". Robustly anonymizing high-dimensional data like user clickstreams is practically impossible, and often knowing a combination of 4-7 websites a user regularly visits is enough to identify him/her in a pool of millions of users (see the talk for details), so I'm highly doubtful about any company that claims it can robustly anonymize such data. If you're confident your data is anonymous why not release a large sample and have researchers look at it? So while I'm not saying Ghostery is also doing that I don't have a lot of good faith in these data collection practices in general (also, I think before Cliqz acquired Ghostery it collected a lot of data like cookies from the users). Again, it's a smart way to collect data but I wouldn't call it very privacy-friendly.
- solso 6y ago
  It is trivial to de-anonymize if records are linkable, which is the case you mention on Dark Data DEFCON25. Another famous case was the de-anonymization of the Netflix data set. However, you are assuming that HumanWeb data collection is record-linkable, which is not the case, precisely to avoid this attack. If what is being collected is linkable: e.g. (user_id, url_1), ... (urser_id, url_n). No matter how you anonymize user_id, it will eventually leak. A single url containing personal identifiable information, e.g. a username, will compromise the whole session. No matter how sophisticated the user_id generation is. The real problem, privacy-wise, is the fact that record can be linked to the same origin. An attacker (or the collector) has the ability to know if two records have the same origin. The anonymization of HumanWeb, however, ensures that linkability across data points is not present. Hence, an attacker cannot know if two records come from the same origin. As a consequence, the fact that one url might give away user data, for instance a username, it would not compromise all the urls sent by that person. If you are interested in more details I recommend this article: https://0x65.dev/blog/2019-12-03/human-web-collecting-data-in-a-socially-responsible-manner.html https://0x65.dev/blog/2019-12-03/human-web-collecting-data-i... [Disclaimer I'm one of the authors]