3 ms·

The Safe Browsing distribution is for efficiency (less bandwidth, less in-memory data to store). The Better Ads Standard list can be obtained here (unhashed): h

The Safe Browsing distribution is for efficiency (less bandwidth, less in-memory data to store). The Better Ads Standard list can be obtained here (unhashed): https://developers.google.com/ad-experience-report/ https://developers.google.com/ad-experience-report/

kodablah 9y ago
Efficiency is distributing the entire list (and they do distribute an indexed and unindexed full EasyList, you can see it in your user's Chome data folder). I doubt it's too big to download in indexed form for a desktop and keep updating with deltas. At the least, I'd like the option. EDIT: Updating from previous statement saying I couldn't find where to get the list. I have now obtained it from [0] and put it at [1] (caution, it's a large gist). 0 - https://developers.google.com/ad-experience-report/v1/reference/rest/v1/violatingSites/list https://developers.google.com/ad-experience-report/v1/refere... 1 - https://gist.github.com/cretz/18594176f791fc0ede26078f76cf1202 https://gist.github.com/cretz/18594176f791fc0ede26078f76cf12...
- evmar 9y ago
  (Disclaimer: haven't worked on Chrome in ~7 years.) This weird hashing scheme comes from safe browsing (which blacklists sites that install malware etc.). I guess (without specific knowledge of it) it was just reused for this ads thing because they had all the code handy for it, both the browser-side code and serving code. For safe browsing, as I recall the data format was designed with Mozilla -- that tech predated the existence of Chrome. There's some history about it here: https://wiki.mozilla.org/Security/Safe_Browsing https://wiki.mozilla.org/Security/Safe_Browsing and https://wiki.mozilla.org/Phishing_Protection:_Design_Documentation https://wiki.mozilla.org/Phishing_Protection:_Design_Documen... I recall the weird hashing scheme was carefully designed to balance some concerns. For example when it phones home, it phones home with a hash of the current URL so that it doesn't reveal the current URL to the server (unless the URL is already in the server-side blacklist). I also think it was intentional that the client didn't get a list of all known-malware URLs. I can't find any design docs for it at the moment better than https://developers.google.com/safe-browsing/v4/ https://developers.google.com/safe-browsing/v4/ . It may well be the case that the hashing scheme doesn't make sense at all in this context.
  - kodablah 9y ago
    The hashing scheme definitely does address privacy concerns. It's that the whole list isn't present and relies on a Google API that troubles me. Surely the list is not too large to download on desktop, but maybe it is. I'd like the option of instead having it all on my desktop and downloading deltas instead of the hash check. The best "design docs" I've found is the golang impl at https://github.com/google/safebrowsing/ https://github.com/google/safebrowsing/.

kodablah 9y ago

Efficiency is distributing the entire list (and they do distribute an indexed and unindexed full EasyList, you can see it in your user's Chome data folder). I doubt it's too big to download in indexed form for a desktop and keep updating with deltas. At the least, I'd like the option. EDIT: Updating from previous statement saying I couldn't find where to get the list. I have now obtained it from [0] and put it at [1] (caution, it's a large gist). 0 - https://developers.google.com/ad-experience-report/v1/reference/rest/v1/violatingSites/list https://developers.google.com/ad-experience-report/v1/refere... 1 - https://gist.github.com/cretz/18594176f791fc0ede26078f76cf1202 https://gist.github.com/cretz/18594176f791fc0ede26078f76cf12...

evmar 9y ago
(Disclaimer: haven't worked on Chrome in ~7 years.) This weird hashing scheme comes from safe browsing (which blacklists sites that install malware etc.). I guess (without specific knowledge of it) it was just reused for this ads thing because they had all the code handy for it, both the browser-side code and serving code. For safe browsing, as I recall the data format was designed with Mozilla -- that tech predated the existence of Chrome. There's some history about it here: https://wiki.mozilla.org/Security/Safe_Browsing https://wiki.mozilla.org/Security/Safe_Browsing and https://wiki.mozilla.org/Phishing_Protection:_Design_Documentation https://wiki.mozilla.org/Phishing_Protection:_Design_Documen... I recall the weird hashing scheme was carefully designed to balance some concerns. For example when it phones home, it phones home with a hash of the current URL so that it doesn't reveal the current URL to the server (unless the URL is already in the server-side blacklist). I also think it was intentional that the client didn't get a list of all known-malware URLs. I can't find any design docs for it at the moment better than https://developers.google.com/safe-browsing/v4/ https://developers.google.com/safe-browsing/v4/ . It may well be the case that the hashing scheme doesn't make sense at all in this context.
- kodablah 9y ago
  The hashing scheme definitely does address privacy concerns. It's that the whole list isn't present and relies on a Google API that troubles me. Surely the list is not too large to download on desktop, but maybe it is. I'd like the option of instead having it all on my desktop and downloading deltas instead of the hash check. The best "design docs" I've found is the golang impl at https://github.com/google/safebrowsing/ https://github.com/google/safebrowsing/.