5 ms·

AFAIK Mozilla does the same.

by ahoka 7y ago

amluto 7y ago
No, it doesn’t. The equivalent Firefox feature does not reveal your URLs to Mozilla. https://support.mozilla.org/en-US/kb/how-does-phishing-and-malware-protection-work#w_what-information-is-sent-to-mozilla-or-its-partners-when-phishing-and-malware-protection-are-enabled https://support.mozilla.org/en-US/kb/how-does-phishing-and-m...
jchw 7y ago
Mozilla does not send full URLs to a third party. As mentioned in the thread, they use “4-byte URL hash prefixes.” Documentation: https://developers.google.com/safe-browsing/v4/urls-hashing https://developers.google.com/safe-browsing/v4/urls-hashing (Disclosure: I work for Google but have nothing to do with the safe browsing API.)
- 7373737373 7y ago
  Wait, this is not happening locally, against bloom filters or something?
  - est31 7y ago
    There is a first step which uses bloom filters, and if the filter gives a result, it is hashed and the hash is sent to the google service to double-check. That being said, a four byte / 32 bit hash is enough to almost uniquely identify a website. There number of 32 bit numbers and websites is roughly the same order of magnitude. It's a problem without a good solution because if you create many collisions then you also generate plenty of sites falsely reported as phishing and which admin would want that to happen on their site. If you avoid creation of collisions, you have this identifyability problem. There is this CRLite proposal [1] using layered bloom filters to stop reliance on web services. Maybe it can be adopted for phishing sites, as well. [1]: https://ieeexplore.ieee.org/document/7958597 https://ieeexplore.ieee.org/document/7958597
    - gruez 7y ago
      >That being said, a four byte / 32 bit hash is enough to almost uniquely identify a website. There number of 32 bit numbers and websites is roughly the same order of magnitude. Why not send less bits (eg. 24 bit) of the hash?
      - cyphar 7y ago
        The utility of the hash for a user and some potential attacker is the same. Yes, passing 8 fewer bits means that they have 256 times more possible sites you might have visited. But it also means that you get 256 times as many false positives of websites being labeled as phising. This is what GP meant by it being problem without a good solution. One possible option is to do what haveibeenpwned does, where you give fewer bits and then locally check. That would be a good improvement to the system's privacy, but you probably want to avoid downloading the hashes of every malicious website that starts with the given 3 bytes (I'd assume the list is quite large) for every page load.
        tialaramex 7y ago
        SafeBrowsing already does the thing you're describing as a "possible option". Say I go to https://fakebank.example/security/login https://fakebank.example/security/login and Google has decided all of fakebank.example is a phishing site. My browser computes [among other things] SHA256('fakebank.example') and then it snips off the first four bytes and compares that to a large dataset it got from Google. It fetches updates to this dataset every few hours. Sure enough the four byte prefix is present in the dataset. So, we've got an alarm - it calls Google, but it doesn't tell them it's thinking about https://fakebank.example/security/login https://fakebank.example/security/login at all, it just tells them the 4 byte prefix. Google responds with a list of full SHA256 hashes beginning with that prefix that it considers _right now_ to be phishing. The list might be empty (maybe fakebank.example was actually a Greek yoghurt company subject to a PHP 4.x attack, and they upgraded PHP and removed the phishing site so now it's fine) but if it has the entire SHA256 hash we calculated then I get an alert telling me that my browser thinks this is a phishing site and I might want to not visit.
      - deleted 7y ago

AFAIK Mozilla does the same.

by ahoka 7y ago

AFAIK Mozilla does the same.

No, it doesn’t. The equivalent Firefox feature does not reveal your URLs to Mozilla. https://support.mozilla.org/en-US/kb/how-does-phishing-and-malware-protection-work#w_what-information-is-sent-to-mozilla-or-its-partners-when-phishing-and-malware-protection-are-enabled https://support.mozilla.org/en-US/kb/how-does-phishing-and-m...

jchw 7y ago

Mozilla does not send full URLs to a third party. As mentioned in the thread, they use “4-byte URL hash prefixes.” Documentation: https://developers.google.com/safe-browsing/v4/urls-hashing https://developers.google.com/safe-browsing/v4/urls-hashing (Disclosure: I work for Google but have nothing to do with the safe browsing API.)

7373737373 7y ago
Wait, this is not happening locally, against bloom filters or something?
- est31 7y ago
  There is a first step which uses bloom filters, and if the filter gives a result, it is hashed and the hash is sent to the google service to double-check. That being said, a four byte / 32 bit hash is enough to almost uniquely identify a website. There number of 32 bit numbers and websites is roughly the same order of magnitude. It's a problem without a good solution because if you create many collisions then you also generate plenty of sites falsely reported as phishing and which admin would want that to happen on their site. If you avoid creation of collisions, you have this identifyability problem. There is this CRLite proposal [1] using layered bloom filters to stop reliance on web services. Maybe it can be adopted for phishing sites, as well. [1]: https://ieeexplore.ieee.org/document/7958597 https://ieeexplore.ieee.org/document/7958597
  - gruez 7y ago
    >That being said, a four byte / 32 bit hash is enough to almost uniquely identify a website. There number of 32 bit numbers and websites is roughly the same order of magnitude. Why not send less bits (eg. 24 bit) of the hash?
    - cyphar 7y ago
      The utility of the hash for a user and some potential attacker is the same. Yes, passing 8 fewer bits means that they have 256 times more possible sites you might have visited. But it also means that you get 256 times as many false positives of websites being labeled as phising. This is what GP meant by it being problem without a good solution. One possible option is to do what haveibeenpwned does, where you give fewer bits and then locally check. That would be a good improvement to the system's privacy, but you probably want to avoid downloading the hashes of every malicious website that starts with the given 3 bytes (I'd assume the list is quite large) for every page load.
      - tialaramex 7y ago
        SafeBrowsing already does the thing you're describing as a "possible option". Say I go to https://fakebank.example/security/login https://fakebank.example/security/login and Google has decided all of fakebank.example is a phishing site. My browser computes [among other things] SHA256('fakebank.example') and then it snips off the first four bytes and compares that to a large dataset it got from Google. It fetches updates to this dataset every few hours. Sure enough the four byte prefix is present in the dataset. So, we've got an alarm - it calls Google, but it doesn't tell them it's thinking about https://fakebank.example/security/login https://fakebank.example/security/login at all, it just tells them the 4 byte prefix. Google responds with a list of full SHA256 hashes beginning with that prefix that it considers _right now_ to be phishing. The list might be empty (maybe fakebank.example was actually a Greek yoghurt company subject to a PHP 4.x attack, and they upgraded PHP and removed the phishing site so now it's fine) but if it has the entire SHA256 hash we calculated then I get an alert telling me that my browser thinks this is a phishing site and I might want to not visit.
    - deleted 7y ago