5 ms·

Yeah... this kinda seems like an overly sensationalized piece that's just much ado about nothing. From what the article claims vs. what Apple responded with, it

Yeah... this kinda seems like an overly sensationalized piece that's just much ado about nothing. From what the article claims vs. what Apple responded with, it sounds like they're interpreting Apple's statement in the most negative manner possible. What's probably actually happening, if Apple is being honest about how similar it is to Google's use of the prefixes, is that the prefixes are sent to Google/Tencent and a bulk list is sent back. Safari checks the list against the actual hash completely locally and blocks whatever matches the list. So, for example, if someone wanted to visit freehongkong.com with their language set to mainland Chinese, Safari would hash the URL which would give something like "ba7816bf 8f01cfea 414140de 5dae2223 b00361a3 96177a9c b410ff61 f20015ad" and then Google/Tencent would send back every hash and the URL that matches only "ba7816bf" which could be in the order of thousands of URL, or even potentially millions. Safari would then check the URLs that are returned to see which one is the actual matching URL and it blocks the content in the browser without sending any more information to Google/Tencent. Unless Tencent keeps an active profile and uses machine-learning to create models for every single user of its systems and attempts to tie those models to IP addresses that are assumed to be static, this still maintains complete privacy for the end user. I fail to see how this method is in any way equivalent to what the article is claiming.

lallysingh 7y ago
Wouldn't tencent, having a list of politically undesirable domains, already know their hashes? Now the browsers that visit them phone tencent to let them know. This is useful when the actual http(s) transfer is under vpn wrapping. Maybe it won't matter anymore.
- dpkonofa 7y ago
  They would definitely have a list of the domains but a hash isn't just generated from the domain root. They would have to have a hash generated for every single URL and then they would have to collect every single instance where a hash came up as a "hit" for that hash. They'd then have to model these out to determine which hashes were being hit that were related from a specific IP address which, in my opinion, would be very, very computationally expensive.
  - lallysingh 7y ago
    A few permutations of each would do it. http://foo.com http://foo.com, with /index.html, /favicon.ico, etc. You don't have to match all of them to mark a user (who in China is 1:1 to IP, IIRC) as visitor of undesirable site.
    - dpkonofa 7y ago
      That doesn't make sense. http://foo.com/index.html http://foo.com/index.html would hash to something completely different from http://foo.com/favicon.ico http://foo.com/favicon.ico. On top of that, the prefix, which is only the first 32 or 48 bits, would match for thousands of other URLs whose total hash would be completely different. Additionally, hash collisions are a thing so 2 completely different inputs could potentially return the same full hash, much less just the prefix. You would need a very, very, very large database to keep track of all the permutations necessary to determine a domain name.
      - r00fus 7y ago
        But wouldn’t favicon.ico be called from any visit to any of the subpages off the domain’s roots?
      - lallysingh 7y ago
        Or the hashes of all the resources loaded by those pages. That collection of prefixes is going to start getting unique quickly. Edit: example: P(baduser | hashA )= ~0 P(baduser | hashA and hashB and ... hashF) = 0.75 One prefix doesn't do it, but a bunch together in a smallish time period can fingerprint a site quite well. This is analagous to browser fingerprinting.
      - deleted 7y ago
        [deleted]
      - kccqzy 7y ago
        The safe browsing hashing algorithm takes that into account. You are supposed to canonicalize and simplify the URL. For example https://example.com/some/path?k=w#hash https://example.com/some/path?k=w#hash would have the hash removed, query parameter optionally removed, then path components optionally removed, and finally sub domains removed up to the public suffix plus one, and produce multiple hashes (up to 30 of them).
chewbacha 7y ago
In my mind, the big deal is that it means a company is actively facilitating governmental mass surveillance. And yes, I believe China does maintain a record for every citizen, and is using machine learning to score them. https://www.theatlantic.com/international/archive/2018/02/china-surveillance/552203/ https://www.theatlantic.com/international/archive/2018/02/ch...
- dpkonofa 7y ago
  How do you get that from what's happening here? Apple is actually doing the exact opposite of that. They're obfuscating traffic prior to sending it out to the government entity (Tencent) and only sending partial matches of the obfuscation and then processing the return completely on the local machine. If anything, that appears to be them actively fighting governmental mass surveillance. Even if they do maintain a record of IP addresses, the model for something like that based on URL hash prefixes alone would need to be staggeringly large. They'd probably need more storage and computing power than is available right now. I'm really curious how you've come to that conclusion and I mean that sincerely. What makes you say that?
  - zwaps 7y ago
    They are sending data to what is essentially a state-owned company that can facilitate building user profiles (with enough compute) and is, besides that, explicitly used to censor the web. What is not to understand? Apple should send precisely nothing to the CCP if they care about their users privacy on principle.
    - dpkonofa 7y ago
      That’s the entire purpose of this system. It’s useless for creating user profiles.
      - zwaps 7y ago
        The purpose of this system is to censor the internet, no? Isn't that enough? But that being said, I do not believe that hashes are enough, because all they really need is to know you visited one (hash of) a certain website out of thousands. They do not need proof, and they don't need it to work reliably, and they do not need the model to be precise or good. For the system to work in China, this is sufficient. It need not be perfect, it just needs to loom in the background. People will do the rest themselves. This, I think, one should not forget, especially if you really, really don't see an issue with what Apple is doing.
peteretep 7y ago
> this kinda seems like an overly sensationalized piece From The Register?!?! Surely you’re joking!
- dpkonofa 7y ago
  Fair point. lol
SomeOldThrow 7y ago
It’s absolutely not an “overly sensationalized piece” when apple does nothing to offer ip-based controls, such as offering the user a choice of which ips to connect to. You have to install little snitch or get super comfortable with packet fence. Apple certainly has the budget to figure this interface out and has for decades but has not. The issue is not capability but chosen business models.
- millstone 7y ago
  "Offering the user a choice of which IPs to connect to" is absurd given Apple's target market. If you would like to disable this particular check, you can do so in settings on both Mac and iOS.
  - SomeOldThrow 7y ago
    > “Offering the user a choice of which IPs to connect to" is absurd given Apple's target market. How do you figure? What is Apple’s target market anyway, Mr Confidently Speaking On My Grandmother’s Behalf?
jrockway 7y ago
I think it's reasonable to be a little concerned. The IP address does not have to be the identifier, it's just another datapoint for the model. For example, how many people in your residential DHCP pool visit Github and Hacker News every day? Whichever IP has those hashes in its pool is probably "you", so eventually a model can be built that watches you move from IP to IP. (Eventually there could be enough data that it doesn't even need to use the "same DHCP pool" as a heuristic, and it can watch you move between home and work or even on your cross-country trips.) I think it would be difficult to get a strong signal as to what sort of browsing you're doing from this information, but I also think that if life and death ride on what websites you visit, you should turn this off.
- jrockway 7y ago
  Something I should have added to the comment during the editing window: You also have to be careful about targeted malware, which can give up far more information than these hashes. So I am not really sure where the balance is. It is more concerning when Tencent is running the service instead of Google. A well-targeted ad is better than being disappeared.

lallysingh 7y ago

Wouldn't tencent, having a list of politically undesirable domains, already know their hashes? Now the browsers that visit them phone tencent to let them know. This is useful when the actual http(s) transfer is under vpn wrapping. Maybe it won't matter anymore.

dpkonofa 7y ago
They would definitely have a list of the domains but a hash isn't just generated from the domain root. They would have to have a hash generated for every single URL and then they would have to collect every single instance where a hash came up as a "hit" for that hash. They'd then have to model these out to determine which hashes were being hit that were related from a specific IP address which, in my opinion, would be very, very computationally expensive.
- lallysingh 7y ago
  A few permutations of each would do it. http://foo.com http://foo.com, with /index.html, /favicon.ico, etc. You don't have to match all of them to mark a user (who in China is 1:1 to IP, IIRC) as visitor of undesirable site.
  - dpkonofa 7y ago
    That doesn't make sense. http://foo.com/index.html http://foo.com/index.html would hash to something completely different from http://foo.com/favicon.ico http://foo.com/favicon.ico. On top of that, the prefix, which is only the first 32 or 48 bits, would match for thousands of other URLs whose total hash would be completely different. Additionally, hash collisions are a thing so 2 completely different inputs could potentially return the same full hash, much less just the prefix. You would need a very, very, very large database to keep track of all the permutations necessary to determine a domain name.
    - r00fus 7y ago
      But wouldn’t favicon.ico be called from any visit to any of the subpages off the domain’s roots?
    - lallysingh 7y ago
      Or the hashes of all the resources loaded by those pages. That collection of prefixes is going to start getting unique quickly. Edit: example: P(baduser | hashA )= ~0 P(baduser | hashA and hashB and ... hashF) = 0.75 One prefix doesn't do it, but a bunch together in a smallish time period can fingerprint a site quite well. This is analagous to browser fingerprinting.
    - deleted 7y ago
      [deleted]
    - kccqzy 7y ago
      The safe browsing hashing algorithm takes that into account. You are supposed to canonicalize and simplify the URL. For example https://example.com/some/path?k=w#hash https://example.com/some/path?k=w#hash would have the hash removed, query parameter optionally removed, then path components optionally removed, and finally sub domains removed up to the public suffix plus one, and produce multiple hashes (up to 30 of them).

chewbacha 7y ago

In my mind, the big deal is that it means a company is actively facilitating governmental mass surveillance. And yes, I believe China does maintain a record for every citizen, and is using machine learning to score them. https://www.theatlantic.com/international/archive/2018/02/china-surveillance/552203/ https://www.theatlantic.com/international/archive/2018/02/ch...

dpkonofa 7y ago
How do you get that from what's happening here? Apple is actually doing the exact opposite of that. They're obfuscating traffic prior to sending it out to the government entity (Tencent) and only sending partial matches of the obfuscation and then processing the return completely on the local machine. If anything, that appears to be them actively fighting governmental mass surveillance. Even if they do maintain a record of IP addresses, the model for something like that based on URL hash prefixes alone would need to be staggeringly large. They'd probably need more storage and computing power than is available right now. I'm really curious how you've come to that conclusion and I mean that sincerely. What makes you say that?
- zwaps 7y ago
  They are sending data to what is essentially a state-owned company that can facilitate building user profiles (with enough compute) and is, besides that, explicitly used to censor the web. What is not to understand? Apple should send precisely nothing to the CCP if they care about their users privacy on principle.
  - dpkonofa 7y ago
    That’s the entire purpose of this system. It’s useless for creating user profiles.
    - zwaps 7y ago
      The purpose of this system is to censor the internet, no? Isn't that enough? But that being said, I do not believe that hashes are enough, because all they really need is to know you visited one (hash of) a certain website out of thousands. They do not need proof, and they don't need it to work reliably, and they do not need the model to be precise or good. For the system to work in China, this is sufficient. It need not be perfect, it just needs to loom in the background. People will do the rest themselves. This, I think, one should not forget, especially if you really, really don't see an issue with what Apple is doing.

peteretep 7y ago

> this kinda seems like an overly sensationalized piece From The Register?!?! Surely you’re joking!

dpkonofa 7y ago
Fair point. lol

SomeOldThrow 7y ago

It’s absolutely not an “overly sensationalized piece” when apple does nothing to offer ip-based controls, such as offering the user a choice of which ips to connect to. You have to install little snitch or get super comfortable with packet fence. Apple certainly has the budget to figure this interface out and has for decades but has not. The issue is not capability but chosen business models.

millstone 7y ago
"Offering the user a choice of which IPs to connect to" is absurd given Apple's target market. If you would like to disable this particular check, you can do so in settings on both Mac and iOS.
- SomeOldThrow 7y ago
  > “Offering the user a choice of which IPs to connect to" is absurd given Apple's target market. How do you figure? What is Apple’s target market anyway, Mr Confidently Speaking On My Grandmother’s Behalf?

jrockway 7y ago

I think it's reasonable to be a little concerned. The IP address does not have to be the identifier, it's just another datapoint for the model. For example, how many people in your residential DHCP pool visit Github and Hacker News every day? Whichever IP has those hashes in its pool is probably "you", so eventually a model can be built that watches you move from IP to IP. (Eventually there could be enough data that it doesn't even need to use the "same DHCP pool" as a heuristic, and it can watch you move between home and work or even on your cross-country trips.) I think it would be difficult to get a strong signal as to what sort of browsing you're doing from this information, but I also think that if life and death ride on what websites you visit, you should turn this off.

jrockway 7y ago
Something I should have added to the comment during the editing window: You also have to be careful about targeted malware, which can give up far more information than these hashes. So I am not really sure where the balance is. It is more concerning when Tencent is running the service instead of Google. A well-targeted ad is better than being disappeared.