3 ms·

I’m guessing the primary use case for this will be setting cookies. (Google fonts or whatever sends different dictionaries to different clients, the decompress

by hedora 2y ago

I’m guessing the primary use case for this will be setting cookies. (Google fonts or whatever sends different dictionaries to different clients, the decompression result is unique per client). I wonder what percentage of http traffic is redundant compression dictionaries. How much could this actually help in theory?

JoshTriplett 2y ago
> I’m guessing the primary use case for this will be setting cookies. There's a huge difference between "this could potentially be used to fingerprint a client" and "this will primarily be used to fingerprint a client". There are many potential fingerprinting mechanisms that rely on tracking what the client already has in its cache. Clients could easily partition their cached dictionaries by origin, just as they could partition other cached information by origin, and that would prevent using this for cross-origin tracking. This proposal would help substantially with smaller requests, where a custom dictionary would provide better compression but not better enough to offset the full size of the dictionary.
derf_ 2y ago
> I wonder what percentage of http traffic is redundant compression dictionaries. How much could this actually help in theory? A lot of times you will send a (possibly large) blob of text repeatedly with a few minor changes. One example I've used in practice is session descriptions (SDP) for WebRTC. Any time you add/remove/change a stream, you have to renegotiate the session, and this is done by passing an SDP blob that describes all of the streams in a session in and out of the Javascript Session Establishment Protocol (JSEP) API on both sides of the connection. A video conference with dozens of participants each with separate audio and video streams joining one at a time might require exchanging hundreds or even thousands of SDP messages, and in large sessions these can grow to be hundreds of kB each, even though only a tiny portion of the SDP changes each time. Now, you could do a lot of work to parse the SDP locally, figure out exactly what changed, send just that difference to the other side, and have it be smart enough to patch its local idea of the current SDP with that difference to feed into JSEP, test it on every possible browser, make it robust to future changes in the SDP the browser will generate, etc. OR You could just send each SDP message compressed using the last SDP you sent as the initial dictionary. It will compress really well. Even using gzip with the first ~31 kB of the previous SDP will get you in the neighborhood of 200:1 compression[0]. Now your several-hundred kB SDP fits in a single MTU. I'm sure WebRTC is not the only place you will encounter this kind of pattern. [0] Not that either gzip or using part of a document as a dictionary are supported by this draft.
- toast0 2y ago
  > Now, you could do a lot of work to parse the SDP locally, figure out exactly what changed, send just that difference to the other side, and have it be smart enough to patch its local idea of the current SDP with that difference to feed into JSEP, test it on every possible browser, make it robust to future changes in the SDP the browser will generate, etc. I work with WebRTC outside a browser context, and we signal calls with simpler datastructures and generate the SDP right near the WebRTC Api. Our SFU doesn't ever see an SDP, because SDP is a big messy format around simpler parts --- it's easier to just communicate the simpler parts, and marshal into SDP at the API border. Even for 1:1 calls, we signal the parts, and then both ends generate the SDP to feed to WebRTC. IMHO, you're going to have to test your generated SDPs everywhere anyway, regardless of if clients or servers generate them. Well managed compression would certainly help reduce SDP size in transit, but reduce, reuse, recompress in that order.
bawolff 2y ago
Sheesh, the google conspiracy theories are getting out of hand. Why would you use this for setting cookies when you could just use cookies? If for some reason you dont want to use real cookies, why wouldnt you just use the cache side channel directly? This doesn't really add any fingerprinting that doesnt already exist.
- giantrobot 2y ago
  Because groups want to track people without cookies. This re-introduces the problem of shared caches. Site A can know if someone visited Site B by whether or not they have the dictionary for some shared resource. Site A never has to show a cookie banner to do this tracking because there's no cookie set. This is just reintroducing the "feature" of cross-site tracking with shared caches in the browser.
  - bawolff 2y ago
    Are you sure? I would assume that available dictionaries would be partioned by site (eTLD+1).
  - patrickmeenan 2y ago
    Nope. The dictionaries are partitioned the same way as the caches and cookies (whichever is partitioned more aggressively for a given browser). Usually by site and frame so there are no cross-site vectors that it opens.
jepler 2y ago
this was my first thought as well. Authors just acknowledge it and move on; it's not like shopify and google care whether there's another way to successfully track users online. 10. Privacy Considerations Since dictionaries are advertised in future requests using the hash of the content of the dictionary, it is possible to abuse the dictionary to turn it into a tracking cookie.
- patrickmeenan 2y ago
  Which is why they are treated as if they are cookies and are cleared any time the cache or cookies are cleared so that they can not provide an additional tracking vector beyond what cookies can do (and when 3rd party cookies are partitioned by site/frame, they are also partitioned the same). There are LOTS of privacy teams within the respective companies, W3C and IETF that have looked it over to make sure that it does not open any new abuse vectors. It's worth noting that Google, Mozilla and Apple are all supportive of the spec and have all been involved over the last year.
  - patrickmeenan 2y ago
    Sorry, I should provide more context. The language in the IETF draft is a bit generic because it is a HTTP spec intended to be used more broadly than just web content in browsers and each should evaluate the risks for their use case. For browsers specifically, the fetch spec changes will be explicit about the cache clearing and partitioning (partitioned by both top-level document site and frame origin). You can see Chrome's implementation here: https://source.chromium.org/chromium/chromium/src/+/main:net/shared_dictionary/shared_dictionary_isolation_key.cc https://source.chromium.org/chromium/chromium/src/+/main:net... The fetch spec changes are in progress (just documenting, the discussions have already happened). You can follow along here if you'd like: https://github.com/whatwg/fetch/issues/1739 https://github.com/whatwg/fetch/issues/1739
SahAssar 2y ago
This follows the same origin rules as cookies and caching, so it is not useful for tracking anymore than those already are. As the when this is useful there are many examples: * Map tiling (like OSM, mapbox, google maps, etc.) often have many repeated tokens/data but the tiles are served as individual requests * Code splitting of CSS/JS, where they share a lot of the same tokens but you don't want to have to load the full bundle on every load since much of it wont be used * Any site where the same terms and/or parts are recurring across multiple pages (for many sites like IMDB or similar I'd guess 80% of the HTML is non-unique per request) This has been tried before with SDCH which unfortunately died, hopefully this goes better.
- patrickmeenan 2y ago
  FWIW, this addresses the BREACH/CRIME issues that killed SDCH by only operating on CORS-readable content. It also solves the problem that SDCH had where the dictionary would be forced on the response and the client would have to go fetch it if it didn't have it before it could process the response. The tooling for generating dictionaries is also WAY better than it was back in the SDCH days (and they are a much cleaner format, being arbitrary byte strings that Brotli and ZStandard can back-reference). Lots of people involved in working on it were also involved with SDCH and have been trying to find a workable solution since it had to be turned down.
magicalist 2y ago
> I’m guessing the primary use case for this will be setting cookies. The cache is partitioned by document and resource origins, so you might as well just use first party cookies at that point (or etags if you insist on being sneaky).

by hedora 2y ago

JoshTriplett 2y ago

> I’m guessing the primary use case for this will be setting cookies. There's a huge difference between "this could potentially be used to fingerprint a client" and "this will primarily be used to fingerprint a client". There are many potential fingerprinting mechanisms that rely on tracking what the client already has in its cache. Clients could easily partition their cached dictionaries by origin, just as they could partition other cached information by origin, and that would prevent using this for cross-origin tracking. This proposal would help substantially with smaller requests, where a custom dictionary would provide better compression but not better enough to offset the full size of the dictionary.

derf_ 2y ago

> I wonder what percentage of http traffic is redundant compression dictionaries. How much could this actually help in theory? A lot of times you will send a (possibly large) blob of text repeatedly with a few minor changes. One example I've used in practice is session descriptions (SDP) for WebRTC. Any time you add/remove/change a stream, you have to renegotiate the session, and this is done by passing an SDP blob that describes all of the streams in a session in and out of the Javascript Session Establishment Protocol (JSEP) API on both sides of the connection. A video conference with dozens of participants each with separate audio and video streams joining one at a time might require exchanging hundreds or even thousands of SDP messages, and in large sessions these can grow to be hundreds of kB each, even though only a tiny portion of the SDP changes each time. Now, you could do a lot of work to parse the SDP locally, figure out exactly what changed, send just that difference to the other side, and have it be smart enough to patch its local idea of the current SDP with that difference to feed into JSEP, test it on every possible browser, make it robust to future changes in the SDP the browser will generate, etc. OR You could just send each SDP message compressed using the last SDP you sent as the initial dictionary. It will compress really well. Even using gzip with the first ~31 kB of the previous SDP will get you in the neighborhood of 200:1 compression[0]. Now your several-hundred kB SDP fits in a single MTU. I'm sure WebRTC is not the only place you will encounter this kind of pattern. [0] Not that either gzip or using part of a document as a dictionary are supported by this draft.

toast0 2y ago
> Now, you could do a lot of work to parse the SDP locally, figure out exactly what changed, send just that difference to the other side, and have it be smart enough to patch its local idea of the current SDP with that difference to feed into JSEP, test it on every possible browser, make it robust to future changes in the SDP the browser will generate, etc. I work with WebRTC outside a browser context, and we signal calls with simpler datastructures and generate the SDP right near the WebRTC Api. Our SFU doesn't ever see an SDP, because SDP is a big messy format around simpler parts --- it's easier to just communicate the simpler parts, and marshal into SDP at the API border. Even for 1:1 calls, we signal the parts, and then both ends generate the SDP to feed to WebRTC. IMHO, you're going to have to test your generated SDPs everywhere anyway, regardless of if clients or servers generate them. Well managed compression would certainly help reduce SDP size in transit, but reduce, reuse, recompress in that order.

bawolff 2y ago

Sheesh, the google conspiracy theories are getting out of hand. Why would you use this for setting cookies when you could just use cookies? If for some reason you dont want to use real cookies, why wouldnt you just use the cache side channel directly? This doesn't really add any fingerprinting that doesnt already exist.

giantrobot 2y ago
Because groups want to track people without cookies. This re-introduces the problem of shared caches. Site A can know if someone visited Site B by whether or not they have the dictionary for some shared resource. Site A never has to show a cookie banner to do this tracking because there's no cookie set. This is just reintroducing the "feature" of cross-site tracking with shared caches in the browser.
- bawolff 2y ago
  Are you sure? I would assume that available dictionaries would be partioned by site (eTLD+1).
- patrickmeenan 2y ago
  Nope. The dictionaries are partitioned the same way as the caches and cookies (whichever is partitioned more aggressively for a given browser). Usually by site and frame so there are no cross-site vectors that it opens.

jepler 2y ago

this was my first thought as well. Authors just acknowledge it and move on; it's not like shopify and google care whether there's another way to successfully track users online. 10. Privacy Considerations Since dictionaries are advertised in future requests using the hash of the content of the dictionary, it is possible to abuse the dictionary to turn it into a tracking cookie.

patrickmeenan 2y ago
Which is why they are treated as if they are cookies and are cleared any time the cache or cookies are cleared so that they can not provide an additional tracking vector beyond what cookies can do (and when 3rd party cookies are partitioned by site/frame, they are also partitioned the same). There are LOTS of privacy teams within the respective companies, W3C and IETF that have looked it over to make sure that it does not open any new abuse vectors. It's worth noting that Google, Mozilla and Apple are all supportive of the spec and have all been involved over the last year.
- patrickmeenan 2y ago
  Sorry, I should provide more context. The language in the IETF draft is a bit generic because it is a HTTP spec intended to be used more broadly than just web content in browsers and each should evaluate the risks for their use case. For browsers specifically, the fetch spec changes will be explicit about the cache clearing and partitioning (partitioned by both top-level document site and frame origin). You can see Chrome's implementation here: https://source.chromium.org/chromium/chromium/src/+/main:net/shared_dictionary/shared_dictionary_isolation_key.cc https://source.chromium.org/chromium/chromium/src/+/main:net... The fetch spec changes are in progress (just documenting, the discussions have already happened). You can follow along here if you'd like: https://github.com/whatwg/fetch/issues/1739 https://github.com/whatwg/fetch/issues/1739

SahAssar 2y ago

This follows the same origin rules as cookies and caching, so it is not useful for tracking anymore than those already are. As the when this is useful there are many examples: * Map tiling (like OSM, mapbox, google maps, etc.) often have many repeated tokens/data but the tiles are served as individual requests * Code splitting of CSS/JS, where they share a lot of the same tokens but you don't want to have to load the full bundle on every load since much of it wont be used * Any site where the same terms and/or parts are recurring across multiple pages (for many sites like IMDB or similar I'd guess 80% of the HTML is non-unique per request) This has been tried before with SDCH which unfortunately died, hopefully this goes better.

patrickmeenan 2y ago
FWIW, this addresses the BREACH/CRIME issues that killed SDCH by only operating on CORS-readable content. It also solves the problem that SDCH had where the dictionary would be forced on the response and the client would have to go fetch it if it didn't have it before it could process the response. The tooling for generating dictionaries is also WAY better than it was back in the SDCH days (and they are a much cleaner format, being arbitrary byte strings that Brotli and ZStandard can back-reference). Lots of people involved in working on it were also involved with SDCH and have been trying to find a workable solution since it had to be turned down.

magicalist 2y ago

> I’m guessing the primary use case for this will be setting cookies. The cache is partitioned by document and resource origins, so you might as well just use first party cookies at that point (or etags if you insist on being sneaky).