6 ms·
Cache across domains with a file hash
- finnw 14y agoWhy all the worry about collisions? They will be rare enough that they could be worked around with a hard-coded blacklist in the browser.
- gojomo 14y agoHaving URI types that are content-centric rather than location-centric has been typical of P2P networks for over a decade... but web browser support has been spotty. (It's sometimes been faked via a localhost HTTP proxy.) A new chance for this to change soon may be come alongside the forthcoming browser-to-browser connectivity standards under the heading "WebRTC". Then you won't just have to hope a matching file might be in your local cache... you can cooperate with other nearby nodes to find reliable replicas (like the P2P networks do), triggered by a content-hash-specifying URI scheme.
- Jailbird 14y agoWould this allow some kind of mine-by-SHA cross-site attacks? (sidenote: I've not thought this through, this is just an off-the-cuff thought.)
- alexchamberlain 14y agoIf you could find a malicious file that had the same hash as the current jQuery before the next one was released, then yes. I don't think this is possible, but I'm no expert!
- Jailbird 14y agoI guess I was thinking it'd be used for much more than just jQuery, and fast. Then for anything that any site wanted cached, some other site could essentially grab it if all they had was the hash.
- alexchamberlain 14y agoNot sure I understand. How would another site grab it? And if they imported it into their page, why would that matter?
- saalweachter 14y agoIf I'm reading it correctly, I think the idea is, imagine your bank uses a hash-cache of an AJAX request containing your bank statements (stupid, yeah, but everyone is stupid sometimes). A malicious website could then include a request to download a bogus URL from their site with a hash which matched that of the bank statement AJAX response. The browser would then return the cached bank statement, without ever noticing the URLs were bogus or didn't even point to a file with the desired hash. The malicious website could then upload the bank statements back to itself. This seems like technically an attack path, but it relies on being able to guess a 256-bit hash in a relatively small number of guesses; if you can do that, you might as well just start guessing password hashes and log in to a target website directly.
- RyanMcGreal 14y ago"If you fear SHA-256 collisions but do not keep with you a loaded shotgun at all times, then you are getting your priorities wrong." http://stackoverflow.com/a/4681221 http://stackoverflow.com/a/4681221
- jefffoster 14y agoI think that quote is appropriate if you fear collisions with your own data set. The worry with a cache scheme like this is people would be actively seeking to find cache collisions.
- alexchamberlain 14y agoThat answer clearly explains why you can't, using current techniques and hardware.
- Dylan16807 14y ago> current techniques We're talking about a long-term standard.
- sp332 14y agoDropbox used to be vulnerable to this because of their back-end deduplication. Users could know if a file had been uploaded already by seeing if the dropbox client had to upload it, or if the service "recognized" the hash. http://paranoia.dubfire.net/2011/04/how-dropbox-sacrifices-user-privacy-for.html http://paranoia.dubfire.net/2011/04/how-dropbox-sacrifices-u... Actually an FTC complaint was filed and Dropbox had to give a up a few of the benefits of deduplication.
- __alexs 14y agoAnything like this is going to need a mandatory Cache-control header that indicates whether the content can be shared cross-domain. Otherwise you can just probe the cache for arbitrary resources, similar to stuff like the a:visited browser history attack. Or have we just given up trying to deal with all the privacy leaking side-channels in browsers these days? Another way to do this would be to allow multiple alternative src urls for a resource. Then the browser could pick the best (for as yet undetermined values of best) source the resource. This has the extra bonus of advantage of looking mostly like a normal <link> or <script> tag and doesn't introduce yet another way to probe across domains.
- alexchamberlain 14y agoI'm sure you're right, but I'm not sure I understand. It would only apply to scripts you add the `sha256` attribute to and you would probably only do this for public libraries.
- __alexs 14y agoI can add whatever SHA I want to the tag though. This means I can tell if you've viewed specific images or other resources on 3rd party domains that wouldn't otherwise by in my browser history. Unless the original content source has opted into this kind of behaviour this seems like a privacy leak to me.
- SoftwareMaven 14y agoOnly links with a hash initially get put in the addressable cache. So if I download foo.js without a hash in the script tag, it is effectively a different document than foo.js with the script tag. The cache address hash would include the file hash, if you will. This would effectively mean the person serving the files gets to determine if you are succeptible to this. If I serve you foo.js and don't want others to know you've seen it, I just make sure not to put a hash. Although, overloading the hash this way sounds wrong. I'd rather see a "shared_cache='public|domain|none' property. I still like the consuming HTML being responsible for deciding that, though.
- sehrope 14y agoI'm apprehensive to the idea of a cached version of X for Y based on the hash value (could be secure but assuming eventual collisions in SHA-256 it wouldn't be hard to setup a poisoned copy of a library that would live on for a long time). An alternative use of this would be to validate the script that is being downloaded from the remote source. If you're delegating to Google or any other CDN for a javascript library you really expect it to be the one you requested. Assuming you're not pointing at the HEAD/latest version versus a specific (which would already be pretty amateur for a real site) then you expect it to always be exactly the same (ex: jquery-1.8.1.min.js should always be the same file ... if a bug is fixed it's a new version). It'd be a security improvement to have a way to have the browser verify that. <script src="https://some-cdn.example.com/some-library-1.2.3.js https://some-cdn.example.com/some-library-1.2.3.js hash="[secure hash of contents]"> If the downloaded file's hash does not match the expected one then the browser would flag a security warning or simply not load the file.
- alexchamberlain 14y agoGreat idea! Could the browser include the libraries and authenticate them in some way?
- sehrope 14y agoThat's kind of the idea. The hash would be there so the browser could verify it is what is expected. Assuming the source page is delivered via HTTPS (w/ compression disabled :D) then the client could verify the hash against what it received to confirm it's valid.
- deleted 14y ago[deleted]
- asdfaoeu 14y agoShould be used for static assets over HTTPS. Where the main page could specify the sha for the various resources and avoid bothering with encryption and letting http proxies cache them.
- alexchamberlain 14y agoAre there any security experts that could confirm whether, given current estimates and knowledge, whether a collision attack by a botnet would likely succeed?
- agl 14y agoI've coded something similar up in WebKit and considered landing it in Chrome, although I was more concerned with being able to use HTTP resources securely in HTTPS pages. Firstly you need a way to incrementally verify a file: you don't want to have to download and buffer the whole file before figuring out whether it's good. Thankfully you can do this with Merkle trees: just make them `degenerate' (every left child is a leaf node). If you make the browser cache content addressable (as the post suggests) then you need to do more than look at the caching header. Consider a site with Content Security Policy enabled: If an attacker found an XSS, they could inject <script href="http://correct.origin.com/foo.js http://correct.origin.com/foo.js digest="degen-hash:sha256:123abc...">. That would match the CSP origin restrictions without the server ever having to serve foo.js. (Thanks to abarth for pointing that out.) So I believe content addressable caches would need a separate hash:// URL scheme for this. But I didn't attempt to make the cache content addressable in my code. I don't know what to do about img srcsets[1]. There's no obviously good place to put the hashes in the HTML. And lastly, and possibly fatally for the "secure HTTP resources in HTTPS pages" use case, many networks will now transcode images on the fly (and possibly other files too). So matching a known hash will result in broken pages for users on those networks. [1] http://www.whatwg.org/specs/web-apps/current-work/multipage/embedded-content-1.html#attr-img-srcset http://www.whatwg.org/specs/web-apps/current-work/multipage/...
- deleted 14y ago[deleted]
- alexchamberlain 14y agoI deliberately avoided images in my proposition, because they are rarely shared across domains and are frequently targeted by dodgy proxies.
- agl 14y agoI'm not certain that Javascript isn't similarly `optimised' by such networks, but images are significant for the mixed content case. I believe that we collected some data in Chrome about the efficacy of a content addressable cache in improving density that was underwhelming, despite expectations. I don't have it to hand, but I suspect that it'll appear in a paper at some point. I've not seen data about how well it reduces network loads yet, although I believe the same colleagues were intending to collect that too.
- pornel 14y agoI've proposed that to W3C HTML WG few months ago: http://lists.w3.org/Archives/Public/public-html/2012Jun/0094.html http://lists.w3.org/Archives/Public/public-html/2012Jun/0094...
- sp332 14y agoIf you want to use content-addressable resources, why not use a content-addressable network, like Kademlia?
- WimLeers 14y agoAssuming this is safe and works well, an interesting side-effect is that the need/recommendation of aggregating CSS files would be at least partially obviated.
- 7952 14y agoThis is just another addressing system for the Internet. At some level this has to be about one URL for one resource. The new indexing system just tells you which URL to use. At that point why not just use normal URLs, and implement everything else in Javascript and ETags?
- patio11 14y agoThis trivially exposes browser history to any site on the Internet. That's probably a non-starter. (Edit: I see alex beat me to this, below. He's detailed the attack, too.)
- yason 14y agoSomething like a lightweight low-latency bittorrent for browsers would be the ultimate caching solution.