5 ms·

A short time later they started checking Referer headers; but as I pointed out to them, sending a fake Referer header is easy, so that only helps against the mo

by chubot 12y ago

A short time later they started checking Referer headers; but as I pointed out to them, sending a fake Referer header is easy, so that only helps against the most careless attackers. I suggested several options for solving this, and they chose the simplest: Use a cron job to delete all the generated graphs every minute. If I'm understanding correctly, I think this problem can be elegantly solved with Macaroons: http://research.google.com/pubs/pub41892.html http://research.google.com/pubs/pub41892.html And it's relatively common issue with auth on the web. I think Facebook has (or had) this problem too. You can generally right click and "copy link" to get a .jpg URL, and send around people's pictures without any auth. Basically the problem is when there are two web servers, a "dynamic" one with auth, and a static one that serves images. The static one is often a CDN. Macaroons are basically a simple technique for decentralized auth, involving HMAC chaining. In this setting, the static server would first give the dynamic server a macaroon M authorizing ALL pictures. At serving time, the dynamic server authorizes the user for a particular request. That request will have an <img src="acct123.jpg"> link. The dynamic server will take the macaroon M, and add a CAVEAT that the file must be "acct123.jpg", yielding Macaroon M2. The client gets the restricted macaroon M2 with the HTML, and sends it back to the static server to retrieve the .jpg. The server can 1) verify that M2 is derived from the original M, and 2) read the caveat from the dynamic server, proving that the user was authorized for the image acct123.jpg (and only that image). The HMAC chain is constructed so that the client can't remove the caveat and get access to all pictures. Basically what happened is that the static server DELEGATED auth logic for its resources to the dynamic server. In figure 2 of the paper, the static server would be TS (target service), and the dynamic server is IS (intermediate service). The static server still needs extra code for Macaroons, which existing CDNs and static servers don't currently have. It would be cool to have an Nginx plugin that does this. But the key point is that it is preserving the original intention behind the static/dynamic split: performance. In less performance sensitive context, you would have a web server perform custom auth logic, and then just read() the static file from disk and serve it over HTTP. This is likely to be many times slower than say Nginx. With the Macaroons, you can authorize ONCE in the dynamic server, and then PROVE to the static server that the auth decision was made. So all the .jpg requests can be fast and only hit the static server. The HMAC calculations are just hashing so they are cheap. It is symmetric crypto, with the shared HMAC secret. The paper has some other use cases and is definitely worth a read. I'm thinking about using this technique for a project. I'm interested in opinions from crypto/security folks.

jacquesm 12y ago
That's a neat trick but it requires a lot of underwater stuff that can break and will to some extent defeat the original purpose (speed).
- chubot 12y ago
  I don't think performance is the issue. A heavy static file serving workload (like a CDN) is either I/O bound or memory bound, depending on whether data set can fit in cache and the network situation. The HMAC verification is just doing MD5/SHA1 hashes and comparing for equality, which all CPU (and really CPU, not CPU + memory, since the data is so small). It's tens of microseconds of CPU implemented in JavaScript (Table II). I'm sure it will be single digit microseconds or less implemented in C, so that's at least 100K - 1M req/s. The CPU will be negligible compared to the rest of the workload. The performance issues with naive solutions for auth of static files are pretty different: hitting databases for auth checks, copying data through two process, context switches, etc. Those are things likely to slow down a static file serving workload. There is some extra "complexity", but I think it's almost the simplest solution you can think of for auth, even ignoring performance. It's a lot simpler and more robust than say putting a database in the request path. The implementation isn't as big as it may appear. There is an open source library that is less than 2K lines of C: https://github.com/rescrv/libmacaroons/blob/master/macaroons.c https://github.com/rescrv/libmacaroons/blob/master/macaroons... What I described is probably 200 lines of C or less. Again you are just verifying an HMAC chain. And the code for hash functions is stuff that probably already appears in all web servers anyway. It's using only the simpler "first party caveats" and not the "third party caveats".
DenisM 12y ago
Why not just give every file a name that's a guid or a hash of the content? this works fine with any CDN and it's impossible to guess/fudge. You can only obtain a hash if you have the rights, and while you could share it with others it doesn't seem a bigger security problem than just sharing the picture directly instead of the link. The only problem I see with such approach is lack of expiration policy (other than re-loading the file under a different name every now and again).

by chubot 12y ago

jacquesm 12y ago

That's a neat trick but it requires a lot of underwater stuff that can break and will to some extent defeat the original purpose (speed).

chubot 12y ago
I don't think performance is the issue. A heavy static file serving workload (like a CDN) is either I/O bound or memory bound, depending on whether data set can fit in cache and the network situation. The HMAC verification is just doing MD5/SHA1 hashes and comparing for equality, which all CPU (and really CPU, not CPU + memory, since the data is so small). It's tens of microseconds of CPU implemented in JavaScript (Table II). I'm sure it will be single digit microseconds or less implemented in C, so that's at least 100K - 1M req/s. The CPU will be negligible compared to the rest of the workload. The performance issues with naive solutions for auth of static files are pretty different: hitting databases for auth checks, copying data through two process, context switches, etc. Those are things likely to slow down a static file serving workload. There is some extra "complexity", but I think it's almost the simplest solution you can think of for auth, even ignoring performance. It's a lot simpler and more robust than say putting a database in the request path. The implementation isn't as big as it may appear. There is an open source library that is less than 2K lines of C: https://github.com/rescrv/libmacaroons/blob/master/macaroons.c https://github.com/rescrv/libmacaroons/blob/master/macaroons... What I described is probably 200 lines of C or less. Again you are just verifying an HMAC chain. And the code for hash functions is stuff that probably already appears in all web servers anyway. It's using only the simpler "first party caveats" and not the "third party caveats".

DenisM 12y ago

Why not just give every file a name that's a guid or a hash of the content? this works fine with any CDN and it's impossible to guess/fudge. You can only obtain a hash if you have the rights, and while you could share it with others it doesn't seem a bigger security problem than just sharing the picture directly instead of the link. The only problem I see with such approach is lack of expiration policy (other than re-loading the file under a different name every now and again).