6 ms·
Would it be possible to extend this system to allow you to serialize and deserialize the closures? Then you could store them externally in a "real cache" and le
by IgorPartola 12y ago
Would it be possible to extend this system to allow you to serialize and deserialize the closures? Then you could store them externally in a "real cache" and let that take care of the expiration, etc. You could then also open up a nice security hole :)
- vilhelm_s 12y agoI thought the same thing. There certainly have been Scheme systems developed that can serialize closures and continuations (e.g. I found [1], and the systems they cite in the related work section). But it seems tricky to get the garbage collection to work right... somehow you need to tell the program which data should be serialized, and which should be stored in RAM and somehow marked as still being live. For the security aspect, maybe the server could have a secret key and store a MAC along with the serialized string? [1] http://www.iro.umontreal.ca/~feeley/papers/PierardFeeleySW07.pdf http://www.iro.umontreal.ca/~feeley/papers/PierardFeeleySW07...
- pshc 12y agoOh! Taking that a step further, what if you mmap'd an empty file first, say 1MB of zeroes. Then, just start writing closures to it, one after another. When you hit 1MB, use mremap to add another 1MB. And just keep going! The pagefile would become a fossil record of the webserver's access history. The earlier in the file, the older the request, and the less likely it would ever be mapped into physical RAM ever again. On a 64-bit machine with a modern hard drive, you could probably go forever :)
- espeed 12y agoChronicle Map would be great for that: http://openhft.net/products/chronicle-map/ http://openhft.net/products/chronicle-map/
- derefr 12y agoOr you could just URLencode the serialized closure, and sign that URL. Now as long as the page exists, the closure exists!
- rcfox 12y agoThat sounds really dangerous. It probably wouldn't be too hard to inject arbitrary data into the server.
- amock 12y agoSigning the URL should prevent that, but it still seems overly complicated.
- nitrogen 12y agoSigned URLs is how AWS and CloudFront handle access permissions without actually storing the permissions on their servers.
- amock 12y agoYes, but that's a very different kind of system. AWS has to deal with a very large number of servers handling authentication for client that aren't trusted by the credential owner. HN has clients with credentials accessing a single server. It would be much easier to just store the serialized data in Memcached or Redis since the system is already centralized and use a token to look it up. Doing so requires less bandwidth and less messing with cryptography.
- IgorPartola 12y agoPersonally, I would be ok with a system I am responsible for giving the client a signed or encrypted token. I would not trust many (any?) crypto systems to have signed/encrypted arbitrary code passed to me from an untreated client.
- derefr 12y agoNote that the client is passing you a token with your signature on it, not the client's signature. This isn't PKI, or even shared-secret encryption like RSA; this is the client receiving an opaque blob and then passing it directly back to the server, and the server verifying the URL's HMAC to prove that A. the non-HMAC part of the URL is byte-for-byte identical to the one the HMAC claims it is; and B. the HMAC contains the server's secret. I really like this approach, myself (it works really well in Erlang, where closures can be serialized like any other term), so I'll argue in favor of it for a bit: 1. OS package managers (especially those that provide automatic security updates) are, effectively, arbitrary code execution limited solely by signature verification. If you don't trust signature verification, you basically can't trust OS update infrastructure. And these are actually less secure than URL signing, when you think about it: with a signed URL, you are the signatory, and it's very easy to know if you are you. With OS updates, you have to trust the OS manufacturer has itself granted trusts only to the right entities. (Microsoft could put an update signing key from law enforcement into Windows, letting them push automatic wiretap/rootkit "updates" to selected individuals, etc.) In other words, the security of a system is derived from its weakest link—and there are links far weaker than URL signing. 2. People already do this a ton—deserializing an opaque blob of data signed by the server and then treating it as if it was something just sitting in the server's memory to begin with. Where? In "signed-cookie session storage", the default session mechanism of both Rails and Django. The only difference is that you're putting the information in the URL (where it belongs, in this case) instead of the session—although you could just as well store a continuation table in the session, and then reference it from the URL, if you liked. Okay, there's also the fact that you're storing a serialized closure instead of a public route—but in business terms, that's no more dangerous to e.g. the valuable information in your database than storing the user's effective UID in signed-cookie session storage, presuming you have administrator-role users in your system with the ability to delete that data. The one difference might be if the limited set of all your public API endpoints acts as a slapshod "sandbox" for your server, with you trusting that sandbox to protect your system. Which is to say, if your server can do more harm by executing arbitrary code than an administrator user can do by sending messages to it, you should really look into Docker/BSD jails/etc.