4 ms·
> The more serious issue is that I can't search encrypted email, whether I sent it or received it. The tools to do this simply don't currently exist, but GnuN
by wtbob 13y ago
> The more serious issue is that I can't search encrypted email, whether I sent it or received it.
The tools to do this simply don't currently exist, but GnuNet has developed an approach to convergent encryption of keywords which points to how one might implement this in an email system.
The first step to do this would be to run the email through an extractor. It might do things like extract keywords from the email text (perhaps all non-common words and their frequency), attachment file names, MIME types &c. This extractor would then encrypt or hash these keywords and add an entry into the index for each keyword/email ID pair.
As a simple example, let's take your first paragraph. It might reduce down to the following words/weights:
((convince 1) (couple 1) (decided 1) (effort 2) (encryption.html 1) (fairly 1) (http://www.ocf.berkeley.edu/ http://www.ocf.berkeley.edu/ 1) (http://www.ocf.berkeley.edu/~rwl/ http://www.ocf.berkeley.edu/~rwl/ 1) (http://www.ocf.berkeley.edu/~rwl/encryption.html http://www.ocf.berkeley.edu/~rwl/encryption.html 1) (linked 1) (make 1) (others 1) (pgp 2) (problems 1) (quickly 1) (recently 1) (regularly 1) (run 1) (signature 1) (solve 1) (too 1) (use 3))
You might then hash each element with your keyword hashing key, to get the following:
(("bea916398ce5fafc4497d378fc4555d9" 1) ("c81dc4aa3b1ae95487f288b297f51a6b" 1) ("8b801b4290ce10372b400330c737858b" 1) ("80b3e9090ee572563e67ebdecdb9b936" 2) ("cdd4c4efda80451e4dd1ff340dd2ed17" 1) ("9d4bcdc88159f8d830384f1696c32c19" 1) ("cb7baeb2e45e5e9d69fcf37c4df25669" 1) ("707f687e23bbbce127970361fd6bda10" 1) ("e6656976f2c6944f7aefd16b5786d6cd" 1) ("df9c284344194e1887e56478ec655dda" 1) ("ec8e0c526c105fc8ebb1ebdf293cf717" 1) ("f2d031d514795444be32f18690f6961f" 1) ("270273a76f0d974c2a06f96d40e2430a" 2) ("46bd41b18d2de638bbd3c126cb4ed716" 1) ("d96c629e01788ed763529620f308bd0c" 1) ("2d5fe4b610334160f178a1977b7d1db1" 1) ("211d0c274e8fb59bbcfe6e74241c317a" 1) ("ae5360545c4e008ed683df468136a8b0" 1) ("d6a5babaa2da167ef4548e7c34b49f40" 1) ("adcf3cd322f7ce16ad015f4725ddbda8" 1) ("e7e89d9c445263f9835c483aabf4f453" 1) ("84f0d11faa98a86f9624df86bd71901d" 3))
And then you'd store your document ID (in this case, the paragraph ID) and the weight in the index under each keyword.
To search for "use PGP", you'd extract the keywords "use" and "pgp", then hash them with your hash key to produce "84f0d11faa98a86f9624df86bd71901d" and "270273a76f0d974c2a06f96d40e2430a", look those up in your index, see that paragraph ID and the weights 3 and 1, retrieve the paragraph, decrypt it with your decryption key and you're in business.
Interestingly, this same architecture works securely if the index lives on another system, so long as the keyword hashing is executed on your system and your hash key is itself secure. Thus, there's no good reason that Google couldn't offer keyword-based searching on messages that only you can decrypt—no reason except that they do not wish to.
- malandrew 13y agoI would hope that such a security system also generates noise in the keyword index so that there is plausible deniability built in just like there is with TrueCrypt and fake/hidden partitions.
- wtbob 13y agoYou surely could, although since the hashes are unpredictable to an attacker (due to the secret hashing key) I don't know why it'd matter, except possibly for traffic analysis.
- rwl 13y agoHmm, very interesting! > The tools to do this simply don't currently exist, but GnuNet has developed an approach to convergent encryption of keywords which points to how one might implement this in an email system. What you describe sounds a lot like "fully homomorphic encryption" (given my pretty amateur understanding of that term). Is the technique you describe one that could be layered on top of PGP? Or does it need an entirely different crypto scheme?
- wtbob 13y ago> What you describe sounds a lot like "fully homomorphic encryption" (given my pretty amateur understanding of that term). Well, not really from a technical standpoint. Homomorphic encryption is about me encrypting 1 and 2, handing you #x8495 and #x7835, you executing (homomorphic-add #x8495 #x7835) to get #x1932 and handing it back to me, whereupon I'm able to decrypt it to get 3. You can imagine why this could be cool: there's the potential for someone to be able to perform work on one's behalf without compromising one's privacy (imagine TurboTax working on encrypted data, with Intuit unable to know what one's finances are). The technique I describe is different, but it does enable a similar end result: someone else can run an indexing/search engine on my behalf, and I can consult it to search my data without ever revealing that data to the engine-runner. The engine isn't really able to perform any calculations though. > Is the technique you describe one that could be layered on top of PGP? Well, it would work in parallel to PGP (or any other data-at-rest scheme). PGP would encrypt the plaintext, and this would be used to index the plaintext. GNUnet uses this technique so that two different people can arrive at the same encryption for keyword searches (which of course has vulnerabilities, since one's adversary can perform those same searches); I use it so that the same person at two different times can index and search by keyword, and protect against an adversary through the use of a secret hash key.