3 ms·

Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet. The

Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet. The most private AI is the one running on my own hardware, not in some giant data center.

froh 2mo ago
where does this factor "1000x" come from? I have doubts.
- raggi 2mo ago
  https://www.jeremykun.com/2024/05/04/fhe-overview/#the-highest-level-view https://www.jeremykun.com/2024/05/04/fhe-overview/#the-highe... > Fourth, there is a bandwidth concern. FHE encryption schemes generally increase the size of the data being encrypted, and the user must send the server a special set of encryption keys to enable the computation, which are relatively large as well. The special keys need only be generated and sent once and can be used for all future computations, but they can easily be gigabytes in size. In one example FHE scheme with lightweight keys, a ciphertext encrypting a single integer is on the order of 25 KB, and the special keys are about 0.5 GB. In others, 16,000 or more integers are packed into a single ciphertext of similar size, but the keys can be 10s of GiBs.
  - pamcake 2mo ago
    I'm still a bit skeptic - 1000x sounds overly optimistic and only looking at encryption transformation without any operation is already at least 10000x with scheme mentioned in that quote. Sibling comment estimates lower bounds of current research at minimun 10^6 overhead which sounds more realistic.
- pamcake 2mo ago
  Most likely from above in thread. There is no reason to believe it should be lower than that - or even that low. Or do you have access to research claiming such achievements?
  - froh 2mo ago
    maybe I simply don't know how this works ;-) I was very much surprised and asked. give me demerits for the way of asking. but the question stays: how come an encryption scheme inflates data by this order of magnitude and needs GB sized keys? where can I learn about this? not the nutty gritty details proofs and all but an overview. assume I did my CS masters in the 1990s and worked as SW eng ever since. NVM, I asked Gemini https://share.google/aimode/pqZO1VF3cGTeetamq https://share.google/aimode/pqZO1VF3cGTeetamq
- meindnoch 2mo ago
  Yeah, I have doubts too. It's much higher than that.
Eueudhsbsj32 2mo ago
Aren't there already much more efficient ways to make inference private? Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/ https://tinfoil.sh/
- siddthesquid 2mo ago
  If I used regular encryption to send my credit card information to an AI with fraud detection, the provider still needs to decrypt that data on their side at some point before it goes into the AI. Using this other encryption, the provider has neither need nor capability to decrypt it on their end, so the user gets extra security.
  - llleeeoooh 2mo ago
    this is not entirely true, I think. secure enclaves can provide guarantee such that even the host machine cannot inspect the contents within the VM. so even though the AI model itself needs to see plaintext, all is happening in the enclave which the provider cannot see. the main difference is where the guarantee comes from. for FHE, it comes from math, which we trust. for secure enclave, the guarantee comes from Intel/AMD's promise that their hardware is bugless/backdoorless, and that your adversary cannot directly inspect bits in the hardware
    - siddthesquid 2mo ago
      - If an AI provider has control over the AI algorithm running in the secure enclave, they can easily have functions that provide them the plaintext through a separate channel. secure enclave does not prevent that - The output can reveal information to the provider, which homomorphic encryption would have protected - Inference is running on GPUs - so its moreso nvidia than amd/intel, but this is just a nit So homomorphic encryption exists so the user doesn't need to do work to figure out if the provider could be adversarial.
      - llleeeoooh 2mo ago
amelius 2mo ago
> The most private AI is the one running on my own hardware, not in some giant data center. I want that too, but you gotta ask yourself the question how efficient that is compared to running it in a datacenter shared with everybody else.
- lupire 2mo ago
  It doesn't matter how efficient it is because the user base for that workflow is microscopic
- adgjlsfhk1 2mo ago
  it's more efficient than paying 1000x for fhe
- Chris2048 2mo ago
  Energy efficient, yes, but when you want to keep a query/data private it's maybe worth the extra $ KW. Chicken pie recipes and google AI-search can still go though the datacentres. As an aside: The computation might also not be the same e.g. ever-changing hidden pre-prompts, security/safety checks blocking or degrading responses, unavoidable verbosity to simple questions, watermarking, collection of prompt data to build user profiles for the purpose of advertising - and we haven't even seen in-response adverts, or sponsor-biased responses yet, but no doubt it's coming.
deleted 2mo ago
[deleted]

froh 2mo ago

where does this factor "1000x" come from? I have doubts.

raggi 2mo ago
https://www.jeremykun.com/2024/05/04/fhe-overview/#the-highest-level-view https://www.jeremykun.com/2024/05/04/fhe-overview/#the-highe... > Fourth, there is a bandwidth concern. FHE encryption schemes generally increase the size of the data being encrypted, and the user must send the server a special set of encryption keys to enable the computation, which are relatively large as well. The special keys need only be generated and sent once and can be used for all future computations, but they can easily be gigabytes in size. In one example FHE scheme with lightweight keys, a ciphertext encrypting a single integer is on the order of 25 KB, and the special keys are about 0.5 GB. In others, 16,000 or more integers are packed into a single ciphertext of similar size, but the keys can be 10s of GiBs.
- pamcake 2mo ago
  I'm still a bit skeptic - 1000x sounds overly optimistic and only looking at encryption transformation without any operation is already at least 10000x with scheme mentioned in that quote. Sibling comment estimates lower bounds of current research at minimun 10^6 overhead which sounds more realistic.
pamcake 2mo ago
Most likely from above in thread. There is no reason to believe it should be lower than that - or even that low. Or do you have access to research claiming such achievements?
- froh 2mo ago
  maybe I simply don't know how this works ;-) I was very much surprised and asked. give me demerits for the way of asking. but the question stays: how come an encryption scheme inflates data by this order of magnitude and needs GB sized keys? where can I learn about this? not the nutty gritty details proofs and all but an overview. assume I did my CS masters in the 1990s and worked as SW eng ever since. NVM, I asked Gemini https://share.google/aimode/pqZO1VF3cGTeetamq https://share.google/aimode/pqZO1VF3cGTeetamq
meindnoch 2mo ago
Yeah, I have doubts too. It's much higher than that.

Eueudhsbsj32 2mo ago

Aren't there already much more efficient ways to make inference private? Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/ https://tinfoil.sh/

siddthesquid 2mo ago
If I used regular encryption to send my credit card information to an AI with fraud detection, the provider still needs to decrypt that data on their side at some point before it goes into the AI. Using this other encryption, the provider has neither need nor capability to decrypt it on their end, so the user gets extra security.
- llleeeoooh 2mo ago
  this is not entirely true, I think. secure enclaves can provide guarantee such that even the host machine cannot inspect the contents within the VM. so even though the AI model itself needs to see plaintext, all is happening in the enclave which the provider cannot see. the main difference is where the guarantee comes from. for FHE, it comes from math, which we trust. for secure enclave, the guarantee comes from Intel/AMD's promise that their hardware is bugless/backdoorless, and that your adversary cannot directly inspect bits in the hardware
  - siddthesquid 2mo ago
    - If an AI provider has control over the AI algorithm running in the secure enclave, they can easily have functions that provide them the plaintext through a separate channel. secure enclave does not prevent that - The output can reveal information to the provider, which homomorphic encryption would have protected - Inference is running on GPUs - so its moreso nvidia than amd/intel, but this is just a nit So homomorphic encryption exists so the user doesn't need to do work to figure out if the provider could be adversarial.
    - llleeeoooh 2mo ago

amelius 2mo ago

> The most private AI is the one running on my own hardware, not in some giant data center. I want that too, but you gotta ask yourself the question how efficient that is compared to running it in a datacenter shared with everybody else.

lupire 2mo ago
It doesn't matter how efficient it is because the user base for that workflow is microscopic
adgjlsfhk1 2mo ago
it's more efficient than paying 1000x for fhe
Chris2048 2mo ago
Energy efficient, yes, but when you want to keep a query/data private it's maybe worth the extra $ KW. Chicken pie recipes and google AI-search can still go though the datacentres. As an aside: The computation might also not be the same e.g. ever-changing hidden pre-prompts, security/safety checks blocking or degrading responses, unavoidable verbosity to simple questions, watermarking, collection of prompt data to build user profiles for the purpose of advertising - and we haven't even seen in-response adverts, or sponsor-biased responses yet, but no doubt it's coming.

deleted 2mo ago

[deleted]