3 ms·

Aren't there already much more efficient ways to make inference private? Using regular encryption and secure enclaves, there are already providers that are rou

Aren't there already much more efficient ways to make inference private? Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/ https://tinfoil.sh/

siddthesquid 1mo ago
If I used regular encryption to send my credit card information to an AI with fraud detection, the provider still needs to decrypt that data on their side at some point before it goes into the AI. Using this other encryption, the provider has neither need nor capability to decrypt it on their end, so the user gets extra security.
- llleeeoooh 1mo ago
  this is not entirely true, I think. secure enclaves can provide guarantee such that even the host machine cannot inspect the contents within the VM. so even though the AI model itself needs to see plaintext, all is happening in the enclave which the provider cannot see. the main difference is where the guarantee comes from. for FHE, it comes from math, which we trust. for secure enclave, the guarantee comes from Intel/AMD's promise that their hardware is bugless/backdoorless, and that your adversary cannot directly inspect bits in the hardware
  - siddthesquid 1mo ago
    - If an AI provider has control over the AI algorithm running in the secure enclave, they can easily have functions that provide them the plaintext through a separate channel. secure enclave does not prevent that - The output can reveal information to the provider, which homomorphic encryption would have protected - Inference is running on GPUs - so its moreso nvidia than amd/intel, but this is just a nit So homomorphic encryption exists so the user doesn't need to do work to figure out if the provider could be adversarial.
    - llleeeoooh 1mo ago
      yes I agree. although just to nitpick, there are ways to prevent side channel attacks within the enclave using software tricks. and output can remain in ciphertext via diffie hellman with client, etc. etc. but you're right that FHE provides a much more concrete guanrantee.
    - Eueudhsbsj32 1mo ago
      > they can easily have functions that provide them the plaintext through a separate channel That's why they provide cryptographic attestation that the open model they're running is exactly what they advertise without any modifications. That combined with GPU confidential compute should protect your LLM prompt and output.
      - catlifeonmars 1mo ago
        They’re not equivalent. One provides a best effort guarantee and is not trivial to implement correctly (it’s the Secure Enclave). I agree that you _could_ do all those things and do them correctly. However there are a lot more points of failure. E2E encryption (including homomorphic encryption) have the nice property that there are much fewer ways for things to fail. (Tangentially, attestation is basically trying to ensure that faults are obvious, but that doesn’t reduce the probability of the faults in the first place).
delusional 1mo ago
There's a much more simple and much more efficient way to do it: Policy and Legal contract.
- Eueudhsbsj32 1mo ago
  A contract doesn't protect your data from government searchs, with or without warrant.
  - epwr 1mo ago
    Wouldn’t a zero data retention agreement be pretty close to this?
    - traceroute66 1mo ago
      > Wouldn’t a zero data retention agreement be pretty close to this? 1. A ZDR clause is "trust me bro". You have zero way of verifying their pinky-promise. 2. A ZDR clause is still subject to the old-classic "government, court or administrative order" catch-all clause. :) 3. "Even with ZDR enabled, Anthropic may retain data where required by law or to address Usage Policy violations. If a session is flagged for a policy violation, Anthropic may retain the associated inputs and outputs for up to 2 years, consistent with Anthropic’s standard ZDR policy." (I quoted Anthropic, I'm sure all the others have similar).
whyage 1mo ago
Check out opaque.co
catlifeonmars 1mo ago
> aren’t there more efficient ways Heh yes absolutely, but there is some nuance. Secure Enclave still requires you to trust the operator and also trust that it’s configured properly, supply chain is secure, etc. The beauty of FHE is that it doesn’t rely on the compute being secure. All you need to secure are things you already have control over as a client. I agree with you it’s still way too slow to be generally useful. (By general, I mean practical for arbitrary computation — you can relax the requirement and have fast homomorphic encryption if you only do specific kinds of operations).
rahkiin 1mo ago
Yes, by running models locally

siddthesquid 1mo ago

If I used regular encryption to send my credit card information to an AI with fraud detection, the provider still needs to decrypt that data on their side at some point before it goes into the AI. Using this other encryption, the provider has neither need nor capability to decrypt it on their end, so the user gets extra security.

llleeeoooh 1mo ago
this is not entirely true, I think. secure enclaves can provide guarantee such that even the host machine cannot inspect the contents within the VM. so even though the AI model itself needs to see plaintext, all is happening in the enclave which the provider cannot see. the main difference is where the guarantee comes from. for FHE, it comes from math, which we trust. for secure enclave, the guarantee comes from Intel/AMD's promise that their hardware is bugless/backdoorless, and that your adversary cannot directly inspect bits in the hardware
- siddthesquid 1mo ago
  - If an AI provider has control over the AI algorithm running in the secure enclave, they can easily have functions that provide them the plaintext through a separate channel. secure enclave does not prevent that - The output can reveal information to the provider, which homomorphic encryption would have protected - Inference is running on GPUs - so its moreso nvidia than amd/intel, but this is just a nit So homomorphic encryption exists so the user doesn't need to do work to figure out if the provider could be adversarial.
  - llleeeoooh 1mo ago
    yes I agree. although just to nitpick, there are ways to prevent side channel attacks within the enclave using software tricks. and output can remain in ciphertext via diffie hellman with client, etc. etc. but you're right that FHE provides a much more concrete guanrantee.
  - Eueudhsbsj32 1mo ago
    > they can easily have functions that provide them the plaintext through a separate channel That's why they provide cryptographic attestation that the open model they're running is exactly what they advertise without any modifications. That combined with GPU confidential compute should protect your LLM prompt and output.
    - catlifeonmars 1mo ago
      They’re not equivalent. One provides a best effort guarantee and is not trivial to implement correctly (it’s the Secure Enclave). I agree that you _could_ do all those things and do them correctly. However there are a lot more points of failure. E2E encryption (including homomorphic encryption) have the nice property that there are much fewer ways for things to fail. (Tangentially, attestation is basically trying to ensure that faults are obvious, but that doesn’t reduce the probability of the faults in the first place).

delusional 1mo ago

There's a much more simple and much more efficient way to do it: Policy and Legal contract.

Eueudhsbsj32 1mo ago
A contract doesn't protect your data from government searchs, with or without warrant.
- epwr 1mo ago
  Wouldn’t a zero data retention agreement be pretty close to this?
  - traceroute66 1mo ago
    > Wouldn’t a zero data retention agreement be pretty close to this? 1. A ZDR clause is "trust me bro". You have zero way of verifying their pinky-promise. 2. A ZDR clause is still subject to the old-classic "government, court or administrative order" catch-all clause. :) 3. "Even with ZDR enabled, Anthropic may retain data where required by law or to address Usage Policy violations. If a session is flagged for a policy violation, Anthropic may retain the associated inputs and outputs for up to 2 years, consistent with Anthropic’s standard ZDR policy." (I quoted Anthropic, I'm sure all the others have similar).

whyage 1mo ago

Check out opaque.co

catlifeonmars 1mo ago

> aren’t there more efficient ways Heh yes absolutely, but there is some nuance. Secure Enclave still requires you to trust the operator and also trust that it’s configured properly, supply chain is secure, etc. The beauty of FHE is that it doesn’t rely on the compute being secure. All you need to secure are things you already have control over as a client. I agree with you it’s still way too slow to be generally useful. (By general, I mean practical for arbitrary computation — you can relax the requirement and have fast homomorphic encryption if you only do specific kinds of operations).

rahkiin 1mo ago

Yes, by running models locally