9 ms·

Google is making private AI practical with homomorphic encryption

498 points by u1hcw9nx 2mo ago

FloatArtifact 2mo ago
Encryption or not, if it's on somebody else's server, it isn't yours. I don't believe Google has my best interest.
- u1hcw9nx 2mo ago
  With Fully Homomorphic Encryption it's nobody elses. The basic idea of of the project is to remove the need for trust.
  - unsungNovelty 2mo ago
    It's google. They are good at engineering. Not at creating trust. After the bizillionth time they have broken trust, there is no need for benefit of the doubt.
    - tonyhart7 2mo ago
      so what's the option ??? Meta ?? xAi ?? or OpenAI ???
      - unsungNovelty 2mo ago
        Why should there by any reason to look at the above ones? We are talking about this B2C company which effects a lot of our lives. The discussion isn't about that or this. The discussion is about JUST this specific company.
      - deleted 2mo ago
        [deleted]
    - throwaway27448 2mo ago
      There's more to life than branding.
    - Plont 2mo ago
      Yeah. Google is an ad business. Their entire motive for getting invested in AI is ad revenue. We're supposed to believe they just... won't turn on the money fountain? After going into the red for their data center investments? Hell nah. I'd expect this to be something like the Google Ad ID: technically separated from what Google considers personal information, but trivially easy to tie back to an individual person and to other information about that person. They're continually breaking trust by illegally scraping up the internet to feed to their plagiarism machine, which they are now asking us to trust with more data. It's not a compelling arguement.
      - ameliaquining 2mo ago
        This blog post is about an open source project that you can look at for yourself and decide whether it's suitable for your use case.
        unsungNovelty 2mo ago
        Sure. But it will be trickled down into project by google which will effect us all and we won't have a say.
  - thih9 2mo ago
    To what end?
    - u1hcw9nx 2mo ago
      Biometric authentication, credential checking, blind auctions, threat matching (search zero-day signatures without revealing the vectors), private dna matching or running queries of medical symptoms and health records against databases. I think the most immediately useful is checking if a username/password hash appears in a leaked database without revealing which account or password hash is being checked.
      - thih9 2mo ago
        I do like that list and thank you for the explanation. I still worry about how an ad company is going to approach this. Each of these advancements could be used to put ads in more places.
  - FloatArtifact 2mo ago
    It was not my intent to undercut cryptography as a field or in its use. Consider client side scanning for photos before they're encrypted and uploaded to the cloud. I see no difference AI's future. Content, maybe encrypted, but a form of client-side scanning, or the model itself, will report users.
- krunck 2mo ago
  It's FHE for "cryptographically-secure private AI inference" not for every other service where they snoop into your behavioral information.
- kccqzy 2mo ago
  This kind of attitude is really disrespectful of decades of progress in cryptography. Without even considering homomorphic encryption, classic encryption is specifically designed to make intermediate nodes such as ISP dumb pipes that do not know the contents of communication. The ISP can store your communications on their server however they want. If you don’t agree with this model, I’m afraid modern cryptography doesn’t have anything to offer.
  - jewel 2mo ago
    Even if your files are properly encrypted, if you keep them on, say, Google Drive, then you might lose access to them if your account gets inadvertently blocked. I think GP is trying to say something along those lines. Not that encryption isn't trustworthy, but that even with homomorphic encryption you still give up autonomy.
    - kccqzy 2mo ago
      That’s not a problem solvable by cryptography. Your ISP can’t read your traffic, but can cut off your traffic. You can use end-to-end encryption to encrypt your files to upload to Google Drive, but Google can choose to delete your files.
- yjftsjthsd-h 2mo ago
  It can give you confidentiality without availability. Not sure about integrity? Anyways, still useful.
- bitpush 2mo ago
  Unless all of your money is under your mattress, you're already making such decisions with a far more valuable asset.
- furkanturan 2mo ago
  FHE is not always for using somebody else's server; but for sure you do not share keys with that somebody else. Besides, FHE is not always about using somebody else's server. At Belfort, in addition to FHE acceleration, we also explore such uses cases; https://belfortlabs.com/blog/encrypted-fraud-detection-with-swift https://belfortlabs.com/blog/encrypted-fraud-detection-with-... https://belfortlabs.com/blog/belfort-partners-with-lg-on-encrypted-advertising-recommendations https://belfortlabs.com/blog/belfort-partners-with-lg-on-enc...
noident 2mo ago
Does this rely on the Trust Me Bro model, or is there some way for the client to verify that the provider actually isn't able to see your inputs? I want to read a whitepaper but all I can find is the tl;dw conference presentation
- eslaught 2mo ago
  The linked project page [1] claims to be fully homomorphic. Assuming the claim holds (I haven't verified it), then there is provably no way for Google or anyone else to obtain any information from the encrypted data or computation performed on them. FHE is traditionally horrifically slow, so it's hard to imagine running anything beyond toy models with it. They list some applications on the original article page, but (presumably) they must be dramatically stripped down in order to run within any reasonable time budget. This is not going to run anything like a Sol/Opus any time soon. [1]: https://heir.dev/ https://heir.dev/
- boilerupnc 2mo ago
  Related, I'd seen this blog [0] posted on HN a few years back that gave a nice run down on the "programmable cryptography" space which introduce FHE and a few other neat concepts. Really enjoyed the read and learned some new terms. [0] https://0xparc.org/blog/programmable-cryptography-1 https://0xparc.org/blog/programmable-cryptography-1
- joshuamorton 2mo ago
  > Does this rely on the Trust Me Bro model, or is there some way for the client to verify that the provider actually isn't able to see your inputs? The point of FHE is that you only ever send encrypted data. So if you trust that Google hasn't broken the encryption algorithm entirely, they never access the plaintext.
LoganDark 2mo ago
One flaw with FHE is that it guarantees only that you need the key to see the inputs or outputs of the computation, but not necessarily that the computation is the one you want. For example, the computation could be adversarial for certain inputs, or an adversary could insert their own computation first (or last).
- noman-land 2mo ago
  100% not an expert but my understanding was that part of what you are proving by signing the computation is that the computation itself was performed specifically as agreed to. I may be mixing this up with zero knowledge proofs.
  - LoganDark 2mo ago
    If this were the case it would be necessary to send the entire model weights in response to every request which would be a bit inconvenient.
    - drdeca 2mo ago
      Hmm, could one instead of sending the model weights, send like, a merkle tree root for them, not exactly specifying the model, but at least demonstrating that the same model is used each time?
      - LoganDark 2mo ago
        I feel like that'd have to be generated as a side effect of the computation somehow?
- mswphd 2mo ago
  note that this is even true for an honest server. Roughly, FHE computations often require certain bounds on the (encrypted) messages for things like tuning polynomial approximation domains etc. If your messages are out of distribution for the tuned polynomial approximations you'll get back garbage as as result.
Deukhoofd 2mo ago
It sounds neat, but I do wonder how viable this is commercially. How high do we rate the chances that governments around the world will step in before another kind of E2E is rolled out.
- deleted 2mo ago
  [deleted]
filup 2mo ago
Proper encryption means the ciphertext is indistinguishable from noise. So...in order to be able to process on it, you have to make it not indistinguishable from noise. So I take offense to the term FHE. It's a oxymoron. The whole thing immidiatly stands out as a sham to build trust where it's gone.
- pluto_modadic 2mo ago
  they could have gone with an oblivious transfer approach (where it's working on what looks like multiple problems at once, you don't know which)
- drdeca 2mo ago
  Eh? With secret sharing one can do computation on a shared secret where it is provable that no individual party can recover any information about the data with their share alone. I don’t see why you conclude that FHE couldn’t be close to as secure as that. (Like, not information theoretically, but with computationally bounded adversaries.)
  - filup 2mo ago
    I'm not saying that you can't design a system for secure cloud computing.
    - drdeca 2mo ago
      You said that FHE is an oxymoron, seemingly on the basis that in proper encryption the ciphertext is indistinguishable from noise, and you think this can’t be true of FHE. I am arguing to the contrary. In the secret sharing multi-party computation schemes, the individual shares of the secret are random and have no information about the plaintext. I see no reason that FHE can’t have ciphertexts indistinguishable from noise.
- Ar-Curunir 2mo ago
  No, that’s not what proper encryption means. Security for encryption means that cipher texts encrypting distinct messages are indistinguishable. This is called IND-CPA, and FHE satisfies this.
sabretooth1405 2mo ago
My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.
- clayhacks 2mo ago
  Do you think that’s like a fundamental limit or something that will improve with time and new algorithms?
  - bhu8 2mo ago
    I wouldn't be very bullish. Homomorphic encryption got significantly efficient with the first few iterations, but I don't really see the necessary orders of magnitude savings coming soon. You could reduce this by some partial encryption schemes (e.g., for LLMs you need a handful of basic operations) but a better alternative already exists: multi-party computation. Source: I did research in this area in the past.
- dietr1ch 2mo ago
  Exactly my concern, and worse overhead that what I recalled. Cost-wise the only viable private compute is local compute. It's more expensive than cloud, but true private compute in the cloud is definitely pricier.
- abetusk 2mo ago
  1000x slowdown is bad but not a complete deal breaker. Do you have a sense of what a reasonable achievable factor is? Do you have sense for how long before we get to that achievable factor?
  - jacquesm 2mo ago
    It's a ridiculous waste of energy, just use local compute.
    - 2mo ago
Cider9986 2mo ago
This is the same Google that doesn't have e2ee on their password manager by default. Like WTF, it's a password manager.
- amazingamazing 2mo ago
  If you think about it for a moment you will realize the average user does not want that.
  - rexpop 2mo ago
    I suspect that your one moment of thought follows many years of contemplation. Maybe you can fill us in on some of that background.
    - amazingamazing 2mo ago
      How do you make e2e password management work on multiple devices as convenient as not? You cannot. Most people who use Google trust them. And if you do not trust them then why are you using their password manager? You see? Hence why they do not bother. Of course one might read this and say, multi device e2ee is possible! Sure, but can you allow for recovery when people inevitably lose their recovery codes at scale? If you really care about this it is not necessary to go with Google. I stand by my original comment: The average user does not want it.
      - Cider9986 2mo ago
        Literally all major password managers are E2EE except Google's. The UX is fine.
    - giovannibonetti 2mo ago
      E2E encryption means that if the user loses the keys, there is no way to recover that even if they contact support and prove the data belongs to them.
      - 2mo ago
jrm4 2mo ago
Correct. I appreciate the theoretical technology here, but I believe a great deal of harm is done by the fact that people are not likely to understand exactly what this means. Which is to say, I believe that google is strongly implying the falsehood of "no one at Google can read your stuff."
oulipo 2mo ago
Zama.ai is also a player in this space
- furkanturan 2mo ago
  Yes, it is. But it’s important to recognize that there are different FHE schemes. Zama has invested heavily in TFHE and achieved impressive results. However, time has shown that TFHE is not the best fit for encrypted AI workloads, with CKKS and related schemes gaining an advantage with mathematical and algorithmic advances continue. At Belfort we have foot on both sides; FPGA acceleration for integrating to Zama's TFHE-rs, and GPU acceleration for CKKS.
bmenrigh 2mo ago
Has FHE really progressed so far that it's now so efficient that doing computation on an encrypted prompt is feasible? I thought even basic operations like FHE addition were still thousands of times more complex. The only mention in the article I see is: > But while homomorphic encryption has a nontrivial cost overhead, it shifts the capability/privacy trade-off to a question of cost. And the cost of homomorphic encryption is rapidly decreasing. Which doesn't spell out exactly hon "nontrivial" the cost overhead still is.
- sean_pedersen 2mo ago
  There are LLM models which compute only using addition no?
- mswphd 2mo ago
  addition is easy/essentially the same cost as standard (not really, because you have to compute mod p addition rather than mod 2^32, but ignoroing that it's roughly the same). as a general rule multiplication is the difficult part. it's hard to accurately quantify what "nontrivial" cost overheads are because they're very application dependent. for example, things that require encrypted control flow are very hard under FHE. so an encrypted hashmap sounds roughly unimplemnetable. but things that do not require encrypted control flow (e.g. many ML applications) are less bad. this can still be quite bad though. for example, relu is trivial in plaintext. it is hard homomorphically, because the trivial way to write it uses private control flow.
- nl 2mo ago
  It's not running LLMs.
lsb 2mo ago
Google is making private AI practical with Gemma4 something that you can run without an Internet connection. All of the proofs of privacy rely on us getting the math right. All of the privacy from unplugging your internet cable is there by default.
NyxWulf 2mo ago
Maybe I'm not understanding this, but how is it that you can know enough about the data to process it without undermining the fundamental concept of encryption? Isn't encrypted data supposed to be just random noise without the key? The more you know about the underlying data the easier it gets to decrypt? Does this mean someone can just steal your encrypted data and use that to steal your identity without even needing to decrypt it anymore?
- luckydata 2mo ago
  you can produce a correct result without knowing what that result means as long as the person you then provide that result to can decrypt it. So the way it works is the server does the job on data it can't read, sends an answer to the device, the device decrypts it. It's very useful for things like healthcare data for example but also for products like "home assistants", and it's one of the use cases that the team hiring internally was giving in their job req. I tried to apply too but didn't get it unfortunately, this was a very interesting product to work on imho, congrats to whoever got my job :)
- mswphd 2mo ago
  the basic encryption scheme used here is fairly straightforward actually, at least the symmetric encryption version. Let s be a uniformly random, 512-dimensional u32 vector. To encrypt a message m (say a 512-dimensional bit vector for simplicity), you 1. generate a 512 x 512 random (u32) matrix A, and 2. generate a 512-dimensional rounded (to the nearest integer) Gaussian, say of standard deviation 10, e. The ciphertext is then [A, b :=As + e + 2^8 m]. To decrypt, you compute b - As to recover 2^8 m + e. You can then recover m, as e << 2^8 with high probability. Anyway, if you have two of these ciphertexts, you can sum them together to get [A1 + A2, (A1 + A2)s + (e1 + e2) + 2^8 (m1 + m2)] this decrypts to m1 + m2, so you can recover homomorphic sums (or scalings by small integers). Multiplication is more complex, so I won't get into it here. But the high level from the above example is that you could have someone compute arbitrary linear functions of your data without them knowing what your data is.
  - Chris2048 2mo ago
sheesdev 2mo ago
I've published two papers on using HE for ML and it's nice, but also alarming, to see big players like Google and Microsoft making decent tools for performing HE. Although the technology is still much farther out from being commercially viable, it does pose an interesting problem about how these data aggregation companies will utilise a tech that is inherently private.
- tolugenius 2mo ago
  I guess I'm curious from your perspective what and why makes the tech from being commercially viable, and the concerns of big players adopting this (which was always going to happen I imagine, but when shapes a lot of the future of the field).
- j2kun 2mo ago
  One potentially simple answer is: to make things that are otherwise illegal (for regulatory reasons) possible. There are some instances of hand-built cryptographic protocols used in production at Google that use FHE in small doses (e.g., computing a single dot product as part of an interactive protocol), so the big question is whether having a general-purpose tool will expand the feasibility of applying these to other, maybe less critical projects that still want some privacy guarantees but can't afford to hire cryptographers.
traceroute66 2mo ago
Quoted from the blog post: > user-data can be protected from data breaches, but then the service provider cannot provide features that depend on the data, such as spam or virus detection I think they forgot "or advertising" at the end. I don't trust Google. I would much prefer to use on-prem or - at most - one of the secure-enclave providers like Tinfoil[1] or Private Mode[2] [1] https://tinfoil.sh/ https://tinfoil.sh/ [2] https://www.privatemode.ai/ https://www.privatemode.ai/
- flambo 2mo ago
  Secure Enclaves are broken all the time though [1]. They’re also not secure against attacks from a quantum computer (including store now decrypt later attacks). At least with symmetric FHE, anything that happens after encryption requires zero trust. [1] https://sgx.fail/ https://sgx.fail/
  - j2kun 2mo ago
    Indeed, Google itself has a large research group that repeatedly breaks TEEs with side-channels and such.
mohamedkoubaa 2mo ago
Anything to keep people from running local models
nazgulsenpai 2mo ago
Why is the bold letter n lighter than the rest in this page's font?
p0w3n3d 2mo ago
Ideally: we will have private AI Reality: our computers will be used as distributed AI calculators
- hamper653 2mo ago
  Freenet, but for AI inference.
  - p0w3n3d 2mo ago
    That's not what I meant. I meant botnet taken over by Google
nullc 2mo ago
Private AI is practical by running the model locally, every much more so than any homomorphic encryption scheme. So essentially the headline sells this as work to keep your data private, but really it's work to keep the AI-- which was trained on your code and your writing-- private.
drblast 2mo ago
I did some amount of research into the feasibility of PHE and FHE about 20 years ago, and my conclusion at the time was that the space overhead of the encrypted output was a massive bottleneck, which meant that while it was potentially useful in a small number of niche cases it wasn't ever going to be practical for general-purpose computations without a major breakthrough. The gist was I could do an encrypted (int)x + (int)y = (int)z computation, I could encrypt the inputs and then get a result back that was correct, secure, and decryptable, but was like 1MB in size. So, for someone whose knowledge is 20 years outdated and is about Pallier crypto, has that major breakthrough happened?
meindnoch 2mo ago
Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet. The most private AI is the one running on my own hardware, not in some giant data center.
- froh 2mo ago
  where does this factor "1000x" come from? I have doubts.
  - raggi 2mo ago
    https://www.jeremykun.com/2024/05/04/fhe-overview/#the-highest-level-view https://www.jeremykun.com/2024/05/04/fhe-overview/#the-highe... > Fourth, there is a bandwidth concern. FHE encryption schemes generally increase the size of the data being encrypted, and the user must send the server a special set of encryption keys to enable the computation, which are relatively large as well. The special keys need only be generated and sent once and can be used for all future computations, but they can easily be gigabytes in size. In one example FHE scheme with lightweight keys, a ciphertext encrypting a single integer is on the order of 25 KB, and the special keys are about 0.5 GB. In others, 16,000 or more integers are packed into a single ciphertext of similar size, but the keys can be 10s of GiBs.
    - pamcake 2mo ago
      I'm still a bit skeptic - 1000x sounds overly optimistic and only looking at encryption transformation without any operation is already at least 10000x with scheme mentioned in that quote. Sibling comment estimates lower bounds of current research at minimun 10^6 overhead which sounds more realistic.
  - pamcake 2mo ago
    Most likely from above in thread. There is no reason to believe it should be lower than that - or even that low. Or do you have access to research claiming such achievements?
weatherlite 2mo ago
> Google is making private AI practical with homomorphic encryption I don't understand why we need to bring LGBTQ+ into everything
hellohello2 2mo ago
Have all the skeptics in this thread somehow forgot about Moore's law?
- pinkmuffinere 2mo ago
  What, how is that relevant? the transistors on a microchip double about every two years. That’s descriptive, not prescriptive, so whatever you’re trying to imply about the future is kindof a toss up — there’s no promise the parents hold. And, it doesn’t really have to do with tfa, as far as I can tell? Do you perhaps mean Murphy’s law, “whatever can go wrong will go wrong”?
  - hellohello2 2mo ago
    The top comment mentions a ~10^3 overhead. Am I crazy (or already old?) for thinking this is not a very large constant?
    - pinkmuffinere 2mo ago
      Oh I see, you’re arguing that computer growth will make this more attractive. I think it’s a decent argument, although personally I find it unconvincing. More compute will be allocated between all the existing options, and I suspect it will go towards faster/better inference, instead of encryption. For most use cases people just don’t care much about encryption, and when feature prioritization comes around it always gets a back seat
      - hellohello2 2mo ago
        Yes, I think it really depends on the task. Clearly we won't be willing to spend 1000x on tokens for our everyday needs. But what about for the police monitoring conversations? I'd very much like my chats to stay encrypted, but there have been laws pushing against this. Technology like this provides a real solution i.e. you can scan chats for keywords without actually reading them, and get a warrant if you find something.
- 2mo ago
gaigalas 2mo ago
I have an idea for AI companies for tremendous scale, with privacy and all nice things accounted for: - Unstarve the GPU and RAM consumer market. - Let enthusiasts and volunteers quickly ramp up local AI. - Reap the results that the community will most certainly achieve. Don't repeat Microsoft's earlier mistakes. It flourished when it embraced the community and open source. If it had made that move earlier, it would have been unstoppable. Yes, if you can control the entire market that's probably awesome. But it's also full of nonsense risks.
shevy-java 2mo ago
I do not want to make Google more powerful than it already is.
cantalopes 2mo ago
It's a step to provide targeted advertising with mathematically provable "no sensitive info stored" approach. The google must fight really hard because this is the only source of income that makes sense for their position. And it /is/ quite evil tbh
- FartyMcFarter 2mo ago
  How would advertising be implemented in the scenario of using this technique?
sarjann 2mo ago
I think at 1000x it's probably way cheaper to do an "AWS outpost" style thing. Although not sure if it's a fundamental constraint or something that could be optimised.
josefritzishere 2mo ago
Why is Homomorphic Encryption necessary at all? The article seems to lose the plot in the process of tooting their own horn.
deleted 2mo ago
[deleted]
Tepix 2mo ago
> Google’s history of innovations in privacy technology Ah yes, their history of innovations! Thanks to Gemini for making a list for me over at https://share.gemini.google/vx9lE5dEJ1L7 https://share.gemini.google/vx9lE5dEJ1L7 The list contains more than 6 chapters with multiple entries each. Google has truly been very innovative in destroying privacy online.
Havoc 2mo ago
Can't say I have much faith in "send it to google it'll be private thanks to crypto magic". nah bro...the solution here is to not trust google
jacquesm 2mo ago
Private AI is perfectly practical already and it really doesn't need Google at all. Besides the word 'practical' here is really stretching things.
- furkanturan 2mo ago
  FHE doesn’t need Google too. Google research is investing in to it. Like we are at Belfort, various teams work on making FHE practical for organizations that want to run it for their own use cases, including on infrastructure they control themselves. Even in these setups, FHE can enable applications that would otherwise be blocked by regulatory or privacy requirements. Check out; https://belfortlabs.com/blog/belfort-partners-with-lg-on-encrypted-advertising-recommendations https://belfortlabs.com/blog/belfort-partners-with-lg-on-enc... https://belfortlabs.com/blog/encrypted-fraud-detection-with-swift https://belfortlabs.com/blog/encrypted-fraud-detection-with-...
maxo133 2mo ago
Funny to read this, but google is currently number one anti-privacy big tech company. No other company makes it as difficult for its users using anonymization/privacy software to access their services
- kccqzy 2mo ago
  Pure hyperbole. I can’t even browse Facebook or Instagram anonymously for 15 seconds; I get a hard login popup that would require me to be identified in order to continue. Same with X. I’m also starting to see this with Reddit. Google is the tamest of the bunch; I still do all my Google searches while logged out. Occasionally I get a CAPTCHA but that’s about it.
  - stackskipton 2mo ago
    Google generally doesn't need you logged in unless you are doing heavy privacy cleaning. Even in that case, they see data they are getting to be worthwhile enough for their other lines of business.
deleted 2mo ago
[deleted]
magios 2mo ago
https://fhetextbook.github.io/ https://fhetextbook.github.io/ which i believe was linked on this website previous is what i used to learn about how this stuff works.
- s20n 2mo ago
  Also this lecture by Vinod Vaikuntanathan is really helpful if you want to learn about FHE: https://www.youtube.com/watch?v=5LGwaICJ5sw https://www.youtube.com/watch?v=5LGwaICJ5sw
Danox 2mo ago
The only question is do you trust Google? No…
- furkanturan 2mo ago
  That is the key advantage of FHE: you encrypt your data locally, using keys that you own, and never need to share the plaintext—or the keys—with anyone. But there is more to it. Teams like us at Belfort, we are working to make FHE practical for organizations that want to deploy it for their own use cases, including on infrastructure they control themselves. Even in these setups, FHE can enable applications that would otherwise be blocked by regulatory, privacy, or compliance requirements.
chrisan 2mo ago
Man oh man did I read that title wrong and was completely confused
CurbStomper 2mo ago
[dead]
cryptographical 2mo ago
all we need now is indistinguishable obfuscation
never_inline 2mo ago
I think you folks are reading too much into it. I think the people working on FHE need to publish an AI-oriented pitch to retain funding from AI-pilled execs. Must be the same case with the golang post few days ago.
- fedpost 2mo ago
  Yeah I don't think they actually want this. There's far too much deeply personal and useful data being freely feed to the companies running these models. People are giving them everything. Their thoughts, feelings, intentions, company IP... It's a fucking gold mine. There's just too much value in retraining on that data and even if there wasn't, the feds lean so heavily on Google search histories to get convictions I am certain there will be pressure to prevent any mass market services from adopting this.
  - oofbey 2mo ago
    All true. HE will never be used for anything real because it’s way to slow and inefficient, meaning you can only run the stupidest models on it. And there’s no commercial incentive to make it work because collecting data is too valuable. But at the same time HE is classic Google promotion fodder. Really difficult technical problem! That’s what gets people promoted at Google. Nobody actually cares about it or will use it? That also is what gets people promoted at Google. Because Google only employs “the smartest people on earth” which means people who solve difficult problems. Difficult problems - not problems that need solving.
  - akoboldfrying 2mo ago
    When McDonalds introduced salads to their menu, purchases shot up -- but ~no one actually bought the salads. They just felt better about buying the same old unhealthy food from McDonalds when they knew that healthier options were available that they could potentially buy. This is the same kind of thing. The crazy overhead of FHE means it's not practical for almost any kind of real world computational work. But having it available at all is undeniably technically impressive, and some of that sense of technical refinement bleeds over into your opinion of Google as a whole. Incidentally, being a business client of Google's who actually uses this (or, say, quantum computing) for $BUSINESS_FEATURE would garner the same kind of positive attention, independently of whether it's technically a good idea. Finally, if the overheads come down to the level where they are still significant but not insane, GCP is in a great position.
stephen_cagle 2mo ago
I see a lot of comments here about how private AI is much cheaper than doing this, and that is currently true. But I am curious if anyone can think of any use cases for this as it currently stands? For every idea I can think of, I can think of another solution that would probably be a better solution at current slowdown in cost from using homomorphic encryption. For their two examples... 1) For the "will they click the add" one, how would anyone verify that the model I am uploading of myself is remotely accurate? And can't they learn my characteristics by running them against very specific ads that pinpoint my preferences? 2) For the "Do you have heart disease" one, why not just generating a billion datapoints (or more). The user can download the compressed data of this huge sample of points and then just enter their data on their own side and find the point that is closest (if not an exact match) to their own data? I'm not pulling apart the idea of homorphic encryption, I'm just curious if anyone can think of a really compelling use case for it given it 10^3 to 10^6 greater compute cost?
- catlifeonmars 2mo ago
  If the bottleneck is data transfer rate or volume, then I think you’ll find more use from today’s FHE. Incidentally there is a fair amount of overlap in the use case space between homomorphisms and oblivious transfer.
  - stephen_cagle 2mo ago
    Huh, that is an interesting thought. My belief is that a FHE scales at least linearly to the size of the memory. My belief for this is that it if didn't I could "learn" what computation you are doing by analyzing what parts of memory you did not look in. So, by that logic, we are potentially looking for a problem space where their is a large amount of data that is too large to even be paged through by the client, and where the FHE would actually make financial sense relative to downloading it? I'll think about that, but nothing immediately comes to mind.
fillihp 2mo ago
homoerotic fan cryption - after dark.
Watney-0717 2mo ago
Can we slash Google's 1000x FHE cost by 50%+ using a brain-inspired RC router like Project Ares-TCO?
r3trohack3r 2mo ago
Does this also solve provable compute as a side effect? Since the compute is fully encrypted end-to-end, the host wouldn’t be able to modify the program and still give valid results? So a caller could run a program on untrusted hardware and trust that the hardware actually ran the program?
- tsimionescu 2mo ago
  No, with FHE you don't know what program is running on the data, you just know that it can't access your data.
gigatexal 2mo ago
Maybe Apple will put its watches behind this and the computational complexity will be solved by some breakthrough.
- gigatexal 2mo ago
  s/watches/war chest/g
myshapeprotocol 2mo ago
[flagged]
Frannky 2mo ago
A homomorphic encryption blogpost without any mention of the computational overhead...
jijji 2mo ago
gaslighting people by pretending that encrypting reasoning traces between agent and client is a win for "private AI"... the title should be rewritten as it is plainly conceding -- Closed AI providers use encrypted reasoning traces to hide what the model is doing to come to a conclusion. This is not scientific progress, this is molopoly protectionism. No person using these models wants the reasoning traces hidden from them, it prevents the user from learning how conclusions about a question are derived.
senand 2mo ago
I‘m new to this, but mildly skeptical. In complex agentic engineering, there are lots of intermediate steps that depend on each other. Even if you disregard the significant encryption cost, this makes it impractical. So seems like this will stay a niche.
- furkanturan 2mo ago
  Thanks a lot. We have some road ahead for using encrypted AI for your agentic work. Today, I would not suggest thinking that as the primary target. Instead, there are many untapped applications (e.g. inter-bank transfers, industry, healthcare) where regulations, privacy laws and compliance requirements restrict institutions from touching data. These will initially be our key enablers, and over time we hope to extend the range of applications. At Belfort we are exploring such use cases that will benefit our FHE acceleration.
HlessClaudesman 2mo ago
"hotword" detection, is this basically sniffing encrypted data for words the authorities don't want you to speak? Seems counterproductive to their stated aim of privacy.
- xg15 2mo ago
  Not even just by the authorities. This sounds like the wet dream of Google itself or any other data mining companies. You can just ship the most sensitive data around and extract signals to your hearts content - and even if it would be protected by law, you're good because hey, it's always encrypted, and the signals, ad assignments and profile markers are encrypted too! This is what I don't like about all this "I'm not touching you" privacy tech, also including the new trend of nonvoluntary on-device processing. They pretend "privacy" was a primary goal of it's own, completely detached from any reasons why you would want to have it. Then they go on gloriously holding up the flag of privacy, while at the same time enabling all the things that you wanted to have privacy against in the first place.
  - HlessClaudesman 2mo ago
    And if they can detect "words", surely a determined attacker can reconstruct the whole thing.
- hyusap 2mo ago
  from what I understand, only the holder of the private key would be able to see if a hotword was detected or not. even if Google ran hotword detection over your data, they’d only get a encrypted output on if the hotword was detected or not
vkaku 2mo ago
I really want nobody to waste cycles on my AI than me. This is useless because now I need to pay for wasted inference AND encryption cycles on top of wire encryption.
- furkanturan 2mo ago
  While encrypted AI chatbots seem within reach, we don’t see them as the primary target today. There is a set of untapped applications, such as inter-bank transfers, industrial use cases, and healthcare; where regulations, privacy laws, and compliance requirements prevent institutions from directly accessing or processing sensitive data. This is where we believe Belfort can make a real difference: enabling organizations to compute on sensitive data while preserving strong privacy guarantees. And FHE doesn’t necessarily mean relying on Google, AWS, or Azure for encrypted compute. Companies can run FHE on their own infrastructure, allowing them to process their clients’ data privately while maintaining control over their systems and keys. These applications are our initial focus, and over time, we aim to expand the range of use cases that FHE can unlock.
  - vkaku 2mo ago
    I think that encrypted/homomorphic inference has overstated benefits and won't work as well for financial/ regulatory cases, where explainability is a bigger problem and AI has that accuracy issue.
snovv_crash 2mo ago
So much inefficiency just to run it on someone else's untrusted hardware. Private AI is already possible today with local open-weight models running on hardware you control. Homomorphic encryption is cool technology, but I'm really not sure what problem it solves.
- firtoz 2mo ago
  It's so that they can sell more things to enterprise customers
- furkanturan 2mo ago
  There are many challenges we need to solve for privacy preserving compute. Legislations, legal matters, key management, HSM like devices that has limited compute capabilities. At Belfort, we are accelerating FHE, besides exploring such uses cases for FHE. Like this one with Google, we also have publishes use cases; https://belfortlabs.com/blog/encrypted-fraud-detection-with-swift https://belfortlabs.com/blog/encrypted-fraud-detection-with-... https://belfortlabs.com/blog/belfort-partners-with-lg-on-encrypted-advertising-recommendations https://belfortlabs.com/blog/belfort-partners-with-lg-on-enc...
  - snovv_crash 2mo ago
    The advertising case seems to be solved by local models as well, then the data doesn't even need to be sent to the server. The Swift one is interesting, but I don't understand why the transaction needs to be encrypted, or more specifically why the model can only run in the encrypted part of the pipeline. Super cool tech, but I guess I just don't see the application yet. And wow, I worked with Marc a decade ago, small world.
    - furkanturan 2mo ago
      Regarding the SWIFT example, transactions between banks across countries go through SWIFT’s infrastructure and are encrypted for regulatory and security reasons. Banks and financial institutions operating at this scale are also required to use HSMs (Hardware Security Modules) to meet security and compliance standards, ensuring that sensitive data is processed securely. Suggesting alternative approaches to such large corporations is not straightforward. Their IT and security teams cannot afford to risk handling sensitive financial data in plaintext. No manager would readily approve a solution that offers additional functionality at the cost of deviating, even slightly, from well-established compliance practices. Encryption is therefore fundamental to how these systems operate. The challenge arises when these institutions need to perform computations or checks on such encrypted data. Decrypting the data, even under controlled conditions and with trained personnel, is operationally complex, introduces additional risk, and can create a significant compliance burden. This is where FHE offers a compelling solution. We demonstrate how a check can be performed directly on encrypted transactions against a known blacklist, without requiring the underlying transaction data to be decrypted. P.S. It is great to work with Marc :)
tjpnz 2mo ago
Sounds like a fantastic waste of compute. The future of private AI is local inference rendering this application of it pointless.
bilsbie 2mo ago
I think this could be an effective privacy preserving approach: Run the encoding and first and last layer of the LLM on the users machine. They could still get some idea of the content but really couldn’t easily get the actual text back. Anyone know if that’s viable?
childintime 2mo ago
It seems to me this tech still presumes the data sits in data warehouses, which i don't like to start with. The tech succeeds in packing the data in identical black boxes, so they all seem equal to the map-reduce function that runs over them. A separate identification layer knows which of the boxes is yours. But who knows, maybe from the results you can be fingerprinted anyway. As in the game, how many questions before you can guess the thing I'm thinking of? Answer: not that many.
mmeyerlein 2mo ago
[flagged]
wseqyrku 2mo ago
It's "private" "cloud" computing all over again, this time brought to you by jensen et al.
hulitu 2mo ago
> Google is making private (in the voice of George Bush Jr.) And it feels good doing it. Google doing something private is like Microsoft doing something secure.
beyondscale-yes 2mo ago
[dead]

498 points by u1hcw9nx 2mo ago

FloatArtifact 2mo ago

Encryption or not, if it's on somebody else's server, it isn't yours. I don't believe Google has my best interest.

u1hcw9nx 2mo ago
With Fully Homomorphic Encryption it's nobody elses. The basic idea of of the project is to remove the need for trust.
- unsungNovelty 2mo ago
  It's google. They are good at engineering. Not at creating trust. After the bizillionth time they have broken trust, there is no need for benefit of the doubt.
  - tonyhart7 2mo ago
    so what's the option ??? Meta ?? xAi ?? or OpenAI ???
    - unsungNovelty 2mo ago
      Why should there by any reason to look at the above ones? We are talking about this B2C company which effects a lot of our lives. The discussion isn't about that or this. The discussion is about JUST this specific company.
    - deleted 2mo ago
      [deleted]
  - throwaway27448 2mo ago
    There's more to life than branding.
  - Plont 2mo ago
    Yeah. Google is an ad business. Their entire motive for getting invested in AI is ad revenue. We're supposed to believe they just... won't turn on the money fountain? After going into the red for their data center investments? Hell nah. I'd expect this to be something like the Google Ad ID: technically separated from what Google considers personal information, but trivially easy to tie back to an individual person and to other information about that person. They're continually breaking trust by illegally scraping up the internet to feed to their plagiarism machine, which they are now asking us to trust with more data. It's not a compelling arguement.
    - ameliaquining 2mo ago
      This blog post is about an open source project that you can look at for yourself and decide whether it's suitable for your use case.
      - unsungNovelty 2mo ago
        Sure. But it will be trickled down into project by google which will effect us all and we won't have a say.
- thih9 2mo ago
  To what end?
  - u1hcw9nx 2mo ago
    Biometric authentication, credential checking, blind auctions, threat matching (search zero-day signatures without revealing the vectors), private dna matching or running queries of medical symptoms and health records against databases. I think the most immediately useful is checking if a username/password hash appears in a leaked database without revealing which account or password hash is being checked.
    - thih9 2mo ago
      I do like that list and thank you for the explanation. I still worry about how an ad company is going to approach this. Each of these advancements could be used to put ads in more places.
- FloatArtifact 2mo ago
  It was not my intent to undercut cryptography as a field or in its use. Consider client side scanning for photos before they're encrypted and uploaded to the cloud. I see no difference AI's future. Content, maybe encrypted, but a form of client-side scanning, or the model itself, will report users.
krunck 2mo ago
It's FHE for "cryptographically-secure private AI inference" not for every other service where they snoop into your behavioral information.
kccqzy 2mo ago
This kind of attitude is really disrespectful of decades of progress in cryptography. Without even considering homomorphic encryption, classic encryption is specifically designed to make intermediate nodes such as ISP dumb pipes that do not know the contents of communication. The ISP can store your communications on their server however they want. If you don’t agree with this model, I’m afraid modern cryptography doesn’t have anything to offer.
- jewel 2mo ago
  Even if your files are properly encrypted, if you keep them on, say, Google Drive, then you might lose access to them if your account gets inadvertently blocked. I think GP is trying to say something along those lines. Not that encryption isn't trustworthy, but that even with homomorphic encryption you still give up autonomy.
  - kccqzy 2mo ago
    That’s not a problem solvable by cryptography. Your ISP can’t read your traffic, but can cut off your traffic. You can use end-to-end encryption to encrypt your files to upload to Google Drive, but Google can choose to delete your files.
yjftsjthsd-h 2mo ago
It can give you confidentiality without availability. Not sure about integrity? Anyways, still useful.
bitpush 2mo ago
Unless all of your money is under your mattress, you're already making such decisions with a far more valuable asset.
furkanturan 2mo ago
FHE is not always for using somebody else's server; but for sure you do not share keys with that somebody else. Besides, FHE is not always about using somebody else's server. At Belfort, in addition to FHE acceleration, we also explore such uses cases; https://belfortlabs.com/blog/encrypted-fraud-detection-with-swift https://belfortlabs.com/blog/encrypted-fraud-detection-with-... https://belfortlabs.com/blog/belfort-partners-with-lg-on-encrypted-advertising-recommendations https://belfortlabs.com/blog/belfort-partners-with-lg-on-enc...

noident 2mo ago

Does this rely on the Trust Me Bro model, or is there some way for the client to verify that the provider actually isn't able to see your inputs? I want to read a whitepaper but all I can find is the tl;dw conference presentation

eslaught 2mo ago
The linked project page [1] claims to be fully homomorphic. Assuming the claim holds (I haven't verified it), then there is provably no way for Google or anyone else to obtain any information from the encrypted data or computation performed on them. FHE is traditionally horrifically slow, so it's hard to imagine running anything beyond toy models with it. They list some applications on the original article page, but (presumably) they must be dramatically stripped down in order to run within any reasonable time budget. This is not going to run anything like a Sol/Opus any time soon. [1]: https://heir.dev/ https://heir.dev/
boilerupnc 2mo ago
Related, I'd seen this blog [0] posted on HN a few years back that gave a nice run down on the "programmable cryptography" space which introduce FHE and a few other neat concepts. Really enjoyed the read and learned some new terms. [0] https://0xparc.org/blog/programmable-cryptography-1 https://0xparc.org/blog/programmable-cryptography-1
joshuamorton 2mo ago
> Does this rely on the Trust Me Bro model, or is there some way for the client to verify that the provider actually isn't able to see your inputs? The point of FHE is that you only ever send encrypted data. So if you trust that Google hasn't broken the encryption algorithm entirely, they never access the plaintext.

LoganDark 2mo ago

One flaw with FHE is that it guarantees only that you need the key to see the inputs or outputs of the computation, but not necessarily that the computation is the one you want. For example, the computation could be adversarial for certain inputs, or an adversary could insert their own computation first (or last).

noman-land 2mo ago
100% not an expert but my understanding was that part of what you are proving by signing the computation is that the computation itself was performed specifically as agreed to. I may be mixing this up with zero knowledge proofs.
- LoganDark 2mo ago
  If this were the case it would be necessary to send the entire model weights in response to every request which would be a bit inconvenient.
  - drdeca 2mo ago
    Hmm, could one instead of sending the model weights, send like, a merkle tree root for them, not exactly specifying the model, but at least demonstrating that the same model is used each time?
    - LoganDark 2mo ago
      I feel like that'd have to be generated as a side effect of the computation somehow?
mswphd 2mo ago
note that this is even true for an honest server. Roughly, FHE computations often require certain bounds on the (encrypted) messages for things like tuning polynomial approximation domains etc. If your messages are out of distribution for the tuned polynomial approximations you'll get back garbage as as result.

Deukhoofd 2mo ago

It sounds neat, but I do wonder how viable this is commercially. How high do we rate the chances that governments around the world will step in before another kind of E2E is rolled out.

deleted 2mo ago
[deleted]

filup 2mo ago

Proper encryption means the ciphertext is indistinguishable from noise. So...in order to be able to process on it, you have to make it not indistinguishable from noise. So I take offense to the term FHE. It's a oxymoron. The whole thing immidiatly stands out as a sham to build trust where it's gone.

pluto_modadic 2mo ago
they could have gone with an oblivious transfer approach (where it's working on what looks like multiple problems at once, you don't know which)
drdeca 2mo ago
Eh? With secret sharing one can do computation on a shared secret where it is provable that no individual party can recover any information about the data with their share alone. I don’t see why you conclude that FHE couldn’t be close to as secure as that. (Like, not information theoretically, but with computationally bounded adversaries.)
- filup 2mo ago
  I'm not saying that you can't design a system for secure cloud computing.
  - drdeca 2mo ago
    You said that FHE is an oxymoron, seemingly on the basis that in proper encryption the ciphertext is indistinguishable from noise, and you think this can’t be true of FHE. I am arguing to the contrary. In the secret sharing multi-party computation schemes, the individual shares of the secret are random and have no information about the plaintext. I see no reason that FHE can’t have ciphertexts indistinguishable from noise.
Ar-Curunir 2mo ago
No, that’s not what proper encryption means. Security for encryption means that cipher texts encrypting distinct messages are indistinguishable. This is called IND-CPA, and FHE satisfies this.

sabretooth1405 2mo ago

My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.

clayhacks 2mo ago
Do you think that’s like a fundamental limit or something that will improve with time and new algorithms?
- bhu8 2mo ago
  I wouldn't be very bullish. Homomorphic encryption got significantly efficient with the first few iterations, but I don't really see the necessary orders of magnitude savings coming soon. You could reduce this by some partial encryption schemes (e.g., for LLMs you need a handful of basic operations) but a better alternative already exists: multi-party computation. Source: I did research in this area in the past.
dietr1ch 2mo ago
Exactly my concern, and worse overhead that what I recalled. Cost-wise the only viable private compute is local compute. It's more expensive than cloud, but true private compute in the cloud is definitely pricier.
abetusk 2mo ago
1000x slowdown is bad but not a complete deal breaker. Do you have a sense of what a reasonable achievable factor is? Do you have sense for how long before we get to that achievable factor?
- jacquesm 2mo ago
  It's a ridiculous waste of energy, just use local compute.
  - 2mo ago

Cider9986 2mo ago

This is the same Google that doesn't have e2ee on their password manager by default. Like WTF, it's a password manager.

amazingamazing 2mo ago
If you think about it for a moment you will realize the average user does not want that.
- rexpop 2mo ago
  I suspect that your one moment of thought follows many years of contemplation. Maybe you can fill us in on some of that background.
  - amazingamazing 2mo ago
    How do you make e2e password management work on multiple devices as convenient as not? You cannot. Most people who use Google trust them. And if you do not trust them then why are you using their password manager? You see? Hence why they do not bother. Of course one might read this and say, multi device e2ee is possible! Sure, but can you allow for recovery when people inevitably lose their recovery codes at scale? If you really care about this it is not necessary to go with Google. I stand by my original comment: The average user does not want it.
    - Cider9986 2mo ago
      Literally all major password managers are E2EE except Google's. The UX is fine.
  - giovannibonetti 2mo ago
    E2E encryption means that if the user loses the keys, there is no way to recover that even if they contact support and prove the data belongs to them.
    - 2mo ago

jrm4 2mo ago

Correct. I appreciate the theoretical technology here, but I believe a great deal of harm is done by the fact that people are not likely to understand exactly what this means. Which is to say, I believe that google is strongly implying the falsehood of "no one at Google can read your stuff."

oulipo 2mo ago

Zama.ai is also a player in this space

furkanturan 2mo ago
Yes, it is. But it’s important to recognize that there are different FHE schemes. Zama has invested heavily in TFHE and achieved impressive results. However, time has shown that TFHE is not the best fit for encrypted AI workloads, with CKKS and related schemes gaining an advantage with mathematical and algorithmic advances continue. At Belfort we have foot on both sides; FPGA acceleration for integrating to Zama's TFHE-rs, and GPU acceleration for CKKS.

bmenrigh 2mo ago

Has FHE really progressed so far that it's now so efficient that doing computation on an encrypted prompt is feasible? I thought even basic operations like FHE addition were still thousands of times more complex. The only mention in the article I see is: > But while homomorphic encryption has a nontrivial cost overhead, it shifts the capability/privacy trade-off to a question of cost. And the cost of homomorphic encryption is rapidly decreasing. Which doesn't spell out exactly hon "nontrivial" the cost overhead still is.

sean_pedersen 2mo ago
There are LLM models which compute only using addition no?
mswphd 2mo ago
addition is easy/essentially the same cost as standard (not really, because you have to compute mod p addition rather than mod 2^32, but ignoroing that it's roughly the same). as a general rule multiplication is the difficult part. it's hard to accurately quantify what "nontrivial" cost overheads are because they're very application dependent. for example, things that require encrypted control flow are very hard under FHE. so an encrypted hashmap sounds roughly unimplemnetable. but things that do not require encrypted control flow (e.g. many ML applications) are less bad. this can still be quite bad though. for example, relu is trivial in plaintext. it is hard homomorphically, because the trivial way to write it uses private control flow.
nl 2mo ago
It's not running LLMs.

lsb 2mo ago

Google is making private AI practical with Gemma4 something that you can run without an Internet connection. All of the proofs of privacy rely on us getting the math right. All of the privacy from unplugging your internet cable is there by default.

NyxWulf 2mo ago

Maybe I'm not understanding this, but how is it that you can know enough about the data to process it without undermining the fundamental concept of encryption? Isn't encrypted data supposed to be just random noise without the key? The more you know about the underlying data the easier it gets to decrypt? Does this mean someone can just steal your encrypted data and use that to steal your identity without even needing to decrypt it anymore?

luckydata 2mo ago
you can produce a correct result without knowing what that result means as long as the person you then provide that result to can decrypt it. So the way it works is the server does the job on data it can't read, sends an answer to the device, the device decrypts it. It's very useful for things like healthcare data for example but also for products like "home assistants", and it's one of the use cases that the team hiring internally was giving in their job req. I tried to apply too but didn't get it unfortunately, this was a very interesting product to work on imho, congrats to whoever got my job :)
mswphd 2mo ago
the basic encryption scheme used here is fairly straightforward actually, at least the symmetric encryption version. Let s be a uniformly random, 512-dimensional u32 vector. To encrypt a message m (say a 512-dimensional bit vector for simplicity), you 1. generate a 512 x 512 random (u32) matrix A, and 2. generate a 512-dimensional rounded (to the nearest integer) Gaussian, say of standard deviation 10, e. The ciphertext is then [A, b :=As + e + 2^8 m]. To decrypt, you compute b - As to recover 2^8 m + e. You can then recover m, as e << 2^8 with high probability. Anyway, if you have two of these ciphertexts, you can sum them together to get [A1 + A2, (A1 + A2)s + (e1 + e2) + 2^8 (m1 + m2)] this decrypts to m1 + m2, so you can recover homomorphic sums (or scalings by small integers). Multiplication is more complex, so I won't get into it here. But the high level from the above example is that you could have someone compute arbitrary linear functions of your data without them knowing what your data is.
- Chris2048 2mo ago

sheesdev 2mo ago

I've published two papers on using HE for ML and it's nice, but also alarming, to see big players like Google and Microsoft making decent tools for performing HE. Although the technology is still much farther out from being commercially viable, it does pose an interesting problem about how these data aggregation companies will utilise a tech that is inherently private.

tolugenius 2mo ago
I guess I'm curious from your perspective what and why makes the tech from being commercially viable, and the concerns of big players adopting this (which was always going to happen I imagine, but when shapes a lot of the future of the field).
j2kun 2mo ago
One potentially simple answer is: to make things that are otherwise illegal (for regulatory reasons) possible. There are some instances of hand-built cryptographic protocols used in production at Google that use FHE in small doses (e.g., computing a single dot product as part of an interactive protocol), so the big question is whether having a general-purpose tool will expand the feasibility of applying these to other, maybe less critical projects that still want some privacy guarantees but can't afford to hire cryptographers.

traceroute66 2mo ago

Quoted from the blog post: > user-data can be protected from data breaches, but then the service provider cannot provide features that depend on the data, such as spam or virus detection I think they forgot "or advertising" at the end. I don't trust Google. I would much prefer to use on-prem or - at most - one of the secure-enclave providers like Tinfoil[1] or Private Mode[2] [1] https://tinfoil.sh/ https://tinfoil.sh/ [2] https://www.privatemode.ai/ https://www.privatemode.ai/

flambo 2mo ago
Secure Enclaves are broken all the time though [1]. They’re also not secure against attacks from a quantum computer (including store now decrypt later attacks). At least with symmetric FHE, anything that happens after encryption requires zero trust. [1] https://sgx.fail/ https://sgx.fail/
- j2kun 2mo ago
  Indeed, Google itself has a large research group that repeatedly breaks TEEs with side-channels and such.

mohamedkoubaa 2mo ago

Anything to keep people from running local models

nazgulsenpai 2mo ago

Why is the bold letter n lighter than the rest in this page's font?

p0w3n3d 2mo ago

Ideally: we will have private AI Reality: our computers will be used as distributed AI calculators

hamper653 2mo ago
Freenet, but for AI inference.
- p0w3n3d 2mo ago
  That's not what I meant. I meant botnet taken over by Google

nullc 2mo ago

Private AI is practical by running the model locally, every much more so than any homomorphic encryption scheme. So essentially the headline sells this as work to keep your data private, but really it's work to keep the AI-- which was trained on your code and your writing-- private.

drblast 2mo ago

I did some amount of research into the feasibility of PHE and FHE about 20 years ago, and my conclusion at the time was that the space overhead of the encrypted output was a massive bottleneck, which meant that while it was potentially useful in a small number of niche cases it wasn't ever going to be practical for general-purpose computations without a major breakthrough. The gist was I could do an encrypted (int)x + (int)y = (int)z computation, I could encrypt the inputs and then get a result back that was correct, secure, and decryptable, but was like 1MB in size. So, for someone whose knowledge is 20 years outdated and is about Pallier crypto, has that major breakthrough happened?

meindnoch 2mo ago

Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet. The most private AI is the one running on my own hardware, not in some giant data center.

froh 2mo ago
where does this factor "1000x" come from? I have doubts.
- raggi 2mo ago
  https://www.jeremykun.com/2024/05/04/fhe-overview/#the-highest-level-view https://www.jeremykun.com/2024/05/04/fhe-overview/#the-highe... > Fourth, there is a bandwidth concern. FHE encryption schemes generally increase the size of the data being encrypted, and the user must send the server a special set of encryption keys to enable the computation, which are relatively large as well. The special keys need only be generated and sent once and can be used for all future computations, but they can easily be gigabytes in size. In one example FHE scheme with lightweight keys, a ciphertext encrypting a single integer is on the order of 25 KB, and the special keys are about 0.5 GB. In others, 16,000 or more integers are packed into a single ciphertext of similar size, but the keys can be 10s of GiBs.
  - pamcake 2mo ago
    I'm still a bit skeptic - 1000x sounds overly optimistic and only looking at encryption transformation without any operation is already at least 10000x with scheme mentioned in that quote. Sibling comment estimates lower bounds of current research at minimun 10^6 overhead which sounds more realistic.
- pamcake 2mo ago
  Most likely from above in thread. There is no reason to believe it should be lower than that - or even that low. Or do you have access to research claiming such achievements?

weatherlite 2mo ago

> Google is making private AI practical with homomorphic encryption I don't understand why we need to bring LGBTQ+ into everything

hellohello2 2mo ago

Have all the skeptics in this thread somehow forgot about Moore's law?

pinkmuffinere 2mo ago
What, how is that relevant? the transistors on a microchip double about every two years. That’s descriptive, not prescriptive, so whatever you’re trying to imply about the future is kindof a toss up — there’s no promise the parents hold. And, it doesn’t really have to do with tfa, as far as I can tell? Do you perhaps mean Murphy’s law, “whatever can go wrong will go wrong”?
- hellohello2 2mo ago
  The top comment mentions a ~10^3 overhead. Am I crazy (or already old?) for thinking this is not a very large constant?
  - pinkmuffinere 2mo ago
    Oh I see, you’re arguing that computer growth will make this more attractive. I think it’s a decent argument, although personally I find it unconvincing. More compute will be allocated between all the existing options, and I suspect it will go towards faster/better inference, instead of encryption. For most use cases people just don’t care much about encryption, and when feature prioritization comes around it always gets a back seat
    - hellohello2 2mo ago
      Yes, I think it really depends on the task. Clearly we won't be willing to spend 1000x on tokens for our everyday needs. But what about for the police monitoring conversations? I'd very much like my chats to stay encrypted, but there have been laws pushing against this. Technology like this provides a real solution i.e. you can scan chats for keywords without actually reading them, and get a warrant if you find something.
2mo ago

gaigalas 2mo ago

I have an idea for AI companies for tremendous scale, with privacy and all nice things accounted for: - Unstarve the GPU and RAM consumer market. - Let enthusiasts and volunteers quickly ramp up local AI. - Reap the results that the community will most certainly achieve. Don't repeat Microsoft's earlier mistakes. It flourished when it embraced the community and open source. If it had made that move earlier, it would have been unstoppable. Yes, if you can control the entire market that's probably awesome. But it's also full of nonsense risks.

shevy-java 2mo ago

I do not want to make Google more powerful than it already is.

cantalopes 2mo ago

It's a step to provide targeted advertising with mathematically provable "no sensitive info stored" approach. The google must fight really hard because this is the only source of income that makes sense for their position. And it /is/ quite evil tbh

FartyMcFarter 2mo ago
How would advertising be implemented in the scenario of using this technique?

sarjann 2mo ago

I think at 1000x it's probably way cheaper to do an "AWS outpost" style thing. Although not sure if it's a fundamental constraint or something that could be optimised.

josefritzishere 2mo ago

Why is Homomorphic Encryption necessary at all? The article seems to lose the plot in the process of tooting their own horn.

deleted 2mo ago

[deleted]

Tepix 2mo ago

> Google’s history of innovations in privacy technology Ah yes, their history of innovations! Thanks to Gemini for making a list for me over at https://share.gemini.google/vx9lE5dEJ1L7 https://share.gemini.google/vx9lE5dEJ1L7 The list contains more than 6 chapters with multiple entries each. Google has truly been very innovative in destroying privacy online.

Havoc 2mo ago

Can't say I have much faith in "send it to google it'll be private thanks to crypto magic". nah bro...the solution here is to not trust google

jacquesm 2mo ago

Private AI is perfectly practical already and it really doesn't need Google at all. Besides the word 'practical' here is really stretching things.

furkanturan 2mo ago
FHE doesn’t need Google too. Google research is investing in to it. Like we are at Belfort, various teams work on making FHE practical for organizations that want to run it for their own use cases, including on infrastructure they control themselves. Even in these setups, FHE can enable applications that would otherwise be blocked by regulatory or privacy requirements. Check out; https://belfortlabs.com/blog/belfort-partners-with-lg-on-encrypted-advertising-recommendations https://belfortlabs.com/blog/belfort-partners-with-lg-on-enc... https://belfortlabs.com/blog/encrypted-fraud-detection-with-swift https://belfortlabs.com/blog/encrypted-fraud-detection-with-...

maxo133 2mo ago

Funny to read this, but google is currently number one anti-privacy big tech company. No other company makes it as difficult for its users using anonymization/privacy software to access their services

kccqzy 2mo ago
Pure hyperbole. I can’t even browse Facebook or Instagram anonymously for 15 seconds; I get a hard login popup that would require me to be identified in order to continue. Same with X. I’m also starting to see this with Reddit. Google is the tamest of the bunch; I still do all my Google searches while logged out. Occasionally I get a CAPTCHA but that’s about it.
- stackskipton 2mo ago
  Google generally doesn't need you logged in unless you are doing heavy privacy cleaning. Even in that case, they see data they are getting to be worthwhile enough for their other lines of business.

deleted 2mo ago

[deleted]

magios 2mo ago

https://fhetextbook.github.io/ https://fhetextbook.github.io/ which i believe was linked on this website previous is what i used to learn about how this stuff works.

s20n 2mo ago
Also this lecture by Vinod Vaikuntanathan is really helpful if you want to learn about FHE: https://www.youtube.com/watch?v=5LGwaICJ5sw https://www.youtube.com/watch?v=5LGwaICJ5sw

Danox 2mo ago

The only question is do you trust Google? No…

furkanturan 2mo ago
That is the key advantage of FHE: you encrypt your data locally, using keys that you own, and never need to share the plaintext—or the keys—with anyone. But there is more to it. Teams like us at Belfort, we are working to make FHE practical for organizations that want to deploy it for their own use cases, including on infrastructure they control themselves. Even in these setups, FHE can enable applications that would otherwise be blocked by regulatory, privacy, or compliance requirements.

chrisan 2mo ago

Man oh man did I read that title wrong and was completely confused

CurbStomper 2mo ago

[dead]

cryptographical 2mo ago

all we need now is indistinguishable obfuscation

never_inline 2mo ago

I think you folks are reading too much into it. I think the people working on FHE need to publish an AI-oriented pitch to retain funding from AI-pilled execs. Must be the same case with the golang post few days ago.

fedpost 2mo ago
Yeah I don't think they actually want this. There's far too much deeply personal and useful data being freely feed to the companies running these models. People are giving them everything. Their thoughts, feelings, intentions, company IP... It's a fucking gold mine. There's just too much value in retraining on that data and even if there wasn't, the feds lean so heavily on Google search histories to get convictions I am certain there will be pressure to prevent any mass market services from adopting this.
- oofbey 2mo ago
  All true. HE will never be used for anything real because it’s way to slow and inefficient, meaning you can only run the stupidest models on it. And there’s no commercial incentive to make it work because collecting data is too valuable. But at the same time HE is classic Google promotion fodder. Really difficult technical problem! That’s what gets people promoted at Google. Nobody actually cares about it or will use it? That also is what gets people promoted at Google. Because Google only employs “the smartest people on earth” which means people who solve difficult problems. Difficult problems - not problems that need solving.
- akoboldfrying 2mo ago
  When McDonalds introduced salads to their menu, purchases shot up -- but ~no one actually bought the salads. They just felt better about buying the same old unhealthy food from McDonalds when they knew that healthier options were available that they could potentially buy. This is the same kind of thing. The crazy overhead of FHE means it's not practical for almost any kind of real world computational work. But having it available at all is undeniably technically impressive, and some of that sense of technical refinement bleeds over into your opinion of Google as a whole. Incidentally, being a business client of Google's who actually uses this (or, say, quantum computing) for $BUSINESS_FEATURE would garner the same kind of positive attention, independently of whether it's technically a good idea. Finally, if the overheads come down to the level where they are still significant but not insane, GCP is in a great position.

stephen_cagle 2mo ago

I see a lot of comments here about how private AI is much cheaper than doing this, and that is currently true. But I am curious if anyone can think of any use cases for this as it currently stands? For every idea I can think of, I can think of another solution that would probably be a better solution at current slowdown in cost from using homomorphic encryption. For their two examples... 1) For the "will they click the add" one, how would anyone verify that the model I am uploading of myself is remotely accurate? And can't they learn my characteristics by running them against very specific ads that pinpoint my preferences? 2) For the "Do you have heart disease" one, why not just generating a billion datapoints (or more). The user can download the compressed data of this huge sample of points and then just enter their data on their own side and find the point that is closest (if not an exact match) to their own data? I'm not pulling apart the idea of homorphic encryption, I'm just curious if anyone can think of a really compelling use case for it given it 10^3 to 10^6 greater compute cost?

catlifeonmars 2mo ago
If the bottleneck is data transfer rate or volume, then I think you’ll find more use from today’s FHE. Incidentally there is a fair amount of overlap in the use case space between homomorphisms and oblivious transfer.
- stephen_cagle 2mo ago
  Huh, that is an interesting thought. My belief is that a FHE scales at least linearly to the size of the memory. My belief for this is that it if didn't I could "learn" what computation you are doing by analyzing what parts of memory you did not look in. So, by that logic, we are potentially looking for a problem space where their is a large amount of data that is too large to even be paged through by the client, and where the FHE would actually make financial sense relative to downloading it? I'll think about that, but nothing immediately comes to mind.

fillihp 2mo ago

homoerotic fan cryption - after dark.

Watney-0717 2mo ago

Can we slash Google's 1000x FHE cost by 50%+ using a brain-inspired RC router like Project Ares-TCO?

r3trohack3r 2mo ago

Does this also solve provable compute as a side effect? Since the compute is fully encrypted end-to-end, the host wouldn’t be able to modify the program and still give valid results? So a caller could run a program on untrusted hardware and trust that the hardware actually ran the program?

tsimionescu 2mo ago
No, with FHE you don't know what program is running on the data, you just know that it can't access your data.

gigatexal 2mo ago

Maybe Apple will put its watches behind this and the computational complexity will be solved by some breakthrough.

gigatexal 2mo ago
s/watches/war chest/g

myshapeprotocol 2mo ago

[flagged]

Frannky 2mo ago

A homomorphic encryption blogpost without any mention of the computational overhead...

jijji 2mo ago

gaslighting people by pretending that encrypting reasoning traces between agent and client is a win for "private AI"... the title should be rewritten as it is plainly conceding -- Closed AI providers use encrypted reasoning traces to hide what the model is doing to come to a conclusion. This is not scientific progress, this is molopoly protectionism. No person using these models wants the reasoning traces hidden from them, it prevents the user from learning how conclusions about a question are derived.

senand 2mo ago

I‘m new to this, but mildly skeptical. In complex agentic engineering, there are lots of intermediate steps that depend on each other. Even if you disregard the significant encryption cost, this makes it impractical. So seems like this will stay a niche.

furkanturan 2mo ago
Thanks a lot. We have some road ahead for using encrypted AI for your agentic work. Today, I would not suggest thinking that as the primary target. Instead, there are many untapped applications (e.g. inter-bank transfers, industry, healthcare) where regulations, privacy laws and compliance requirements restrict institutions from touching data. These will initially be our key enablers, and over time we hope to extend the range of applications. At Belfort we are exploring such use cases that will benefit our FHE acceleration.

HlessClaudesman 2mo ago

"hotword" detection, is this basically sniffing encrypted data for words the authorities don't want you to speak? Seems counterproductive to their stated aim of privacy.

xg15 2mo ago
Not even just by the authorities. This sounds like the wet dream of Google itself or any other data mining companies. You can just ship the most sensitive data around and extract signals to your hearts content - and even if it would be protected by law, you're good because hey, it's always encrypted, and the signals, ad assignments and profile markers are encrypted too! This is what I don't like about all this "I'm not touching you" privacy tech, also including the new trend of nonvoluntary on-device processing. They pretend "privacy" was a primary goal of it's own, completely detached from any reasons why you would want to have it. Then they go on gloriously holding up the flag of privacy, while at the same time enabling all the things that you wanted to have privacy against in the first place.
- HlessClaudesman 2mo ago
  And if they can detect "words", surely a determined attacker can reconstruct the whole thing.
hyusap 2mo ago
from what I understand, only the holder of the private key would be able to see if a hotword was detected or not. even if Google ran hotword detection over your data, they’d only get a encrypted output on if the hotword was detected or not

vkaku 2mo ago

I really want nobody to waste cycles on my AI than me. This is useless because now I need to pay for wasted inference AND encryption cycles on top of wire encryption.

furkanturan 2mo ago
While encrypted AI chatbots seem within reach, we don’t see them as the primary target today. There is a set of untapped applications, such as inter-bank transfers, industrial use cases, and healthcare; where regulations, privacy laws, and compliance requirements prevent institutions from directly accessing or processing sensitive data. This is where we believe Belfort can make a real difference: enabling organizations to compute on sensitive data while preserving strong privacy guarantees. And FHE doesn’t necessarily mean relying on Google, AWS, or Azure for encrypted compute. Companies can run FHE on their own infrastructure, allowing them to process their clients’ data privately while maintaining control over their systems and keys. These applications are our initial focus, and over time, we aim to expand the range of use cases that FHE can unlock.
- vkaku 2mo ago
  I think that encrypted/homomorphic inference has overstated benefits and won't work as well for financial/ regulatory cases, where explainability is a bigger problem and AI has that accuracy issue.

snovv_crash 2mo ago

So much inefficiency just to run it on someone else's untrusted hardware. Private AI is already possible today with local open-weight models running on hardware you control. Homomorphic encryption is cool technology, but I'm really not sure what problem it solves.

firtoz 2mo ago
It's so that they can sell more things to enterprise customers
furkanturan 2mo ago
There are many challenges we need to solve for privacy preserving compute. Legislations, legal matters, key management, HSM like devices that has limited compute capabilities. At Belfort, we are accelerating FHE, besides exploring such uses cases for FHE. Like this one with Google, we also have publishes use cases; https://belfortlabs.com/blog/encrypted-fraud-detection-with-swift https://belfortlabs.com/blog/encrypted-fraud-detection-with-... https://belfortlabs.com/blog/belfort-partners-with-lg-on-encrypted-advertising-recommendations https://belfortlabs.com/blog/belfort-partners-with-lg-on-enc...
- snovv_crash 2mo ago
  The advertising case seems to be solved by local models as well, then the data doesn't even need to be sent to the server. The Swift one is interesting, but I don't understand why the transaction needs to be encrypted, or more specifically why the model can only run in the encrypted part of the pipeline. Super cool tech, but I guess I just don't see the application yet. And wow, I worked with Marc a decade ago, small world.
  - furkanturan 2mo ago
    Regarding the SWIFT example, transactions between banks across countries go through SWIFT’s infrastructure and are encrypted for regulatory and security reasons. Banks and financial institutions operating at this scale are also required to use HSMs (Hardware Security Modules) to meet security and compliance standards, ensuring that sensitive data is processed securely. Suggesting alternative approaches to such large corporations is not straightforward. Their IT and security teams cannot afford to risk handling sensitive financial data in plaintext. No manager would readily approve a solution that offers additional functionality at the cost of deviating, even slightly, from well-established compliance practices. Encryption is therefore fundamental to how these systems operate. The challenge arises when these institutions need to perform computations or checks on such encrypted data. Decrypting the data, even under controlled conditions and with trained personnel, is operationally complex, introduces additional risk, and can create a significant compliance burden. This is where FHE offers a compelling solution. We demonstrate how a check can be performed directly on encrypted transactions against a known blacklist, without requiring the underlying transaction data to be decrypted. P.S. It is great to work with Marc :)

tjpnz 2mo ago

Sounds like a fantastic waste of compute. The future of private AI is local inference rendering this application of it pointless.

bilsbie 2mo ago

I think this could be an effective privacy preserving approach: Run the encoding and first and last layer of the LLM on the users machine. They could still get some idea of the content but really couldn’t easily get the actual text back. Anyone know if that’s viable?

childintime 2mo ago

It seems to me this tech still presumes the data sits in data warehouses, which i don't like to start with. The tech succeeds in packing the data in identical black boxes, so they all seem equal to the map-reduce function that runs over them. A separate identification layer knows which of the boxes is yours. But who knows, maybe from the results you can be fingerprinted anyway. As in the game, how many questions before you can guess the thing I'm thinking of? Answer: not that many.

mmeyerlein 2mo ago

[flagged]

wseqyrku 2mo ago

It's "private" "cloud" computing all over again, this time brought to you by jensen et al.

hulitu 2mo ago

> Google is making private (in the voice of George Bush Jr.) And it feels good doing it. Google doing something private is like Microsoft doing something secure.

beyondscale-yes 2mo ago

[dead]