4 ms·
I love homomorphic encryption, but why can't they just do a neural search locally? - iOS Photos -> Vectors - Search Query "Dog photos" -> Vectors - Result (C
by williamtrask 2y ago
I love homomorphic encryption, but why can't they just do a neural search locally?
- iOS Photos -> Vectors
- Search Query "Dog photos" -> Vectors
- Result (Cosine Similarity): Look some dog photos!
iPhones have plenty of local storage and compute power for doing this kind of thing when the phone is idle. And cosine similarity can work quickly at runtime.
- orf 2y agoThat’s what they do
- dialup_sounds 2y agoApparently they only do the cloud HE song and dance for landmarks, which is too big of a data set to realistically keep on-device.
- deleted 2y ago[deleted]
- woadwarrior01 2y agoHere's an open source iOS app[1] that does that. Incidentally, it's built using Apple's own MobileCLIP[2] models. [1]: https://github.com/fguzman82/CLIP-Finder2 https://github.com/fguzman82/CLIP-Finder2 [2]: https://github.com/apple/ml-mobileclip https://github.com/apple/ml-mobileclip
- jerf 2y agoBecause the blog post needs some sort of concrete example to explain, but all concrete examples of fully-homeomorphic encryption are generally done better locally at the moment due to the extreme costs of FHE.
- internetter 2y agoI discuss this in the post: > This seems like a lot of data the client is getting anyway. I don’t blame you for questioning if the server is actually needed. The thing is, the stored vectors that are compared against are by far the biggest storage user. Each vector can easily be multiple kilobytes. The paper discusses a database of 35 million entries divided across 8500 clusters.