4 ms·

Yes, but it doesn't scale as your photo collection scales up, since if the algorithm is updated, you need all the data on device to reprocess every photo. You

Yes, but it doesn't scale as your photo collection scales up, since if the algorithm is updated, you need all the data on device to reprocess every photo. You need something like CryptoDL: deep learning over homomorphic encryption, that would let the photos stay encrypted in the cloud, but still allow DL computations to be run.

aikinai 8y ago
Apple has the same functionality and does all processing locally, though I'm not sure if that's an argument for or against your assertion that it doesn't scale. It does work, but it's not nearly as seamless as Google's cloud implementation. I have many tens of thousands of photos in my Apple Photos library and I think it takes over a month to process everything if I start from scratch on a new computer. Somehow it seems much faster on the phone, and I'm not sure if that's because the processing is better hidden from me, it's much more efficient on the custom chip, or maybe a bit of both.
- jsjohnst 8y ago
  > I'm not sure if that's because the processing is better hidden from me, it's much more efficient on the custom chip, or maybe a bit of both. The later, a bit of both. It’s also probable that your phone is turned on and plugged in more often than your computer (especially for laptops).
- toomuchtodo 8y ago
  Regardless of how it's done, I treasure my iPhone performing the facial recognition of my Photos library locally (10+ years of photos) as I continue to extricate my data from Google services.
personjerry 8y ago
That's a problem for the big software companies, not for the consumer. Namely, to optimize for on-device training and patches.
- cromwellian 8y ago
  The problem isn't ondevice training. The problem is, if I "free up space" by deleting my local copy, and I have 10 years of photos backed up in the cloud, then an update to the ML model requires redownloading all of those photos and running the inference on them. It doesn't matter how clever you are: if you disallow processing the photos on the server, then any index you have on the client that is cached will be invalidated by an algorithm update, and there is no way to rebuild the index without downloading all the photos once again. I'm not talking about federated learning, that's a different problem. Let's say we solved that problem using on-device learning and differential privacy. The problem still remains that retrained/updated models on devices require the photos to be processed locally again. So if for example, you can now detect "hugs" in photos, and I search for "photos of my me hugging my daughter", Google Photos gives me what I want, and your local-only model means I have to wait a long time to redownload and reindex my entire collection. It's a huge user problem. A workable solution to this problem has to somehow entail homomorphic encryption, if you somehow want to keep the photos encrypted on the server.
  - reaperducer 8y ago
    Doesn't seem to be a problem on the iPhone. But then I usually search on "cats" and it gives me the option of "adult cats" or "kittens". But it certainly works on large libraries because it shows me recent photos mixed with cats that died years ago.

aikinai 8y ago

Apple has the same functionality and does all processing locally, though I'm not sure if that's an argument for or against your assertion that it doesn't scale. It does work, but it's not nearly as seamless as Google's cloud implementation. I have many tens of thousands of photos in my Apple Photos library and I think it takes over a month to process everything if I start from scratch on a new computer. Somehow it seems much faster on the phone, and I'm not sure if that's because the processing is better hidden from me, it's much more efficient on the custom chip, or maybe a bit of both.

jsjohnst 8y ago
> I'm not sure if that's because the processing is better hidden from me, it's much more efficient on the custom chip, or maybe a bit of both. The later, a bit of both. It’s also probable that your phone is turned on and plugged in more often than your computer (especially for laptops).
toomuchtodo 8y ago
Regardless of how it's done, I treasure my iPhone performing the facial recognition of my Photos library locally (10+ years of photos) as I continue to extricate my data from Google services.

personjerry 8y ago

That's a problem for the big software companies, not for the consumer. Namely, to optimize for on-device training and patches.

cromwellian 8y ago
The problem isn't ondevice training. The problem is, if I "free up space" by deleting my local copy, and I have 10 years of photos backed up in the cloud, then an update to the ML model requires redownloading all of those photos and running the inference on them. It doesn't matter how clever you are: if you disallow processing the photos on the server, then any index you have on the client that is cached will be invalidated by an algorithm update, and there is no way to rebuild the index without downloading all the photos once again. I'm not talking about federated learning, that's a different problem. Let's say we solved that problem using on-device learning and differential privacy. The problem still remains that retrained/updated models on devices require the photos to be processed locally again. So if for example, you can now detect "hugs" in photos, and I search for "photos of my me hugging my daughter", Google Photos gives me what I want, and your local-only model means I have to wait a long time to redownload and reindex my entire collection. It's a huge user problem. A workable solution to this problem has to somehow entail homomorphic encryption, if you somehow want to keep the photos encrypted on the server.
- reaperducer 8y ago
  Doesn't seem to be a problem on the iPhone. But then I usually search on "cats" and it gives me the option of "adult cats" or "kittens". But it certainly works on large libraries because it shows me recent photos mixed with cats that died years ago.