6 ms·

Neat. Is my phone classifying all this stuff, or is it sending all my photos to some Apple server to do it? I watched the video in the OP but it didn't go into

by Lammy 6y ago

Neat. Is my phone classifying all this stuff, or is it sending all my photos to some Apple server to do it? I watched the video in the OP but it didn't go into any technical details.

spullara 6y ago
I don't know of any ML stuff that Apple does serverside. That is why they have invested so much in ML processing on their devices.
- user-the-name 6y ago
  I guess Siri does work at least partially serverside?
  - spullara 6y ago
    The ML part is local for the voice transcription but the questions almost always need to be answered from the server. But if you ask to play music or set a timer for example that is all local. Also works in Airplane mode.
- yazaddaruvala 6y ago
  All ML on device definitely seems like a goal for Apple[1]. However, right now Siri doesn't work in airplane mode. Maybe its still too power hungry to run locally? Maybe they still need the data from the requests server side to improve the model? I hope both Siri and Maps, and especially their integration, i.e. "Siri navigate to ____." "Siri find restaurants nearby (or along the way)" works on device in the near future. Its annoying for the CarPlay integration to stop working when you're in a remote area and need the navigational help. [1] https://appleinsider.com/articles/18/11/15/apple-considering-offline-mode-for-siri-that-could-process-voice-locally-on-an-iphone https://appleinsider.com/articles/18/11/15/apple-considering...
  - alblue 6y ago
    Siri’s voice translation works in your phone but the answers it needs are all calculated from the server (on the whole). Maps, for example, are all stored server side and route calculation are server side processes as well, as are the locations of restaurants. There’s a difference been decoding voice requests and being able to answer any questions you may have; you wouldn’t expect Google web search or Wikipedia to work in offline mode, so why do you expect it to work for Siri?
    - yazaddaruvala 5y ago
      I don’t expect it to work :) I expect that it will one day soon work offline. Likely not a full web search. By 202X, while offline my device should have: 1. Enough storage for voice to text models, translation models, local Maps data auto-downloaded when I have reception (including restaurants, reviews, image thumbnails, etc), Wikipedia, wallets (car keys, credit cards, concert/movie tickets, email, daily news, music/podcasts/YouTube that I’ve subscribed to. All of this should be automatically synced when I have reception (podcasts and music already do es this). 2. Enough compute to translate voice to text (and between languages), set a simple timer/alarm, unlock my car, search through Wikipedia, search for restaurants, directions between points on the map (without using traffic data), open an App or play downloaded music/podcasts/YouTube (including the option to chrome-cast without wifi - using BlueTooth (or similar with high bandwidth), NFC/UWB for simple pairing). 3. Auto-upgrade to use the network when it’s available, like use current traffic data for directions, hi-res images/audio/video. Bonus: When offline, Apps like Spotify, Gmail, etc should also run some local search options for downloaded data. Bonus: When offline/online, restaurants should have NFC/UWB servers for their menu. My iPhone should just “download” the menu from the POS system. No need for “signal” or wifi.
      - spullara 5y ago
        1. Some are possible now: https://apps.apple.com/us/app/minipedia-offline-wikipedia/id473512078 https://apps.apple.com/us/app/minipedia-offline-wikipedia/id.... 2. Airplay doesn't require a wifi network. Just wifi on, it uses the same thing as Airdrop. Bonus 1: you can search your offline library in spotify. Bonus 2: this used to be common back in the 00s but it was generally decided to be mostly spam. Bonus: local search for downloaded spotify stuff works without a connection.
- propogandist 6y ago
  yet they can't manage offline Maps and directions without internet connectivity.
banana_giraffe 6y ago
It runs on the phone, no doubt using whatever model they use to enable the keyword searching in the camera-roll. Just verified it all works in airplane mode.
machello13 6y ago
It's your phone.

by Lammy 6y ago

Neat. Is my phone classifying all this stuff, or is it sending all my photos to some Apple server to do it? I watched the video in the OP but it didn't go into any technical details.

spullara 6y ago

I don't know of any ML stuff that Apple does serverside. That is why they have invested so much in ML processing on their devices.

user-the-name 6y ago
I guess Siri does work at least partially serverside?
- spullara 6y ago
  The ML part is local for the voice transcription but the questions almost always need to be answered from the server. But if you ask to play music or set a timer for example that is all local. Also works in Airplane mode.
yazaddaruvala 6y ago
All ML on device definitely seems like a goal for Apple[1]. However, right now Siri doesn't work in airplane mode. Maybe its still too power hungry to run locally? Maybe they still need the data from the requests server side to improve the model? I hope both Siri and Maps, and especially their integration, i.e. "Siri navigate to ____." "Siri find restaurants nearby (or along the way)" works on device in the near future. Its annoying for the CarPlay integration to stop working when you're in a remote area and need the navigational help. [1] https://appleinsider.com/articles/18/11/15/apple-considering-offline-mode-for-siri-that-could-process-voice-locally-on-an-iphone https://appleinsider.com/articles/18/11/15/apple-considering...
- alblue 6y ago
  Siri’s voice translation works in your phone but the answers it needs are all calculated from the server (on the whole). Maps, for example, are all stored server side and route calculation are server side processes as well, as are the locations of restaurants. There’s a difference been decoding voice requests and being able to answer any questions you may have; you wouldn’t expect Google web search or Wikipedia to work in offline mode, so why do you expect it to work for Siri?
  - yazaddaruvala 5y ago
    I don’t expect it to work :) I expect that it will one day soon work offline. Likely not a full web search. By 202X, while offline my device should have: 1. Enough storage for voice to text models, translation models, local Maps data auto-downloaded when I have reception (including restaurants, reviews, image thumbnails, etc), Wikipedia, wallets (car keys, credit cards, concert/movie tickets, email, daily news, music/podcasts/YouTube that I’ve subscribed to. All of this should be automatically synced when I have reception (podcasts and music already do es this). 2. Enough compute to translate voice to text (and between languages), set a simple timer/alarm, unlock my car, search through Wikipedia, search for restaurants, directions between points on the map (without using traffic data), open an App or play downloaded music/podcasts/YouTube (including the option to chrome-cast without wifi - using BlueTooth (or similar with high bandwidth), NFC/UWB for simple pairing). 3. Auto-upgrade to use the network when it’s available, like use current traffic data for directions, hi-res images/audio/video. Bonus: When offline, Apps like Spotify, Gmail, etc should also run some local search options for downloaded data. Bonus: When offline/online, restaurants should have NFC/UWB servers for their menu. My iPhone should just “download” the menu from the POS system. No need for “signal” or wifi.
    - spullara 5y ago
      1. Some are possible now: https://apps.apple.com/us/app/minipedia-offline-wikipedia/id473512078 https://apps.apple.com/us/app/minipedia-offline-wikipedia/id.... 2. Airplay doesn't require a wifi network. Just wifi on, it uses the same thing as Airdrop. Bonus 1: you can search your offline library in spotify. Bonus 2: this used to be common back in the 00s but it was generally decided to be mostly spam. Bonus: local search for downloaded spotify stuff works without a connection.
propogandist 6y ago
yet they can't manage offline Maps and directions without internet connectivity.

banana_giraffe 6y ago

It runs on the phone, no doubt using whatever model they use to enable the keyword searching in the camera-roll. Just verified it all works in airplane mode.

machello13 6y ago

It's your phone.