3 ms·

I hope Apple gives us both a web API (that can be used on any platform at a low-ish cost, similar to OpenAI) and a smaller, private model that's optimally sized

I hope Apple gives us both a web API (that can be used on any platform at a low-ish cost, similar to OpenAI) and a smaller, private model that's optimally sized to run on Mac, iPad, and Phone hardware.

deleted 3y ago
[deleted]
TillE 3y ago
Does Apple do general-purpose web APIs? The only one I can think of is WeatherKit. Seems like any AI stuff will be designed to add value to its hardware exclusively, probably in conjunction with its on-device Neural Engine (tm).
- ch_sm 3y ago
  The only other one I know is the Apple Maps API. I think it‘s really likely that this will just be an improved version of Siri and not stand-alone product.
  - capableweb 3y ago
    Some more from the top of my mind: - Apple Pay JS - MusicKit JS - Sign in with Apple JS - App Store Connect API (not really user facing, but still)
- StackOverlord 3y ago
  https://www.icloud.com/ https://www.icloud.com/
cubefox 3y ago
"Smaller" models still require excessive RAM while not producing acceptable results for conversation. LLMs will probably always run in the cloud, anything else will just be too weak for most applications.
- zmarty 3y ago
  I have been trying the small llama 2 models (7B and 13B) and they are now good enough for most of my NLP tasks. Plus, Qualcomm will support llama models on mobile devices: https://www.qualcomm.com/news/releases/2023/07/qualcomm-works-with-meta-to-enable-on-device-ai-applications-usi https://www.qualcomm.com/news/releases/2023/07/qualcomm-work... soon. I think Apple will come up with some crazy hardware to run good quality LLMs.
  - cubefox 3y ago
    What are those NLP tasks, if I may ask? (I was thinking above about using it as a chatbot like ChatGPT or Bard, which currently seems the only application for end-users.)
    - zmarty 3y ago
      News summarization, news data extraction, news question answering, news filtering. I can assure you that older 7B/13B models had trouble following directions and outputting (for example) JSON.
      - cubefox 3y ago
        I'm pretty sure Apple won't offer those things locally on iPhones. The hardware requirement is too high and the value to average Apple customers too small.
        LoganDark 3y ago
        > The hardware requirement is too high if you use commodity GPUs, sure. if you use TPUs (which Apple is already building into their chips) the efficiency improvements are massive. seriously look at some Coral Edge TPUs and what they can do at power levels completely unheard of for GPUs. then look at how much faster M1/M2 Macs are than normal desktop GPUs for machine learning tasks because they have an onboard accelerator

deleted 3y ago

[deleted]

TillE 3y ago

Does Apple do general-purpose web APIs? The only one I can think of is WeatherKit. Seems like any AI stuff will be designed to add value to its hardware exclusively, probably in conjunction with its on-device Neural Engine (tm).

ch_sm 3y ago
The only other one I know is the Apple Maps API. I think it‘s really likely that this will just be an improved version of Siri and not stand-alone product.
- capableweb 3y ago
  Some more from the top of my mind: - Apple Pay JS - MusicKit JS - Sign in with Apple JS - App Store Connect API (not really user facing, but still)
StackOverlord 3y ago
https://www.icloud.com/ https://www.icloud.com/

cubefox 3y ago

"Smaller" models still require excessive RAM while not producing acceptable results for conversation. LLMs will probably always run in the cloud, anything else will just be too weak for most applications.

zmarty 3y ago
I have been trying the small llama 2 models (7B and 13B) and they are now good enough for most of my NLP tasks. Plus, Qualcomm will support llama models on mobile devices: https://www.qualcomm.com/news/releases/2023/07/qualcomm-works-with-meta-to-enable-on-device-ai-applications-usi https://www.qualcomm.com/news/releases/2023/07/qualcomm-work... soon. I think Apple will come up with some crazy hardware to run good quality LLMs.
- cubefox 3y ago
  What are those NLP tasks, if I may ask? (I was thinking above about using it as a chatbot like ChatGPT or Bard, which currently seems the only application for end-users.)
  - zmarty 3y ago
    News summarization, news data extraction, news question answering, news filtering. I can assure you that older 7B/13B models had trouble following directions and outputting (for example) JSON.
    - cubefox 3y ago
      I'm pretty sure Apple won't offer those things locally on iPhones. The hardware requirement is too high and the value to average Apple customers too small.
      - LoganDark 3y ago
        > The hardware requirement is too high if you use commodity GPUs, sure. if you use TPUs (which Apple is already building into their chips) the efficiency improvements are massive. seriously look at some Coral Edge TPUs and what they can do at power levels completely unheard of for GPUs. then look at how much faster M1/M2 Macs are than normal desktop GPUs for machine learning tasks because they have an onboard accelerator