4 ms·

Apple has the best consumer local hardware for AI, and then ... Siri. How did the gap between hardware and software that could fully utilise it grow so large?

• wilg 2y ago
  I think pretty much because the neural engine hardware stuff was to accelerate various existing ML features (face detection, voice recognition, etc) and not any kind of foresight about how ML would become relevant for LLM-type AI features (which can mostly use the cloud anyway so local isn't that important).
  • astrange 2y ago
    It's not "accelerate", that isn't important on a battery powered device and you shouldn't primarily think about it as a developer. Everything is about perf/power ratio. The LLM does run on the neural engine though. LLMs are simple architecturally.
    • wilg 2y ago
      doing less work = less power = accelerate, no?
      • astrange 2y ago
        Either slower or faster can use less power, depends on the details.
• Apocryphon 2y ago
  Siri has always been of dubious quality, the gap has been growing for a decade.