10 ms·

I really think you have hit the nail on the head here. Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised L

I really think you have hit the nail on the head here. Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. We must be talking orders of magnitude differences in operational cost, not to mention completely unique features like privacy. The very definition of disruption, waiting in the wings. Not only that, but this model of 'wait in the wings and pounce when the tech is really there and absolutely nail it' is Apple's forté, aligned completely with the cultural and strategic DNA of the company.

jorvi 4y ago
Siri was supposed to become locally processed but sometimes can’t even set a timer because of a connection time-out. Or she’ll use her fancy ML speech recognition model to turn “set a timer for 3 minutes 10 seconds” into “search for trinity tensor”. So much for an “unfathomable moat”.
- getcrunk 4y ago
  You see the only reason it does that is because your … holding your phone wrong
- belugacat 4y ago
  Siri is so bad it needs to be scrapped and rethought from the ground up (it can’t even give me the time when internet is down at my rural house). No one at Apple who wants a career there would dare propose that. The switch to a LLM architecture could be the perfect transition point for this.
  - 015a 4y ago
    If Siri can't give you the time without the internet, I think you need to update iOS. There are definitely two tiers of Siri queries. There are queries like "set the brightness to 10%" or "set a timer for 5 minutes" which absolutely and consistently work without internet, and have for several years, and if you're legitimately having a different experience then its possible a cosmic ray hit your iPhone (or, realistically, you're running into a strange and rare bug which is not indicative of the general experience and will be fixed in three to five years or maybe never). There are also queries like "create a new note" (from the Notes app) which should be able to work offline, but don't. And, naturally, there are queries like "when did Resident Evil 4 come out" which wouldn't reasonably work without internet (but, if you're curious, she does get it right). In other words, Siri clearly does some local guessing as to whether she can answer a query without the internet, and some queries appear to be miscategorized into the second bucket. My leading theory on why this happens, which may be incorrect, but it seems like: if an app has any Siri functionality which requires the internet to answer, all of the queries which are responded to by that app have to require the internet. It doesn't matter where the processing ends up actually happening to respond to the query, it just shuts the query down. Its weird, but its consistent with the behavior I've seen. The more important point: Siri's real and weird limitations don't seem to have much to do with limitations in local processing. They've said that the speech interpretation all happens on-device. They encrypt practically all of your data that does get shipped to their servers, so a query like "Open the note titled 'Hello World'" is probably also being processed on-device. But: Siri still requires the internet for that query. That doesn't seem like a significant limitation with their ML algorithms or silicon or anything meaningful; it seems like just a case of dumb coding, which is certainly something Apple is no stranger to.
    - king_magic 4y ago
      > If Siri can't give you the time without the internet, I think you need to update iOS. interesting - latest iOS in airplane mode - "hey siri what time is it?" - "you need to turn off airplane mode to do that" but - "hey siri set a time for 5 minutes" - works just fine
      - smoldesu 4y ago
        They must be using the esteemed time delivery API, https://www.whattimeisitrightnow.com/ https://www.whattimeisitrightnow.com/
- deleted 4y ago
  [deleted]
highwaylights 4y ago
Maybe, but Apple doesn’t have a search to rival Google (or even an assistant, given the state of Siri). Focusing on privacy and on-device learning is great, but when the strength of these models is in consuming all the data they can hoover up your motive is at odds with your philosophy.
- fnordpiglet 4y ago
  I think this is wrong because you can always build a search engine - it’s not hard - and you can supplement the learned models with cryptographically secured private data to build a private personalized model. In fact this seems in all ways superior to any alternative, and strikes at the heart of the Google spy ware ad spam business model. And seriously, did you really try to assert the Google whatever it is has outperformed Siri? I’d love to meet someone that uses it so I can verify this statement. But with the advent of Chatgpt Alexa, Siri, and Google whatever all look like freshman projects. The next generation of assistants have just begun, and I assure you, the state of the past has nothing to do with what comes next. OpenAI hit a giant reset button across an awful lot of stuff.
  - coolspot 4y ago
    > And seriously, did you really try to assert the Google whatever it is has outperformed Siri? Anything outperforms Siri. Google Assistant is head and shoulders above Alexa and Siri.
    - schrodinger 4y ago
      I have both and find that 90% of the time my commands are “remind me ___”, “wake me up at ___”, “what’s the weather today”, and “play ___ on Spotify”. For these, I can’t tell the difference between google and Siri in terms of quality (but Alexa was worse).
      - bigDinosaur 4y ago
        I was just setting reminders today and had Siri repeatedly fail to do what I want. The only use case I have for it is setting timers, which is does very well. I'd use it for much more if it was as good as ChatGPT.
arthurcolle 4y ago
> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give her a shot, she disappoints me. It can do cookie-cutter, super well-traveled code paths that were engineered together for demos, basically just one off tricks without cohesion, but the whole infrastructure around Siri is not open and has not been improved in any way in many OS versions in any way that I've discerned... But I am not an expert with these systems or APIs. The Neural Engine architecture especially considering M1/M2 hardware leaps (which are absolutely mind-bogglingly impressive) are both really superb technologies from my layman view as a mere software engineer, with improved battery life as well as performance, a real improvement over Intel x86 architectures - I just don't see Apple as a serious AI player right now, despite head start in AI w/ Siri acq, despite these hardware leaps that may make it easier to do cool stuff in the future like this post. The OCR stuff in Photos is cool and useful and I use it every day, usually to look up my Known Traveler Number. Ramblings over, just wanted to ask you to opine as to Apple LLM related information if you have any additional context!
- feifan 4y ago
  The part that Siri is bad at will be commoditized — someone will open-source a GPT-4-level language model. And Apple's moat will be being able to run that on-device with all the attendant benefits (privacy, zero marginal cost to the company, availability in more scenarios, etc)
  - arthurcolle 4y ago
    yeah, makes sense. At least these integrations at a lower level are happening now. I just can't help but feel disappointed that we didn't see earlier cohesion at least a few years ago, maybe around the early or pre-M1 era when there were all those A7, A8 whatever SoCs. But in retrospect, that is basically 10 minutes ago in the history of the universe so yeah, I'm just overly excitable!
  - 4y ago
ThorsBane 4y ago
Yeah, Apple is slower to adopt some things but I noticed when they hit it...they knock the ball not just out of the park...they elevate the entire game. I dominated a 13" mbp m1 for like two solid years on hardcore startup software engineering. The base model. It was almost entirely perfect until the last few months when I upgraded to the new 16". These are alien devices. They're just not possible yet here I am using one.
wetpaws 4y ago
Apple's exceptional hardware can only be rivaled by the absolute shitfest quality of their software. So no, I'm not buying it.
- lunatuna 4y ago
  Exactly. You can see Apple struggle to build anything of quality with their software stack. Good luck making anything from this. So much great HW sadly wasted.
yyyk 4y ago
> Apple has a ridiculous, almost unfathomably deep moat for.. running personalised... Apple has decent hw but no LLM software. Unfortunately, in LLM space software changes are the ones driving performance for now, since the space hasn't stabilized yet. Since their competitors control the software, they get to adapt it to their hardware. That is, Google and Microsoft are going to adapt GPT/Bard to Qualcomm ARM etc. while Apple is being ignored. Unless Apple gets in with their own LLM (quite possible), their hw advantage will end up not mattering one bit.
zitterbewegung 4y ago
They might not even require the need to train. There are many open implementations of models that don’t have an academic requirement. Right now you can use tensorflow.js which runs on the gpu using webgl of a phone. If you build the API out developers will use it.
josephjrobison 4y ago
In addition, Apple is notoriously slower at adopting the most cutting edge technologies but historically they tend to consider things very thoughtfully and rolls them out in beautiful launches (not always). Examples would be the fact that Samsung almost always beats them to the punch on camera technology and other whiz-bang features, but Apple eventually adopts to much mass market consumer acclaim and groans from Android techies. Another example is they’re just now considering touch screen on laptops. A counter example is the TouchBar - which was “innovative” but many didn’t like.
waffletower 4y ago
iCloud has screamed ineptitude for decades. I don't see how Apple can possibly develop successful products that can't be held in the hand and fetishized by myopic designers. Much like a stereotypical grandparent raised during the Great Depression, Tim Cook still leads like a supply chain miser trying to save a company that has long prospered -- they tragically squander the unprecedented capital at their disposal. Change my mind, Apple please.

jorvi 4y ago

Siri was supposed to become locally processed but sometimes can’t even set a timer because of a connection time-out. Or she’ll use her fancy ML speech recognition model to turn “set a timer for 3 minutes 10 seconds” into “search for trinity tensor”. So much for an “unfathomable moat”.

getcrunk 4y ago
You see the only reason it does that is because your … holding your phone wrong
belugacat 4y ago
Siri is so bad it needs to be scrapped and rethought from the ground up (it can’t even give me the time when internet is down at my rural house). No one at Apple who wants a career there would dare propose that. The switch to a LLM architecture could be the perfect transition point for this.
- 015a 4y ago
  If Siri can't give you the time without the internet, I think you need to update iOS. There are definitely two tiers of Siri queries. There are queries like "set the brightness to 10%" or "set a timer for 5 minutes" which absolutely and consistently work without internet, and have for several years, and if you're legitimately having a different experience then its possible a cosmic ray hit your iPhone (or, realistically, you're running into a strange and rare bug which is not indicative of the general experience and will be fixed in three to five years or maybe never). There are also queries like "create a new note" (from the Notes app) which should be able to work offline, but don't. And, naturally, there are queries like "when did Resident Evil 4 come out" which wouldn't reasonably work without internet (but, if you're curious, she does get it right). In other words, Siri clearly does some local guessing as to whether she can answer a query without the internet, and some queries appear to be miscategorized into the second bucket. My leading theory on why this happens, which may be incorrect, but it seems like: if an app has any Siri functionality which requires the internet to answer, all of the queries which are responded to by that app have to require the internet. It doesn't matter where the processing ends up actually happening to respond to the query, it just shuts the query down. Its weird, but its consistent with the behavior I've seen. The more important point: Siri's real and weird limitations don't seem to have much to do with limitations in local processing. They've said that the speech interpretation all happens on-device. They encrypt practically all of your data that does get shipped to their servers, so a query like "Open the note titled 'Hello World'" is probably also being processed on-device. But: Siri still requires the internet for that query. That doesn't seem like a significant limitation with their ML algorithms or silicon or anything meaningful; it seems like just a case of dumb coding, which is certainly something Apple is no stranger to.
  - king_magic 4y ago
    > If Siri can't give you the time without the internet, I think you need to update iOS. interesting - latest iOS in airplane mode - "hey siri what time is it?" - "you need to turn off airplane mode to do that" but - "hey siri set a time for 5 minutes" - works just fine
    - smoldesu 4y ago
      They must be using the esteemed time delivery API, https://www.whattimeisitrightnow.com/ https://www.whattimeisitrightnow.com/
deleted 4y ago
[deleted]

highwaylights 4y ago

Maybe, but Apple doesn’t have a search to rival Google (or even an assistant, given the state of Siri). Focusing on privacy and on-device learning is great, but when the strength of these models is in consuming all the data they can hoover up your motive is at odds with your philosophy.

fnordpiglet 4y ago
I think this is wrong because you can always build a search engine - it’s not hard - and you can supplement the learned models with cryptographically secured private data to build a private personalized model. In fact this seems in all ways superior to any alternative, and strikes at the heart of the Google spy ware ad spam business model. And seriously, did you really try to assert the Google whatever it is has outperformed Siri? I’d love to meet someone that uses it so I can verify this statement. But with the advent of Chatgpt Alexa, Siri, and Google whatever all look like freshman projects. The next generation of assistants have just begun, and I assure you, the state of the past has nothing to do with what comes next. OpenAI hit a giant reset button across an awful lot of stuff.
- coolspot 4y ago
  > And seriously, did you really try to assert the Google whatever it is has outperformed Siri? Anything outperforms Siri. Google Assistant is head and shoulders above Alexa and Siri.
  - schrodinger 4y ago
    I have both and find that 90% of the time my commands are “remind me ___”, “wake me up at ___”, “what’s the weather today”, and “play ___ on Spotify”. For these, I can’t tell the difference between google and Siri in terms of quality (but Alexa was worse).
    - bigDinosaur 4y ago
      I was just setting reminders today and had Siri repeatedly fail to do what I want. The only use case I have for it is setting timers, which is does very well. I'd use it for much more if it was as good as ChatGPT.

arthurcolle 4y ago

> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices. Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give her a shot, she disappoints me. It can do cookie-cutter, super well-traveled code paths that were engineered together for demos, basically just one off tricks without cohesion, but the whole infrastructure around Siri is not open and has not been improved in any way in many OS versions in any way that I've discerned... But I am not an expert with these systems or APIs. The Neural Engine architecture especially considering M1/M2 hardware leaps (which are absolutely mind-bogglingly impressive) are both really superb technologies from my layman view as a mere software engineer, with improved battery life as well as performance, a real improvement over Intel x86 architectures - I just don't see Apple as a serious AI player right now, despite head start in AI w/ Siri acq, despite these hardware leaps that may make it easier to do cool stuff in the future like this post. The OCR stuff in Photos is cool and useful and I use it every day, usually to look up my Known Traveler Number. Ramblings over, just wanted to ask you to opine as to Apple LLM related information if you have any additional context!

feifan 4y ago
The part that Siri is bad at will be commoditized — someone will open-source a GPT-4-level language model. And Apple's moat will be being able to run that on-device with all the attendant benefits (privacy, zero marginal cost to the company, availability in more scenarios, etc)
- arthurcolle 4y ago
  yeah, makes sense. At least these integrations at a lower level are happening now. I just can't help but feel disappointed that we didn't see earlier cohesion at least a few years ago, maybe around the early or pre-M1 era when there were all those A7, A8 whatever SoCs. But in retrospect, that is basically 10 minutes ago in the history of the universe so yeah, I'm just overly excitable!
- 4y ago

ThorsBane 4y ago

Yeah, Apple is slower to adopt some things but I noticed when they hit it...they knock the ball not just out of the park...they elevate the entire game. I dominated a 13" mbp m1 for like two solid years on hardcore startup software engineering. The base model. It was almost entirely perfect until the last few months when I upgraded to the new 16". These are alien devices. They're just not possible yet here I am using one.

wetpaws 4y ago

Apple's exceptional hardware can only be rivaled by the absolute shitfest quality of their software. So no, I'm not buying it.

lunatuna 4y ago
Exactly. You can see Apple struggle to build anything of quality with their software stack. Good luck making anything from this. So much great HW sadly wasted.

yyyk 4y ago

> Apple has a ridiculous, almost unfathomably deep moat for.. running personalised... Apple has decent hw but no LLM software. Unfortunately, in LLM space software changes are the ones driving performance for now, since the space hasn't stabilized yet. Since their competitors control the software, they get to adapt it to their hardware. That is, Google and Microsoft are going to adapt GPT/Bard to Qualcomm ARM etc. while Apple is being ignored. Unless Apple gets in with their own LLM (quite possible), their hw advantage will end up not mattering one bit.

zitterbewegung 4y ago

They might not even require the need to train. There are many open implementations of models that don’t have an academic requirement. Right now you can use tensorflow.js which runs on the gpu using webgl of a phone. If you build the API out developers will use it.

josephjrobison 4y ago

In addition, Apple is notoriously slower at adopting the most cutting edge technologies but historically they tend to consider things very thoughtfully and rolls them out in beautiful launches (not always). Examples would be the fact that Samsung almost always beats them to the punch on camera technology and other whiz-bang features, but Apple eventually adopts to much mass market consumer acclaim and groans from Android techies. Another example is they’re just now considering touch screen on laptops. A counter example is the TouchBar - which was “innovative” but many didn’t like.

waffletower 4y ago

iCloud has screamed ineptitude for decades. I don't see how Apple can possibly develop successful products that can't be held in the hand and fetishized by myopic designers. Much like a stereotypical grandparent raised during the Great Depression, Tim Cook still leads like a supply chain miser trying to save a company that has long prospered -- they tragically squander the unprecedented capital at their disposal. Change my mind, Apple please.