6 ms·

Weird how the title says, "macOS Sequoia is available today, bringing iPhone Mirroring, Apple Intelligence, and more to Mac" Then you look at the paragraph for

Weird how the title says, "macOS Sequoia is available today, bringing iPhone Mirroring, Apple Intelligence, and more to Mac" Then you look at the paragraph for this and it says, "Coming Soon: Apple Intelligence" They announced this in June, and it's just loosely coming "this fall." I have a feeling they are finding it very difficult to do anything with any kind of consistency or value on their 8gb RAM phones / Macbooks. Their models they have demonstrated so far are a relatively low parameter count with LoRA adapters for various use cases. I have a feeling that the majority of Apple Intelligence will eventually just be farmed out to their "secure cloud" or whatever they were calling it.

bredren 2y ago
A commonly cited analyst described lower than expected sales of iPhone 16 Pro due to AI features not rolling out until October. https://www.macrumors.com/2024/09/15/iphone-16-pro-demand-estimate/ https://www.macrumors.com/2024/09/15/iphone-16-pro-demand-es... I find that a little hard to believe. I would suppose the vast majority of customers don't think a lot about the timing of software features in a purchase decision.
- freeone3000 2y ago
  It’s not as if the hardware features on the 16 Pro are groundbreaking — a slightly nicer camera, a slightly nicer display, a slightly nicer battery life, and a $600 upgrade fee. No, thank you, the software features are what make-or-break the upgrade.
  - shinycode 2y ago
    Phones nowadays are made to last years, even 6+ years easily. So the update this year is not for the people who bought it last year but for people with an iPhone X or 11. They will see major improvements. But it’s silly to update (any) flagship phone every year.
    - freeone3000 2y ago
      Okay, but then, why upgrade to a 16 Pro instead of the much cheaper 15 Pro? The price difference applies there too — there needs to be some reason to get the newest, most expensive one. This generation it is not hardware.
      - shinycode 2y ago
        Well this applies to everything then. Why buy the latest AMD or Intel CPU or Nvidia GPU ? Some people upgrade from iPhone X to 15 this year because it’s cheaper. Some have an old iPhone and just want the latest model to go for 6 or 7 more years. Even incremental upgrades like this year are welcome. Before there was an S model every other year. This year it’s the « iPhone 15S pro ». Slightly better and more powerful. The other thing is, I think it’s pretty hard to say « well last year model was good enough, we’ll go on a 2 year cycle this time ». It could send the wrong message to investors so they’re trapped in the race maybe ?
        freeone3000 2y ago
        Usually because there’s some advantage — you actually need/want the performance offered by Zen4 over Zen3, you need four-cycle AXV512, you have a target framerate in a game, you need RTX or CUDA14… because if you don’t, obviously buying older and used saves you money and you don’t need the new stuff anyway.
- philistine 2y ago
  Ming-Chi Kuo is the golden standard for reports on Apple's production apparatus. He has the best sources, the best leaks, the best insight into very specific discussions regarding parts acquisition for Apple. When it comes to what Apple does with those parts, or market segmentation, or sale projection, he's got no special knowledge. In this specific case, pardon the technical language, but he's full of ...
- yunwal 2y ago
  I would guess that the vast majority of customers don’t even know when a new iPhone is released and therefore don’t buy in the first few weeks after it’s released. Of the customers that are aware of the release, I would assume that most are aware because they’re anticipating new features
- eli 2y ago
  If you're not interested in AI features I'm not sure why you would wait at all rather than just buy a 15 Pro
- toofy 2y ago
  absolutely anecdotal, but from my experience, people who are normally excited for iPhones really don’t care about AI at all. i think Apple hyping AI so much was a mistake, doesn’t have nearly the same impact to them as hyping up a new camera feature or a new screen or a new color or something.
- llm_nerd 2y ago
  The same AI features are coming to the entire iPhone 16 line. The major benefit of the 16 Pro, just as with prior generations, is a faster SoC and better cameras. The only reason the iPhone 15 Pro was the only of the 15 generation to get the AI stuff is that they happened to put 8GB in that device, giving it enough headroom to not majorly impact user enjoyment trying to cram a multi-GB model into memory. But all of the iPhone 16s have 8GB (+?)
  - drexlspivey 2y ago
    Isn't RAM super cheap? Why can't they ship an iPhone with 16GB RAM to run all the LLMs locally?
- latexr 2y ago
  > I would suppose the vast majority of customers don't think a lot about the timing of software features in a purchase decision. The ones who buy on day one do. I don’t find it surprising that the people who even know about and anticipate the release date of a product are the ones who are eager for specific features. If those aren’t there yet, there’s no rush to buy. I’m no analyst but I expect demand in the EU won’t be that (comparatively) great either. When the flagship feature isn’t available, there’s little reason to spend the extra money.
llm_nerd 2y ago
iOS 18.1 and macOS 15.1 -- the versions with the full Apple Intelligence -- are both in betas, and there are enormous numbers of people blogging and vlogging about their experiences. And yes we all realize that on-device models aren't going to be competing with trillion parameter models, but people are finding it pretty useful. They just don't feel like it's release ready so they're refining. Apple has done this with major releases over several iterations now where a couple of features are held back for the .1 release while it's perfected.
- andrewinardeer 2y ago
  'perfected' is an interesting word choice. I'd be more inclined to use 'iterated'.
  - whyenot 2y ago
    "iterated" implies that there is no improvement. Why do you thing 15.1 wouldn't be an improvement over 15.0? I do agree with you that "perfected" is also not the correct word choice. I think I would have gone with "refined" or "improved"
- avidphantasm 2y ago
  Unless it can be completely turned off I will never upgrade and I guess I will be selling my year-old M3 Max in favor of some shitty PC (or I’ll eventually just run Asahi once it supports my hardware well).
  - dcre 2y ago
    Why is this the line for you? macOS is already doing plenty of things it sounds like you wouldn't like.
    - avidphantasm 2y ago
      I don’t want an LLM taking 8GB of RAM to do things I don’t value.
  - 2y ago
someNameIG 2y ago
[flagged]
- lnxg33k1 2y ago
  This is such an Apple brainwash, like how many apps in background you think has everyone else, my linuxbox has almost none, the rest is just non-linux from third party, unless you only install apps from Apple and like having a subpar experience, everything you install is just from third party on which they have no control on. And killing apps doesn't mean anything for AI, it doesn't run anything useful on 8GB, doesn't run anything useful on 8GB, no matter if you kill all the processes. I wish I could just have a conversation where people doesn't cyborgically repeats Apple's marketing bullshit and turn on the human brain
  - someNameIG 2y ago
    I'm not sure how comparing a mobile OS like iOS to a linux box is useful. iOS can kill background apps from memory outside the app developers control.
    - lnxg33k1 2y ago
      Because other OSes don't kill OOM apps? Or any app they want? It is only Apple and only possible on mobile? The only thing that apple has, is the inability to have background apps, so that if you want to use third party sync services, like syncthing, you can't and have to stick to apple products, the rest is just like anyone else
      - veilrap 2y ago
        iOS isn’t killing OOM apps. It’s killing inactive apps. It’s something that wouldn’t fly on a desktop or server OS under general use, but works reasonably well in the mobile space.
- vbezhenar 2y ago
  I think it's actually easier on macOS because you can just swap everything if necessary. For some reason I don't understand, iOS does not employ swap, so while it can kill background apps, it can't kill foreground app.
halJordan 2y ago
Apple doesn't say how big the server Foundational model is, but it scores around Llama 3 70B, so that range should be it.
Retr0id 2y ago
I wouldn't be surprised if they're struggling full-stop. Doing anything with "apple-level" polish and reliability on current-gen LLMs is a challenge. It looks great in demos and maybe even 95% of real-world use cases, but the remaining 5% tends to be embarrassing. And it's not just a case of "the last 5% takes 95% of the work", it's more like "the last 5% is an open research problem".
- minot 2y ago
  > "the last 5% is an open research problem". That is the biggest hurdle, in my opinion. If we could even reply with, "sorry, I don't know about that", it would be such an improvement over what we have today. Sadly, from what I understand, the only way to say "sorry, I don't know about that" is to just say that to every single question?
  - Filligree 2y ago
    There's no specific reason why LLMs couldn't be trained to say "Don't know" when they don't know. Indeed, some close examination shows separate calculation patterns when it's telling the truth, when it's making a mistake and when it's deliberately bullshitting, with the latter being painfully common. The problem is we don't train them that way. They're trained on what data is on the internet, and people... people really aren't good at saying "I don't know". Applying RLHF on top of that at least helps reduce the deliberate lies, but it isn't normal to give a thumbs-up to an "I don't know" response either. ... Of course, all this stuff does seem fixable.
    - mikepurvis 2y ago
      Do you think it's really a training set problem? I don't think you learn to say that you don't understand by observing people say it, you learn to say it by being introspective about how much you have actually comprehended, understanding when your thinking is going in multiple conflicting directions and you don't know which is correct, etc. Kids learn to express confusion and uncertainty in an environment where their parents are always very confident of everything. Overall though, I agree that this is the biggest issue right now in the AI space; instead of being able to cut itself off, the system just rambles and hallucinates and makes stuff up out of whole cloth.
zozbot234 2y ago
My understanding is that 8GB is plenty enough to play with small models (7B or so) in a lightly quantized version. (Especially since model params are kept on disk and only really "mapped" to RAM, with it acting as a cache.) 4GB is where it gets dicey, as you're constrained to tiny models that just don't do anything very interesting.
- jazzyjackson 2y ago
  7B llama 3.1 takes up 5GB ram loaded up in LM Studio, Ive never seen macos idle below 5GB on its own but maybe it can pull some swap magic.
- fhdsgbbcaA 2y ago
  Yes but if it is running system-wide services the issue is what is left over? They should not be selling 8GB machines, it was always about being greedy with upgrades. Now they painted themselves in a corner.
- int_19h 2y ago
  Thing is, you do need all the params usually, so if the model is only partially mapped to RAM, it's the equivalent of an app swapping in and out as it runs. Which is to say, it means that inference is much slower. Local Apple models are likely in the 2-3B range, but fine-tuned to specific tasks.
deleted 2y ago
[deleted]

bredren 2y ago

A commonly cited analyst described lower than expected sales of iPhone 16 Pro due to AI features not rolling out until October. https://www.macrumors.com/2024/09/15/iphone-16-pro-demand-estimate/ https://www.macrumors.com/2024/09/15/iphone-16-pro-demand-es... I find that a little hard to believe. I would suppose the vast majority of customers don't think a lot about the timing of software features in a purchase decision.

freeone3000 2y ago
It’s not as if the hardware features on the 16 Pro are groundbreaking — a slightly nicer camera, a slightly nicer display, a slightly nicer battery life, and a $600 upgrade fee. No, thank you, the software features are what make-or-break the upgrade.
- shinycode 2y ago
  Phones nowadays are made to last years, even 6+ years easily. So the update this year is not for the people who bought it last year but for people with an iPhone X or 11. They will see major improvements. But it’s silly to update (any) flagship phone every year.
  - freeone3000 2y ago
    Okay, but then, why upgrade to a 16 Pro instead of the much cheaper 15 Pro? The price difference applies there too — there needs to be some reason to get the newest, most expensive one. This generation it is not hardware.
    - shinycode 2y ago
      Well this applies to everything then. Why buy the latest AMD or Intel CPU or Nvidia GPU ? Some people upgrade from iPhone X to 15 this year because it’s cheaper. Some have an old iPhone and just want the latest model to go for 6 or 7 more years. Even incremental upgrades like this year are welcome. Before there was an S model every other year. This year it’s the « iPhone 15S pro ». Slightly better and more powerful. The other thing is, I think it’s pretty hard to say « well last year model was good enough, we’ll go on a 2 year cycle this time ». It could send the wrong message to investors so they’re trapped in the race maybe ?
      - freeone3000 2y ago
        Usually because there’s some advantage — you actually need/want the performance offered by Zen4 over Zen3, you need four-cycle AXV512, you have a target framerate in a game, you need RTX or CUDA14… because if you don’t, obviously buying older and used saves you money and you don’t need the new stuff anyway.
philistine 2y ago
Ming-Chi Kuo is the golden standard for reports on Apple's production apparatus. He has the best sources, the best leaks, the best insight into very specific discussions regarding parts acquisition for Apple. When it comes to what Apple does with those parts, or market segmentation, or sale projection, he's got no special knowledge. In this specific case, pardon the technical language, but he's full of ...
yunwal 2y ago
I would guess that the vast majority of customers don’t even know when a new iPhone is released and therefore don’t buy in the first few weeks after it’s released. Of the customers that are aware of the release, I would assume that most are aware because they’re anticipating new features
eli 2y ago
If you're not interested in AI features I'm not sure why you would wait at all rather than just buy a 15 Pro
toofy 2y ago
absolutely anecdotal, but from my experience, people who are normally excited for iPhones really don’t care about AI at all. i think Apple hyping AI so much was a mistake, doesn’t have nearly the same impact to them as hyping up a new camera feature or a new screen or a new color or something.
llm_nerd 2y ago
The same AI features are coming to the entire iPhone 16 line. The major benefit of the 16 Pro, just as with prior generations, is a faster SoC and better cameras. The only reason the iPhone 15 Pro was the only of the 15 generation to get the AI stuff is that they happened to put 8GB in that device, giving it enough headroom to not majorly impact user enjoyment trying to cram a multi-GB model into memory. But all of the iPhone 16s have 8GB (+?)
- drexlspivey 2y ago
  Isn't RAM super cheap? Why can't they ship an iPhone with 16GB RAM to run all the LLMs locally?
latexr 2y ago
> I would suppose the vast majority of customers don't think a lot about the timing of software features in a purchase decision. The ones who buy on day one do. I don’t find it surprising that the people who even know about and anticipate the release date of a product are the ones who are eager for specific features. If those aren’t there yet, there’s no rush to buy. I’m no analyst but I expect demand in the EU won’t be that (comparatively) great either. When the flagship feature isn’t available, there’s little reason to spend the extra money.

llm_nerd 2y ago

iOS 18.1 and macOS 15.1 -- the versions with the full Apple Intelligence -- are both in betas, and there are enormous numbers of people blogging and vlogging about their experiences. And yes we all realize that on-device models aren't going to be competing with trillion parameter models, but people are finding it pretty useful. They just don't feel like it's release ready so they're refining. Apple has done this with major releases over several iterations now where a couple of features are held back for the .1 release while it's perfected.

andrewinardeer 2y ago
'perfected' is an interesting word choice. I'd be more inclined to use 'iterated'.
- whyenot 2y ago
  "iterated" implies that there is no improvement. Why do you thing 15.1 wouldn't be an improvement over 15.0? I do agree with you that "perfected" is also not the correct word choice. I think I would have gone with "refined" or "improved"
avidphantasm 2y ago
Unless it can be completely turned off I will never upgrade and I guess I will be selling my year-old M3 Max in favor of some shitty PC (or I’ll eventually just run Asahi once it supports my hardware well).
- dcre 2y ago
  Why is this the line for you? macOS is already doing plenty of things it sounds like you wouldn't like.
  - avidphantasm 2y ago
    I don’t want an LLM taking 8GB of RAM to do things I don’t value.
- 2y ago

someNameIG 2y ago

[flagged]

lnxg33k1 2y ago
This is such an Apple brainwash, like how many apps in background you think has everyone else, my linuxbox has almost none, the rest is just non-linux from third party, unless you only install apps from Apple and like having a subpar experience, everything you install is just from third party on which they have no control on. And killing apps doesn't mean anything for AI, it doesn't run anything useful on 8GB, doesn't run anything useful on 8GB, no matter if you kill all the processes. I wish I could just have a conversation where people doesn't cyborgically repeats Apple's marketing bullshit and turn on the human brain
- someNameIG 2y ago
  I'm not sure how comparing a mobile OS like iOS to a linux box is useful. iOS can kill background apps from memory outside the app developers control.
  - lnxg33k1 2y ago
    Because other OSes don't kill OOM apps? Or any app they want? It is only Apple and only possible on mobile? The only thing that apple has, is the inability to have background apps, so that if you want to use third party sync services, like syncthing, you can't and have to stick to apple products, the rest is just like anyone else
    - veilrap 2y ago
      iOS isn’t killing OOM apps. It’s killing inactive apps. It’s something that wouldn’t fly on a desktop or server OS under general use, but works reasonably well in the mobile space.
vbezhenar 2y ago
I think it's actually easier on macOS because you can just swap everything if necessary. For some reason I don't understand, iOS does not employ swap, so while it can kill background apps, it can't kill foreground app.

halJordan 2y ago

Apple doesn't say how big the server Foundational model is, but it scores around Llama 3 70B, so that range should be it.

Retr0id 2y ago

I wouldn't be surprised if they're struggling full-stop. Doing anything with "apple-level" polish and reliability on current-gen LLMs is a challenge. It looks great in demos and maybe even 95% of real-world use cases, but the remaining 5% tends to be embarrassing. And it's not just a case of "the last 5% takes 95% of the work", it's more like "the last 5% is an open research problem".

minot 2y ago
> "the last 5% is an open research problem". That is the biggest hurdle, in my opinion. If we could even reply with, "sorry, I don't know about that", it would be such an improvement over what we have today. Sadly, from what I understand, the only way to say "sorry, I don't know about that" is to just say that to every single question?
- Filligree 2y ago
  There's no specific reason why LLMs couldn't be trained to say "Don't know" when they don't know. Indeed, some close examination shows separate calculation patterns when it's telling the truth, when it's making a mistake and when it's deliberately bullshitting, with the latter being painfully common. The problem is we don't train them that way. They're trained on what data is on the internet, and people... people really aren't good at saying "I don't know". Applying RLHF on top of that at least helps reduce the deliberate lies, but it isn't normal to give a thumbs-up to an "I don't know" response either. ... Of course, all this stuff does seem fixable.
  - mikepurvis 2y ago
    Do you think it's really a training set problem? I don't think you learn to say that you don't understand by observing people say it, you learn to say it by being introspective about how much you have actually comprehended, understanding when your thinking is going in multiple conflicting directions and you don't know which is correct, etc. Kids learn to express confusion and uncertainty in an environment where their parents are always very confident of everything. Overall though, I agree that this is the biggest issue right now in the AI space; instead of being able to cut itself off, the system just rambles and hallucinates and makes stuff up out of whole cloth.

zozbot234 2y ago

My understanding is that 8GB is plenty enough to play with small models (7B or so) in a lightly quantized version. (Especially since model params are kept on disk and only really "mapped" to RAM, with it acting as a cache.) 4GB is where it gets dicey, as you're constrained to tiny models that just don't do anything very interesting.

jazzyjackson 2y ago
7B llama 3.1 takes up 5GB ram loaded up in LM Studio, Ive never seen macos idle below 5GB on its own but maybe it can pull some swap magic.
fhdsgbbcaA 2y ago
Yes but if it is running system-wide services the issue is what is left over? They should not be selling 8GB machines, it was always about being greedy with upgrades. Now they painted themselves in a corner.
int_19h 2y ago
Thing is, you do need all the params usually, so if the model is only partially mapped to RAM, it's the equivalent of an app swapping in and out as it runs. Which is to say, it means that inference is much slower. Local Apple models are likely in the 2-3B range, but fine-tuned to specific tasks.

deleted 2y ago

[deleted]