3 ms·

Lunar Lake is very clearly a response to the M1, not its larger siblings: the core counts, packaging, and power delivery changes all line up with the M1 and suc

Lunar Lake is very clearly a response to the M1, not its larger siblings: the core counts, packaging, and power delivery changes all line up with the M1 and successors. Lunar Lake isn't intended to scale up to the power (or price) ranges of Apple's Pro/Max chips. So this is definitely not the product where you could expect Intel to start using a wider memory bus. And there's very little benefit to widening the memory bus past 128-bit unless you have a powerful GPU to make good use of that bandwidth. There are comparatively few consumer workloads for CPUs that are sufficiently bandwidth-hungry.

nox101 2y ago
with all of the local ML being introduced by Apple and Google and Microsoft this thinking seems close to "640k is all you need" I suspect consumer workloads to rise
- throwuxiytayq 2y ago
  I think the number of people interested in running ML models locally might be greatly overestimated [here]. There is no killer app in sight that needs to run locally. People work and store their stuff in the cloud. Most people just want a lightweight laptop, and AI workloads would drain the battery and cook your eggs in a matter of minutes, assuming you can run them. Production quality models are pretty much cloud only, and I don’t think open source models, especially ones viable for local inference will close the gap anytime soon. I’d like all of those things to be different, but I think that’s just the way things are. Of course there are enthusiasts, but I suspect that they prefer and will continue to prefer dedicated inference hardware.
  - Onavo 2y ago
    > I think the number of people interested in running ML models locally might be greatly overestimated [here]. There is no killer app in sight that needs to run locally. People work and store their stuff in the cloud. Most people just want a lightweight laptop, and AI workloads would drain the battery and cook your eggs in a matter of minutes, assuming you can run them. Production quality models are pretty much cloud only, and I don’t think open source models, especially ones viable for local inference will close the gap anytime soon. I’d like all of those things to be different, but I think that’s just the way things are. Of course there are enthusiasts, but I suspect that they prefer and will continue to prefer dedicated inference hardware. Do you use FTP instead of Dropbox?
    - cvs268 2y ago
      Do you use Dropbox instead of rsync? ( Couldn't resist (^.^) )
  - tucnak 2y ago
    > AI workloads would drain the battery and cook your eggs in a matter of minutes, assuming you can run them M2 Max is passively cooled... and does 1/2 of 4090's token bandwidth in inference.
    - MindSpunk 2y ago
      Um, no? The MBP has fans? The base level M2 in a MacBook Air is passively cooled and thermal throttles very aggressively when you push it.
      - tucnak 2y ago
        That sounds right, I was judging by mac studio I own that is passively cooled.
  - 0x000xca0xfe 2y ago
    Microsoft wants to bring Recall back. When ML models come as part of the OS suddenly there are hundreds of millions of users.
    - layer8 2y ago
      It remains to be seen whether users want what Microsoft wants.
    - throwuxiytayq 2y ago
      I have some difficulty with estimating how heavy Recall’s workload is, but either way, I have little faith in Microsoft’s ability to implement this feature efficiently. They struggle with much simpler features, such as search. I wouldn’t be surprised if a lot of people disable the feature to save battery life and improve system performance.
  - fasa99 2y ago
    Huh? All the files are local and models are gonna be made to have a lot of or ultimately all of them in the "context window". You can't really have an AI for your local documents on the cloud because the cloud doesn't have access. Same logic for businesses. The use case follows from data availability and barriers. We've observed the same on web pages where more and more functionality gets pushed to the frontend. One could push the last (x) layers of the neural net for example, to the frontend, for lower expense and if rightly engineered, better speed and scalability. AIs will be local, super-AIs still in the cloud. Local AIs will be proprietary and they will have strings to the mothership The strings will have business value both related to the consumer and for further AI training.
    - throwuxiytayq 2y ago
      > All the files are local From what I’ve seen, most people tend to use cloud document platforms. Microsoft has made Office into one. This has been the steady direction for the last few years; they’ll keep pushing for it, because it gives them control. Native apps with local files is an inconvenient model for them. This sadly applies to most other types of apps. On many of these cloud platforms, you can’t even download the project files. > You can't really have an AI for your local documents on the cloud because the cloud doesn't have access Yes, up to the cloud it all goes. They don’t mind, they can charge you for it. Microsoft literally openly wants to move the whole OS to the cloud. > Same logic for businesses Businesses hate local files. They’re a huge liability. When firing people it’s very convenient that you can just cut someone off by blocking their cloud credentials. > We've observed the same on web pages where more and more functionality gets pushed to the frontend It will never go all the way. > One could push the last (x) layers of the neural net for example, to the frontend, for lower expense and if rightly engineered, better speed and scalability I’ll believe it when I see it. I don’t think the incentive is there. Sounds like a huge complicating factor. It’s much simpler to keep everything running in the cloud, and software architects strongly prefer simple designs. How much do these layers weigh? How many MB/GB of data will I need to transfer? How often? Does that really give me better latency than just transferring a few KBs of the AIs output?
- wtallis 2y ago
  Local ML isn't a CPU workload. The NPUs in mobile processors (both laptop and smartphone) are optimized for low power and low precision, which limit how much memory bandwidth they can demand. So as I said, demand for more memory bandwidth depends mainly on how powerful the GPU is.
epolanski 2y ago
The few reviews we have seen now show that lunar lake is competitive with m3s too depending on the application.
formerly_proven 2y ago
Is the full memory bandwidth actually available to the CPU on M-series CPUs? Because that would seem like a waste of silicon to me, to have 200+ GB/s of past-LLC bandwidth for eight cores or so.
- adrian_b 2y ago
  I do not know for the latest models, but at least in M1 the CPU was limited to a fraction of the total memory throughput, IIRC to about 100 GB/s.
- inkyoto 2y ago
  100 Gb/sec per a CPU core. 200+ Gb/sec (IIRC Anandtech measured 240 Gb/sec) per a cluster. Pro is one cluster, Max is two clusters and Ultra is four clusters, so the accumulative bandwidth is 200, 400 and 800 Gb/sec respectively[*]. The bandwidth is also shared with GPU and NPU cores within the same cluster, so on combined loads it is plausible that the memory bus may become fully saturated. [*] Starting with M3, Apple has pivoted Pro models into more Pro and less Pro versions that have differing memory bus widths.

nox101 2y ago

with all of the local ML being introduced by Apple and Google and Microsoft this thinking seems close to "640k is all you need" I suspect consumer workloads to rise

throwuxiytayq 2y ago
I think the number of people interested in running ML models locally might be greatly overestimated [here]. There is no killer app in sight that needs to run locally. People work and store their stuff in the cloud. Most people just want a lightweight laptop, and AI workloads would drain the battery and cook your eggs in a matter of minutes, assuming you can run them. Production quality models are pretty much cloud only, and I don’t think open source models, especially ones viable for local inference will close the gap anytime soon. I’d like all of those things to be different, but I think that’s just the way things are. Of course there are enthusiasts, but I suspect that they prefer and will continue to prefer dedicated inference hardware.
- Onavo 2y ago
  > I think the number of people interested in running ML models locally might be greatly overestimated [here]. There is no killer app in sight that needs to run locally. People work and store their stuff in the cloud. Most people just want a lightweight laptop, and AI workloads would drain the battery and cook your eggs in a matter of minutes, assuming you can run them. Production quality models are pretty much cloud only, and I don’t think open source models, especially ones viable for local inference will close the gap anytime soon. I’d like all of those things to be different, but I think that’s just the way things are. Of course there are enthusiasts, but I suspect that they prefer and will continue to prefer dedicated inference hardware. Do you use FTP instead of Dropbox?
  - cvs268 2y ago
    Do you use Dropbox instead of rsync? ( Couldn't resist (^.^) )
- tucnak 2y ago
  > AI workloads would drain the battery and cook your eggs in a matter of minutes, assuming you can run them M2 Max is passively cooled... and does 1/2 of 4090's token bandwidth in inference.
  - MindSpunk 2y ago
    Um, no? The MBP has fans? The base level M2 in a MacBook Air is passively cooled and thermal throttles very aggressively when you push it.
    - tucnak 2y ago
      That sounds right, I was judging by mac studio I own that is passively cooled.
- 0x000xca0xfe 2y ago
  Microsoft wants to bring Recall back. When ML models come as part of the OS suddenly there are hundreds of millions of users.
  - layer8 2y ago
    It remains to be seen whether users want what Microsoft wants.
  - throwuxiytayq 2y ago
    I have some difficulty with estimating how heavy Recall’s workload is, but either way, I have little faith in Microsoft’s ability to implement this feature efficiently. They struggle with much simpler features, such as search. I wouldn’t be surprised if a lot of people disable the feature to save battery life and improve system performance.
- fasa99 2y ago
  Huh? All the files are local and models are gonna be made to have a lot of or ultimately all of them in the "context window". You can't really have an AI for your local documents on the cloud because the cloud doesn't have access. Same logic for businesses. The use case follows from data availability and barriers. We've observed the same on web pages where more and more functionality gets pushed to the frontend. One could push the last (x) layers of the neural net for example, to the frontend, for lower expense and if rightly engineered, better speed and scalability. AIs will be local, super-AIs still in the cloud. Local AIs will be proprietary and they will have strings to the mothership The strings will have business value both related to the consumer and for further AI training.
  - throwuxiytayq 2y ago
    > All the files are local From what I’ve seen, most people tend to use cloud document platforms. Microsoft has made Office into one. This has been the steady direction for the last few years; they’ll keep pushing for it, because it gives them control. Native apps with local files is an inconvenient model for them. This sadly applies to most other types of apps. On many of these cloud platforms, you can’t even download the project files. > You can't really have an AI for your local documents on the cloud because the cloud doesn't have access Yes, up to the cloud it all goes. They don’t mind, they can charge you for it. Microsoft literally openly wants to move the whole OS to the cloud. > Same logic for businesses Businesses hate local files. They’re a huge liability. When firing people it’s very convenient that you can just cut someone off by blocking their cloud credentials. > We've observed the same on web pages where more and more functionality gets pushed to the frontend It will never go all the way. > One could push the last (x) layers of the neural net for example, to the frontend, for lower expense and if rightly engineered, better speed and scalability I’ll believe it when I see it. I don’t think the incentive is there. Sounds like a huge complicating factor. It’s much simpler to keep everything running in the cloud, and software architects strongly prefer simple designs. How much do these layers weigh? How many MB/GB of data will I need to transfer? How often? Does that really give me better latency than just transferring a few KBs of the AIs output?
wtallis 2y ago
Local ML isn't a CPU workload. The NPUs in mobile processors (both laptop and smartphone) are optimized for low power and low precision, which limit how much memory bandwidth they can demand. So as I said, demand for more memory bandwidth depends mainly on how powerful the GPU is.

epolanski 2y ago

The few reviews we have seen now show that lunar lake is competitive with m3s too depending on the application.

formerly_proven 2y ago

Is the full memory bandwidth actually available to the CPU on M-series CPUs? Because that would seem like a waste of silicon to me, to have 200+ GB/s of past-LLC bandwidth for eight cores or so.

adrian_b 2y ago
I do not know for the latest models, but at least in M1 the CPU was limited to a fraction of the total memory throughput, IIRC to about 100 GB/s.
inkyoto 2y ago
100 Gb/sec per a CPU core. 200+ Gb/sec (IIRC Anandtech measured 240 Gb/sec) per a cluster. Pro is one cluster, Max is two clusters and Ultra is four clusters, so the accumulative bandwidth is 200, 400 and 800 Gb/sec respectively[*]. The bandwidth is also shared with GPU and NPU cores within the same cluster, so on combined loads it is plausible that the memory bus may become fully saturated. [*] Starting with M3, Apple has pivoted Pro models into more Pro and less Pro versions that have differing memory bus widths.