4 ms·

You'd need a fairly drastic shift in the memory architecture of CPUs for that. Not something unheard of, such as Intel's new Xeon Max beast with HBM 2e on the C

You'd need a fairly drastic shift in the memory architecture of CPUs for that. Not something unheard of, such as Intel's new Xeon Max beast with HBM 2e on the CPU module. But it's definitely not an issue of just throwing some big SIMD blocks onto the die & calling it a day. That is, after all, basically what AVX-512 is. And while it has it's place, it's also not eating anyone's lunch money. And also, as weird as it is, 1.5TFlops isn't actually that ridiculous. We had that performance 14 years ago at 150w with desktop GPUs. 14 years to reduce from 150w to what, 5w?, is cool but also honestly pretty par for the course is it not? Especially for a fixed-function block?

r00fus 4y ago
Aside from Apple's processors is 1.5TFlops in 5w possible with other archs?
- kllrnohj 4y ago
  It sounds pretty similar to Qualcomm's hexagon ( https://www.anandtech.com/show/13680/snapdragon-855-going-into-detail/2 https://www.anandtech.com/show/13680/snapdragon-855-going-in... ) and probably whatever Google is doing with their Pixel SoC But otherwise that's less than what mobile GPUs can do, although those take more power they're not in entirely different ballparks either
  - deleted 4y ago
    [deleted]
sliken 4y ago
"You'd need a fairly drastic shift in the memory architecture of CPU". You mean like selling laptops (at 400GB/sec) and desktops (at 800GB/sec) with much improved memory systems. I don't want to give up SO-DIMMs for a few mm thinner laptop, but going from the intel/amd standard 70GB/sec to 400GB/sec is a pretty big incentive.
- kllrnohj 4y ago
  M1's CPUs can't achieve that rate fwiw. It's also generic LPDDR5, not actually anything particularly new or exciting. The 800GB/s isn't to a single compute unit, either, it's 2x 400GB/s controllers. Regardless, no that's not the drastic shift necessary. It's not even a shift at all. 4, 6, and 8 channel memory architectures have all existed for a while now in data center SKUs. It hasn't changed the desire for GPU compute in the data center
  - sliken 4y ago
    A comment here on benchmarking the M1 managed to get 360GB/sec observed out of a 400GB/sec memory system, so I'd consider that pretty good and certainly a better percentage of peak than I see out of the x86-64 systems I benchmarked. Well the 4, 6, and 8 channel memory architectures have been dramatically slower than the Apple 800GB/sec with dramatically less memory transactions in flight (1 per channel) than Apple's 800GB/sec with 64 channels. The push for Apple like memory systems with HBM from Intel, AMD, Fujitsu and others does show that there's a need for more compute bandwidth and Fujitsu in particular has shown that a healthy CPU with a GPU like memory system can get great performance on real world codes without the hassle of rewriting for CUDA. With all that said, yes there's a healthy demand for GPUs and codes that run best on GPUs. However I find it refreshing that at last one company is providing significantly improved memory systems in laptops and small desktops.
    - kllrnohj 4y ago
      > A comment here on benchmarking the M1 managed to get 360GB/sec observed out of a 400GB/sec memory system, so I'd consider that pretty good and certainly a better percentage of peak than I see out of the x86-64 systems I benchmarked. "the M1 Max isn’t able to fully saturate the SoC bandwidth from just the CPU side; From a single core perspective [..] it’s able to stress the memory fabric to up to 102GB/s. [..] Adding a third thread there’s a bit of an imbalance across the clusters, DRAM bandwidth goes to 204GB/s, but a fourth thread lands us at 224GB/s and this appears to be the limit on the SoC fabric that the CPUs are able to achieve" https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc... So peak CPU bandwidth on the M1 Max is 224GB/s. Which is really good for a CPU to be sure, but we're still talking bandwidth numbers far below what you'd need to eliminate the GPU. > Well the 4, 6, and 8 channel memory architectures have been dramatically slower than the Apple 800GB/sec with dramatically less memory transactions in flight (1 per channel) Huh? I guarantee you Xeons and Epycs are not limited to 1 transaction in flight, nor is Graviton2. The AMD APUs in things like the steam deck, PS4, and PS5 (448GB/s fwiw) are also definitely not limited like that, either. > than Apple's 800GB/sec with 64 channels M1 ultra is 32 channels. Well, it's really 2x 16 channels since it's a NUMA configuration, like a dual-socket system. > The push for Apple like memory systems with HBM from Intel, AMD, Fujitsu lol? Apple's memory system isn't like HBM. It's more like a traditional GPU's but with GDDR swapped out for LPDDR. M1 Max is 512-bit memory bus, just like many GPUs have done over the years. 512-bit bus and it can only hit 400GB/s shows you just how much less bandwidth LPDDR has vs. GDDR, too. AMD & NVidia are hitting 1TB/s with 384-bit buses these days. Which is also what the HBM2e Xeon's are claiming.
roxgib 4y ago
Apple Silicon chips share memory between the CPU and GPU, would that play into any calculation of the relative benefits? Presumably the GPU isn't getting the full benefits of a GPU optimised memory set up so the difference would be smaller?
- kllrnohj 4y ago
  That's been doable on other products for over a decade. AMD's APUs and Intel's CPUs, for example, along with nearly every mobile phone SoC from the last 10 years It makes some GPGPU workloads viable that otherwise wouldn't be, but it doesn't reduce the bandwidth needs of the GPU for traditional GPU workloads so net-net you're worse off overall. You either use DDR and penalize your GPU performance, or you use GDDR (like consoles do) and sacrifice CPU memory latency. Also power efficiency of GDDR is worse, especially compared to Apple Silicon's choice of using LPDDR HBM2e gets you everything except it's expensive and limited capacity. But it'll be interesting to see how that plays out with the Xeon Max, which can also still supplement the 64GB HBM2e with some absurd channel count DDR5

r00fus 4y ago

Aside from Apple's processors is 1.5TFlops in 5w possible with other archs?

kllrnohj 4y ago
It sounds pretty similar to Qualcomm's hexagon ( https://www.anandtech.com/show/13680/snapdragon-855-going-into-detail/2 https://www.anandtech.com/show/13680/snapdragon-855-going-in... ) and probably whatever Google is doing with their Pixel SoC But otherwise that's less than what mobile GPUs can do, although those take more power they're not in entirely different ballparks either
- deleted 4y ago
  [deleted]

sliken 4y ago

"You'd need a fairly drastic shift in the memory architecture of CPU". You mean like selling laptops (at 400GB/sec) and desktops (at 800GB/sec) with much improved memory systems. I don't want to give up SO-DIMMs for a few mm thinner laptop, but going from the intel/amd standard 70GB/sec to 400GB/sec is a pretty big incentive.

kllrnohj 4y ago
M1's CPUs can't achieve that rate fwiw. It's also generic LPDDR5, not actually anything particularly new or exciting. The 800GB/s isn't to a single compute unit, either, it's 2x 400GB/s controllers. Regardless, no that's not the drastic shift necessary. It's not even a shift at all. 4, 6, and 8 channel memory architectures have all existed for a while now in data center SKUs. It hasn't changed the desire for GPU compute in the data center
- sliken 4y ago
  A comment here on benchmarking the M1 managed to get 360GB/sec observed out of a 400GB/sec memory system, so I'd consider that pretty good and certainly a better percentage of peak than I see out of the x86-64 systems I benchmarked. Well the 4, 6, and 8 channel memory architectures have been dramatically slower than the Apple 800GB/sec with dramatically less memory transactions in flight (1 per channel) than Apple's 800GB/sec with 64 channels. The push for Apple like memory systems with HBM from Intel, AMD, Fujitsu and others does show that there's a need for more compute bandwidth and Fujitsu in particular has shown that a healthy CPU with a GPU like memory system can get great performance on real world codes without the hassle of rewriting for CUDA. With all that said, yes there's a healthy demand for GPUs and codes that run best on GPUs. However I find it refreshing that at last one company is providing significantly improved memory systems in laptops and small desktops.
  - kllrnohj 4y ago
    > A comment here on benchmarking the M1 managed to get 360GB/sec observed out of a 400GB/sec memory system, so I'd consider that pretty good and certainly a better percentage of peak than I see out of the x86-64 systems I benchmarked. "the M1 Max isn’t able to fully saturate the SoC bandwidth from just the CPU side; From a single core perspective [..] it’s able to stress the memory fabric to up to 102GB/s. [..] Adding a third thread there’s a bit of an imbalance across the clusters, DRAM bandwidth goes to 204GB/s, but a fourth thread lands us at 224GB/s and this appears to be the limit on the SoC fabric that the CPUs are able to achieve" https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc... So peak CPU bandwidth on the M1 Max is 224GB/s. Which is really good for a CPU to be sure, but we're still talking bandwidth numbers far below what you'd need to eliminate the GPU. > Well the 4, 6, and 8 channel memory architectures have been dramatically slower than the Apple 800GB/sec with dramatically less memory transactions in flight (1 per channel) Huh? I guarantee you Xeons and Epycs are not limited to 1 transaction in flight, nor is Graviton2. The AMD APUs in things like the steam deck, PS4, and PS5 (448GB/s fwiw) are also definitely not limited like that, either. > than Apple's 800GB/sec with 64 channels M1 ultra is 32 channels. Well, it's really 2x 16 channels since it's a NUMA configuration, like a dual-socket system. > The push for Apple like memory systems with HBM from Intel, AMD, Fujitsu lol? Apple's memory system isn't like HBM. It's more like a traditional GPU's but with GDDR swapped out for LPDDR. M1 Max is 512-bit memory bus, just like many GPUs have done over the years. 512-bit bus and it can only hit 400GB/s shows you just how much less bandwidth LPDDR has vs. GDDR, too. AMD & NVidia are hitting 1TB/s with 384-bit buses these days. Which is also what the HBM2e Xeon's are claiming.

roxgib 4y ago

Apple Silicon chips share memory between the CPU and GPU, would that play into any calculation of the relative benefits? Presumably the GPU isn't getting the full benefits of a GPU optimised memory set up so the difference would be smaller?

kllrnohj 4y ago
That's been doable on other products for over a decade. AMD's APUs and Intel's CPUs, for example, along with nearly every mobile phone SoC from the last 10 years It makes some GPGPU workloads viable that otherwise wouldn't be, but it doesn't reduce the bandwidth needs of the GPU for traditional GPU workloads so net-net you're worse off overall. You either use DDR and penalize your GPU performance, or you use GDDR (like consoles do) and sacrifice CPU memory latency. Also power efficiency of GDDR is worse, especially compared to Apple Silicon's choice of using LPDDR HBM2e gets you everything except it's expensive and limited capacity. But it'll be interesting to see how that plays out with the Xeon Max, which can also still supplement the 64GB HBM2e with some absurd channel count DDR5