3 ms·

> I struggle to imagine a regular workload that benefits from 32gbps of bandwidth, much less a scenario where the average layman could saturate a 200gbps or 400

> I struggle to imagine a regular workload that benefits from 32gbps of bandwidth, much less a scenario where the average layman could saturate a 200gbps or 400gbps bus. Since this is HN, many of us like… compiling, you know, stuff. It is a regular workload for many of us on here. The main beneficiary of the M1 Max wide memory architecture appears to be the Haskell GHC compiler, which is very, very voracious when it comes to the memory consumption and the data bus width. Regular Haskell builds shift the data around at 35-40 Gbps on average, with the Haskell optimiser hovering around 50 Gpbs and easily peaking out at 65 Gbps on medium size projects. Since cabal (the Haskell build toolchain) for some reason can't fully utilise the «-j10» parameter, roughly a 1/2 of CPU cores are not utilised. Which means we can kick off another large Haskell build in parallel without affecting memory transfer speeds of the first build. Rust release builds are a runner-up with 30-35 Gbps transfer speeds on average. Static type analysis and type inference in Haskell and Rust is hard, is complex and requires a lot of fast memory and benefits vastly from large CPU caches. I have not attemped a Rust LTO build yet, but I would expect the LTO builds to like the wide memory bus, like, A LOT. In general, a LTO build (e.g. large C++ builds) is heavily memory bound and shifts data blocks, small and large, around extensively. But an application is not very useful with a dataset, so many of us then like training our data models. Data model training can easily saturate 200 Gbps per a M1 Max CPU cluster. Then the application needs to load the data in and usually do something with it. For example, issue complex queries to a graph database. Large CPU caches coupled with a wide data bus is a winner here. All of the above are examples of real, practical and regular workloads.

smoldesu 5y ago
I'd be inclined to agree, but my 16gbps i5 520m compiles Rust just fine. Small and medium-sized desktop applications aren't constrained by bandwidth as much as they are CPU-bound. I could easily imagine AI model training blowing up a memory bus, but every machine that isn't an M1 Pro or M1 Max is going to delegate that to a dGPU that is bound by PCIe bandwidth, not memory. Even then, properly-optimized CUDA programs will calculate most of the work on your GPU itself, constraining you to the 300-something gbps that your GPU is capable of. Regardless, neither of these workloads are something that I'd imagine the average Macbook user doing on a regular basis. The only "real world" workload I can imagine that would exercise it is insanely high-res video encoding, which again, is only really going to be taken advantage of by a handful of users.
- inkyoto 5y ago
  Discussing needs of an average Macbook user is a moot point as they would not purchase a maxed out M1 Max. I was referring to specific use cases of a specific target group of users with specific examples who can benefit from specific hardware architecture solutions and about being able to put them into a practical use today. > I'd be inclined to agree, but my 16gbps i5 520m compiles Rust just fine. Small and medium-sized desktop applications aren't constrained by bandwidth as much as they are CPU-bound. Gbps refers to the memory throughput, not to the memory footprint. A Haswell CPU equipped laptop with 16 Gb of RAM can compile Rust applications just fine; the difference is the time. Compile times are the single important delineating factor for software engineers. > Even then, properly-optimized CUDA programs will calculate most of the work on your GPU itself, constraining you to the 300-something gbps that your GPU is capable of. With all due respect, it is exceptionally unusual to train ML models directly in CUDA, which is a low-level proprietory Nvidia API. Of which CUDA is one, and there are several others as well (ROCm, OpenCL, CoreML, Metal etc), and they are swapped out for one another depending on what the underlying hardware actually is. ML training is done via the use of high-level ML frameworks (Tensorflow, NumPy, SpaCy, Pandas, SPARK and others), but not in CUDA – it simply almost does not exist. Lastly, whilst the ML training undertaken on a high performant and an exorbitantly expensive dGPU itself would be substantially faster compared to 32 M1 Max GPU cores, one would yet be constrained with the PCIe bus data transfers speeds: ~32Gbps for PCIe v4.0 in the x16 lane setup (the high end of the commodity PC hardware today), and ~64 Gpbs for PCIe v5.0 in the x16 lane setup (not even unavailable for purchase today in PC laptops and desktops). With both being a far cry from the 200 Gbps of a single M1 Max CPU cluster.

smoldesu 5y ago

I'd be inclined to agree, but my 16gbps i5 520m compiles Rust just fine. Small and medium-sized desktop applications aren't constrained by bandwidth as much as they are CPU-bound. I could easily imagine AI model training blowing up a memory bus, but every machine that isn't an M1 Pro or M1 Max is going to delegate that to a dGPU that is bound by PCIe bandwidth, not memory. Even then, properly-optimized CUDA programs will calculate most of the work on your GPU itself, constraining you to the 300-something gbps that your GPU is capable of. Regardless, neither of these workloads are something that I'd imagine the average Macbook user doing on a regular basis. The only "real world" workload I can imagine that would exercise it is insanely high-res video encoding, which again, is only really going to be taken advantage of by a handful of users.

inkyoto 5y ago
Discussing needs of an average Macbook user is a moot point as they would not purchase a maxed out M1 Max. I was referring to specific use cases of a specific target group of users with specific examples who can benefit from specific hardware architecture solutions and about being able to put them into a practical use today. > I'd be inclined to agree, but my 16gbps i5 520m compiles Rust just fine. Small and medium-sized desktop applications aren't constrained by bandwidth as much as they are CPU-bound. Gbps refers to the memory throughput, not to the memory footprint. A Haswell CPU equipped laptop with 16 Gb of RAM can compile Rust applications just fine; the difference is the time. Compile times are the single important delineating factor for software engineers. > Even then, properly-optimized CUDA programs will calculate most of the work on your GPU itself, constraining you to the 300-something gbps that your GPU is capable of. With all due respect, it is exceptionally unusual to train ML models directly in CUDA, which is a low-level proprietory Nvidia API. Of which CUDA is one, and there are several others as well (ROCm, OpenCL, CoreML, Metal etc), and they are swapped out for one another depending on what the underlying hardware actually is. ML training is done via the use of high-level ML frameworks (Tensorflow, NumPy, SpaCy, Pandas, SPARK and others), but not in CUDA – it simply almost does not exist. Lastly, whilst the ML training undertaken on a high performant and an exorbitantly expensive dGPU itself would be substantially faster compared to 32 M1 Max GPU cores, one would yet be constrained with the PCIe bus data transfers speeds: ~32Gbps for PCIe v4.0 in the x16 lane setup (the high end of the commodity PC hardware today), and ~64 Gpbs for PCIe v5.0 in the x16 lane setup (not even unavailable for purchase today in PC laptops and desktops). With both being a far cry from the 200 Gbps of a single M1 Max CPU cluster.