4 ms·
He points to Intel’s Sapphire Rapids as an example of hype-driven engineering versus the kind that could pay real dividends for broader HPC. “Maybe in their sec
by KKKKkkkk1 6y ago
He points to Intel’s Sapphire Rapids as an example of hype-driven engineering versus the kind that could pay real dividends for broader HPC. “Maybe in their second generation they’ll put high-bandwidth memory in the CPU, but why did that take so long? It’s been available for years. We think this is attributed to LINPACK, for needing higher rankings. And now they’re just thinking about dense linear algebra and not about memory or throughput or how the memory and cache can allow these vector engines to work effectively. That’s just bad engineering. If we continue like this, we’re just going to keep wasting resources.”
To be fair, NVIDIA is also guilty of this. Their marketing is highlighting their tensor core units, claiming order-of-magnitude speedups on deep learning workloads, when in practice, workloads that involve convolutions are actually memory bandwidth bound and will not see a significant benefit.
- dragontamer 6y agoIntel had a competitor to high bandwidth RAM on Intel Xeon Phi: the Hybrid Memory Cube. HMC was interesting: the RAM operations could complete out-of-order (if one section of RAM was refreshing, maybe it'd be faster to out-of-order the responses). Too bad it never became popular. HBM / HBM2 still needs to account for refreshes, and still takes place in-order.
- rbanffy 6y agoIt's a shame the Phi failed the way it did. I believe single thread performance has some relationship to the old "there is no substitute for the cubic inch" principle that's applicable to internal combustion engines. The Phi with 16 GB of memory onboard would probably make a very interesting developer workstation. Nothing will make code become more parallel faster than making developers plow their fields with chickens instead of oxen. ;-) (forgive-me Seymour)
- lumost 6y agoI'd bet a Threadripper with 128 GB of memory + NVMe storage would crush a Phi as a workstation.
- dragontamer 6y agoXeon Phi was 2014-era, and had 352 GB/s memory speeds. Threadripper only has ~100GB/s, best case, across its 4x memory channels. (25.6GB/s at 3200 MT/s). Upgrade to Threadripper Pro or EPYC (8x memory channels) and you're still only ~200 GB/s to main memory bandwidth. -------- I'm sure Threadripper has more compute (and cache) than Xeon Phi, but a lot of that is 22nm process vs 7nm process.
- lumost 6y agoThere were 14nm phi's. While there are likely a few workloads the phi performed well on, it's days as a dedicated accelerator were clearly numbered with the increased core counts we're seeing today. The phi only lists at 112 GB/s memory bandwidth. https://versus.com/en/amd-ryzen-threadripper-3990x-vs-intel-xeon-phi-7290f https://versus.com/en/amd-ryzen-threadripper-3990x-vs-intel-...
- dragontamer 6y agoThere were many models of Phi. https://insidehpc.com/2016/01/mcdram/ https://insidehpc.com/2016/01/mcdram/ This one with MCDRAM (aka: HMC RAM from Micron) was well known to hit 400GBps bandwidth. It does seem like this model is 14nm, as you've said. So I stand corrected on the process node for Xeon Phi: 14nm is more realistic representation, at least for the HMC model I was thinking about.
- rbanffy 6y agoI was thinking about the MCDRAM models when I thought about a Phi developer workstation. Having no external RAM would allow a very simple design. It would be no NUC, however - it's still a 300+W part.
- marmaduke 6y agoBets aren't worth much. It depends on the workload. Throughput-wise the Xeon Phi could handle 4-way SMT, so we ran the 68-core KNL node with AVX512 workloads and did quite well, compared to equivalent conventional Xeon nodes.
- inasio 6y agoThis recent Anandtech article is relevant [0]. Intel added 128 MB of fast on-die memory to the first 14nm cpu (5 years ago?), and then never did it again. For some (gaming) workloads, those chips work very well, even today. [0] https://www.anandtech.com/show/16195/a-broadwell-retrospective-review-in-2020-is-edram-still-worth-it https://www.anandtech.com/show/16195/a-broadwell-retrospecti...
- rbanffy 6y agoSimilarly, IBM z15 (and LinuxONE III) mainframes have 940 MB of L4 cache (implemented in DRAM close to the external memory bus) for each 4-socket processor cluster.
- ekelsen 6y agoStandard convolutions are not memory bandwidth bound for common sizes/shapes. Convolutions run much faster using the TC than not using them. Depthwise convolutions are different though - they usually are bandwidth bound.
- easde 6y agoIt's true that depthwise convolutions are bandwidth bound, but most networks that use them in combination with a kernel size 1 "convolution". If those two operations are tiled and fused together, the result is often compute bound again. This is usually not the case in most ML frameworks and libraries though, including CuDNN.