8 ms·
I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional
by atty 5y ago
I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I think that’s a smart thing on their part, because Nvidia really has a chokehold on the deep learning field right now with the A100 and CUDA/CUDNN software stack. I also find it interesting (and a good sign) that their recent supercomputer deals include 100 million for software development for ROCm/HIP.
However, considering how robust CUDA is compared to ROCm, I feel like all Nvidia would need to do to take the HPC market back completely would be to get close to AMD’s current FP64 performance. I don’t think anyone would buy AMD if prices were comparable and FP64 performance was anywhere in the ballpark. It’ll be very interesting to see Nvidia’s new cards, hopefully next year.
- baybal2 5y agoSupercomputer market is still there. Intel Sapphire rapids owes its existence to the next US nuke design (to be made on Aurora supercomputer.)
- kolbusa 5y agoI don't think so. The new ISA that is in the SPR is mostly about deep learning: it supports int8 and bfloat16 (https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-extension-amx-brings-matrix-operations-to-debut-with-sapphire-rapids/ https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-...). You can emulate higher precision using bfloat16 (https://arxiv.org/abs/1904.06376 https://arxiv.org/abs/1904.06376), but I have not seen this used in the wild.
- freemint 5y agoYou know about surrogate models being run in simulations (physics informed neural network and all that stuff) the US is investing a lot in this. Maybe then it makes sense why bfloat16 exist in SPR.
- mhh__ 5y agoNot sure I really trust AMD not to trip over their own feet when it comes to software unfortunately. Hats off to them if they manage to really land a blow on NV here. Also potentially keep an eye on Intel a few years down the line? Since they can actually do software in my experience (and, get this AMD, document the software!)
- bayindirh 5y ago> I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I develop scientific simulation software, and I can't tell you how happy I'm about it. Because while doing high precision work, GPUs fall flat fast. I also work at a HPC center, and there's mountains of FP64 dependent applications running on CPUs. Moving them to GPUs will bring a lot of improvements in a lot of disciplines. It's not uncommon to let things run for a week on multiple nodes for meaningful results.
- pixelpoet 5y agoIf I could in any way afford it I'd buy one of these just for doing (hobby) n-body simulations and fractal rendering. I bought a Radeon VII specifically for this and it died a month or so after I got it, with no replacement possible because they were EOL :( So badly want a replacement, but with GPU prices as they are now, there's just no way to justify buying a 2nd hand one. Right now I think the best you can do for FP64/$ is AVX-512 CPUs and Radeon GPUs.
- Retric 5y agoNvidia can’t improve FP64 performance to near parity without sacrificing a lot in the process, which would make them more vulnerable on the deep learning side of things.
- hyperbovine 5y ago?? Build two separate product lines.
- Retric 5y agoWith which engineers, managers, etc? Intel has long faced the same issues, they want their best R&D on x86 CPU’s so they had major issues expanding into other product lines.