4 ms·
Unless Nvidia can somehow massively increase memory capacity, it is looking bleak for them in the consumer AI inference space. From the left Apple has a fully i
by imtringued 3y ago
Unless Nvidia can somehow massively increase memory capacity, it is looking bleak for them in the consumer AI inference space. From the left Apple has a fully integrated SoC with insane memory bandwidth and capacity, from the right AMD is tackling the FLOPs advantage using their XDNA AI Engines that they got from their Xilinx acquisition and they are going to open source the compiler for their AI Engines. The only competitive advantage that NVidia has left is its high memory bandwidth, but even that is being threatened by Strix Point so they will need to adopt GDDR7 with 32 GB to 64GB VRAM fast or they will become irrelevant except for training. Oh and by the way AMD GPUs will stay completely irrelevant for AI so that explains why they didn't want to waste so much time on ROCm for consumer GPUs. Nobody is going to buy those for AI anymore by the end of the year.
- rubatuga 3y agoAMD was the first to introduce consumer HBM cards.
- account42 3y agoWell, on "consumer" cards that were AMDs desperate attempt to stay relevant in that space by rebadging their enterprise compute-focused design for consumers. They haven't used HBM in any of their actual consumer designs unfortunately.
- oasisaimlessly 3y agoAre you sure? https://www.techpowerup.com/gpu-specs/radeon-r9-fury-x.c2677 https://www.techpowerup.com/gpu-specs/radeon-r9-fury-x.c2677 > AMD has paired 4 GB HBM memory with the Radeon R9 FURY X, which are connected using a 4096-bit memory interface.
- wmf 3y agoIt's not clear that the "consumer (local) AI inference space" is a real market. Ultimately Nvidia has access to all the same technologies as their competitors and better software so anything they can do Nvidia can do better... if they want to.
- IshKebab 3y agoI mean... CUDA. nVidia is fine.
- lloeki 3y agoToo big to fail? Sure, they have an edge in being the current de facto go to solution, but that can be changing very rapidly if they rest on their laurels.
- mschuster91 3y ago> Unless Nvidia can somehow massively increase memory capacity, it is looking bleak for them in the consumer AI inference space. Who cares about consumer inference? The money is in training because that's where utterly insane amounts of compute capacity are needed, and as long as no competition comes even close to CUDA, NVIDIA has a cash cow to milk. As for potential competition, Apple doesn't sell their silicon to anyone else, and AMD lacks the available manufacture capacity (keep in mind, they also supply the game console market), the driver/tooling quality and most importantly developer trust/ecosystem quality. Everyone is using CUDA.
- coffeebeqn 3y agoIs there any reason they couldn’t make a card with lots of GDDR5 for example? Isn’t it “just” memory chips that you could put a lot of on a board ?
- adgjlsfhk1 3y agothey do. the reason Nvidia doesn't do it for consumer cards is that they do it for their server cards and raise the price 40x.