4 ms·
M3/M4 Max MacBooks with 128GB RAM are already way better than an A6000 for very large local LLMs. So even if the GPU is as slow as the one in M3/M4 Max (<3070),
by treprinum 2y ago
M3/M4 Max MacBooks with 128GB RAM are already way better than an A6000 for very large local LLMs. So even if the GPU is as slow as the one in M3/M4 Max (<3070), and using some basic RAM like LPDDR5x it would still be way faster than anything from NVidia.
- deleted 2y ago[deleted]
- kevingadd 2y agoAre you suggesting that Intel 'just' release a GPU at the same price point as an M4 Max SOC? And that there would be a large market for it if they did so? Seems like an extremely niche product that would be demanding to manufacture. The M4 Max makes sense because it's a complete system they can sell to Apple's price-insensitive audience, Intel doesn't have a captive market like that to sell bespoke LLM accelerator cards to yet. If this hypothetical 128GB LLM accelerator was also a capable GPU that would be more interesting but Intel hasn't proven an ability to execute on that level yet.
- m00x 2y agoIt's also impossible and it would need to be a CPU. CPUs and GPUs access memory very differently.
- treprinum 2y agoNothing in my comment says about pricing it at the M4 Max level. Apple charges as much because they can (typing this on an $8000 M3 Max). 128GB LPDDR5 is dirt cheap these days just Apple adds its premium because they like to. Nothing prevents Intel from releasing a basic GPU with that much RAM for under $1k.
- wtallis 2y agoYou're asking for a GPU die at least as large as NVIDIA's TU102 that was $1k in 2018 when paired with only 11GB of RAM (because $1k couldn't get you a fully-enabled die to use 12GB of RAM). I think you're off by at least a factor of two in your cost estimates.
- treprinum 2y agoIntel has Xeon Phi which was a spin-off of their first attempt at GPU so they have a lot of tech in place they can reuse already. They don't need to go with GDDRx/HBMx designs that require large dies.
- ksec 2y agoI don't want to further this discussions but may be you dont realise some of the people who replied to you either design hardware for a living or has been in the hardware industry for longer than 20 years.
- treprinum 2y agoFor some reason Apple did it with M3/M4 Max likely by folks that are also on HN. The question is how many of the years spent designing HW were spent also by educating oneselves on the latest best ways to do it.
- ksec 2y ago>For some reason..... They already replied with an answer.
- ryao 2y agoWhile it is not a GPU, Qualcomm already made an inferencing card with 128GB RAM: https://www.qualcomm.com/news/onq/2023/11/introducing-qualcomm-cloud-ai-100-ultra https://www.qualcomm.com/news/onq/2023/11/introducing-qualco... It would be interesting if those saying that a regular GPU with 128GB of VRAM cannot be made would explain how Qualcomm was able to make this card. It is not a big stretch to imagine a GPU with the same memory configuration. Note that Qualcomm did not use HBM for this.
- wtallis 2y agoEven LPDDR requires a large die. It only takes things out of the realm of technologically impossible to merely economically impractical. A 512-bit bus is still very inconveniently large for a single die.
- anon373839 2y ago> release a GPU at the same price point as an M4 Max SOC Why would it need to be introduced at Apple's high-margin pricing?
- jsheard 2y agoThe M4 Max needs an enormous 512bit memory bus to extract enough bandwidth out of those LPDDR5x chips, while the GPUs that Intel just launched are 192/160bit and even flagships rarely exceed 384bit. They can't just slap more memory on the board, they would need to dedicate significantly more silicon area to memory IO and drive up the cost of the part, assuming their architecture would even scale that wide without hitting weird bottlenecks.
- p1esk 2y agoApple could do it. Why can’t Intel?
- jsheard 2y agoBecause Apple isn't playing the same game as everyone else. They have the money and clout to buy out TSMCs bleeding-edge processes and leave everyone else with the scraps, and their silicon is only sold in machines with extremely fat margins that can easily absorb the BOM cost of making huge chips on the most expensive processes money can buy.
- p1esk 2y agoBleeding edge processes is what Intel specializes in. Unlike Apple, they don’t need TSMC. This should have been a huge advantage for Intel. Maybe that’s why Gelsinger got the boot.
- AlotOfReading 2y agoIntel Arc hardware is manufactured by TSMC, specifically on N6 and N5 for this latest announcement. Intel doesn't currently have nodes competitive with TSMC or excess capacity in their better processes.
- knowitnone 2y agoSerious question, why don't they have excess capacity? They aren't producing many CPUs...that people want.
- wtallis 2y agoThat would basically mean Intel doubling the size of their current GPU die, with a different memory PHY. They're clearly not ready to make that an affordable card. Maybe when they get around to making a chiplet-based GPU.