5 ms·

Really confused why the Intel and AMD both continue to struggle and yet still refuse to offer what Nvidia wont, i.e. high ram consumer GPUs. I'd much prefer pay

by mythz 1y ago

Really confused why the Intel and AMD both continue to struggle and yet still refuse to offer what Nvidia wont, i.e. high ram consumer GPUs. I'd much prefer paying 3x cost for 3x VRAM (48GB/$1047), 6x cost for 6x VRAM (96GB/$2094), 12x cost for 12x VRAM (192GB/$4188), etc. They'd sell like hotcakes and software support would quickly improve. At 16GB I'd still prefer to pay a premium for NVidia GPUs given its superior ecosystem, I really want to get off NVidia but Intel/AMD isn't giving me any reason to.

doctorpangloss 1y ago
they don't manufacture RAM, so none of the margin goes to them
- nullc 1y ago
  Even if they put out some super high memory models and just pass the ram through at cost it would increase sales -- potentially quite dramatically and increase their total income a lot and have a good chance of transitioning to being a market leader rather than an also-ran. AMD has lagged so long because of the software ecosystem but the climate now is that they'd only need to support a couple popular model architectures to immediately grab a lot of business. The failure to do so is inexplicable. I expect we will eventually learn that this was about yet another instance of anti-competitive collusion.
  - doctorpangloss 1y ago
    the whole RAM industry was twice sanctioned for price fixing, so I agree: any business that deals with RAM has, more likely than other industries by a lot, anti-competitive collusion
  - zargon 1y ago
    > The failure to do so is inexplicable. Lisa and Jensen are cousins. I think that explains it. Lisa can easily prove me wrong by releasing a high-memory GPU that significantly undercuts Nvidia's RTX 6000 Pro.
- kube-system 1y ago
  They sell the completed card, which has margin. You can charge more money for a card with more vram.
  - blitzar 1y ago
    Or shrink the margin down to just 50% and sell 10x the number of cards (for the week or two it would take Nvidia to announce a 5090 with 128gb)
    - AnthonyMouse 1y ago
      Nvidia uses VRAM amount for market segmentation. They can't make a 128GB consumer card without cannibalizing their enterprise sales. Which means Intel or AMD making an affordable high-VRAM card is win-win. If Nvidia responds in kind, Nvidia loses a ton of revenue they'd otherwise have available to outspend their smaller competitors on R&D. If they don't, they keep more of those high-margin customers but now the ones who switch to consumer cards are switching to Intel or AMD, which both makes the company who offers it money and helps grow the ecosystem that isn't tied to CUDA. People say things like "it would require higher pin counts" but that's boring. The increase in the amount people would be willing to pay for a card with more VRAM is unambiguously more than the increase in the manufacturing cost. It's more plausible that there could actually be global supply constraints in the manufacture of GDDR, but if that's the case then just use ordinary DDR5 and a wider bus. That's what Apple does and it's fine, and it may even cost less in pins than you save because DDR is cheaper than GDDR. It's not clear what they're thinking by not offering this.
      - singhrac 1y ago
        This is almost true but not quite - I don't think much of the (dollar) spend on enterprise GPUs (H100, B200, etc.) would transfer if there was a 128 GB consumer card. The problem is both memory bandwidth (HBM) and networking (NVLink), which NVIDIA definitely uses to segment consumer vs enterprise hardware. I think your argument is still true overall, though, since there are a lot of "gpu poors" (i.e. grad students) who write/invent in the CUDA ecosystem, and they often work in single card settings. Fwiw Intel did try this with Arctic Sound / Ponte Vecchio, but it was late out the door and did not really perform (see https://chipsandcheese.com/p/intels-ponte-vecchio-chiplets-gone-crazy https://chipsandcheese.com/p/intels-ponte-vecchio-chiplets-g...). It seems like they took on a lot of technical risk; hopefully some of that transfers over to a future project though Falcon Shores was cancelled. They really should should have released some of those chips even at a loss, but I don't know the cost of a tape out.
daemonologist 1y ago
This card does have double the VRAM of the more expensive Nvidia competitor (the A1000, which has 8 GB), but I take your point that it doesn't feel like quite enough to justify giving up the Nvidia ecosystem. The memory bandwidth is also... not great. They also announced a 24 GB B60 and a double-GPU version of the same (saves you physical slots), but it seems like they don't have a release date yet (?).
- HelloNurse 1y ago
  They don't have a release date yet because they need to see sales results to commit to such a new product.
- jjkaczor 1y ago
  Yeah am currently building a new PC , and have been waiting and waiting for B60 Pro release dates from partners - here is an offering from ASRock: https://www.asrock.com/Graphics-Card/Intel/Intel%20Arc%20Pro%20B60%20Creator%2024GB/ https://www.asrock.com/Graphics-Card/Intel/Intel%20Arc%20Pro...
zdw 1y ago
I doubt you'd get linear scaling of price/capacity - the larger capacity modules are more expensive per GB than smaller ones, and in some cases are supply constrained. The number of chips on the bus is usually pretty low (1 or 2 of them on most GPUs), so GPUs tend to have to scale out their memory bus widths to get to higher capacity. That's expensive and takes up die space, and for the conventional case (games) isn't generally needed on low end cards. What really needs to happen is someone needs to make some "system seller" game that is incredibly popular and requires like 48GB of memory on the GPU to build demand. But then you have a chicken/egg problem. Example: https://wccftech.com/nvidia-geforce-rtx-5090-128-gb-memory-gpu-for-ai-price-13200-usd/ https://wccftech.com/nvidia-geforce-rtx-5090-128-gb-memory-g...
cmxch 1y ago
Maxsun does offer a high VRAM (48GB) dual Arc Pro B60, but the only US availability has it on par with a 5090 at ~$3000.
- PostOnce 1y ago
  I think that's actually two GPUs on one card, and not a single GPU with 48GB VRAM
  - hengheng 1y ago
    Needs a PCIe bifurcation chip on the main board for all we know. Compatibility is going to be fun.
Ekaros 1y ago
I am not sure there is significant enough market for those. That is selling enough consumer units to cover all design and other costs. From gamer perspective 16GB is now a reasonable point. 32GB is most one would really want and even that not at more than say 100 more price point. This to me is the gamer perspective. This segment really does not need even 32GB, let alone 64GB or more.
- drra 1y ago
  Never underestimate bragging rights in gamers community. Majority of us run unoptimized systems with that one great piece of gear and as long as the game runs at decent FPS and we have some bragging rights it's all ok.
  - rkomorn 1y ago
    Exactly. A decade ago, I put 64GB of RAM in a PC and my friend asked me why I needed that much and I replied "so I can say I have 64GB RAM". The only time usage was "high" was when I created a VM with 48GB RAM just for kicks. It was useless. But I could say I had 64GB RAM.
    - rvba 1y ago
      My work computer with Widnows, Outlook, few tabs open and Excel already craps put with 16 GB If you hae a private computer, why wpuld you even bug something with 16GB in 2025? My 10 year old laptop had that much. Im looking for a new laptop and Im looking at a 128GB setup - so those 200 chrome tabs can eat it, I have space to run other stuff, like those horrible electron chat apps + a game
      - rkomorn 1y ago
        I have 64GB in my current machine and, TBH, it's more than enough. But if I were to upgrade, I'd still get at least 128GB. Nobody's gonna be impressed with 64GB anymore. I don't need that in my life...
- 1y ago
fredoralive 1y ago
Because the market of people who want huge RAM GPUs for home AI tinkering is basically about 3 Hacker News posters. Who probably won’t buy one because it doesn’t support CUDA. PS5 has something like 16GB unified RAM, and no game is going to really push much beyond that in VRAM use, we don’t really get Crysis style system crushers anymore.
- bilekas 1y ago
  > PS5 has something like 16GB unified RAM, and no game is going to really push much beyond that in VRAM use, we don’t really get Crysis style system crushers anymore. This isn't really true from the recreational card side, nVidia themselves are reducing the number of 8GB models as a sign of market demand [1]. Games these days are regularly maxing out 6 & 8 GB when running anything above 1080p for 60fps. The prevalence of Unreal Engine 5 also recently with a low quality of optimization for weaker hardware is causing games to be released basically unplayable for most. For recreational use the sentiment is that 8GB is scraping the bottom of the requirements. Again this is partly due to bad optimizations, but games are being played in higher resolutions also, which required more memory for larger texture sizes. [1] https://videocardz.com/newz/nvidia-reportedly-reduces-supply-of-rtx-5060-and-rtx-5060-ti-8gb https://videocardz.com/newz/nvidia-reportedly-reduces-supply...
  - deleted 1y ago
    [deleted]
  - pjmlp 1y ago
    As someone that started on 8 bit computing, Tim Sweeny is right the Electron garbage culture when applied to Unreal 5 is one of the reasons so much RAM is needed, with such bad performance. While I dislike some of the handmade hero culture, in one thing they are right, regarding how bad modern hardware happens to be used.
    - anthk 1y ago
      I remember UE1 being playable even software mode, such as the first Deus EX. Now, I think the Surreal Engine (UE1 reimplementation) needs damn GL 3.3 (if not 4.5 and Vulkan) to play games I used to play in an Athlon. Now I can't use surreal to Play DX on my legacy n270 netbook with GL 2.1... something that was more than enough to play the game at 800x600 with everything turned on and much more. A good thing is that I turned myself into libre/indie gaming with games such as Cataclysm DDA:Bright Ness with far less requeriments than a UE5 game and yet being enyojable due to playability and in-game lore (and a proper ending compared to vanilla CDDA).
kristopolous 1y ago
You want an M3 ultra Mac studio
- ginko 1y ago
  That only runs Mac OS so it's useless.
  - kristopolous 1y ago
    for ai workloads? You're wrong. I use mine as a server, just ssh into it. I don't even have a keyboard or display hooked up to it. You can get 96gb of vram and about 40-70% the speed of a 4090 for $4000. Especially when you are running a large number of applications you want to talk to each other it makes sense ... the only way to do it on a 4090 is to hit disk, shut the application down, start up the other applciation, read from disk ... it's slowwww... the other option is a multi-gpu system but then it gets into real money. trust me, it's a gamechanger. I just have it sitting in a closet. Use it all the time. The other nice thing is unlike with any Nvidia product, you can walk into an apple store, pay the retail price and get it right away. No scalpers, no hunting.
  - deleted 1y ago
    [deleted]
YetAnotherNick 1y ago
> I'd much prefer paying 3x cost for 3x VRAM Why not just buy 3 card then? These cards doesn't require active cooling anyways and you can just fit 3 in decent sized case. You will get 3x VRAM speed and 3x compute. And if your usecase is llm inference, it will be a lot faster than 1x card with 3x VRAM.
- _zoltan_ 1y ago
  because then instead of RAM bandwidth now you're dealing with PCIe BW which is way less.
  - mythz 1y ago
    Also less power efficient, takes up more PCI slots and a lot of software doesn't support GPU clustering. Already have 4x 16GB GPUs which is unable to run large models exceeding 16GB. Currently running them different VMs to be able to make full use of them, used to have them running in different docker containers however OOM Exceptions would frequently bring down the whole server, which running in VMs helped resolve.
    - zargon 1y ago
      What’s your application for high-VRAM that doesn’t leverage multiple gpus?
  - YetAnotherNick 1y ago
    For LLM inference of batch size 1, it's hard to be saturate PCIe bandwidth specially for less powerful chips. You would get close to linear performance[1]. The obvious issue is few things on multiple GPU is harder, and many softwares don't fully support it or isn't optimized for it. [1]: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inferen...
- zargon 1y ago
  We will buy 4 cards if they are 48 GB or more. At a measly 16 GB, we’re just going to stick with 3090s, P40s, MI50s, etc. > 3x VRAM speed and 3x compute LLM scaling doesn’t work this way. If you have 4 cards, you may get 2x performance increase if you use vLLM. But you’ll also need enough VRAM to run FP8. 3 cards would only run at 1x performance.
0x500x79 1y ago
I think it's a bit of planned obsolescence as well. The 1080ti has been a monster with it's 11GB VRAM up until this generation. A lot of enthusiasts basically call out that Nvidia won't make that mistake again since it led to longer upgrade cycles.
akvadrako 1y ago
AMD Strix Halo has about 100GB of VRAM for around $1500 if that's all you care about.

by mythz 1y ago

doctorpangloss 1y ago

they don't manufacture RAM, so none of the margin goes to them

nullc 1y ago
Even if they put out some super high memory models and just pass the ram through at cost it would increase sales -- potentially quite dramatically and increase their total income a lot and have a good chance of transitioning to being a market leader rather than an also-ran. AMD has lagged so long because of the software ecosystem but the climate now is that they'd only need to support a couple popular model architectures to immediately grab a lot of business. The failure to do so is inexplicable. I expect we will eventually learn that this was about yet another instance of anti-competitive collusion.
- doctorpangloss 1y ago
  the whole RAM industry was twice sanctioned for price fixing, so I agree: any business that deals with RAM has, more likely than other industries by a lot, anti-competitive collusion
- zargon 1y ago
  > The failure to do so is inexplicable. Lisa and Jensen are cousins. I think that explains it. Lisa can easily prove me wrong by releasing a high-memory GPU that significantly undercuts Nvidia's RTX 6000 Pro.
kube-system 1y ago
They sell the completed card, which has margin. You can charge more money for a card with more vram.
- blitzar 1y ago
  Or shrink the margin down to just 50% and sell 10x the number of cards (for the week or two it would take Nvidia to announce a 5090 with 128gb)
  - AnthonyMouse 1y ago
    Nvidia uses VRAM amount for market segmentation. They can't make a 128GB consumer card without cannibalizing their enterprise sales. Which means Intel or AMD making an affordable high-VRAM card is win-win. If Nvidia responds in kind, Nvidia loses a ton of revenue they'd otherwise have available to outspend their smaller competitors on R&D. If they don't, they keep more of those high-margin customers but now the ones who switch to consumer cards are switching to Intel or AMD, which both makes the company who offers it money and helps grow the ecosystem that isn't tied to CUDA. People say things like "it would require higher pin counts" but that's boring. The increase in the amount people would be willing to pay for a card with more VRAM is unambiguously more than the increase in the manufacturing cost. It's more plausible that there could actually be global supply constraints in the manufacture of GDDR, but if that's the case then just use ordinary DDR5 and a wider bus. That's what Apple does and it's fine, and it may even cost less in pins than you save because DDR is cheaper than GDDR. It's not clear what they're thinking by not offering this.
    - singhrac 1y ago
      This is almost true but not quite - I don't think much of the (dollar) spend on enterprise GPUs (H100, B200, etc.) would transfer if there was a 128 GB consumer card. The problem is both memory bandwidth (HBM) and networking (NVLink), which NVIDIA definitely uses to segment consumer vs enterprise hardware. I think your argument is still true overall, though, since there are a lot of "gpu poors" (i.e. grad students) who write/invent in the CUDA ecosystem, and they often work in single card settings. Fwiw Intel did try this with Arctic Sound / Ponte Vecchio, but it was late out the door and did not really perform (see https://chipsandcheese.com/p/intels-ponte-vecchio-chiplets-gone-crazy https://chipsandcheese.com/p/intels-ponte-vecchio-chiplets-g...). It seems like they took on a lot of technical risk; hopefully some of that transfers over to a future project though Falcon Shores was cancelled. They really should should have released some of those chips even at a loss, but I don't know the cost of a tape out.

daemonologist 1y ago

This card does have double the VRAM of the more expensive Nvidia competitor (the A1000, which has 8 GB), but I take your point that it doesn't feel like quite enough to justify giving up the Nvidia ecosystem. The memory bandwidth is also... not great. They also announced a 24 GB B60 and a double-GPU version of the same (saves you physical slots), but it seems like they don't have a release date yet (?).

HelloNurse 1y ago
They don't have a release date yet because they need to see sales results to commit to such a new product.
jjkaczor 1y ago
Yeah am currently building a new PC , and have been waiting and waiting for B60 Pro release dates from partners - here is an offering from ASRock: https://www.asrock.com/Graphics-Card/Intel/Intel%20Arc%20Pro%20B60%20Creator%2024GB/ https://www.asrock.com/Graphics-Card/Intel/Intel%20Arc%20Pro...

zdw 1y ago

I doubt you'd get linear scaling of price/capacity - the larger capacity modules are more expensive per GB than smaller ones, and in some cases are supply constrained. The number of chips on the bus is usually pretty low (1 or 2 of them on most GPUs), so GPUs tend to have to scale out their memory bus widths to get to higher capacity. That's expensive and takes up die space, and for the conventional case (games) isn't generally needed on low end cards. What really needs to happen is someone needs to make some "system seller" game that is incredibly popular and requires like 48GB of memory on the GPU to build demand. But then you have a chicken/egg problem. Example: https://wccftech.com/nvidia-geforce-rtx-5090-128-gb-memory-gpu-for-ai-price-13200-usd/ https://wccftech.com/nvidia-geforce-rtx-5090-128-gb-memory-g...

cmxch 1y ago

Maxsun does offer a high VRAM (48GB) dual Arc Pro B60, but the only US availability has it on par with a 5090 at ~$3000.

PostOnce 1y ago
I think that's actually two GPUs on one card, and not a single GPU with 48GB VRAM
- hengheng 1y ago
  Needs a PCIe bifurcation chip on the main board for all we know. Compatibility is going to be fun.

Ekaros 1y ago

I am not sure there is significant enough market for those. That is selling enough consumer units to cover all design and other costs. From gamer perspective 16GB is now a reasonable point. 32GB is most one would really want and even that not at more than say 100 more price point. This to me is the gamer perspective. This segment really does not need even 32GB, let alone 64GB or more.

drra 1y ago
Never underestimate bragging rights in gamers community. Majority of us run unoptimized systems with that one great piece of gear and as long as the game runs at decent FPS and we have some bragging rights it's all ok.
- rkomorn 1y ago
  Exactly. A decade ago, I put 64GB of RAM in a PC and my friend asked me why I needed that much and I replied "so I can say I have 64GB RAM". The only time usage was "high" was when I created a VM with 48GB RAM just for kicks. It was useless. But I could say I had 64GB RAM.
  - rvba 1y ago
    My work computer with Widnows, Outlook, few tabs open and Excel already craps put with 16 GB If you hae a private computer, why wpuld you even bug something with 16GB in 2025? My 10 year old laptop had that much. Im looking for a new laptop and Im looking at a 128GB setup - so those 200 chrome tabs can eat it, I have space to run other stuff, like those horrible electron chat apps + a game
    - rkomorn 1y ago
      I have 64GB in my current machine and, TBH, it's more than enough. But if I were to upgrade, I'd still get at least 128GB. Nobody's gonna be impressed with 64GB anymore. I don't need that in my life...
1y ago

fredoralive 1y ago

Because the market of people who want huge RAM GPUs for home AI tinkering is basically about 3 Hacker News posters. Who probably won’t buy one because it doesn’t support CUDA. PS5 has something like 16GB unified RAM, and no game is going to really push much beyond that in VRAM use, we don’t really get Crysis style system crushers anymore.

bilekas 1y ago
> PS5 has something like 16GB unified RAM, and no game is going to really push much beyond that in VRAM use, we don’t really get Crysis style system crushers anymore. This isn't really true from the recreational card side, nVidia themselves are reducing the number of 8GB models as a sign of market demand [1]. Games these days are regularly maxing out 6 & 8 GB when running anything above 1080p for 60fps. The prevalence of Unreal Engine 5 also recently with a low quality of optimization for weaker hardware is causing games to be released basically unplayable for most. For recreational use the sentiment is that 8GB is scraping the bottom of the requirements. Again this is partly due to bad optimizations, but games are being played in higher resolutions also, which required more memory for larger texture sizes. [1] https://videocardz.com/newz/nvidia-reportedly-reduces-supply-of-rtx-5060-and-rtx-5060-ti-8gb https://videocardz.com/newz/nvidia-reportedly-reduces-supply...
- deleted 1y ago
  [deleted]
- pjmlp 1y ago
  As someone that started on 8 bit computing, Tim Sweeny is right the Electron garbage culture when applied to Unreal 5 is one of the reasons so much RAM is needed, with such bad performance. While I dislike some of the handmade hero culture, in one thing they are right, regarding how bad modern hardware happens to be used.
  - anthk 1y ago
    I remember UE1 being playable even software mode, such as the first Deus EX. Now, I think the Surreal Engine (UE1 reimplementation) needs damn GL 3.3 (if not 4.5 and Vulkan) to play games I used to play in an Athlon. Now I can't use surreal to Play DX on my legacy n270 netbook with GL 2.1... something that was more than enough to play the game at 800x600 with everything turned on and much more. A good thing is that I turned myself into libre/indie gaming with games such as Cataclysm DDA:Bright Ness with far less requeriments than a UE5 game and yet being enyojable due to playability and in-game lore (and a proper ending compared to vanilla CDDA).

kristopolous 1y ago

You want an M3 ultra Mac studio

ginko 1y ago
That only runs Mac OS so it's useless.
- kristopolous 1y ago
  for ai workloads? You're wrong. I use mine as a server, just ssh into it. I don't even have a keyboard or display hooked up to it. You can get 96gb of vram and about 40-70% the speed of a 4090 for $4000. Especially when you are running a large number of applications you want to talk to each other it makes sense ... the only way to do it on a 4090 is to hit disk, shut the application down, start up the other applciation, read from disk ... it's slowwww... the other option is a multi-gpu system but then it gets into real money. trust me, it's a gamechanger. I just have it sitting in a closet. Use it all the time. The other nice thing is unlike with any Nvidia product, you can walk into an apple store, pay the retail price and get it right away. No scalpers, no hunting.
- deleted 1y ago
  [deleted]

YetAnotherNick 1y ago

> I'd much prefer paying 3x cost for 3x VRAM Why not just buy 3 card then? These cards doesn't require active cooling anyways and you can just fit 3 in decent sized case. You will get 3x VRAM speed and 3x compute. And if your usecase is llm inference, it will be a lot faster than 1x card with 3x VRAM.

_zoltan_ 1y ago
because then instead of RAM bandwidth now you're dealing with PCIe BW which is way less.
- mythz 1y ago
  Also less power efficient, takes up more PCI slots and a lot of software doesn't support GPU clustering. Already have 4x 16GB GPUs which is unable to run large models exceeding 16GB. Currently running them different VMs to be able to make full use of them, used to have them running in different docker containers however OOM Exceptions would frequently bring down the whole server, which running in VMs helped resolve.
  - zargon 1y ago
    What’s your application for high-VRAM that doesn’t leverage multiple gpus?
- YetAnotherNick 1y ago
  For LLM inference of batch size 1, it's hard to be saturate PCIe bandwidth specially for less powerful chips. You would get close to linear performance[1]. The obvious issue is few things on multiple GPU is harder, and many softwares don't fully support it or isn't optimized for it. [1]: https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inferen...
zargon 1y ago
We will buy 4 cards if they are 48 GB or more. At a measly 16 GB, we’re just going to stick with 3090s, P40s, MI50s, etc. > 3x VRAM speed and 3x compute LLM scaling doesn’t work this way. If you have 4 cards, you may get 2x performance increase if you use vLLM. But you’ll also need enough VRAM to run FP8. 3 cards would only run at 1x performance.

0x500x79 1y ago

I think it's a bit of planned obsolescence as well. The 1080ti has been a monster with it's 11GB VRAM up until this generation. A lot of enthusiasts basically call out that Nvidia won't make that mistake again since it led to longer upgrade cycles.

akvadrako 1y ago

AMD Strix Halo has about 100GB of VRAM for around $1500 if that's all you care about.