16 ms·

The 3060/4060 16GB is Nvidia's answer to "I'm a hobbyist/tiny business and I need an okay AI rig at a low price." If Intel could jam 32GB onto a card with compa

The 3060/4060 16GB is Nvidia's answer to "I'm a hobbyist/tiny business and I need an okay AI rig at a low price." If Intel could jam 32GB onto a card with comparable performance, and still compete on price, they might have a winner. They'd have my money, that's for sure. I want to run Stable Diffusion at stupid resolutions, and I want it cheap.

andrewstuart 3y ago
I'm curious to know what prevents Intel putting 64GB or 128GB on a GPU - is there a technical limitation that makes it impossible to go to 128GB? There would certainly be an awkward silence in response from Nvidia. Given that Intel needs some angle to get ahead in the GPU wars I would have thought adding RAM would be an easy way to do that - especially given Nvidia and AMD seem to dish out RAM like its the most precious substance in the universe.
- ryukoposting 3y ago
  As another commented, it's an easy way to enforce segmentation. So, not a good reason, but a reason. I'd also imagine that the hardware on the smaller/cheaper dies doesn't have an address bus wide enough for 128GB.
- qball 3y ago
  >especially given Nvidia and AMD seem to dish out RAM like its the most precious substance in the universe. Well, it's only really nVidia. And the reason it's only nVidia doing this is because they don't want to self-cannibalize; if the 4080 had 24GB literally no prosumer or small organization wanting to play with the tech would even be considering the 4090 (and to a limited extent, the same is true of the 3080 vs. the 3090, but everyone was too busy mining Ethereum to notice that). As far as the other manufacturers are concerned, there's no objective need for anything over 16GB on midrange cards (2070/3060/4060 equivalents) because that's the kind of GPU in the consoles and nobody releases exclusively for PC any more; you'll have to upgrade your card to have parity when the next generation comes out anyway so why provide more/drive costs up? It's not like AMD or Intel GPUs are in any way able to use that RAM for anything else, and why offer it now and kneecap themselves should they somehow get into a position where they actually can offer a good AI suite and want to sell 24GB cards for 2000+ dollars too? And nVidia has taken full advantage of that fact, and that's why the 4080 only has 16GB even though the 7900XTX is performance-competitive with it and has the expected memory compliment. The 4080, including the Super, is a sub-par product because it kind of has to be.
- metadat 3y ago
  > especially given Nvidia and AMD seem to dribble out RAM like its the most precious substance in the universe. Great quote, and apt description of this mildly infuriating trend.
  - orbisvicis 3y ago
    Now I want to see a hardware mod that doubles the amount of RAM in a 4080.
- 15155 3y ago
  The fact that Samsung and Hynix make it, maybe?
justinclift 3y ago
Doesn't stable diffusion just start tiling things when you use large resolution sizes? Not exactly tiled mind you, just "clearly repeating the same thing, with some variation". Asking because when I was playing with it about some months ago, that's what it did. People explained (at the time) that it was because stable diffusion was trained on um... (from rough memory) 512x512 images.
- MrNeon 3y ago
  It does do that but you can generate an image at the normal resolution and then scale it up to run more high-frequency sampling steps to add detail without losing the composition.
  - justinclift 3y ago
    Ahhh cool. Sounds like I should muck around with it some more at some point then. :)
- ryukoposting 3y ago
  Not sure what you mean. I can tell you it pitches a fit if I tell it to go above ~1800x1800 on my 8GB card. I think automatic1111 has a flag to let it page stuff out to main RAM, but then everything is hideously slow.
  - justinclift 3y ago
    k, sounds like things might have improved since I was last mucking around with that stuff (months ago). :)

andrewstuart 3y ago

I'm curious to know what prevents Intel putting 64GB or 128GB on a GPU - is there a technical limitation that makes it impossible to go to 128GB? There would certainly be an awkward silence in response from Nvidia. Given that Intel needs some angle to get ahead in the GPU wars I would have thought adding RAM would be an easy way to do that - especially given Nvidia and AMD seem to dish out RAM like its the most precious substance in the universe.

ryukoposting 3y ago
As another commented, it's an easy way to enforce segmentation. So, not a good reason, but a reason. I'd also imagine that the hardware on the smaller/cheaper dies doesn't have an address bus wide enough for 128GB.
qball 3y ago
>especially given Nvidia and AMD seem to dish out RAM like its the most precious substance in the universe. Well, it's only really nVidia. And the reason it's only nVidia doing this is because they don't want to self-cannibalize; if the 4080 had 24GB literally no prosumer or small organization wanting to play with the tech would even be considering the 4090 (and to a limited extent, the same is true of the 3080 vs. the 3090, but everyone was too busy mining Ethereum to notice that). As far as the other manufacturers are concerned, there's no objective need for anything over 16GB on midrange cards (2070/3060/4060 equivalents) because that's the kind of GPU in the consoles and nobody releases exclusively for PC any more; you'll have to upgrade your card to have parity when the next generation comes out anyway so why provide more/drive costs up? It's not like AMD or Intel GPUs are in any way able to use that RAM for anything else, and why offer it now and kneecap themselves should they somehow get into a position where they actually can offer a good AI suite and want to sell 24GB cards for 2000+ dollars too? And nVidia has taken full advantage of that fact, and that's why the 4080 only has 16GB even though the 7900XTX is performance-competitive with it and has the expected memory compliment. The 4080, including the Super, is a sub-par product because it kind of has to be.
metadat 3y ago
> especially given Nvidia and AMD seem to dribble out RAM like its the most precious substance in the universe. Great quote, and apt description of this mildly infuriating trend.
- orbisvicis 3y ago
  Now I want to see a hardware mod that doubles the amount of RAM in a 4080.
15155 3y ago
The fact that Samsung and Hynix make it, maybe?

justinclift 3y ago

Doesn't stable diffusion just start tiling things when you use large resolution sizes? Not exactly tiled mind you, just "clearly repeating the same thing, with some variation". Asking because when I was playing with it about some months ago, that's what it did. People explained (at the time) that it was because stable diffusion was trained on um... (from rough memory) 512x512 images.

MrNeon 3y ago
It does do that but you can generate an image at the normal resolution and then scale it up to run more high-frequency sampling steps to add detail without losing the composition.
- justinclift 3y ago
  Ahhh cool. Sounds like I should muck around with it some more at some point then. :)
ryukoposting 3y ago
Not sure what you mean. I can tell you it pitches a fit if I tell it to go above ~1800x1800 on my 8GB card. I think automatic1111 has a flag to let it page stuff out to main RAM, but then everything is hideously slow.
- justinclift 3y ago
  k, sounds like things might have improved since I was last mucking around with that stuff (months ago). :)