4 ms·

Does regular RAM matter if I want to run stable diffusion? I only have 16GB RAM (but obviously upgradable). If I load the weights onto the VRAM, does it work SS

Does regular RAM matter if I want to run stable diffusion? I only have 16GB RAM (but obviously upgradable). If I load the weights onto the VRAM, does it work SSD > RAM > VRAM or does it bypass the RAM?

aseipp 4y ago
Yes, Stable Diffusion performance will improve the more VRAM you give it. This is generally true of almost every ML model, because ML models are often limited by bandwidth. There is a constant need to shove more data into the accelerator, but PCIe is very slow compared to data in VRAM. If you constantly had to push data over PCIe, your GPU would run at 5% of its max performance, and spend 95% of its time waiting on PCIe transfers. Instead, you load a bunch of data over PCIE at once into VRAM, as much as possible, and then you begin working. So the more VRAM you have, the more data you have "in hand" -- so you can have a bigger batch size for your training iterations, for example. So one of the things various releases of Stable Diffusion have tried to optimize, for example, is VRAM usage, because that's the #1 limiting factor for most people. Modern GPUs use virtual memory so they can see very large amounts of data and the GPU and OS will work together to page data in and out, like normal RAM. But if you want good performance, you need to work out of VRAM, so you need as much of it as possible, so you aren't paging data in and out. (More specifically to answer your question, yes it will page data from SSD into RAM and then into VRAM; this is because the data must pass through the memory controller in practice on most designs to go over PCIe, but if you are using fancier Tesla/Quadro cards, they can actually do P2P DMA transfers and go directly from SSD -> GPU, but this requires special care and support from all pieces of hardware inbetween.) Note that in a funny twist of fate, the M1 series from Apple kind of has an advantage here. They use very high-bandwidth LPDDR5X for their designs, and the GPU and CPU share that. Which means that in theory, the Mac Studio for example could have astronomically large amounts of VRAM dedicated to a task like ML. So why not use a Mac? Because the hardware is only good as the software you run it on, and training/tuning ML software on Macs is simply nowhere near as well polished as Linux/Nvidia. That's all it comes down to. And I say "well polished" very liberally; in practice it's all a bit of a hodge-podge nightmare because the job is very hodge-podge...
lostmsu 4y ago
You don't need to care about this.
moandcompany 4y ago
The model checkpoint file (i.e. weights) being used is read from disk, rather than from system memory afaik. 16GB of system RAM is more than sufficient.
sdflhasjd 4y ago
It is fairly RAM heavy, but 16 should be plenty

aseipp 4y ago

Yes, Stable Diffusion performance will improve the more VRAM you give it. This is generally true of almost every ML model, because ML models are often limited by bandwidth. There is a constant need to shove more data into the accelerator, but PCIe is very slow compared to data in VRAM. If you constantly had to push data over PCIe, your GPU would run at 5% of its max performance, and spend 95% of its time waiting on PCIe transfers. Instead, you load a bunch of data over PCIE at once into VRAM, as much as possible, and then you begin working. So the more VRAM you have, the more data you have "in hand" -- so you can have a bigger batch size for your training iterations, for example. So one of the things various releases of Stable Diffusion have tried to optimize, for example, is VRAM usage, because that's the #1 limiting factor for most people. Modern GPUs use virtual memory so they can see very large amounts of data and the GPU and OS will work together to page data in and out, like normal RAM. But if you want good performance, you need to work out of VRAM, so you need as much of it as possible, so you aren't paging data in and out. (More specifically to answer your question, yes it will page data from SSD into RAM and then into VRAM; this is because the data must pass through the memory controller in practice on most designs to go over PCIe, but if you are using fancier Tesla/Quadro cards, they can actually do P2P DMA transfers and go directly from SSD -> GPU, but this requires special care and support from all pieces of hardware inbetween.) Note that in a funny twist of fate, the M1 series from Apple kind of has an advantage here. They use very high-bandwidth LPDDR5X for their designs, and the GPU and CPU share that. Which means that in theory, the Mac Studio for example could have astronomically large amounts of VRAM dedicated to a task like ML. So why not use a Mac? Because the hardware is only good as the software you run it on, and training/tuning ML software on Macs is simply nowhere near as well polished as Linux/Nvidia. That's all it comes down to. And I say "well polished" very liberally; in practice it's all a bit of a hodge-podge nightmare because the job is very hodge-podge...

lostmsu 4y ago

You don't need to care about this.

moandcompany 4y ago

The model checkpoint file (i.e. weights) being used is read from disk, rather than from system memory afaik. 16GB of system RAM is more than sufficient.

sdflhasjd 4y ago

It is fairly RAM heavy, but 16 should be plenty