5 ms·

I started out using my old GTX 1080 on Thursday, could generate 512x512 just fine. That's in 8G of VRAM. It worked well on the hlky branch using webui (built us

I started out using my old GTX 1080 on Thursday, could generate 512x512 just fine. That's in 8G of VRAM. It worked well on the hlky branch using webui (built using gradio). Seeing that training etc. is much more memory intensive, and wanting to get faster results, I bought an RTX 3090, which has 24G of VRAM. However it maxes out at about 1024x512, only twice as many pixels. Observing the card with GPUZ, it never actually allocates more than 13.9G. Using the lstein branch, I can't get above 896x512. Similarly, GPUZ shows allocated VRAM never reaches 14G. The interface isn't as good as the webui on hlky either - never mind the web interface, a bigger problem is it doesn't save all the parameters alongside generated images. This is all running using Miniconda on Windows. On Linux it may be a different story, but my gaming PC is not dual-boot (yet).

WhereWhyWhat 4y ago
How many it/s do you get with 3090 compared to the 1080? My 1080 Ti gets around 2.5it/s with the k_lms sampler.
- capableweb 4y ago
  On a 2080 Ti I get around 8 it/s with k_lms
- barrkel 4y ago
  Where the 1080 would do about 2 it/sec, the 3090 does about 10 it/sec. When I do batches, it slows down but not linearly; if 1x does 10 it/sec, 2x does about 6 it/sec. Batching is the other upside of more VRAM.
pixl97 4y ago
Maybe look in to booting off a USB stick as a means to test this. I wouldn't be surprised if there were some kind of driver reservation I Windows causing this issue.
MattRix 4y ago
Have you tried something like 1024x768? Going to full 1024x1024 would double your VRAM usage so I can see why that wouldn’t work. For my uses, the real benefit of having more VRAM is that you can generate more images simultaneously. My 3080 can generate only one 512x512 in 7 seconds but three 384x384 in that same timeframe. It’s allowed me to generate grids of hundreds of images in just a few minutes.
- the-golden-one 4y ago
  I can squeeze 640x512 out of my Mobile RTX 3080 with 8Gb VRAM (as the secondary GPU). Anything more than that fails.
- barrkel 4y ago
  1024x768 didn't work until I bumped my system commit cap (i.e. increased my paging file max size). I hadn't paid attention to system memory, this box only has 32G.
ladberg 4y ago
Just checking, are you also using the same GPU for rendering your desktop? If so then try switching over to your integrated GPU or the 1080 if it’s still attached so you can leave 100% of the 3090 available to the network.
- glenneroo 4y ago
  That probably won't help much since OP said they aren't even using 14GB of VRAM. I have dual GPUs and use the 2nd one (3060 Ti with no monitors connected) for rendering, which is nice because I have the full 8GB free.
  - barrkel 4y ago
    I was looking at GPU graphs and neglected my physical RAM. This machine only has 32G and I didn't notice I was hitting a ceiling on memory allocations too - I took the error message about GPU memory allocations at face value. I bumped my system commit cap (increased paging file size) by 24G and now I can use all my VRAM.
    - Mossly 4y ago
      I'm doing 576x576 on my 3060 Ti - just wondering the max resolution you can render now using all your VRAM?
tbalsam 4y ago
Recommendations: - Linux w/ Display drivers on CPU (and just...ditch miniconda please) - Use lower FP precision mode if available to use the tensor cores (also to double "effective" memory) - Batch things! - I don't know what the max resolution of the diffusion network is, you may have to just simply tile it past a certain point (with overlap please! ;P Hope that helps somewhat. A 3090 should be more than enough for what you're doing, I'm stuck with P100s at best for me! (Cost :'( ) Best of luck! :D :)
- yoru-sulfur 4y ago
  > and just...ditch miniconda please Why the recommendation to stop using miniconda?
  - r2_pilot 4y ago
    Well, for starters(not that proficient with PyCharm as I only use it for less-professional use), I have tried for a few days to get PyCharm to use the miniconda environment on Windows, to no avail. It at least works on the command line so I get to play around with it. I'm going to spend more time tomorrow trying alternatives.
- cbozeman 4y ago
  I ordered an NVIDIA Tesla K80 off eBay (and the power adapter... and the blower fan shroud, etc.) and intend to install it when it arrives around Thursday or Friday. I'm hoping that after I install the NVIDIA Linux datacenter drivers I'll be able to use the card with SD. My only worry is that because the K80 is two GPUs on one board, that it might only utilize one of them, with only 12 GB of VRAM instead of both chips and all 24 GB. 5992 CUDA cores and 24 GB VRAM would be a pretty decent SD accelerator for only $150.
  - monkmartinez 4y ago
    I really want to put one of these in my Dell Precision workstation, and I share similar concerns. I have an older Quadro in it now and could use a proper upgrade, but I really want to wait for the 4000 series cards due in a few months. Craft Computing on Youtube has the best information from what I have seen so far. I don't like watching Youtube videos for information like this, but I understand why creators have moved to this medium in general. Linux should be much easier to configure for using the K80 to capacity.
TrinaryWorksToo 4y ago
WSL2 is a thing you should look in to.
monkmartinez 4y ago
How have you configured Pytorch in the 'setup' section for your card? The hlky/webui (Shout out to Altryne), is configured for lower end GPU that are memory constrained. The knobs that need a twistn' on these DL models feel infinitesimal.

WhereWhyWhat 4y ago

How many it/s do you get with 3090 compared to the 1080? My 1080 Ti gets around 2.5it/s with the k_lms sampler.

capableweb 4y ago
On a 2080 Ti I get around 8 it/s with k_lms
barrkel 4y ago
Where the 1080 would do about 2 it/sec, the 3090 does about 10 it/sec. When I do batches, it slows down but not linearly; if 1x does 10 it/sec, 2x does about 6 it/sec. Batching is the other upside of more VRAM.

pixl97 4y ago

Maybe look in to booting off a USB stick as a means to test this. I wouldn't be surprised if there were some kind of driver reservation I Windows causing this issue.

MattRix 4y ago

Have you tried something like 1024x768? Going to full 1024x1024 would double your VRAM usage so I can see why that wouldn’t work. For my uses, the real benefit of having more VRAM is that you can generate more images simultaneously. My 3080 can generate only one 512x512 in 7 seconds but three 384x384 in that same timeframe. It’s allowed me to generate grids of hundreds of images in just a few minutes.

the-golden-one 4y ago
I can squeeze 640x512 out of my Mobile RTX 3080 with 8Gb VRAM (as the secondary GPU). Anything more than that fails.
barrkel 4y ago
1024x768 didn't work until I bumped my system commit cap (i.e. increased my paging file max size). I hadn't paid attention to system memory, this box only has 32G.

ladberg 4y ago

Just checking, are you also using the same GPU for rendering your desktop? If so then try switching over to your integrated GPU or the 1080 if it’s still attached so you can leave 100% of the 3090 available to the network.

glenneroo 4y ago
That probably won't help much since OP said they aren't even using 14GB of VRAM. I have dual GPUs and use the 2nd one (3060 Ti with no monitors connected) for rendering, which is nice because I have the full 8GB free.
- barrkel 4y ago
  I was looking at GPU graphs and neglected my physical RAM. This machine only has 32G and I didn't notice I was hitting a ceiling on memory allocations too - I took the error message about GPU memory allocations at face value. I bumped my system commit cap (increased paging file size) by 24G and now I can use all my VRAM.
  - Mossly 4y ago
    I'm doing 576x576 on my 3060 Ti - just wondering the max resolution you can render now using all your VRAM?

tbalsam 4y ago

Recommendations: - Linux w/ Display drivers on CPU (and just...ditch miniconda please) - Use lower FP precision mode if available to use the tensor cores (also to double "effective" memory) - Batch things! - I don't know what the max resolution of the diffusion network is, you may have to just simply tile it past a certain point (with overlap please! ;P Hope that helps somewhat. A 3090 should be more than enough for what you're doing, I'm stuck with P100s at best for me! (Cost :'( ) Best of luck! :D :)

yoru-sulfur 4y ago
> and just...ditch miniconda please Why the recommendation to stop using miniconda?
- r2_pilot 4y ago
  Well, for starters(not that proficient with PyCharm as I only use it for less-professional use), I have tried for a few days to get PyCharm to use the miniconda environment on Windows, to no avail. It at least works on the command line so I get to play around with it. I'm going to spend more time tomorrow trying alternatives.
cbozeman 4y ago
I ordered an NVIDIA Tesla K80 off eBay (and the power adapter... and the blower fan shroud, etc.) and intend to install it when it arrives around Thursday or Friday. I'm hoping that after I install the NVIDIA Linux datacenter drivers I'll be able to use the card with SD. My only worry is that because the K80 is two GPUs on one board, that it might only utilize one of them, with only 12 GB of VRAM instead of both chips and all 24 GB. 5992 CUDA cores and 24 GB VRAM would be a pretty decent SD accelerator for only $150.
- monkmartinez 4y ago
  I really want to put one of these in my Dell Precision workstation, and I share similar concerns. I have an older Quadro in it now and could use a proper upgrade, but I really want to wait for the 4000 series cards due in a few months. Craft Computing on Youtube has the best information from what I have seen so far. I don't like watching Youtube videos for information like this, but I understand why creators have moved to this medium in general. Linux should be much easier to configure for using the K80 to capacity.

TrinaryWorksToo 4y ago

WSL2 is a thing you should look in to.

monkmartinez 4y ago

How have you configured Pytorch in the 'setup' section for your card? The hlky/webui (Shout out to Altryne), is configured for lower end GPU that are memory constrained. The knobs that need a twistn' on these DL models feel infinitesimal.